Superintelligence cannot be contained: Lessons from Computability Theoryhttps://arxiv.org/abs/1607.00913
-
-
Replying to @nyrath
A question I like to ask, is if we build an AGI superintelligence, how can we tell we did it right? You can say "E=MC^2" to your dog every day of its life, and it will never understand. An AGI SI could do the same to us, saying something profound, and we'd hear gibberish.
2 replies 1 retweet 4 likes -
Replying to @ConradTeves @nyrath
Power over the universe (engineering). We obviously know more about electricity than dogs, as proven by electric lights. If a superintelligence made some claim that we could barely comprehend, but couldn't verify the correctness of, it could show the proof by showing the pudding.
2 replies 0 retweets 2 likes -
I get reluctance to accept there's nothing we can't grasp if explained well, but learning is also constrained by time. I'd bet that Magnus Carlsen could play chess as well as Alpha Zero if he played 3.5M games too, but no human lives long enough to walk the path to the knowledge.
1 reply 0 retweets 0 likes -
However, the "proof is in the pudding" runs up against a problem. If we don't understand the claim, how does the proof make sense either? I throw a ball in the air and it arcs over and the dog catches it. Does the dog think, "evidence of spacetime curvature confirmed!"?
1 reply 0 retweets 0 likes -
Replying to @ConradTeves @nyrath
I'm arguing something in the direction of a Zero Knowledge Proof https://en.wikipedia.org/wiki/Zero-knowledge_proof … If a human tells a dog "I have a model of Newtonian physics; set this trebuchet to any counterweight size, any payload mass, and any arm length and I will predict where it lands" >>>
1 reply 0 retweets 0 likes -
the dog, using the dog equivalent of Bayesianism, will sort of trust my claim re Newtonian physics after 1 test, and very much trust it after 20 tests. Likewise us and superintelligences. If they have a model of truths we can't even understand, and they make predictions that >
1 reply 0 retweets 0 likes -
come true, repeatedly, then that doesn't mean that we should trust them that the model X they speak of is the actual mode of the universe, but we should trust them that they have SOME model X or Y or Z that works.
1 reply 0 retweets 1 like -
btw, I 100% understand and agree with your core point > I get reluctance to accept there's nothing we can't grasp if explained well, but learning is also constrained by time nope, no reluctant at all I'm not arguing that we can GRASP all truths, merely that many can be proven
1 reply 0 retweets 1 like -
Yeah, in physics I would think it (hopefully) harder for a mad AI to be yanking our chain because it thought it was funny. "Lookit the humans go! They're gonna build cool stuff with this idea but boy are they in for a surprise once they rely on it! LOL."
#nightmarefuel1 reply 0 retweets 1 like
ok, I think we're in general agreement, then - (a) that in physics an AI can prove certain things (e.g. prove that it knows SOME x in set X, but could lie and say that it knows x1 when it actually knows x2) , (b) in other domains more resistant to proofs, not so much
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.