Q: “Why couldn’t we just shut off a computer if it got too powerful?” A: “A smart AI could predict that we’d want to turn it off if it made us nervous. So it would try hard not to make us nervous, because doing so wouldn’t help it accomplish its goals” https://www.vox.com/future-perfect/2018/12/21/18126576/ai-artificial-intelligence-machine-learning-safety-alignment …
-
-
Replying to @FrankPasquale
Why would we give a computer such a high degree of autonomy, for example, autonomy to self-improve without human intervention or evaluation. This makes no sense from an engineering perspective.
7 replies 2 retweets 10 likes -
Replying to @tdietterich @FrankPasquale
Before we deploy any AI system, we must test it carefully. We wouldn't open a bridge or a skyscraper without first testing the quality of the construction work. The same applies to software. The doomsday scenarios all rely on violating such norms.
3 replies 4 retweets 10 likes -
Replying to @tdietterich @FrankPasquale
A general AI will easily be taken down by a simpler, more specialized AI whose only purpose is to take it down. Just like a simple virus can kill a human. It will be even easier if the simple AI killer has access to the similar computing resources as the general AI.
5 replies 6 retweets 37 likes -
If that was a certainty (not just possible), why did evolution not favor a viral goo over complex organisms?
3 replies 0 retweets 5 likes -
Viruses cannot exist without a (more complex) host organism.
1 reply 0 retweets 2 likes
I agree, a microbial goo would have been a more accurate metaphor. (Your idea that an AGI race might be won by a low complexity solution is not implausible, and was explored by Lem 1988: Peace on Earth. I suspect that low complexity dominance might be only a local optimum.)
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.