I think "AIs make paperclips" has probably obtained more audience than all of my other conceptual originations combined. I guess it's not very surprising that it's an invention that had the potential for easy misunderstanding, and that it's the misunderstanding that spread.
-
-
3/ Misunderstood and widespread meaning: The first AGI ever to arise could show up in a paperclip factory (instead of a research lab specifically trying to do that). And then because AIs just mechanically carry out orders, it does what the humans had in mind, but too much of it.
Show this thread -
4/ The key intended ideas are, (a), it requires no defect of intelligence, rationality, or reflectivity to optimize for paperclips, and (b), if you *don't* manage to align an AI and have it do something interpretable, the result will look as meaningless as paperclips.
Show this thread -
5/ Imagining that an AI just goes on making paperclips because it's mechanical and unreflective sends the opposite message of (a). Supposing that the AI was meant to run a paperclip factory, and deliberately successfully aligned to do that, sends the opposite message of (b).
Show this thread
End of conversation
New conversation -
-
-
AI has a high likelihood of concluding either banal or seemingly brutal truths about human value and identity, to the point there will probably be a programmatic panic to correct the "error"
-
This should be irrelevant to any reasonable design and methodology for alignment. I know some of the people who say they want to do this. They're smarter than that. Not smart enough to survive the real technical problems, but smart enough not to die of cliche movie plots.
End of conversation
New conversation -
-
-
Unless you're a paperclips CEO.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.