I think "AIs make paperclips" has probably obtained more audience than all of my other conceptual originations combined. I guess it's not very surprising that it's an invention that had the potential for easy misunderstanding, and that it's the misunderstanding that spread.
-
Show this thread
-
2/ Original usage and intended meaning: The problem with turning the future over to just any superintelligence is that its utility function may have its attainable maximum at states we'd see as very low-value, even from the most cosmopolitan standpoint.https://arbital.com/p/paperclip/
3 replies 6 retweets 31 likesShow this thread -
3/ Misunderstood and widespread meaning: The first AGI ever to arise could show up in a paperclip factory (instead of a research lab specifically trying to do that). And then because AIs just mechanically carry out orders, it does what the humans had in mind, but too much of it.
2 replies 0 retweets 20 likesShow this thread -
Replying to @ESYudkowsky
Oh, so like the factory programming just accidentally becomes intelligent enough to "explode?"
1 reply 0 retweets 0 likes
That is the MUTATION of the meme which is the OPPOSITE of the scenario I was originally describing, yes. My scenario was that AGI happened in a dedicated research lab, alignment failed, and the resulting utility function had its maximum at tiny molecules shaped like paperclips.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.