Conversation

It's strange how one quirk of wording, on an email you fired off on two minutes' composition to a mailing list, can shape a conversation so much. The trouble is, of course, that we don't know in advance which emails or quirks of wording will shape the conversation for 20 years.
4
133
On the other hand, you *did* manage to capture many people's intuitions and imaginations on the subject this way, and I'm not sure a more abstract concept would have had as much impact
1
19
People who can't be captured by the word "squiggle" instead of "paperclip" because "squiggle" is too abstract are bluntly not people I want in this conversation.
7
13
Show replies
Paperclip maximizer worked as a meme because it was evocative; you can’t stop people from misinterpreting an analogy. It worked for me at least to creative a quick mental shorthand for a particular type of optimizer failure. I don’t think the “more accurate” version would have.
1
69
Show replies
I think people who know enough already to consider the inner/outer alignment distinction are not the people you need to reach with the paperclip maximiser thought experiment. Also, wasn't it Bostrom who first described it? If not, Wikipedia is wrong.
1
3
Show replies
Show replies
Wait, what? I thought you were using the paperclips to illustrate something that a superintelligence might pursue that's clearly not valuable in response to people saying that "whatever a superintelligence might want to do with the lightcone, it must be Good"
1
5
That's part of the concept, but another part is that this happens, not because capitalist fools built a runaway paperclip factory, but because a leading research AI was undeliberately built in a way that caused its actual utility function to max out on tiny squiggles.
2
19
Show replies