Top-level takeaway of tonight's catch-up reading on ML...
I still find it hard to take AI Risk seriously as a special problem here. There are definite risks but they don't seem qualitatively different from other kinds of engineering risk.
Conversation
Replying to
I don't think I've seen this argument. You should tweetstorm it.
Replying to
Yup. I reached basically this conclusion a few years ago. Most conceptualizations of a hostile AI are a "how *I* might be evil" projection. I can guess how Golem allegory is used now. Looks like I might have saved myself some time if I'd persisted with Wiener beyond a quick gloss
Replying to
Even the notionally beyond-good-and-evil ideas (kill all humans as apathetic side effect of paperclip maximization) fall prey to the trap of means-ends nihilism
Replying to
Most safety engineering seems to focus on one (or a small enumerable set) of standard functioning modes, and verifies all parts are rated for that amount of eg torque. AI safety looks more like ensuring humanity is rated for a function space over the reals
1
Replying to
Climate risk, nuclear arms control, are just 2 examples that also look like that. More so in fact.
1
1
Show replies
Replying to
I think one should also consider the grey goo scenarios of AI risk.
E.g. personally targeted advertising (weak AI) convinces people to vote badly (scaled social engineering)
1
1
Replying to
Yeah I take this kind more seriously, but it seems closer to a disease epidemic
1
1
Show replies
Replying to
If you limit your definition of AI to ML and other narrowly scoped methods applied to similarly narrow domains then yes, AI risks probably arent categorically different to other engineering risks. But are you saying that AGI also is in this category?
1
Replying to
Disagree for 2 specific reasons. Other engineered products do not hold the potential to 1. Become independently self replicating 2. Autonomously deviate from initial design.
2
1
Replying to
Computer viruses already have that ability in a limited way (independently self-replicating only within specific computer networks, not the open world). Mutating memes in general qualify if you treat human brains as a replication substrate with low agency




