Conversation

Top-level takeaway of tonight's catch-up reading on ML... I still find it hard to take AI Risk seriously as a special problem here. There are definite risks but they don't seem qualitatively different from other kinds of engineering risk.
9
14
Replying to
Most safety engineering seems to focus on one (or a small enumerable set) of standard functioning modes, and verifies all parts are rated for that amount of eg torque. AI safety looks more like ensuring humanity is rated for a function space over the reals
1
Replying to
That’s why I’m also fairly concerned about those—very complex systems with many cases to check. But 1) AI is this but more so, and 2) we spend billions every year on those two but can list on two hands all AI safety researchers
1
Replying to
That's my point. We actually spend a lot more if you classify right. All infosec research is also "AI risk" research if you conceptualize correctly without anthropomorphizing for example. If you buy an arbitrary religious boundary, there's few people inside it.
Replying to and
Personally I look at AI risk research as consisting of robustness and/or alignment work (infosec is robustness) that can scale to ML-powered systems that are stronger, faster, and harder to understand than today’s (a lot of infosec work won’t).
2
Replying to and
there's a bunch of work that would be useful to mitigate the risk of AI related accidents (e.g. formal verification, infosec), but much less happening that's directly aimed at the big, less understood problems (e.g. value alignment)