Oatmeal Agitante

Armchair critique of "AI Alignment": Defining an aligned utility function is no harder than defining a sufficiently sophisticated goal

Making an AI that is "aligned" is a distraction. Make one that is "not dirt stupid" first

Thought experiments I see to illustrate alignment problems can be trivially reformulated to be about "not being dirt stupid"

I have not seen one where an agent behaving catastrophically badly can't be substituted for behaving stupidly and failing

I think we don't have a good way to make agents behave intelligently in environments without fully-defined affordances

Reinforcement learning is easy in environments w fully-defined affordances, we have no idea how to do it w/o

We aren't going to put a "reward button" in one w/o, and expect an agent to "just be nice", let alone "not dirt stupid"