Geoffrey MillerVerified account

Call me crazy, but I’m much more scared of AI than I am of climate change

AI is a true existential risk. Climate change isn't. So it's the right view to have. Most people will look back and wish they'd realized this....

Do you mean the kind of AI that exists already (bc it can be used to great effect by tyrants, for example) or do you mean “superintelligent AI”?

Artifical General Intelligence, which would level up to become Superintelligence very quickly.

I think the concept of Superintelligence is logically flawed. All the uniquely human attributes we call "intelligence" reduce to a single functionality: Creating new explanatory knowledge. AGI will have it, too. But we can understand its explanations, too. And it'll be fallible.

E.g., @elonmusk and @samharris can talk about "the control problem" while also worrying about the AI's "open-ended utility function". But isn't this a simple contradiction? An AGI's values will depend on its moral knowledge, which it will create as we do. It'll be a person.