The Legendary Dragon Alephwyr, named Alephwyr, BA

Previous thread about this from February. Wish I’d indexed all my threads over the past 5-6 years to see if my positions have evolved or shifted.https://twitter.com/vgr/status/1363965583860011015 …

But to the original QT, a new point I’m adding here is the non-trivial observation that those who most believe in AGIs also happen to be convinced they are the smartest people around (and apparently manage to convince some around them, though Matt appears to be snarking).

Circular like the anthropic principle. You notice that earth is optimized to sustain human life. Your first thought is, a God created this place just for us. Then you have the more sophisticated thought that if it weren’t Goldilocks optimal we wouldn’t be around to wonder why…

But notice that the first thought posits a specific kind of extrapolation — an egocentric extrapolation. “God” is not a random construct but an extrapolation of an egocentric self-image as “cause” of Goldilocks zone. The second thought makes it unnecessary to posit that.

Flip it around to be teleological. In this case, a certain class of people do well in a pattern of civilization. If you assume that pattern is eternal, that class of people suggest evolution to an alluring god-like omega point and a worry that machines will get there first.

But as a skeptic, you wonder… if this civilization didn’t have this pattern, these people wouldn’t be around worrying about superintelligence. Some other group would be. Top dogs always fight imaginary gods.

No accident that the same crowd is also most interested in living forever. A self-perpetuation drive shapes this entire thought space.

This world is your oyster, you’re winning unreasonably easily and feel special. You want it to continue. You imagine going from temporarily successful human to permanently successful superhuman. Attribution bias helps pick out variables to extrapolate and competitors to fear.