You might reply, "Nuh-uh, that wouldn't happen; evolution gave us souls", but that's only because (a) the environment of evolutionary adaptedness was more complicated (war was just one of many adaptive problems), and (b) evolution is a terrible engineer. 4/3
-
-
Replying to @zackmdavis @robbensinger
The actual tip of the spear is only a minor part of what makes an effective war band. The main problem with the near future mil tech scenarios you're describing is that they're *scalable* - better to be big (to produce more of them) and absorb *everything*
1 reply 0 retweets 2 likes -
Once a single society absorbs all the others then it is free to spin into madness as status maximizers short circuit all the evolved systems for keeping them out (and there's no threat of external system destruction to keep them under control).
1 reply 0 retweets 3 likes -
Curtis Yarvin had an interesting proposal on his substack that when an orbital power emerges it should allow unrestricted, constant pre-industrial warfare (no chemical explosives, no compressed gasses).
1 reply 0 retweets 3 likes -
Closed systems tend to break down as people prioritize rising up in the system over the good of the system as a whole - there needs to be some kind of check on that tendency.
1 reply 0 retweets 2 likes -
Replying to @CovfefeAnon @robbensinger
My guys (I'll say "guys" when talking to you, but we don't call ourselves that because we're fake and gay) are worried about a more radical problem. The entire paradigm of some values being more "fit", only applies to a world where evolution is the only optimization process. 1/3
1 reply 0 retweets 2 likes -
Once you have consequentialist agents, values become much more arbitrary! In a Darwinian world with no lookahead, creatures have to intrinsically love (ancestral correlates of) war and reproduction or they'd go extinct. But once you have general intelligence 2/3
1 reply 0 retweets 2 likes -
you can calculate the instrumentally best strategy (for competing with other agents in an open system, or just resource extraction if you've become a closed system) for whatever your values are, and just do that. This is unprecedented in the history of life

3/31 reply 0 retweets 2 likes -
Replying to @zackmdavis @robbensinger
In a closed system the calculation is never really the barrier - the implementation is. The "best" strategy is simple - form a team that can cooperate without direct communication, block coordination and cooperation of anyone who could form a competing team.
1 reply 0 retweets 3 likes -
The "problem" is how to keep defectors (from the common good) from defecting (on your tacit conspiracy). I've become convinced that there could be a real problem with machine intelligence but it's unsolvable as long as every institution is run by an egregore that works that way.
1 reply 0 retweets 3 likes
The egreogore is incapable of acting in any way that doesn't go up the slope of adaptation - since it has no leader, it can't choose to go downward to hit a greater peak.
-
-
Replying to @CovfefeAnon @robbensinger
My guys are thinking along similar lines ("The system isn’t in the best Nash equilibrium because nobody has the power to look over the system and choose good Nash equilibria") but they still don't know about egregores yet (I fear we are ngmi
)https://equilibriabook.com/molochs-toolbox/ …0 replies 1 retweet 3 likesThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.