Conversation

Deleted previous version of the tweet where I mistakenly attributed it to Bret Victor rather than Maciej Cegłowski. That makes much more sense. I was surprised to find myself agreeing with what I thought was Victor. In my head “idlewords” somehow sounds close to “worrydream”
2
25
My diagnosis was always a kind of anti-projection. a) You think in a totalizing (INTJish) way and are impressed by its power b) You see a machine that thinks in analogous ways and looks like it lacks your limits c) You extrapolate its future as you minus biological limits
2
35
Note this is specifically a critique of the Bostrom-LW vision of the future of AI, based on an IQ++ model of what intelligence is. Not of all possible futures for the tech. It’s one that commits to a a sequential evolutionary model where the prefix “super” makes sense.
1
14
The reason I don’t bother engaging with this conversation is that my starting point is ontologically at the opposite pole from IQ++. I don’t find “entrance tests for bureaucratic industrial orgs to test aptitude for their legible functions” to be an interesting place to start.
2
32
Mine is: “the brain is a 100 billion neuron system that from the inside (“mind”) doesn’t *feel* like it has 100 billion elements, but more like dozens to 100s of high level salient emergent phenomena operating on a rich narrative and verbal memory... what else looks like that?”
2
47
The answers are things like markets, ecosystems, weather systems. Billions of atomic moving parts, but quasi-stable macro-phenomenology. There may be nothing it is “like” to be a “market” but setting aside the hard problem of consciousness it is in the brain-class of things.
5
65
The most interesting and salient thing about these systems is that they are coherent and stable in a thermodynamic sense, maintaining boundary integrity and internal structural identity continuity for periods of time ranging from tens to thousands of years.
2
26