People assume human abilities stay constant while automation is improving. They don’t. The general degradation starts the moment you begin trusting the automation for *anything* because unused or rarely used skills degrade at the rate reinforcement schedule slows.
-
-
Show this thread
-
It’s been a long-standing debate in control theory and AI, but based on a false assumption: that such a thing as unmaintained emergency override skills can exist. Ie that humans deploy behavior X *only* in the emergencies where the automation fails. There is no such thing.
Show this thread -
It is somewhere between malpractice and technical stupidity to base a design on this assumption. When the rate of use of a skill is lower than the rate of practice needed to keep it usable, it must be maintained at minimum practice level.
Show this thread -
Driverless cars are the most familiar example. But any high-risk behavior that requires human backstopping of automation qualifies for this death-trap effect. I think a lot of infrastructure is either in, or getting close to, the death trap zone.
Show this thread -
This is a much more serious and real risk than AGI/singularity BS. Too much infrastructure being in the death-trap state at the same time due to correlated automation wave causing systemic collapses via contagion and 2nd order effects as failures cascade.
Show this thread
End of conversation
New conversation -
-
-
Hmm. The gift of automation comes with the curse of being outmaneuvered by that tech because we can’t transcend our evolution. It’s the Monkey Pause.
-
Not outmaneuvered. Crashed.
End of conversation
New conversation -
-
-
The concave shape of the human capability is like that of the projected forgetting curves of the spacing effect. There’s no way to cram for an emergency.
-
oh yeah I didn't think of that
End of conversation
New conversation -
-
-
Can I use this image in a blog post entitled "Venkatesh Rao's Automation Death Trap Effect"?
-
Sure. I’ll eventually blog a more thought through version somewhere
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.