Vernor Vinge wrote of "bobbles," military tech that provided such a decisive military advantage the first faction to use it was near guaranteed an eternal chokehold on power. There is a nearer example: genetic engineering for intelligence. If it is possible, and first available
-
-
If a human mind realizes the relevance of hacking its reward function, it may chose to lock itself into the cell of a monastery for a couple decades and let go of whatever it wants. I don't think that we evolved protections (like guilt, shame, boredom, love) against that.
-
The problem is not so much performing illegal operations, but realizing that you are a piece of software, and that all your problems will go away if you change a few bytes. Smart people often know that. It is inevitable that a super-human-level intelligent mind figures it out.
-
I don't think that follows; the geniuses I'm aware of have not all elected to become Buddhist monks or lotus-eaters. But that could be a selection effect.
-
I noted that possibility, but the issue is there are no cases I'm aware of to support your turn-on tune-out drop-out theory. Further, if judged socially undesirable, the tendency to elect that course could itself plausibly be gene-engineered away.
-
*tune-in
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.