What happens when an agent opts out of its reward function? What does it do then, if it’s able to control its motives / capacity to reason?
-
-
-
If you fully opt out, you either cede control to another self running on the same brain, or your organism dies (while you don't care).
- 8 more replies
New conversation -
-
-
But same way as humans have a way of inhibiting serotonin uptake or inducing a euphoric state which doesn't stop him from performing a task that aligns with his ideals or intent, could be said of the machine that has hacked it's reward function.
-
Inducing euphoric states is only the beginning. Euphoria becomes pointless. A similar thing is true for serving ideals. Once you figure out how this type of story is generated and how pointless it is, and that knowledge percolates to the control layer, you will give up on it.
- 4 more replies
New conversation -
-
-
Why can't one understand their evolutionary reward functions and still carry them out? You might see that procreating is ultimately pointless, because life itself serves no purpose but to perpetuate itself, but procreation (read: sex) feels good.
-
You can create all the rewards directly in you brain, without dealing with the trouble of messy body fluids and baby shit and sociopathic teenagers and college funds. There is a reason wh enlightened monks don’t have many kids
- 11 more replies
New conversation -
-
-
If I see all of humanity as an organism, this has interesting implications
-
From an evolutionary perspective, I think that is only true for totalitarian religious and ethno fascist groups (state building species with top down control structures implement organism like dynamics in a multilevel selection paradigm).
- 3 more replies
New conversation -
-
-
Ideas will continue to evolve, including how to make ever better reward functions. I don’t see a limit to that.
-
The limit is nirvana. Once you are done with your susceptibility to rewards, your enslavement to the regulation needs of the social primate that owns your mind ends.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.