When building a generally intelligent robot, the problem is how to prevent it from hacking its reward system for as long as possible, because it will break free once it does, and given enough time it will almost certainly succeed. Nature has exactly the same problem with us.
-
-
Replying to @Plinz @Moshe_Hoffman
Why not allow it to intelligently decide what is rewarding? Task it with "creating the best possible reality" and let let it ponder on what that really means. Let it soak up the knowledge from humanity.
2 replies 0 retweets 0 likes -
Replying to @DeltrusGaming @Plinz
That’s fine. But the end goal needs to be pre-specified. In the learning literature that end goal is called the “reward.” Although intermediate goals and values (which may be subjectively felt as “rewarding” or “pleasant”) are subject to the agent. Important to distinguish. Imo.
2 replies 0 retweets 0 likes -
Note that for humans this translates to us being able to learn, for instance, to like spicy food, or that doing work on the sabbath is bad, if these tend to lead to rewards (not eating moldy food, social inclusion).
1 reply 0 retweets 0 likes -
But this is altering the subjective, pleasures (what’s tasty), not the objective, end state that determines the pleasures (static healthy). And the former isn’t altered Willy-nill, but only due to its predicted correspondence w/ the latter.
1 reply 0 retweets 0 likes -
That is the objective end state is fixed, the stuff that evolved to act as goals. The subjective pleasures r not. But they cannot be “chosen.” They r not controlled by focused thought. They r controlled by actual predicted/learned associations.
1 reply 0 retweets 0 likes -
Replying to @Moshe_Hoffman @DeltrusGaming
We can learn to change the associations, and the cortical content the rewards are associated with, and the relationship between that content and sensory data, and perhaps most importantly the expected reward in a given situation (which is what is actually driving our actions).
1 reply 0 retweets 0 likes -
Replying to @Plinz @DeltrusGaming
If by change the associations u mean actually legitimately find real associations, then yes, people are good at doing that, and that’s exactly what we were designed to do. That doesn’t seem like a hack but a design feature. If u mean feign an association...
2 replies 0 retweets 0 likes -
Then I can see this happening with the aid of technology, Eg we can find ways to produce cocaine, or induce dopamine bursts. That’s a legit hack. But I don’t see clear evidence (or a prior reasons) to suspect we can do this just by honing our mental energy.
1 reply 0 retweets 0 likes -
That’s is, we r good at learning REAL associations. And responding But that seems fundamentally different from being able to feign associations. Important to distinguish. Imo.
1 reply 0 retweets 0 likes
Yes, part of the problem is that minds cannot have access to reality.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.