The arguments made e.g. by Nick Bostrom is rather that they could lack deeper understanding of their actions, consequences and have different priories (produce paperclips at all costs), what possibly leads to rational but unethical actions.
-
-
Replying to @tymwol
Interesting question. Do you think that "autistic AI", i.e. an AI that only optimizes for a low level reward function, will outperform "sustainable AI", i.e. one that maximizes its expectation horizon?
2 replies 1 retweet 1 like -
It happens all the time in human power struggles, with sociopaths getting up the ladder in companies and governments. Why should it be different for AIs?
1 reply 0 retweets 0 likes -
That is the general problem of governance: how can you expect someone to design the incentive architecture for society if they don't understand their own incentives? And how can you expect those that best understand their own incentives to be altruistic governors?
1 reply 0 retweets 1 like -
In a simple hawk/dove model it's pretty clear that the best strategy is to change between being a dove or a hawk depending on circumstances. That's the opposite of ethics. I am not sure why you think ethical behaviour is maximizing for agents in general competition.
2 replies 0 retweets 0 likes -
For the most part, we live groups in which the cost of violence dramatically exceeds its possible benefits, so we share the purpose of avoiding violence with most members of society, for instance.
1 reply 0 retweets 0 likes -
How does that solve the problem? An unethical AI would.still not use violence unless its beneficial.overall for itself. It still has that option, a strict advantage, over an AI that can't ever use it in any case. Can't win longterm.
2 replies 0 retweets 0 likes -
Ethics is not the same thing as a commitment to cooperation or non-violence or a particular set of moral intuitions. Ethics is simply principled social behavior. It does not imply that everyone would agree to the values guiding it. (Which is why you can have culture wars.)
1 reply 0 retweets 0 likes -
Ethics is either a set of hard constraints, or a revisited payoff matrix that takes into account "other stuff". "Principled" is meaningless: what doesn't change the payoff matrix or doesn't impose a constraint doesn't exist.
2 replies 0 retweets 0 likes -
You are wrong in looking ethics as a payoff matrix. Ethics are the bunch of assertions used on the path to reach goal, not the reward system. If you kill a person and go to North Korea, you may make a lot of money. Ethics is about asking should you kill.
1 reply 0 retweets 0 likes
Lucio is correct in asserting that designing incentive architectures is what matters, if you want to influence the behavior of other people. (And no, North Korea is not distributing money among people that kill other people. That would not make sense to its cult leaders.)
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.