https://overcast.fm/+Ic2hwsH2U/1:10:49 … #AI
“You could build a mind that thought that 51 was a prime number but otherwise had no defect of its intelligence – if you knew what you were doing” —@ESYudkowsky
Is it possible build a mind able to learn but incapable of correcting this error? (Why?)
-
-
It would *not* be simple the way a superintelligent paperclip maximizer is simple and coherent. I don't know exactly how to do it. But I'm confident it could be done by someone with a completed understanding of AI, because thought steps are physical and not metaphysical.
-
It would look to us like a mind with a lot of weird fiddly bits attached to maintain the delusion plus the weird fiddly bits, but the things you'd need to fiddle would be finite. The meta-meta-meta delusion would look a lot like the meta-meta delusion; there'd be a fixed point.
-
On why the things you’d need to fiddle would be finite: Are the meta delusions comparable? I can imagine 51′s not-prime-ness being a solution to many different problems a mind could have. If there’s a potentially unlimited number of these, you’d need unlimited idea-suppression?
-
Finite axioms have infinite consequences; infinite consequences can sometimes be compactly patched for the same reason. The question isn't how large the set is, it's whether the set compresses.
-
Do we have reasons to believe that one can compactly patch a set of consequences following from a problematic axiom without also effectively patching the axiom in that same process?
-
For instance, if the Intelligence is trained to secretly believe that 51 is prime, but also trained to act as if it isn't prime (which would solve all of the extended errors from the belief), and exclaims to everyone that 51 isn't prime, would that really count?
-
This opens up questions like: what does it mean to believe something? Can one believe something in some ways but not others? Some situations but not others? If you can believe different things in different situations, how many situations do you have to cover to suppress a belief?
-
This is a problem with humans too. In different contexts we express different beliefs. Because of that, I think to some extent we have to define belief as what someone acts out, not what they claim. In the context of AI, we may need a similar definition.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.