I think lots of simple problems can serve as appropriate AI benchmarks (including chess) as long as we constrain them to measure *generalization power* and not *skill*.
Show this thread
-
Skill is boring -- of course any dumb algorithm can reach arbitrary skill levels when leveraging arbitrary amounts of compute, data, and human engineering. You're trading off intelligence for experience.
Show this thread
It isn't very interesting that an algorithm can beat any human at chess -- but it would be significant to see an algorithm infer the rules of chess after looking at a few dozens of games, then proceed to develop a human-level chess solver using just a few hundreds of games
-
-
Similarly, I don't want to see a deep learning model beat humans at a MOBA (which of course it can, given enough data), I want to see a model trained on a few thousands of plays, given a *completely new* character, doing just as well as a human in the same situation
Show this thread -
To make progress on AI, we must measure abstraction strength and generalization power, not plain skill (while ignoring *how* that skill was obtained). Focusing on skill is a bit like looking only at the speed of a vehicle but ignoring its energy consumption.
Show this thread -
Intelligence is about efficiently turning experience into generalization power. Skill at a given task, given infinite resources to get there, has no correlation with intelligence.
Show this thread
End of conversation
New conversation -
-
-
Would you consider that a matter of abductive reasoning?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
A 5 year old can learn the rules of chess and play. Is that the "human-level" benchmark? A few hundred games doesn't a GM make. Then is the benchmark a hundreds-of-games human? Any human?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.