Want to help build a new benchmark for AI, that aims to push machines toward a deeper understanding?
If yes, please contribute a few questions at Google Form below, and please pass this along 
Suggestions for improvements welcome!
https://docs.google.com/forms/d/e/1FAIpQLSekOYnhHejR_aeXfB1yqVrYtWAQlDNiMOy7zoXYpDCnJTIc3Q/viewform …
-
As
@fchollet wrote here https://arxiv.org/abs/1911.01547 , machine learning always finds shortcuts. We cannot compare levels of understanding by machines and humans via one simple test. Need to evaluate abilities via the capacity to adapt to new (unseen) skills, like answering these tests.
i certainly don’t think one test will suffice and have been arguing that for years, hence my proposals for a Turing Olympics, but i think we can at least start to focus on facets that are important and understudied.