Is that single human level (which for example would get tired after hours of doing the same thing, might make the occasional inattention error, etc.) Or an "ensemble" or different humans putting their opinions together?
The purpose of a benchmark is to help guide progress. For a dataset to make a good benchmark, the state-of-the-art must be significantly below human level -- pointing to what needs to be done
-
-
-
I think it's just "what everyone reliably agrees is the best we can do." Once you go beyond that, we only know that the model thinks it's improving, not that it's actually improving. Yea or nay?
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.