I would be happy to take on the challenge. Where is the question dataset?
let’s try it! @openAI want to test your full model on those?
@JeffDean want to try? @etzioni @YejinChoinka @ylecun ?
would love to see any general-purpose model (not directly tailored to these examples) that can approach grade-school performance.
open challenge to all.https://twitter.com/sterlingcrispin/status/1188286346487455744 …
-
-
-
start with those that I posted last night including these
- 1 more reply
New conversation -
-
-
I've seen benchmarks with these kind of commonsense questions but they always have multiple choice answers. I don't think any system out there is close to being able to deal with the kind of open-answer format you suggest. I'd be curious to hear if I'm wrong.
-
not interested in multiple choice; i want to see completion ala GPT-2 - and my son.
- 3 more replies
New conversation -
-
-
As a teenager I caught on that disagreeing with smart people made me look smarter myself - if I wasn’t able to contribute, the quick and safe way to stay involved and share the attention was to poke holes. Luckily I grew out of it
-
I think it's time a write piece called "Why pinpoint limits" or something.....
- 2 more replies
New conversation -
-
-
I like the challenges you propose to the field, but let's keep in mind that OpenAI GPT is a language model, not a (commonsense) knowledge model per say.
-
1. as i say, i would be interested in any general-purpose architecture that can succeed; need not be transformer-based, but shouldn’t be tailored specifically to the task. 2. lots of folks are applying transformer architecture to winograd schemas, so it doesn’t seem unfair to ask
End of conversation
New conversation -
-
-
@GaryMarcus the best way to challenge the field is not to highlight anecdotal examples where current technology fail, however valid they are, but do the hard work and come up with an actual benchmark. Will you take that challenge? -
I don't have bandwith to actively run the show myself but made a lot of suggestions here last night & asked for people that might help. Also made a detailed proposal before with
@heuristicity a few years back in AI Mag. If your org would like to lend support, it would be great.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.