Impressive what sheer scale can do. One potential weakness: trained on high-quality, curated data. I wonder what happens when you introduce typos or odd characters into the seed paragraph?https://twitter.com/OpenAI/status/1096092704709070851 …
-
-
Sounds about right. Definitely using *language modeling* in the narrow sense here, though I am curious what happens when you use this to build a BERT-style system that you can more easily manipulate.
-
Loving this discussion. I like Gary’s synthesis: advance in generation, not so much in understanding. The recent paper from Lake and Baroni can give concrete hints on how to move forward.
- 3 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.