serious question - can any GPT-2 experts answer? has anyone studied this?https://twitter.com/bradpwyble/status/1188155924902502400 …
-
is this about "reading the texts off of the model weights," or about them appearing in output? different things: capacity to output X doesn't imply memorization of X (e.g. me typing this sentence). longer discussion at https://nostalgebraist.tumblr.com/post/184913344984/ive-been-fine-tuning-the-345m-gpt-2-on-a-bunch … and cf. https://arxiv.org/abs/1802.08232
-
-
thanks. fwiw i think you may be missing the reason many of us are excited by gpt2. i don't believe gpt2 has profound understanding and represents some royal road to strong AI, i believe it is a scientifically fascinating linguistic/cognitive phenomenon
-
the fascinating thing is precisely that gpt2 can *successfully* imitate so many features of real text while being so *severely* restricted in deep understanding. gpt2's pseudo-writing *is* built on sand, and it thus shows just how much of language needs no firmer foundation
- 2 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.