I'm not sure of all of what you are referring to since I am not an ML researcher, but I am a trained linguist & read the paper out of curiosity. I don't think he is being reasonable to the state of machine transalation and UG in the 2000s. Everyone thought the paradigm wld work.
-
-
Computational ('connectionist') models if used correctly could not only formulate better surface forms, but reveal more of the syntactic architecture even symbolic architecture underlying. He quotes Marcus well:pic.twitter.com/2QkTeV0qFY
1 reply 0 retweets 0 likes -
What I think Marcus and Pinker are resisting now therefore is the idea that ML is now revealing deeper truths of the quasi-computational (or at least extremely logically rigorous, since the Chomsky stack, which was set up to counter bad machine-trans) Universal Grammar paradigm.
1 reply 0 retweets 2 likes -
The surface forms are good, but insights into the presumed universal mode of human manipulation of syntax are gone. Marcus/SP seem fair. But I guess linguists cld be more excited about what GPT-2 does do... FWIW I take it for granted Chomsky would see style as a separate skill.
1 reply 0 retweets 1 like -
Replying to @jmanooch @michael_nielsen
Sounds to me like Nostalgebraist is saying “GPT2 could be used to make discoveries about the universal mode of syntax, but nobody’s doing it”. So maybe you & he don’t disagree?
2 replies 0 retweets 1 like -
Btw am I understanding you right that “syntax”, to a linguist, means the truth claims language refers to, & is separate from style?
1 reply 0 retweets 0 likes -
Replying to @s_r_constantin @michael_nielsen
No. That is semantics. In classical linguistics language is made up of phonology (and orthography) - sounds (and written symbols) morphology - word composition syntax - idea-unit composition (eg sentence) semantics - 'meaning' at all levels, eg word, sentence, paragraph etc.
1 reply 0 retweets 0 likes -
Why Pinker and Marcus are set off is based on the huge syntactic turn caused by Chomsky which they followed: they all believe(d) syntax was the engine of language, and Universal Grammar UG was in every head just churning out good sentences using 'localized' rules as it were.
2 replies 0 retweets 2 likes -
Replying to @jmanooch @michael_nielsen
People actually believed that?!!?!?
2 replies 0 retweets 2 likes -
Replying to @s_r_constantin @michael_nielsen
Yes and some, including me, sort of still do for a very good reason brought to light by Chomsky: https://en.m.wikipedia.org/wiki/Plato%27s_Problem …
1 reply 1 retweet 2 likes
I totally believe there is a hard-coded “general ability to reason symbolically” in the human brain. I think it’s pretty obviously not limited to sentence-level syntax!!
-
-
Because you get things like GPT2’s output which is logically incoherent at the paragraph or essay level, but grammatically just fine at the sentence level. It’s clearly lacking the “general symbolic reasoning” ability. Which may still be a compact thing!
0 replies 1 retweet 0 likesThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.