We also show that kNN-LM can efficiently scale up LMs to larger training sets and allows for effective domain adaptation, by simply varying the nearest neighbor datastore without further training. It seems to be helpful in predicting long tail patterns, such as factual knowledge!
-
-
Prikaži ovu nit
-
Work done at
@facebookai with amazing collaborators@omerlevy_,@LukeZettlemoyer and@ml_perception as well as my@stanfordnlp advisor@jurafsky!! Paper: https://arxiv.org/abs/1911.00172 Code available soon!Prikaži ovu nit
Kraj razgovora
Novi razgovor -
-
-
I guess this means instead of using a large model we can use a medium-sized model and compactify it (e.g. distillation) for faster inference to process a gigantic datastore (> 100B tokens) made from Common Crawl. I'm excited.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
well, Interesting!
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Are you interested on trying more powerful clustering algorithms like IIC https://arxiv.org/abs/1807.06653 ? It is a really interesting line of work.
-
Could I work with you?
Kraj razgovora
Novi razgovor -
-
-
Really nice and inspiring paper! Congratulations. When will the code be available? Do you think to share also the trained model? Thanks
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Looks like an interesting paper. Last year we used k-nnn defined by embeddings similarity for LM too & found improved results on PTB but more so for WikiText-2, for similar reasons (upsampling for terms in the long tail) See here if interested - https://arxiv.org/pdf/1809.05916.pdf …
- Još 1 odgovor
Novi razgovor -
-
-
Hi! Do you mind if I pick your brain over this in some DMs? I'm pretty familiar with kNN layers myself. I'm just curious about your results.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.