John Hewitt

@johnhewtt

Research in NLP: language, structure, bash scripting, neural things.

Stanford, CA
Vrijeme pridruživanja: veljača 2015.

Tweetovi

Blokirali ste korisnika/cu @johnhewtt

Jeste li sigurni da želite vidjeti te tweetove? Time nećete deblokirati korisnika/cu @johnhewtt

  1. Prikvačeni tweet
    10. ruj 2019.

    How do we design probes that give us insight into a representation? In paper with , our "control tasks" help us understand the capacity of a probe to make decisions unmotivated by the repr. paper: blog:

    Prikaži ovu nit
    Poništi
  2. 5. stu 2019.

    I'm giving a talk on designing and interpreting probing methods for understanding neural representations at EMNLP, Hall 2C, today at 1:30!

    Poništi
  3. proslijedio/la je Tweet
    24. lis 2019.

    Over the last few months, a group of recent NLP PhD applicants compiled some thoughts, perspectives, & advice on the application process. We're happy to share this blog post, and hope future applicants will find it helpful for years to come: [1/3]

    Prikaži ovu nit
    Poništi
  4. 10. ruj 2019.

    Lots of hyperparameters when designing probes, and probing results conflate representation, probe, and data, making interpretration difficult. A control task can help design, and help interpret. code:

    Prikaži ovu nit
    Poništi
  5. 10. ruj 2019.

    Selectivity can also help interpret probing results. Does ELMo1 have better part-of-speech representations than ELMo2? The accuracies suggest so, but probes can memorize -- and selectivity results show it's much easier to memorize from ELMo1.

    Prikaži ovu nit
    Poništi
  6. 10. ruj 2019.

    We claim that good probes are "selective," achieving high accuracy on linguistic tasks, and low acc on control tasks. Between probes, small gains in linguistic acc can correspond to big selectivity losses; gains may be from added probe capacity, not repr properties.

    Prikaži ovu nit
    Poništi
  7. 10. ruj 2019.

    Our control tasks randomly partition the vocabulary, and force the probe to make the same output decision for words in the same subset. No linguistic structure, not reflective of repr, but learnable by the probe! Complex probes achieve high test accuracy on these tasks.

    Prikaži ovu nit
    Poništi
  8. 5. lip 2019.

    I'll be excitedly yammering about structural probes and finding syntax in unsupervised representations today at 4:15 in Nicollet B/C . Even if you don't ❤️ parse trees, come by to learn a method to tell if your neural network softly encodes tree structures!

    Poništi
  9. 7. svi 2019.

    I enjoyed chatting with and on about my paper with on finding syntax in word representations. I'm very grateful to have had this opportunity to talk (at length!) about my work!

    Poništi
  10. 5. tra 2019.

    So a lot of people have arrived here; please read 's excellent take on neural net probes and 's comprehensive neural net probing study, both also at Saphra: Liu:

    Prikaži ovu nit
    Poništi
  11. 5. tra 2019.

    This claim, that parse trees are embedded through distances and norms on your word representation space, is a structural claim about the word representation space, like how vector offsets encode word analogies in word2vec/GloVE. We hope people have fun exploring this more! 4/4

    Prikaži ovu nit
    Poništi
  12. 5. tra 2019.

    These distances/norms reconstruct each tree, and are parametrized only by a single linear transformation. What does this mean? In BERT, ELMo, we find syntax trees approximately embedded as a global property of the transformed vector space. (But not in baselines!) 3/4

    Prikaži ovu nit
    Poništi
  13. 5. tra 2019.

    Key idea: Vector spaces have distance metrics (L2); trees do too (# edges between words). Vector spaces have norms (L2); rooted trees do too (# edges between word and ROOT.) Our probe finds a vector distance/norm on word representations that matches all tree distances/norms 2/4

    Prikaži ovu nit
    Poništi
  14. 5. tra 2019.

    Does my unsupervised neural network learn syntax? In new paper with , our "structural probe" can show that your word representations embed entire parse trees. paper: blog: code: 1/4

    Prikaži ovu nit
    Poništi
  15. 1. lis 2018.

    Scott Aaronson's note is a delightful introduction to reasoning about large numbers, leading up to the Busy Beaver numbers. Years after finding that article, what fun to find Busy Beaver numbers in proofs on RNNs!

    Poništi
  16. 17. srp 2018.

    Wondering under what circumstances visual signal is useful in translation? Feeling a desire for multimodal, multilingual NLP? Use our dataset of images representing words across 100 languages, and check out our poster in Session 3E with Daphne Ippolito

    Poništi
  17. 16. srp 2018.

    We modeled derivational morphological transformations separately as orthographic and distributional functions, then combined: go see present our paper on English derivational morphology in oral session 6D today at ACL!

    Poništi
  18. 2. lip 2018.

    Learned a lot about LSTM behavior -- in very different ways -- from two excellent papers: Sharp Nearby, Fuzzy Far Away... by , He He, Peng Qi, and , and LSTM as Dynamically Computed... by , , , .

    Poništi
  19. proslijedio/la je Tweet
    21. tra 2018.

    Very excited to have my first paper at with John Hewitt and Dan Roth: A Distributional and Orthographic Aggregation Model for Derivational Morphology

    Poništi
  20. 24. tra 2018.

    One of two first ACL papers! Daphne Ippolito and I spearheaded "Learning Translations via Images: A Large Multilingual Dataset and Comprehensive Study," to appear at ! Happy to have worked with , Reno Kriz, Derry Wijaya, and .

    Poništi

Čini se da učitavanje traje već neko vrijeme.

Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.

    Možda bi vam se svidjelo i ovo:

    ·