Our control tasks randomly partition the vocabulary, and force the probe to make the same output decision for words in the same subset. No linguistic structure, not reflective of repr, but learnable by the probe! Complex probes achieve high test accuracy on these tasks.
-
-
Prikaži ovu nit
-
We claim that good probes are "selective," achieving high accuracy on linguistic tasks, and low acc on control tasks. Between probes, small gains in linguistic acc can correspond to big selectivity losses; gains may be from added probe capacity, not repr properties.
Prikaži ovu nit -
Selectivity can also help interpret probing results. Does ELMo1 have better part-of-speech representations than ELMo2? The accuracies suggest so, but probes can memorize -- and selectivity results show it's much easier to memorize from ELMo1.
Prikaži ovu nit -
Lots of hyperparameters when designing probes, and probing results conflate representation, probe, and data, making interpretration difficult. A control task can help design, and help interpret. code:https://github.com/john-hewitt/control-tasks …
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.