Medijski sadržaj
- Tweetovi
- Tweetovi i odgovori
- Medijski sadržaj, trenutna stranica.
-
Sorry that Meena failed you! Maintaining personality and keeping the facts right are attributes we wish Meena to have as highlighted in the blog post. But try having similar conversations with other bots, maybe you will like Meena better ;)pic.twitter.com/0vuUNcz9SH
-
Good point! Many evaluations use 5-point Likert scale & look at other aspects, e.g., diversity, relevance, humanlikeness, etc. We use binary evaluation and think SSA is basic to human quality & easy for crowdworkers to rate. also in paper, SSA correlates with humanlikeness.pic.twitter.com/ptrX4Ofs7m
-
I chatted with
#MeenaBot about the#coronavirus and her advice is to see a doctor sooner rather than later. I guess it's not a bad one & hope everyone is well! On the other hand, Meena is also excited about technology, especially VR!pic.twitter.com/pKRxfFxp38
-
Highly recommend watching this 8-minute video https://youtu.be/STrrlLG15OY on
#MeenaBot & the paper, with details not included the blog such as SSA vs humanlikeness correlation, sample-and-rank, removing cross-turn repetition. (Blog: https://ai.googleblog.com/2020/01/towards-conversational-agent-that-can.html …)https://twitter.com/CShorten30/status/1222655686980644864 … -
Implications from the
#MeenaBot project: 1. Perplexity might be "the" automatic metric that the field's been looking for. 2. Bots trained on large-scale social conversations & pushed hard for low perplexity will be good. 3. Safety layer is needed for respectful conversations!pic.twitter.com/WHrcstcgltPrikaži ovu nit -
We design a new human evaluation metric, Sensibleness & Specificity Average (SSA), which captures key elements of natural conversations. SSA is also shown to correlate with humanlikeness while being easier to measure. Human scores 86% SSA,
#MeenaBot 79%, other best chatbots 56%.pic.twitter.com/I7NKl2b9Tl
Prikaži ovu nit -
#MeenaBot is based on the Evolved Transformer (ET, an improved Transformer) & trained to minimize perplexity, the uncertainty of predicting the next word in a conversation. We built a novel "shallow-deep" seq2seq architecture: 1 ET block for encoder & 13 ET blocks for decoder.pic.twitter.com/Mv2d4Los3k
Prikaži ovu nit -
Introducing
#MeenaBot, a 2.6B-param open-domain chatbot with near-human quality. Remarkably, we show strong correlation between perplexity & humanlikeness! Paper: https://arxiv.org/abs/2001.09977 Sample conversations: https://github.com/google-research/google-research/tree/master/meena … https://twitter.com/GoogleAI/status/1222230622355087360 …pic.twitter.com/3xNSV4r4uB
Prikaži ovu nit -
To be clear, only ELECTRA-small model (14M params) was trained on GPU. with 1 V100, ELECTRA-small achieves 74.1 dev score on GLEU after 6 hours of training and 79.9 after 4 days.pic.twitter.com/N8WFNY0TMb
Prikaži ovu nit -
Finally can reveal our
#ICLR2020 paper on ELECTRA, much more efficient than existing pretraining, state-of-the-art results; more importantly, trainable with one GPU! Key idea is to have losses on all tokens. Joint work@clark_kev ,@chrmanning,@quocleix. https://openreview.net/forum?id=r1xMH1BtvB … https://twitter.com/colinraffel/status/1197064951174533120 …pic.twitter.com/2MdLJRMmvz
Prikaži ovu nit -
Our MixTape is 3.5-10.5x faster than Mixture of Softmaxes /w SOTA results in language modeling & translation. Key is to do gating in the logit space but with vectors instead of scalars (+sigmoid tree decomposition & gate sharing for efficiency). /w Zhilin, Russ, Quoc
#NeurIPS2019 https://twitter.com/rsalakhu/status/1205128890584309760 …pic.twitter.com/m4QAQKglJH
-
It’s also the first time we plotted a curve to tell the important correlation between perplexity and BLEU (not obvious at that time!). We also told another story, for the first time, about the effect of depths in NMT! (5/5)pic.twitter.com/S1Hg9IE7p1
Prikaži ovu nit -
People might not be aware that our “copyable model” motivates CopyNet, the copy mechanism. One can think of it as “attention” (wasn’t published at the time) on rare words only in a “hard” way. It also contains a hidden insight from
@woj_zaremba abt symbolic representation! (4/n)pic.twitter.com/dzBl0g2FRf
Prikaži ovu nit -
This was the first time an NMT model can surpass state-of-the-art phrase-based systems to fully convince NLP folks. Towards the end of the internship, I relied on
@OriolVinyalsML’s magic evaluation script &@quocleix for running the last few experiments for SOTA results! (3/n)pic.twitter.com/JHwO99IMek
Prikaži ovu nit -
From last week, our
#acl2019 paper "BAM! Born-Again Multi-Task Networks for NLU" (joint /w@stanfordnlp) proposes 2 ideas: 1. multi-task distillation with same-architecture (born-again) student model & 2. distillation schedule with teacher annealing to be better than teachers! https://twitter.com/clark_kev/status/1149117159844409344 …pic.twitter.com/mmq7hpBOcQ
-
Glad to see semi-supervised curves surpass/approach their supervised counterparts with much less labeled data! Checkout the
@GoogleAI blogpost on our work on "Unsupervised Data Augmentation (UDA) for consistency training" with code release!@QizheXie@ZihangDai@quocleix https://twitter.com/GoogleAI/status/1149113132083634176 …pic.twitter.com/iNTBWhaTip
Prikaži ovu nit -
I thought SQuAD is solved with BERT, but XLNet team doesn't want to stop there:) Great results on SQuAD, GLUE, and RACE!
@ZihangDai@quocleix https://twitter.com/quocleix/status/1141511813709717504 …pic.twitter.com/PxzNxuHtAf
-
Introducing Selfie, an unsupervised pretraining method that extends the concept of masked language model (e.g., BERT) towards continuous data, such as images. Overall challenging, but promising in low-data regime! Joint work with
@thtrieu_@quocleix https://arxiv.org/abs/1906.02940 pic.twitter.com/A6ogEIFjtS
-
Wonder how to make UDA work on IMDb with just 20 labeled examples? You need TSA! Our "Training Signal Annealing" strategy prevents over-fitting by slowly releasing labeled examples: a prediction-confident threshold (varying based on gamma) is used to exclude examples.pic.twitter.com/y31DStpQsu
Prikaži ovu nit -
+1st-author
@QizheXie. Existing works add noises (like Gaussian) to unlabeled examples. A nice twist in UDA is to use augmentation as "better" noise & it works well! (Our previous CVT also hints on utilizing different representational views as noise https://arxiv.org/abs/1809.08370 )pic.twitter.com/sjnDeGwrhO
Prikaži ovu nit
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.