great post! do you have a reference for MEWR? the link seems broken.
-
-
- Kraj razgovora
Novi razgovor -
-
-
This is a nice summary. A couple of notes on ROUGE, which you mention as a possible alternative. Initially it was recall-oriented but was subsequently extended to recall and precision. 1 /
-
More importantly, there is a fair bit of literature showing that ROUGE does not correlate well with human judgements in some domains, particularly on meeting speech. 2 /
- Još 4 druga odgovora
Novi razgovor -
-
-
As Churchill said, BLEU is the worst metric, except for all the other ones.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Another ref: Stent, A., Marge, M. and Singhai, M., 2005, February. Evaluating evaluation methods for generation in the presence of variation. In International Conference on Intelligent Text Processing and Computational Linguistics (pp. 341-351). Springer, Berlin, Heidelberg.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Completely unrelated to the article, I just recognised the picture in my timelinepic.twitter.com/FX9E9W8Sr1
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
bless you.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
Key takeaways from this: I finally understand the Buffalo buffalo buffalo sentence. Mewr sounds promising and kind of similar to what I was thinking of. It puts a lot of faith in embeddings to hold more meaning than they probably do though and ~5000 tokens an hour? Yikes.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
How about a good (I mean near-perfect!) NLI/similarity model judging NLG systems instead of metrics like BLEU & ROUGE?
-
Could you elaborate?
Kraj razgovora
Novi razgovor -
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.
's name is Gustav.) She/her