Useful ideas for the benchmark you are working on?
"A growing body of evidence shows that state-of-the-art models learn to exploit spurious statistical patterns in datasets .. instead of learning meaning in the flexible and generalizable way that humans do" -- Adversarial NLI: A New Benchmark for NLU https://arxiv.org/abs/1910.14599
-
-
-
Not making it a numerical target. Enabling coarse evaluation.
End of conversation
New conversation -
-
-
1. Standard datasets are biased toward a certain data distribution (e.g. photographer bias) 2. Benchmarks don't incentivize generalization tests like out-of-distribution or zero-shot 3. The information bottleneck might be a good way to regularize nets exploiting spurious stats
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
How is this news or research considering how ML models are defined or what they intend?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
"a growing body of evidence shows that humans exploit spurious statiscal patterns in sense input." (Apologies. Learning is an illusion)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
perhaps "spurious statistical pattern" is in the eye of the beholderhttps://twitter.com/dribnet/status/1176952465549230081 …
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Isn’t that the monster we created with NASNet and continued on to use with RL for augments? Oh wait...
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Yawn. AKA "memorising the training set", "polynomial curve fitting with a poly order > no. data points". A real generalisation contains less info than the data/lower level of abstraction.
-
... and pre-processing the data (say, a set of images) into "basics" (e.g. vector edges/planes) both drastically reduces the total information in the set, as well as introduces a layer of abstraction. The reward function then becomes (partly) "dump as much info as possible".
- 1 more reply
New conversation -
-
-
The brain ignores random stuff. It learns only the perfection that is in the world. How? Because it relies exclusively on timing for learning. It rejects purely random correlations between events.
- 6 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.