Great critique of #DeepLearning includes insightful anecdote: @GaryMarcus trained a NN identity function f(x) = x on even numbers (in binary vector representation); testing worked on even numbers but not odd numbers! NN couldn't extrapolate beyond the training space.https://twitter.com/GaryMarcus/status/948364570120335363 …
-
i.i.d. theory says that it wasn't the NNs job to do this extrapolation. sorry!
It’s the fixation on IID that is getting in trouble in the first place. Convolution doesn’t demand IID, it enforces it. If @ylecun had waited, we would still be stuck with fully connected nets that are vastly last efficient.
-
-
I agree i.i.d. is very restrictive, but taking a procedure built within the i.i.d. framework and hoping that it might do something extra-i.i.d. is ... wishful thinking.
-
agreed. But my point is that the field does a lot of wishful thinking when it imagines that problems like natural language understanding can be solved with IID techniques that are really are inappropriate tools. Human language learning is clearly using something different.
- 1 more reply
New conversation -
-
-
This is why we need relational learning. Down with iid!
- 3 more replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.