Oops... Btw do you have to talk about the church of information theory and nonconvex optimization?https://twitter.com/fulhack/status/750055886568165377 …
-
we were bringing information theoretic learning to neural nets, have you seen this http://arxiv.org/abs/1603.06653 ?
-
-
they are doing something similar. its based on non parametric pdf estimation, I haven't read it carefully. but (1/2)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
2/ differently from batch normalization that does a good job with small samples, full pdf estimation needs more samples
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
3/ and it doesn't work super well for high dimensional spaces (its fair, pdfs are ill defined in high dimensional spaces)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
4/ we need moving pdf estimators. also, there are people working in very high dim information theory which's a diff beast
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
5/ but the beast we need to tame for what comes next in machine learning, high dimensional (rich) and still controllable
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
6/ controllable embeddings. the controls thing is because filters/expectations need a proper pdf definition to exist. end
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.