About 10,000 deep learning papers have been written about "hard-coding priors about a specific task into a NN architecture works better than a lack of prior" -- but they're typically being passed as "architecture XYZ offers superior performance for [overly generic task category]"
-
-
An extreme case of this: working with a synthetically-generated dataset where samples follow a "template" (e.g. bAbI), and manually hard-coding that template into your NN architecture
Show this thread -
Fitting parametric models via gradient descent, unsurprisingly, works best when what you are fitting is already a template of the solution. Of course, convnets are an instance of this (but in a good way, since their assumptions generalize to all visual data).
Show this thread
End of conversation
New conversation -
-
-
This, as a meta-result, is very interesting though, isn't it? Having meaningful priors gives you an advantage. Now one would need to find a general language of priors that generalizes well.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I've done this! Not proud of it, but, I've done this. Had to get a paper published as part of a course. Ran out of actual ideas (which didn't actual achieve anything)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.