Ablation studies are crucial for deep learning research -- can't stress this enough. Understanding causality in your system is the most straightforward way to generate reliable knowledge (the goal of any research). And ablation is a very low-effort way to look into causality.
-
-
Those two fun papers come too mind, when talking about "If I can't break it, I can't understand it" http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268 … https://www.cell.com/cancer-cell/fulltext/S1535-6108(02)00133-2 …
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
classic randomized "Dropout" ? or more emphatic surgical removal of nodes?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Ablation studies are also how you can surprise yourself with a far simpler or more efficient implementation too :) Best of all, the faster your model runs, the quicker you can find such models as running many ablations is trivial ^_^
-
Don't wait until your model is already complex for ablations either. The best accident I ever made was trying to improve
@PyTorch's tutorial language model as I wanted it to remain relatively simple and fast for beginners. I kept looking for low hanging fruit until it was ~SotA. - Show replies
New conversation -
-
-
"old" but gold comment.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.