When starting to overfit it's common for mean loss to go up while the per-sample loss distribution is still evolving in a way that helps acc
I see many people incredulous when "my loss is increasing, but accuracy is going up". What matters is the distribution of per-sample losses.
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Also it usually increases the accuracy because it's just overfitting one class which has more presence in the dataset.
-
it can also happen, but this will generally happen even in balanced classification problems.
End of conversation
New conversation -
-
-
Would a per-sample loss histogram be more useful than a lumped sum then? It should be a trivial change with current tooling.
-
if you can interpret it, maybe.
End of conversation
New conversation -
-
-
so what is your function for validation then? How do you evaluate this distribution?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.