@citnaj Some students are going to "present" a bit about your work this Wednesday for my gen-ed ML course! Question: What does "unshock" refer to, e.g. in https://github.com/jantic/DeOldify/blob/master/ColorizeTraining.ipynb …?
That's awesome! "unshock" has been a workaround for the sensitivity the model has when switching sizes in training. The activations change overly-dramatically, and if you're not careful you can bust your weights and get weird artifacts as a result.
-
-
Easing into this with lower lr helps (hence "unshock"). With my latest model I've replaced this with a smooth incline on the learning rate basically starting near 0 and going to target from there. Seems to work well.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
What this actually looks like in practice if things go awry: Usually some sort of neon highlighting of the objects in the scene- appears to be the self-attention layer poking through. Not what you want!
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.