@citnaj Some students are going to "present" a bit about your work this Wednesday for my gen-ed ML course! Question: What does "unshock" refer to, e.g. in https://github.com/jantic/DeOldify/blob/master/ColorizeTraining.ipynb …?
-
That's awesome! "unshock" has been a workaround for the sensitivity the model has when switching sizes in training. The activations change overly-dramatically, and if you're not careful you can bust your weights and get weird artifacts as a result.
What this actually looks like in practice if things go awry: Usually some sort of neon highlighting of the objects in the scene- appears to be the self-attention layer poking through. Not what you want!
