Jason Antic

1/ "ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks" is so packed with great insights. Of particular note: They pretrained their generator on L1 loss, and report that it actually improves quality. https://arxiv.org/abs/1809.00219 

2/ I've been experimenting with pretraining both generator and critic with non-gan losses for DeOldify, it turns out, because I suspected it would lead to not only faster training but better results as well. It's still early but I'll just say it looks promising!

3/ The core concept definitely works functionally, I can tell you that much. Both for colorzation in DeOldify, as well as de-artifacting/super-res. This was part of lesson 7 in http://fast.ai  V3 part 1, which will be released soon.

4/ The way I think of pretraining a GAN is this: A lot of time is wasted in going back and forth with the generator and critic not knowing what they're doing. Why not teach them directly with faster/simpler loss first to get most of the way there, -then- let them go GAN?