Jason Antic

3/ To elaborate on what those bad side effects would be: The model would just pick another default color. Much like a game of whack-a-mole!

I'm not sure what color space you use but if you go to something like CieLAB, does the default color become more grayish?

I haven't attempted a color space outside of RGB yet but I could imagine that would have a significant impact. To be fair, it might be preferable to default to grayish of all things. I hadn't thought about that possibility honestly... Thanks!

Look into CieLAB or similar. Things like YCrCb or YUV don't reflect as well human perceptions. For info, as a challenge based on your feedback I tried to do the opposite: full GAN colorizer (no pretraining). No success though but I won't give up

Interesting challenge there! Were you thinking there would be some sort advantage to doing that potentially?

Yes, I cannot make it work but I feel like in theory it could converge towards an optimal solution. When there is 50/50 chance of blue or yellow, a model with traditional losses will output the average green while you would have wanted either blue or yellow.

You can try to avoid that by projecting the loss in another space or even using different layers like intermediate activation maps, but you still would have an interpolation problem.

When using GAN + other losses, you just work with compromises, even using weight decay is a compromise for stability. With the "unlimited" quantity of data we can have for this problem, a GAN would be a solution with no compromise at all, where you just want a realistic solution