Jason Antic

Can you give us a general lay persons explanation of how this works? Through reading through comments I'm guessing it basically recognizes identifies objects in the picture and compares it to existing database of color photos? Or "something" ?

1/ That's part of it- It has one end that is a resnet image recognizer that picks out what's in the scene. The other half constructs a new image based on that information. I'd say a significant percent of colors chosen are based more on "optics rules" it seems to be learning.

2/ So those optics rules would be things like how light reflects off glass and other objects, etc. This is more speculation on my part at this point but it would explain a lot of how it's very consistent in what it learns, and how it can fill in all these gaps of data.

3/ Because clearly I can't provide it with all the information about all coffee cans, articles of clothing, random knick-knacks and one off pieces of art, etc. Yet it still manages to figure out sensible colors (mostly).

That's very interesting insight Jason, thanks for sharing! I'm wondering whether there could be a way to share physics with the model, because we know the optic rules it learns. Would there be a way to give the model general insights about how light works? (1/2)