Here's another round of colorizations using my latest unreleased DeOldify model. Photos are again from Reddit's r/TheWayWeWere (great source!)pic.twitter.com/bKwZsXVYTO
Show this thread
-
1/ That's part of it- It has one end that is a resnet image recognizer that picks out what's in the scene. The other half constructs a new image based on that information. I'd say a significant percent of colors chosen are based more on "optics rules" it seems to be learning.
2/ So those optics rules would be things like how light reflects off glass and other objects, etc. This is more speculation on my part at this point but it would explain a lot of how it's very consistent in what it learns, and how it can fill in all these gaps of data.
-
-
3/ Because clearly I can't provide it with all the information about all coffee cans, articles of clothing, random knick-knacks and one off pieces of art, etc. Yet it still manages to figure out sensible colors (mostly).
-
That's very interesting insight Jason, thanks for sharing! I'm wondering whether there could be a way to share physics with the model, because we know the optic rules it learns. Would there be a way to give the model general insights about how light works? (1/2)
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.