Jason Antic

(2) [Squarification] That small square center image is what the deep learning generator actually generates now. Before I was just shrinking the images keeping the same aspect ratio. It turns out, the model does better with squares- even if they're distorted in the process!

(3) Note that I tried other things like keeping the core image's aspect ratio the same and doing various types of padding to make a square (reflect, symmetric, 0, etc). None of this worked as well. Two reasons why I think this works. One- model was trained on squares;

(4) Two- at smaller resolutions I think this is particularly significant- you're giving the model more real image to work with if you just stretch it as opposed to padding. And padding wasn't something the model trained on anyway.

(5)Next we have [Chrominance Optimization] . It turns out that the human eye doesn't perceive color (chrominance) with nearly as much sensitivity as it does intensity (luminance). Hence, we can render the color part at much lower resolution compared to the desired target res.

(6) Before, I was having the model render the image at the same size as the end result image that you saw. So you maxed out around 550px (maybe) because the GPU couldn't handle anymore. Now? Colors can be rendered at say a tiny 272x272 (as the image above), then the color part

(7) of the model output is simply resized and stretched to map over the much higher resolution original images's luminance portion (we already have that!). So the end result looks fantastic, because your eyes can't tell the difference with the color anyway!

(8)[Graceful Rendering Degradation] With the above, we're now able to generate much more consistently good looking images, even at different color gpu rendering sizes. Basically, you do generally get a better image if you have the model take up more memory with a bigger render

(9) BUT if you reduce that memory footprint even in half with having the model render a smaller image, the difference in image quality of the end result is often pretty negligible. This effectively means the colorization is usable on a wide variety of machines now!