Jason Antic

I think one of the possible solutions could be adding some noise to the image perhaps? Which is barely perceptible. One other thing could be online learning. Where you constantly feed the model new data(Using open-source movies ?!?)

Noise is already being added in training- gaussian, Poisson and salt/pepper. This does help quite a bit (made video work a lot better in particular). But you're thinking of something different it seems (?)

Yeah, that kinda solution should improve results for videos as there is a natural noise that is barely perceptible frame to frame. Esp. if you don't want to add a recurrent module to your solution. I always thought that a recurrent module should be used for coloring videos though

I'm not adverse to recurrent modeling. It's just that I feel that the simplest methods should be taken as far as possible (image to image) before going that route. Especially with the lack of resources I have right now but also because it'll be beneficial no matter what.

I mean providing the "context" to the model when coloring the next frame seems to be very intuitive. ie in one frame your model decides to color a can of beans as red but a few frames later due to some change in scenery it now decides it is supposed to be blue

adding a recurrent module to the approach would allow the model to remember its choices and be consistent.

*In theory yes but I suspect it's going to be easier said than done. I'm pretty sure I'll try it eventually- I'm just not there yet.

input image along with a context vector(fairly large given that we want it to be consistent over a very large sequence). Every time we generate a frame we pass some kind of encoding of the said frame through an LSTM and get its updated hidden states as the next context vector.