Jason Antic

In case anybody was wondering about how I monitor progress in DeOldify- I don’t pay a huge amount of attention to training/validation loss because they don’t actually capture a lot of the issues I wind up seeing (artifacts, uniformly black renders, etc).https://twitter.com/citnaj/status/1200854194002878464 …

And then I have both the Jupyter notebook “test” image suites and then a batch of 100s of more public domain images. It’s painful and tedious but using my eyes on these directly I’ve found is the only reliable way I have to measure progress. And even the you have to be extremely

careful about perception issues- you misremember stuff easily, and literally see things differently depending on context. So I don’t necessarily take a first go at it as the final say as to whether or not the model is great or not.

And yes this process drives me nuts and please tell me if there’s a better way lol

If you've got a collection of M trained models, you could crowd source the process of finding the best by repeatedly presenting users with the output for a given photo from 2 of them, and ask them which gives the best result, using a duelling bandit algorithm to determine which 2

I'm sure you'd be able to get a large number of people interested