Jason Antic

1/ I had a question about successfully handling high traffic on computationally expensive deep learning models (like DeOldify). I'm pretty new to this myself and therefore may be very wrong but I do have a take and I'm wondering what others think. Here's what I said:

2/ "As far as scaling goes- this is something that we actually wound up “outsourcing” for DeOldify. In that we’re just licensing the model and letting the licensing companies (like MyHeritage) figure out the hard part of scaling. "

Honestly - you saved yourself a huge headache. Deploying GAN models at scale is a big challenge. I found that out the hard way. There is next to nothing written about this since its so new. Most people writing guides assume your servers will deal with millisecond requests.

Here you're looking at 30+ second requests that take GBs of memory on average. Even the big cloud providers don't have a good out of the box solution that doesn't cost an arm and leg.

That's exactly what we found- it's soooo expensive. At the very least it pays to do a lot of research into optimization before you put something up.