1/ I had a question about successfully handling high traffic on computationally expensive deep learning models (like DeOldify). I'm pretty new to this myself and therefore may be very wrong but I do have a take and I'm wondering what others think. Here's what I said:
-
Show this thread
-
2/ "As far as scaling goes- this is something that we actually wound up “outsourcing” for DeOldify. In that we’re just licensing the model and letting the licensing companies (like MyHeritage) figure out the hard part of scaling. "
2 replies 0 retweets 3 likesShow this thread -
3/ "Because we knew that would be hard and quite frankly not something we’re comfortable with taking on as a two person team. Before we decided to go down this route, we were originally going to go with an app that strictly ran on iPhone hardware (6s+)."
2 replies 0 retweets 3 likesShow this thread -
4/ "Yes,this actually worked, and quite well (noticeably better than open source). As you can imagine- getting rid of servers like this solves the scaling problem :) We’ve also considered doing a desktop app (haven’t ruled it out, just not right now). "
1 reply 0 retweets 3 likesShow this thread -
5/ "Again- deferring huge, expensive computation to the users, hence avoiding a lot of complication on our end. And we’d argue it’s better for the users."
3 replies 0 retweets 3 likesShow this thread -
6/ "My hot take on “the cloud” approaches is that they tend to be the default approach to a fault and traditional desktop/local deployments tend to get overlooked even if they make the most sense."
4 replies 1 retweet 15 likesShow this thread -
Replying to @citnaj
Think about it as providing a service. Scheduling your resources for users is more efficient than everyone having idling hardware. You also keep control over your source code which you can update without anyone noticing. The incoming unlabeled data helps to improve your models.
1 reply 0 retweets 0 likes
It's very, very expensive to do it as a cloud service. To the point where it completely changes what kind of business model you can offer. Control over code- do I really need that? I don't think so. And updates to the model only will happen every few months.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.