The number one thing to keep in mind about machine learning is that performance is evaluated on samples from one dataset, but the model is used in production on samples that may not necessarily follow the same characteristics...
Show this thread
-
Wait, are there people that don't know this? Fuzzing tests, input constraints etc.. all remain as important for production inference systems as they are for any piece of software. I have over the years recognized a lack of basic product competence for a lot of ML engineers
It's elementary, but many deep learning researchers are not aware of this, because in 100% of the work they've ever done, the model's score on the test set was the finality. Most DL research starts from an existing dataset and stops at a test set score.
-
-
How do we change this? This is causing a lot of heartburn and headache for us in the industry who aren't DL researchers but are asked to incorporate DL models into solutions we build. Replacing good old if statements.
-
Teach more software engineers how to do ML and teach researchers how to build basic software
- Show replies
New conversation -
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.