The number one thing to keep in mind about machine learning is that performance is evaluated on samples from one dataset, but the model is used in production on samples that may not necessarily follow the same characteristics...
Show this thread
-
So when asking the question, "would you rather use a model that was evaluated as 90% accurate, or a human that was evaluated as 80% accurate", the answer depends on whether your data is typical per the evaluation process. Humans are adaptable, models are not.
Show this thread
If significant uncertainty is involved, go with the human. They may have inferior pattern recognition capabilities (versus models trained on enormous amounts of data), but they understand what they do, they can reason about it, and they can improvise when faced with novelty
-
-
If every possible situation is known and you want to prioritize scalability and cost-reduction, go with the model. Models exist to encode and operationalize human cognition in well-understood situations.
Show this thread -
("well understood" meaning either that it can be explicitly described by a programmer, or that you can amass a dataset that densely samples the distribution of possible situations -- which must be static)
Show this thread
End of conversation
New conversation -
-
-
So you admit machines are basically just memorizing?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
How would the uncertainty be measured? Could a check for whether the data is stationary or not be good enough? I wonder if it would be possible to make more adaptable models by transform non-stationary data to stationary by ARC like priors
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
But is it not engineering/profitability cost trade off? If product is life threatening or billions of dollars loss situation, then we do more robust product development life cycle.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I will go with Expert-Augmented Machine Learning. Machines + Humans may be the optimal learning strategy.https://www.pnas.org/content/early/2020/02/14/1906831117 …
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.