The number one thing to keep in mind about machine learning is that performance is evaluated on samples from one dataset, but the model is used in production on samples that may not necessarily follow the same characteristics...
Show this thread
-
The finance industry has a saying for this: "past performance is no guarantee of future results". Your model scoring X on your test dataset doesn't mean it will perform at level X on the next N situations it encounters in the real world. The future may not be like the past.
Show this thread
So when asking the question, "would you rather use a model that was evaluated as 90% accurate, or a human that was evaluated as 80% accurate", the answer depends on whether your data is typical per the evaluation process. Humans are adaptable, models are not.
-
-
If significant uncertainty is involved, go with the human. They may have inferior pattern recognition capabilities (versus models trained on enormous amounts of data), but they understand what they do, they can reason about it, and they can improvise when faced with novelty
Show this thread -
If every possible situation is known and you want to prioritize scalability and cost-reduction, go with the model. Models exist to encode and operationalize human cognition in well-understood situations.
Show this thread -
("well understood" meaning either that it can be explicitly described by a programmer, or that you can amass a dataset that densely samples the distribution of possible situations -- which must be static)
Show this thread
End of conversation
New conversation -
-
-
This is about the surgeon question? Tbh, after a point, it is all about manuality. I would totally submit myself to a robot surgeon (ex eye surgery is robotized) but keep the diagnosis and treatment plan to human expertize
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Models are adaptable too
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I think the real question was if you would want the AI surgeon to be illegal rather than which one to choose. It’s a tough question and I think it should be legal if everyone understands the risks involved in choosing one over the other
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
The comparison between model and humans also depends on the intelligence of humans
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
And how can we build models that are (more) adaptable? RL?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Accuracy is wrong measure for most real life problems. Survival then growth is correct target. Nothing to do with human vs ML though human writing code loses to human adapting in the game you describe. There is no ML! Only humans doing things at a distance.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.