At the end of a paper titled "The Futility of Decision Making Research", in which Weiss and Shanteau reflect on the uselessness of their research careers, the authors take a side-swipe at the Naturalistic Decision Making research community.
Which, what? sciencedirect.com/science/articl
Conversation
Replying to
I mean everyone shits on Kahnemann and co. about many results not replicating, but is there a similar effort to test and replicate RPDM etc.? One frustration I have with that field (but maybe just haven't read enough) is that the answer to "how to make better decisions"...
2
3
seems to boil down to "become more of an expert" i.e. develop expert intuition by going to realistic sims. Which...fine, okay. But how does an expert evaluate whether he's making the right calls?
1
3
A lot of RPDM seems to me (again, may need to read more) very descriptive of what experts do, then gives you CTA to become more of an expert. But there's little mechanistic explanation that generalizes outside of "be expert in field"
1
1
Replying to
Your understanding is correct! And I think your frustration is reflective of Shanteau and Weiss's frustration in the paper.
To be fair to them (and to you) they (and you!) seem to want an objective function with which to compare decision quality. And that's fair!
1
2
But it's an open question if that's possible in real world decision making scenarios.
NDM's approach doesn't get at this at all, and instead goes 'ok, we want to help people make better decisions in messy real world environments, how do we do that?'
1
1
It's pragmatic to a fault, because the organisations that fund them (the military, industry, etc) want results (from decision tools that work), not theories that produce decision tools that don't work.
My problem with Shanteau and Weiss's dismissal is that it's so ... harsh.
2
1
1
Replying to
Maybe my frustration comes from having read Heuer's "Psychology of Intelligence Analysis" as one of the first treatments of decision making I looked into. Same context as NDM (military), but very different approach.
1
1
Replying to
Oh so this is actually really interesting!
I actually *do* think JDM research is very useful in intelligence analysis, geopolitical forecasting (e.g. Tetlock's work) and also in finance. These are wicked domains, and these tools do much better there!
2
2
Replying to
Right, I think we had this in a previous discussion. NDM is "tactical decision making", like the fireground commanders they write about. Whereas JDM is slower, like in most business contexts.
1
1
Replying to
Yeah to be fair I haven't quite put my finger on when the JDM approach is better and when the NDM approach makes more sense.
I know Kahneman and Klein covered it in their paper, but I keep finding edge cases where their two properties don't apply cleanly.