At the end of a paper titled "The Futility of Decision Making Research", in which Weiss and Shanteau reflect on the uselessness of their research careers, the authors take a side-swipe at the Naturalistic Decision Making research community.
Which, what? sciencedirect.com/science/articl
Conversation
Replying to
I mean everyone shits on Kahnemann and co. about many results not replicating, but is there a similar effort to test and replicate RPDM etc.? One frustration I have with that field (but maybe just haven't read enough) is that the answer to "how to make better decisions"...
2
3
seems to boil down to "become more of an expert" i.e. develop expert intuition by going to realistic sims. Which...fine, okay. But how does an expert evaluate whether he's making the right calls?
1
3
A lot of RPDM seems to me (again, may need to read more) very descriptive of what experts do, then gives you CTA to become more of an expert. But there's little mechanistic explanation that generalizes outside of "be expert in field"
1
1
Replying to
Your understanding is correct! And I think your frustration is reflective of Shanteau and Weiss's frustration in the paper.
To be fair to them (and to you) they (and you!) seem to want an objective function with which to compare decision quality. And that's fair!
1
2
But it's an open question if that's possible in real world decision making scenarios.
NDM's approach doesn't get at this at all, and instead goes 'ok, we want to help people make better decisions in messy real world environments, how do we do that?'
1
1
It's pragmatic to a fault, because the organisations that fund them (the military, industry, etc) want results (from decision tools that work), not theories that produce decision tools that don't work.
My problem with Shanteau and Weiss's dismissal is that it's so ... harsh.
One interpretation of their paper is "maybe we cannot come up with an objective decision function for real world decision making scenarios and the best we have is NDM-like tools."
But, noooo, they don't conclude that. They instead say "NDM is also useless for this purpose."
1
1
Which — sure. They're entitled to their own opinion. But when the military started funding NDM, they did so because they were tired of funding classical JDM research that produced decision tools that nobody used.
Maybe that should tell them something?
I don't know, man.
1
Replying to
Maybe my frustration comes from having read Heuer's "Psychology of Intelligence Analysis" as one of the first treatments of decision making I looked into. Same context as NDM (military), but very different approach.
1
1
Replying to
Oh so this is actually really interesting!
I actually *do* think JDM research is very useful in intelligence analysis, geopolitical forecasting (e.g. Tetlock's work) and also in finance. These are wicked domains, and these tools do much better there!
2
2
Show replies