At the end of a paper titled "The Futility of Decision Making Research", in which Weiss and Shanteau reflect on the uselessness of their research careers, the authors take a side-swipe at the Naturalistic Decision Making research community.
Which, what? sciencedirect.com/science/articl
Conversation
Replying to
I mean everyone shits on Kahnemann and co. about many results not replicating, but is there a similar effort to test and replicate RPDM etc.? One frustration I have with that field (but maybe just haven't read enough) is that the answer to "how to make better decisions"...
2
3
seems to boil down to "become more of an expert" i.e. develop expert intuition by going to realistic sims. Which...fine, okay. But how does an expert evaluate whether he's making the right calls?
1
3
A lot of RPDM seems to me (again, may need to read more) very descriptive of what experts do, then gives you CTA to become more of an expert. But there's little mechanistic explanation that generalizes outside of "be expert in field"
1
1
Replying to
Your understanding is correct! And I think your frustration is reflective of Shanteau and Weiss's frustration in the paper.
To be fair to them (and to you) they (and you!) seem to want an objective function with which to compare decision quality. And that's fair!
1
2
But it's an open question if that's possible in real world decision making scenarios.
NDM's approach doesn't get at this at all, and instead goes 'ok, we want to help people make better decisions in messy real world environments, how do we do that?'
1
1
It's pragmatic to a fault, because the organisations that fund them (the military, industry, etc) want results (from decision tools that work), not theories that produce decision tools that don't work.
My problem with Shanteau and Weiss's dismissal is that it's so ... harsh.
2
1
1
Replying to
Maybe my frustration comes from having read Heuer's "Psychology of Intelligence Analysis" as one of the first treatments of decision making I looked into. Same context as NDM (military), but very different approach.
1
1
Replying to
Oh so this is actually really interesting!
I actually *do* think JDM research is very useful in intelligence analysis, geopolitical forecasting (e.g. Tetlock's work) and also in finance. These are wicked domains, and these tools do much better there!
2
2
But in other perhaps more kind, more bounded domains, e.g. Marine fire squad commanders, firefighting, learning business, nuclear power plant management, getting better at relationships — I think it's good to look at NDM approaches instead.
Replying to
What trips me up and I'd be curious what you think: every time Jocko Willink talks about the training he ran for the Seals, he talks about sims (obviously), but also that he wanted leaders to mentally detach and generate options. Seems to completely fly into the face of this:
1
1
Replying to
This doesn't actually surprise me. One component of *using* RPD is that you break down 'intuition' to smaller components so you can manipulate it.
You have a few approaches for training RPD: you can train more sims to get better cues and more accurate action scripts, and ...
1
1
Show replies