Your understanding is correct! And I think your frustration is reflective of Shanteau and Weiss's frustration in the paper.
To be fair to them (and to you) they (and you!) seem to want an objective function with which to compare decision quality. And that's fair!
Conversation
But it's an open question if that's possible in real world decision making scenarios.
NDM's approach doesn't get at this at all, and instead goes 'ok, we want to help people make better decisions in messy real world environments, how do we do that?'
1
1
It's pragmatic to a fault, because the organisations that fund them (the military, industry, etc) want results (from decision tools that work), not theories that produce decision tools that don't work.
My problem with Shanteau and Weiss's dismissal is that it's so ... harsh.
2
1
1
Replying to
Maybe my frustration comes from having read Heuer's "Psychology of Intelligence Analysis" as one of the first treatments of decision making I looked into. Same context as NDM (military), but very different approach.
1
1
Replying to
Oh so this is actually really interesting!
I actually *do* think JDM research is very useful in intelligence analysis, geopolitical forecasting (e.g. Tetlock's work) and also in finance. These are wicked domains, and these tools do much better there!
2
2
But in other perhaps more kind, more bounded domains, e.g. Marine fire squad commanders, firefighting, learning business, nuclear power plant management, getting better at relationships — I think it's good to look at NDM approaches instead.
1
3
Replying to
What trips me up and I'd be curious what you think: every time Jocko Willink talks about the training he ran for the Seals, he talks about sims (obviously), but also that he wanted leaders to mentally detach and generate options. Seems to completely fly into the face of this:
1
1
Replying to
This doesn't actually surprise me. One component of *using* RPD is that you break down 'intuition' to smaller components so you can manipulate it.
You have a few approaches for training RPD: you can train more sims to get better cues and more accurate action scripts, and ...
1
1
and you can train a habit to do a quick system 2 check on the results generated by RPD. Klein does both with his more recent Shadowbox training method, though the approach is common sense — of course you want to double check the result of your intuitions.
1
2
The point of the original Klein firefighter study is that, under pressure, human brains do not do option comparison.
The nature of the human mind seems to be that it wants to do RPD, not rational choice comparison. But JDM assumes rational choice comparison in all its tools!
1
2
But that's besides the point — I think the 'best practice' here is to do a combination of 'improve the results generated by RPD (system 1)' and also 'do a sanity check (system 2)', though of course under extremely high stress the latter isn't usually possible.
Hmm, maybe the right lesson here is — in domains where you have to do more option comparison (finance, trading, poker, geopolitical analysis), classical JDM tools work much better.
1
1
Replying to
Yes, that would make sense to me. And being able to do System 2 checks in extreme environments is another skill that requires "equally" extreme training to learn – i.e. Navy Seals (can) spend time/effort on this, others less so.
2
Replying to
Which, interestingly, maps pretty close to my experience with "rote" memorization and spaced repetition. The more of the fundamentals you can do automatically without thinking, the more time you have to do sanity checks when using them.
2