1/ Let's say that you have access to an expert, and you want to get at their expertise because you want to learn it.
You don't have time to do a full skill extraction. (These take hours).
What's one question you can ask that generates the most useful information for you?
Conversation
2/ I think I have one. Ask:
"What would a novice get wrong here?"
The 'here' is key. The question must be asked in the context of a story, a case study, or a decision point. You MUST get the expert to talk about interesting cases they've experienced.
Replying to
3/ I've been reflecting on my ACTA experience with . ACTA is a type of cognitive task analysis (a skill extraction technique). It is supposed to be the simplest of the CTAs.
And yet it took us hours!
1
2
14
4/ Like all CTA methods, ACTA is fairly rigorous, and asks many overlapping questions to get at an expert's tacit expertise.
You use it when you want to extract expertise for training design, or user interface design.
But what if you only have a few minutes?
1
5
5/ On reflection I realise nearly all of John's best answers in our ACTA session were in response to "What would a novice get wrong here?"
I'm starting to suspect the bulk of the ACTA method is simply to frame and set up the conversation so that you get to ask this question.
1
7
6/ In the weeks since, I've dropped "what would a novice get wrong here?" in a podcast interview, in a casual conversation with a programmer I respected, and with a writer. It's not perfect, of course, but it produces a LOT of information about their expertise.
Give it a try.
1
1
13
2
4
Some further nuance:
Quote Tweet
Replying to @ejames_c and @wasimlorgat
Full context: the reason you want to ask about a generic novice and not about just yourself is because the expert has likely seen more novices and more failure modes than you can imagine.
This question thus unearths more of the nature of their expertise than you might expect.
4
Replying to
The first two questions we coached students to ask teachers, mentors, coaches, etc. at DevBootcamp were:
1. What are the most cliché mistakes a beginner/novice/someone like me could make when doing X?
2. When did you first have an inkling I was confused about X?
2
3
Likewise, if you believe some mistakes are more diagnostic than others WRT (critical) flaws in students' mental models, you can design the curriculum+facilitation to (1) induce diagnostic mistakes when the flaw is present and (2) minimize non-diagnostic or misleading mistakes.
1