In September we added a "retry" step to Quantum Country. If you forget something, you'll review it again later that session.
Other systems do something like this, and interviews suggested that readers needed to feel seen+supported when forgetting.
Does it help accuracy? (con't)
Conversation
It seems to helps a little… but not as much as I expected!
Considering questions forgotten after a 2 week break, how likely were those q's to be forgotten next time (a few days later)?
With retry: 79 ±4.1% (N=375)
Without retry: 74 ±1.8% (N=2347)
(95% ci's assume binomial iid)
1
1
5
OK, a few pp, but those intervals overlap. And actually, the confidence is likely a bit overstated, since there'll be within-user correlations of samples. I'd guess it'll resolve to 2-3pp.
It'd be nice to have >80% accuracies for lapsed q's. We'll need stronger interventions!
1
3
Instinctively, I suspect the retry behavior is likely worth keeping around as an emotionally important element. More interviews will help us understand that. But there's a time cost to retrying, so we'll want to replace/augment it with something better if we can!
1
4
I was quite unclear! The numbers in this tweet are *accuracies* (not rates of forgetting).
Quote Tweet
It seems to helps a little… but not as much as I expected!
Considering questions forgotten after a 2 week break, how likely were those q's to be forgotten next time (a few days later)?
With retry: 79 ±4.1% (N=375)
Without retry: 74 ±1.8% (N=2347)
(95% ci's assume binomial iid)
Show this thread
Replying to
Update with a few more months of data: experimental interval tightened up; the result holds.
Lapsed-question accuracies:
With retry: 78±2.1% (N=1553)
Without retry: 74±1.8% (N=2373)
Continues to surprise me that retry doesn't help much!
1
5