In September we added a "retry" step to Quantum Country. If you forget something, you'll review it again later that session.
Other systems do something like this, and interviews suggested that readers needed to feel seen+supported when forgetting.
Does it help accuracy? (con't)
Conversation
Replying to
It seems to helps a little… but not as much as I expected!
Considering questions forgotten after a 2 week break, how likely were those q's to be forgotten next time (a few days later)?
With retry: 79 ±4.1% (N=375)
Without retry: 74 ±1.8% (N=2347)
(95% ci's assume binomial iid)
1
1
5
OK, a few pp, but those intervals overlap. And actually, the confidence is likely a bit overstated, since there'll be within-user correlations of samples. I'd guess it'll resolve to 2-3pp.
It'd be nice to have >80% accuracies for lapsed q's. We'll need stronger interventions!
1
3
Instinctively, I suspect the retry behavior is likely worth keeping around as an emotionally important element. More interviews will help us understand that. But there's a time cost to retrying, so we'll want to replace/augment it with something better if we can!
1
4
I was quite unclear! The numbers in this tweet are *accuracies* (not rates of forgetting).
Quote Tweet
It seems to helps a little… but not as much as I expected!
Considering questions forgotten after a 2 week break, how likely were those q's to be forgotten next time (a few days later)?
With retry: 79 ±4.1% (N=375)
Without retry: 74 ±1.8% (N=2347)
(95% ci's assume binomial iid)
Show this thread
1
3
Update with a few more months of data: experimental interval tightened up; the result holds.
Lapsed-question accuracies:
With retry: 78±2.1% (N=1553)
Without retry: 74±1.8% (N=2373)
Continues to surprise me that retry doesn't help much!
1
5
Replying to
I am sorry if it is already available but is there a way i can go through the questions again as part of the text itself? When i re-read, i do not see the questions.
1
I'm afraid there isn't such an option at the moment. Could you say more about why it feels like that might be helpful to you?
1
Show replies
Replying to
I’m going through it now but I’m taking a couple days to finish it. Does it save my progress?
1
Yes, if you register.
1
Show replies
Replying to
What kind of annoyed me and kept me from going back is that I have to review the easy questions as often as the hard ones (i.e. no easy modifier like in Anki). When I have to answer 20 basic questions I never got wrong to also review 10 hard ones it feels like a waste of time.
1
1
That’s helpful feedback, thank you. We recently made the schedule much more aggressive in part to help with this. Do you recall when you tried it? I would be curious if it still felt annoying with the new schedule.
1
2
Show replies