I’ve been studying dynamics of reader memory with the mnemonic medium, running experiments on interventions, etc. A big challenge has been that I'm roughly trying to understand changes in a continuous value (depth of encoding) through discrete measurements (remembered / didn’t).
Conversation
I can approximate a continuous measure by looking at populations: “X% of users in situation Y remembered.” Compare that % for situations Y and Y’ to sorta measure an effect. This works reasonably well when many users are “just on the edge” of remembering, and poorly otherwise…
1
5
It’s a threshold function on the underlying distribution. Imagine that a person will remember something iff their depth-of-encoding (a hidden variable)—plus some random noise (situation)—is greater than some threshold. Our population measure can distinguish A vs A’, not B vs B’.
2
6
So it works pretty well initially, when the distribution’s spread out. e.g.: I’ve been running an RCT on retry mechanics. Of readers who forget an answer while reading an essay, about 20% more will succeed in their first review if the in-essay prompt gave them a chance to retry.
2
1
6
But it doesn’t work well when the distribution’s skewed to one side. eg: I’ve run RCTs manipulating schedules. You might think shortened intervals would help struggling readers, but it has little effect on the population measure—just (likely) nudges some closer to the threshold.
1
2
Lack of a good continuous measure makes it hard to characterize the dynamics of what’s going on, which makes it hard to make iterative improvements. I’ll need to find some good solution here. Unfortunately, response times are (AFAICT) not a strong enough predictor to use.
2
6
Have you looked at Multinomial Processing Tree models?
1
1
No, will look—thanks!
1
1
Sure! MPTs can help with categorical data and simple latent variables (encoded vs. guessed, etc.)
Also, more recent (but complex) models for SRS scheduling based on first estimating users' recall probabilities:
i) pnas.org/content/116/10
ii) arxiv.org/pdf/2102.04174
2
Ah, thanks, the second one is new to me! I believe models like these can improve performance (particularly v. Leitner). On a conceptual level, I worry about probability-of-recall as a modeling scheme. Predictive, maybe, but explanatory? What’s really going on?
2
Related: an inverse-expo probability decay curve is often attributed to Ebbinghaus (incl in the 2nd paper!), but his inverse-expo wasn’t of probability; it was time-savings-to-relearn. Still not an explanatory model, but at least closer to a physical, directly-measurable one.