How should we evaluate tools for thought? There's no simple metric, as far as I can tell. The best tools change your paradigm anyway, so your old metrics (books printed per year?) aren't what matter.
Here's one (vague, but focusing): how much meaning is unlocked on the margin?
Conversation
That is, you can talk about Mathematica's value by asking how many students use it, or if it helps their test scores, or by timing people solving problems using different tools. But its most significant value is in producing marginal profound mathematical insights.
1
1
21
It's all a variant of Kay's "Sistine chapels per generation," I guess! But the marginal meaning doesn't have to be a grand edifice: Twitter's most powerful metric as a tool for thought is in creating transformative (off-platform) personal connections.
3
3
29
It's not clear how to get leading indicators for any of this! As far as I can tell, you want to be on the lookout for very strange stories, like casually making an animation system in Smalltalk at age 12.
Do any of you have good leading indicator stories here?
2
1
15
I'm worried I don't know what you mean by "leading indicator" but it seems to me that you might analyze the discourse surrounding the tool. The shared google doc convo for instance generates a certain kind of conversation. And is it really thought if it's not shared?
1
Replying to
(Leading indicator: something you can measure as an early sign for what you really want to measure)
This is a great point! E.g. impact of many reform teaching ideas in #mtbos discourse before in widespread academic outcomes.
Re: your last q—transcendent spiritual experience?
Replying to
(Thanks -- now I get it!)
Transcendent spiritual experience is interesting. Though if it's really uncommunicable -- in images, words, poetry, etc. -- I start losing confidence that there is anything really experienced. Though I have to admit this is hard for me to think through.
1