Conversation

Andy Matuschak
@andy_matuschak
This has been one of the biggest culture clashes for me since leaving Apple. The difference here is so striking. For the first couple years, I bristled at all the metrics and A/B-ing at other tech companies (and in e.g. academic HCI). Now I have a softer view (thread):
Quote Tweet
jordwalke
@jordwalke
·
For the last ten years, the only user experience that has been capable of beating iOS on an iPhone is the next version of iOS on the next iPhone. It's inspiring. And how many mass user A/B experiments were shipped and measured in order to get here? Given the secrecy, probably ~0.
Show this thread
1
21
104
Andy Matuschak
@andy_matuschak
What matters is the interconnected chain of ideas that leads from [observed need, market state, org capacity, insights about reality, values] to [design decisions, trade-offs, strategic bets]. That map needs to match territory.
2
2
27
Andy Matuschak
@andy_matuschak
Different tools help us survey different parts of the territory and validate different parts of the chain of ideas that represents the map. We choose tools according to our confidence in each link and according to their cost, sensitivity, specificity, field of view, etc.
1
1
10
Andy Matuschak
@andy_matuschak
Metrics and A/B-ing are sometimes useful tools for evaluating pieces of the chain of ideas. I was wrong: when companies mindlessly A/B, the issue is not that this tool is useless; the issue is often that they don't really *have* an interconnected chain of ideas.
1
8
37
Andy Matuschak
@andy_matuschak
Interestingly, Apple's interconnected chain of ideas typically took the form of storytelling. Those stories encapsulated complex beliefs, intuition, and reasoning. The key insights were often not something A/B-able: the territory was revealed by "look at this prototype."
1
3
36