This has been one of the biggest culture clashes for me since leaving Apple. The difference here is so striking. For the first couple years, I bristled at all the metrics and A/B-ing at other tech companies (and in e.g. academic HCI). Now I have a softer view (thread):https://twitter.com/jordwalke/status/1060787524488949760 …
-
Show this thread
-
What matters is the interconnected chain of ideas that leads from [observed need, market state, org capacity, insights about reality, values] to [design decisions, trade-offs, strategic bets]. That map needs to match territory.
2 replies 2 retweets 27 likesShow this thread -
Different tools help us survey different parts of the territory and validate different parts of the chain of ideas that represents the map. We choose tools according to our confidence in each link and according to their cost, sensitivity, specificity, field of view, etc.
1 reply 1 retweet 10 likesShow this thread -
Metrics and A/B-ing are sometimes useful tools for evaluating pieces of the chain of ideas. I was wrong: when companies mindlessly A/B, the issue is not that this tool is useless; the issue is often that they don't really *have* an interconnected chain of ideas.
2 replies 9 retweets 35 likesShow this thread -
Interestingly, Apple's interconnected chain of ideas typically took the form of storytelling. Those stories encapsulated complex beliefs, intuition, and reasoning. The key insights were often not something A/B-able: the territory was revealed by "look at this prototype."
1 reply 3 retweets 34 likesShow this thread -
Some longer thoughts from me on the limits of A/B-ing and metrics here: http://blog.andymatuschak.org/post/159340765257/exalting-data-missing-meaning ….
3 replies 10 retweets 40 likesShow this thread -
(In conclusion it’s actually just all about Popper; everything else falls out of that)
3 replies 0 retweets 10 likesShow this thread -
Replying to @andy_matuschak
Can you clarify a bit what "Territory" means in this context?
1 reply 0 retweets 0 likes
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.