That would be a great metric to track. It reminds me of the Bezos quote “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.” With tools like Optimizely and Google Optimize today, there’s no reason not to.
Investors should ask about “experimentation metrics” during diligence but don’t: 1. Velocity: how many experiments do you ship daily/weekly/monthly/yearly? 2. Quality: what % of experiments lead to meaningful insights? 3. Roadmap: what experiments will you run this year?
-
-
-
Agreed — most startups don’t track experimentation velocity internally either. Requires discipline and empowered teams to create this culture.
Kraj razgovora
Novi razgovor -
-
-
Oh man, at what size company? What size audience? But also: How many people in these experiments? How do you know if they are statistically valid? Do experiments overlap? How do they overlap with cohorts, or do they? If you are seed and think you have answers I will laugh.
-
You should still have an experimentation culture at seed. Those questions are pretty far in the weeds. You’re trying to learn quickly & use velocity as your advantage being a startup - not necessarily trying to design perfect stat sig tests.
- Još 2 druga odgovora
Novi razgovor -
-
-
Dislike the term, but you might be interested in googling “innovation accounting” — some good breakdowns of the concept (and bad ones, too)
-
Love it and will dig into the idea.
- Još 1 odgovor
Novi razgovor -
-
-
I think the culture is imp but the sophisticated metrics at early stage is hard. Many startups simply trying to get their backend data in sync w fronted analytics so they can do simple cohort analysis let alone this.
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
-
-
@rabois disagrees and so do I.
- Još 1 odgovor
Novi razgovor -
-
-
4: Threshold: what strength of result is required to consider a hypothesis supported? 5: Novelty: what % of experiments are replicating established results (either sci or other co's)?
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.