That’s my feeling too. What I think the misunderstanding is if one needs that in A/B testing to make a design decision... or... do they just need some insight from a reduced level of A/B testing to get enough confidence to move forward in a direction for now.
-
-
Replying to @nickf @lukechesser and
I guess you’re right, if your goal is to not worry about whether your decision was because people actually favored one choice over the other, or if the winner was just “dumb luck.” I you don’t care which, then statistical confidence is a lot of extra work.
1 reply 0 retweets 0 likes -
Of course, by avoiding the work of calculating statistical confidence, you’re leaving yourself open to having made a decision that makes the design worse, not better. But hey, life’s a gamble, right?
1 reply 1 retweet 0 likes -
Replying to @jmspool @lukechesser and
I guess to phrase it another way; don’t let “perfect” get in the way of shipping.
1 reply 0 retweets 0 likes -
Replying to @nickf @lukechesser and
Sure. Though, if you’re not going to use statistical confidence to guide your “perfection”, I’m wondering why you’re using A/B tests at all? They take effort and since the results will be technically inconclusive, why not just flip a coin or look for a sign from the Gods?
1 reply 0 retweets 1 like -
They’re equally accurate methods and will end up with the same outcome. Much less work and expense than those messy A/B test things.
1 reply 0 retweets 0 likes -
Replying to @jmspool @lukechesser and
I see your point. So what would you suggest in the case of Kevin’s (
@vernalkick) predicament here?2 replies 0 retweets 0 likes -
Replying to @nickf @lukechesser and
What I’d recommend to any other designer: Talk to your customers and users.
1 reply 0 retweets 1 like -
Replying to @jmspool @lukechesser and
It amazes me how many businesses fail, or even outright refuse, to talk to their customers. A nightmare job is to be in research and/or design and be barred from talking directly with customers <eyes a few Fortune 500 companies>.
2 replies 0 retweets 4 likes -
Replying to @nickf @lukechesser and
Businesses are not immune to Sturgeon’s Law.
1 reply 0 retweets 0 likes
90% of everything may be crap, but we get to choose if we strive to be in the 10%. It's
often an explicit choice to stay in the 90%. 
-
-
Replying to @DaveHogue @nickf and
I agree and have said the same many times.
0 replies 0 retweets 0 likesThanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.