As someone who has a lot of love for A/B testing, much like science doesn't generally start from proving the existence of gravity and the chemical composition of water, I'd encourage you to reserve your testing budget for high-leverage interesting decisions.
-
-
"You mean most professionals can just extemporize better copy than most places ship?" Well another articulation would be that being steeped in a problem for days or weeks gives you a lot of unconscious understanding of it which the ritual of writing the copy didn't entirely tap.
Show this thread -
But I don't think most shipped copy is better than what anybody could scratchwrite, no.
Show this thread -
Hilarious experimental proof on this which I don't feel bad about quoting because I'll be the only one embarrassed: I was once giving a coding school a pro-bono lecture on A/B testing and used my home page for live coding exercise.
Show this thread -
Just to proceed with the exercise I said something to the effect of "I need a new headline here. Just shout something out; you're not going to beat the thing I've been iterating on for years but it will demonstrate the mechanics."
Show this thread -
I left the rest running for a few weeks, mostly by accident, and eventually sent the class a sheepish email congratulating them on beating my control by 10%.
Show this thread -
"This sounds curiously convenient and non-specific." I know, because I no longer run the business and therefore can't quote you the exact details, but trust me on pain of losing Internet karma: it actually happened substantially as described.
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.