As I understand it, the point of an A/B test is typically that a user is not aware they're subject to such a test, and that it amounts to observing some variable based on feature/game mechanic/whatever A, versus feature/game mechanic/whatever B.
A question for tech folks, particularly in terms of analytics and telemetry: From what I remember during my brief time at King Games, there was an emphasis on A/B tests. I have some questions about my understanding of A/B tests versus what they actually are (thread).
-
-
Näytä tämä ketju
-
I vaguely recall from reading a description from someone involved in the field is that the purpose is usually to measure a given KPI based on some change in user experience, but beyond that, I'm not entirely clear on how A/B criteria are selected, that kind of thing.
Näytä tämä ketju -
Per the explanation that I read, it amounted to something along the lines of examining the KPI for a specific change, i.e.: A: Raising the value of in-game Widgets from X to Y results in +0.5% ARPPU. B: Raising the value of in-game Widgets from X to Z results in +0.75% ARPPU.
Näytä tämä ketju -
If that's the case, though, where do control groups fit in? Based solely on the sorts of user-experience-damaging changes that companies like Twitter appear to make in such a blithe manner, is there any consideration given to other, less measurable effects?
Näytä tämä ketju
Keskustelun loppu
Uusi keskustelu -
Lataaminen näyttää kestävän hetken.
Twitter saattaa olla ruuhkautunut tai ongelma on muuten hetkellinen. Yritä uudelleen tai käy Twitterin tilasivulla saadaksesi lisätietoja.