A question for tech folks, particularly in terms of analytics and telemetry: From what I remember during my brief time at King Games, there was an emphasis on A/B tests. I have some questions about my understanding of A/B tests versus what they actually are (thread).
As I understand it, the point of an A/B test is typically that a user is not aware they're subject to such a test, and that it amounts to observing some variable based on feature/game mechanic/whatever A, versus feature/game mechanic/whatever B.
-
-
I vaguely recall from reading a description from someone involved in the field is that the purpose is usually to measure a given KPI based on some change in user experience, but beyond that, I'm not entirely clear on how A/B criteria are selected, that kind of thing.
Näytä tämä ketju -
Per the explanation that I read, it amounted to something along the lines of examining the KPI for a specific change, i.e.: A: Raising the value of in-game Widgets from X to Y results in +0.5% ARPPU. B: Raising the value of in-game Widgets from X to Z results in +0.75% ARPPU.
Näytä tämä ketju -
If that's the case, though, where do control groups fit in? Based solely on the sorts of user-experience-damaging changes that companies like Twitter appear to make in such a blithe manner, is there any consideration given to other, less measurable effects?
Näytä tämä ketju
Keskustelun loppu
Uusi keskustelu -
-
-
That is how understand it. I see A/B test come up more in UX testing though, or backend stuff the user wouldn't even know about (i.e. new function accessing a database, to see if faster or something)
Kiitos. Käytämme tätä aikajanasi parantamiseen. KumoaKumoa
-
Lataaminen näyttää kestävän hetken.
Twitter saattaa olla ruuhkautunut tai ongelma on muuten hetkellinen. Yritä uudelleen tai käy Twitterin tilasivulla saadaksesi lisätietoja.