Elea McDonnell Feit

@eleafeit

Marketing Analytics Prof . Bayesian. Philadelphian.

Philadelphia
Vrijeme pridruživanja: siječanj 2010.

Tweetovi

Blokirali ste korisnika/cu @eleafeit

Jeste li sigurni da želite vidjeti te tweetove? Time nećete deblokirati korisnika/cu @eleafeit

  1. Prikvačeni tweet
    27. sij

    Have you ever calculated the sample size for an and come up with a sample size that is bigger than you can ever practically get? Does this mean you shouldn't run the test? No! A paper thread for 1/17

    Prikaži ovu nit
    Poništi
  2. 1. velj

    Saturday night at the Feit’s

    Poništi
  3. 31. sij

    Don't forget that the deadline to submit to 's Advanced Research Techniques Forum is approaching on 2/9. If you are doing cool data science work in marketing, we would love to hear about it. Conference will be in Rochester June 3-5.

    Poništi
  4. 29. sij

    Wouldn’t even weak evidence in favor of the null be worrisome in a randomization check?

    Prikaži ovu nit
    Poništi
  5. 29. sij

    I think I know the answer to this, but why do randomization checks or placebo tests use the cutoff p=0.05 instead of something more conservative?

    Prikaži ovu nit
    Poništi
  6. 29. sij

    Do I know anyone that works with one of the private ad exchanges in the US? E.g., Concert (VOX Media), Technorati, CBS, IDG TechNetwork, NBCUniversal

    Poništi
  7. 29. sij
    Poništi
  8. 27. sij

    TFW you post a thread on your recent paper and misspell your co-authors twitter handle. Sorry,

    Poništi
  9. 27. sij

    What else will you find in the paper? We show how to compute profit-maximizing sample sizes when one treatment is likely to perform better than the other. That is, we can tell you the optimal size of holdout in media incrementally tests. Read the paper for the details. 17/17

    Prikaži ovu nit
    Poništi
  10. 27. sij

    The paper is called “Test & Roll: Profit Maximizing A/B tests” and just came out @MarketingScience: . wrote more about it at . 16/17

    Prikaži ovu nit
    Poništi
  11. 27. sij

    Those familiar with multi-armed bandits may be wondering how this compares to a dynamic approach. We looked into that and found that an optimal-size test & roll (orange in plots) achieves nearly the same level of expected profit as Thompson Sampling (purple). 15/17

    Prikaži ovu nit
    Poništi
  12. 27. sij

    Computing a test & roll sample size requires information about the range of average response you expect from your treatments. Using data on the past performance of similar treatments, you can estimate priors with an HB model using tools like and . 14/17

    Prikaži ovu nit
    Poništi
  13. 27. sij

    Our sample size formula also scales with the size of your population. So, if you have lots of customers/traffic, then you should run a bigger test, but marketers with small populations should still run tests. Tests always improve profit! 13/17

    Prikaži ovu nit
    Poništi
  14. 27. sij

    A test & roll usually deploys the wrong treatment at a greater rate than an NHST. Is this bad? No, because the sample size has been optimized for profit. Errors are most likely to happen when the difference between treatments is small and doesn't affect profit much. 12/17

    Prikaži ovu nit
    Poništi
  15. 27. sij

    Why should stop doing hypothesis tests? Profit-maximizing tests are much smaller and generate more profit! In one of our case studies a hypothesis test requires a sample size of 18,468 per group while the profit-maximizing size is 2,284 and makes more profit. 11/17

    Prikaži ovu nit
    Poništi
  16. 27. sij

    developed a similar approach with encouragement from . He frames the problem around discounting, but the idea of trading opportunity cost against deployment errors is essentially the same. 10/17

    Prikaži ovu nit
    Poništi
  17. 27. sij

    So, what do you do when you get the results of the test? Simple: pick the treatment with the higher average response. No inconclusive tests! We test then roll in an intuitive way. 9/17

    Prikaži ovu nit
    Poništi
  18. 27. sij

    If you don’t like formulas, you can compute the profit-maximizing test & roll sample size at . We also provide R code at . 8/17

    Prikaži ovu nit
    Poništi
  19. 27. sij

    So, how big is the profit-maximizing test & roll? @marketsensi and I came up with a new sample size formula that maximizes expected profit. Here it is: 7/17

    Prikaži ovu nit
    Poništi
  20. 27. sij

    In a test & roll, there is a fundamental trade-off between the opportunity cost of the test and the potential to deploy the wrong treatment. With a few distributional assumptions, we can compute the expected profit of a test & roll for different test sizes. 6/17

    Prikaži ovu nit
    Poništi
  21. 27. sij

    Marketers do this type of experiment all the time. For example, here is an email A/B testing tool from . , and all have similar A/B testing tools. 5/17

    Prikaži ovu nit
    Poništi

Čini se da učitavanje traje već neko vrijeme.

Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.

    Možda bi vam se svidjelo i ovo:

    ·