@NYTnickc wrote about our work at @anotheracronym to bring fast & rigorous measurement to online political advertising
https://www.nytimes.com/2020/04/28/us/politics/Facebook-Acronym-adverstising.html ….
-
-
So what’s your overall program effectiveness? People who saw our ads over the past two months have 3.6% [95% CI: 1%, 6.2%] lower approval of Donald Trump compared with our control/holdout.
Show this thread -
Tell me about your methods! We run surveys *on FB.* That means that we know that nearly everyone we survey saw our ads (except those in control). Most attempts to measure social media advertising cannot cleanly match people who took the survey to people who actually saw the ad.
Show this thread -
Aren’t most ad experiments noisy & under-powered? Yes! This is why we generally issue pre- & post surveys—dramatically more precision. & we use indices for our key outcome measures. See
@dbroockman,@j_kalla, & Jas Sekhon https://web.stanford.edu/~dbroock/published%20paper%20PDFs/broockman_kalla_sekhon_design_of_field_experiments_with_survey_outcomes.pdf …Show this thread -
Who are these people? What’s your sample frame? We use the voter file to find people in battleground states, who are more likely to be politically moderate, & less likely to follow politics. These folks are most likely to respond to key facts, but they are hard to reach.
Show this thread -
What about bias from the FB algorithm/ad auction? We take measures to mitigate bias documented by
@sapiezynski,@mbogen,@korolova & others https://arxiv.org/abs/1904.02095 . 1, we don’t include hardcore partisans & 2, we reach the vast majority of people in our sample frame.Show this thread -
We are also relying on an array of ad optimization techniques to get a diverse set of people when they are are most likely to see the ad, rather than just optimizing for clicks
Show this thread -
What about non-response bias? FB's population base for age and gender counterbalance typical response bias patterns. Still, we oversample younger and non-white audiences and constantly rebalance our advertising budgets to make sure we are recruiting from these subpopulations.
Show this thread -
Great thread from
@taraemcg on this!https://twitter.com/taraemcg/status/1255135755757748224?s=20 …Show this thread -
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
alert: One of the hardest things to do in electoral politics is to persuade voters who are on the fence or leaning towards your opponent to change their opinion. It's why billions of
are spent every single cycle with very little evidence of what actually works