Quantitative Editor, @FiveThirtyEight | PhD, @LSEGovernment | 
in 
, via 
| she/her
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
I *was* going to write an article about herding in Austrian polls this week... but things got weirdly busy, so here's a thread instead. Austria's having an election this Sunday, and the polls have been incredibly stable. So stable, in fact, that there's good evidence of herding. https://twitter.com/laurabronner/status/1177284463396556803 …pic.twitter.com/6TKKFzUMvM
To check for herding, I took the 18 polls released since Aug. 1, and simulated the standard deviation you'd expect based on the polling average and sample sizes. I compared that to the actual standard deviation of the polls, which turned out to be *a lot* smaller.pic.twitter.com/c7rNfp8bon
While the above chart shows the ÖVP, this is true for *all* major parties in Austria; in some cases, the actual standard deviation was less than half of what we'd expect.pic.twitter.com/hjcSYSPQhJ
It turns out that at least two major parties showed evidence of herding in *all* of the last six elections, and in some cases almost all parties did.pic.twitter.com/fqUmvB3n7J
The problem with herding is twofold: (1) it reduces the variance of the estimate -- preventing us from getting an accurate measure of the uncertainty in the race -- and (2) it can potentially bias results if pollsters are converging on the wrong number.
But despite evidence of herding, the mean absolute polling error -- the mean difference between the result and the last-4-week polling average, by party -- is quite low. Despite herding, Austrian polls are quite predictive!pic.twitter.com/dHMfhBG08j
What does this mean for Sunday's election? Well, things are probably more uncertain than the polls make them seem. We have good reason to believe that the polls aren't adequately incorporating the uncertainty in the race, leaving room for surprises.
What method did you use to simulate standard deviation?
I simulated new poll results for all 18 polls by sampling from a normal distribution around the polling average assuming SRS and each poll's sample size, then did that a bunch of times and took the standard deviation each time.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.
