Conversation

Completely agreed with Spencer here. Probably the thing I've learned the most throughout my thousands of questions and feedback is exactly this - and despite my experience, I still find it very hard to guarantee question design that definitely won't hit any of these problems. 1/
Quote Tweet
The exact wording in polls, surveys and psychology studies matters more than most people seem to realize. This is a big deal if you want to learn from polls and academic papers, or if you conduct studies yourself. Here are some dramatic real examples:
Show this thread
8
170
I've also had the experience of reading about studies or measures, and then when I see their actual questions I am shocked at how easily misinterpretable they are. Regularly they use questions I would never ever use in my own research. 2/
2
51
I thought for a while that they must have their reasons, it's gotta make sense somehow, they can't be just that bad at question design. But lately I'm starting to suspect they just outright *are* bad at it? Which freaks me out a bit and feels like it has huge implications. 3/
Replying to
How much published research is based on terribly worded questions, that lead to really inaccurate results? Why haven't I heard more people talking about this?
14
94
Replying to
Professional election pollster here. In the field of opinion polling, there is a roughly equal division between “Asks questions properly” “Bad at it because stupid” “Bad at it on purpose to get results they like”.
1
Replying to
Still not Cynical enough. There’s a YES MINISTER bit where a poll comes back supporting a policy the minister opposes: “Well get a poll that’s against it,” and he explains all these tricks That was Television! in the 80s! By now controlling poll results is an exact science