Conversation

Aella
@Aella_Girl
One thing that bothers me is when ppl critique my data as "unscientific" due to sampling bias. First, y'all better not look into a lot of what you think are "scientific" studies Second, there's nothing wrong with sampling bias if you acknowledge limitations and don't extrapolate.
11
276
Aella
@Aella_Girl
Yes, I know most of my responders are white cis men between 25-40 who are associated with STEM, vaguely liberal/centrist/libertarian. There's nothing about collecting data from a particular demographic that's unscientific as long as you're upfront about the demographic.
3
93
Alex Mizrahi
@killerstorm
Replying to
@Aella_Girl
The problem isn't that they are "white cis men etc.". The problem is that you only have answers from people who are motivated enough to give you answers. And you have absolutely no idea how different it is from the general population unless you have an ability to sample the
1
Alex Mizrahi
@killerstorm
Replying to
@killerstorm
and
@Aella_Girl
general population without this bias. I.e. if you ask people opinion on X and get results, you only get results from people who are actively thinking about X. You don't get responses from people who are just not interested, or maybe never heard, etc. So the problem is that
1
Alex Mizrahi
@killerstorm
Replying to
@killerstorm
and
@Aella_Girl
the sample is influenced by the question itself. There's a simple way to address that - ask people to commit to reply before they hear the question. E.g. "I'll give you $10 to answer my question". And only after they agree you show them the question.
2
Aella
@Aella_Girl
Replying to
@killerstorm
This would incentivize ppl to say yes for the $10 without being accurate on the test, would select for ppl who need $10 more. You wouldn't get any busy right people. There's no way to remove "motivation to answer the question" from answers to the questions.
Show replies