Conversation

Aella
@Aella_Girl
Selection bias is when you get misleading effects or conclusions or whatever due to how you selected the ppl you're asking about. As in, if you're measuring how nice ppl are and then ask ppl to volunteer for your study, you'll only be measuring ppl nice enough to volunteer.
3
42
Aella
@Aella_Girl
And to be clear, a lot of those studies you believe and use to shape your view of reality are drawing from pretty narrow samples anyway: college kids tryin to get good grades by participating in studies, or the classic WEIRD demographics.
1
51
Aella
@Aella_Girl
My twitter followers are obvs not a representative sample of the population, I am VIVIDLY AWARE OF THIS i am not an idiot. They tend to be male, white, 25-35yo, disproportionately libertarian (and otherwise politically split), living in mostly the US.
5
69
Aella
@Aella_Girl
And if I did something dumb like asking my twitter followers if they're into FPS video games and then using that number to extrapolate to the general population then yes your critique would be good, however unfortunately I am not in fact this dumb
1
80
Aella
@Aella_Girl
And it's *okay* to ask selected populations about information likely to be heavily biased by the selection (like video games) as long as you interpret it in context and don't fail to consider that selection effects might have modified your results.
1
63
Aella
@Aella_Girl
Things that *are* interesting and have higher chances of extrapolating are lots of types of correlations; in this case, I asked about BDSM and abusive childhoods. Maybe the correlation that showed up is due to selection bias, but how would that work?
1
43
Aella
@Aella_Girl
I would have had to, somehow, disproportionately selected for not just people into BDSM, but for ppl into BDSM who were abused. My twitter must have some way of being that sort of dissuades non-abused BDSM people from following me (and *doesn't* dissuade non-abused non-BDSM ppl)
2
57
Aella
@Aella_Girl
So we have two possible realities here - either I'm somehow having a strange selection effect that's very precise and cuts across two separate questions (bdsm and abuse), *or* there's an actual effect that correlates BDSM and childhood abuse. Of the two, the latter is less absurd
52
Likes
Aella
@Aella_Girl
Replying to
@Aella_Girl
But this is the thing that bothers me - people don't realize that proposing a selection bias is proposing another theory for the results, and that depending on the type of question asked, this theory they're proposing is often extremely unlikely.
1
91
Aella
@Aella_Girl
It's like ppl just register that I'm asking through twitter, know that my twitter account doesn't have a good sampling of the population, and then assume this means that selection bias is a good theory for any sort of twitter poll finding. No! This isn't how it works!
2
64
Aella
@Aella_Girl
And I'm NOT saying that my childhood abuse/bdsm poll is conclusive, there's a lot of possible explanations for it and I'd want to replicate this in a better, more careful and detailed survey to check on other causes. But yelling 'selection bias' just makes me wanna punch you
2
60
Aella
@Aella_Girl
the way it's delivered to me is often a bad, lazy critique that doesnt understand how bias works or how it impacts things we check for. If you tell me *how* it might be selection bias then im happy to engage: what is your theory for the selection pressures on my respondents?
3
69
Aella
@Aella_Girl
If you're like 'hey aella, i think this might be mostly just due to selection bias because you have a high following in this one BDSM recovery from childhood abuse forum that you didn't know about", I'd be like whoah, that's a great point I had no idea
10
101
blue_sun_lover
@blue_sun_lover
Replying to
@Aella_Girl
This is so dumb, you could literally extend this argument to handwave away any effects of selection bias because "well, how likely does it 𝑠𝑒𝑒𝑚 that the sample I chose 𝑗𝑢𝑠𝑡 𝑠𝑜 ℎ𝑎𝑝𝑝𝑒𝑛𝑒𝑑 to cause a sampling bias that matches exactly the results I got?"
1
1
Aella
@Aella_Girl
Replying to
@blue_sun_lover
no you cant use it to handwave away other effects of selection bias, if this were true you should predict that i 'never' think an effect is due to selection bias in my results, whereas I do sometimes think results are due to selection bias
This Tweet was deleted by the Tweet author. Learn more
Aella
@Aella_Girl
yes i did also think about this
Quote Tweet
Aella
@Aella_Girl
Replying to @sashachapin
yeah there's a lot of potential confounders like that, including ppl into bdsm are more likely to be in cultures that frame childhood experiences as abusive, but the difference is large enough that I think there's a good chance some of it is directly causal.