Conversation

the amount of 'but your sample has bias' i get any time i faintly breathe about my data makes me wanna delete twitter. imo most people are wrong about when and why sampling bias is bad, and also assigning unfair scrutiny to me vs other published research because of credentialism
48
467
my samples are likely more reliable than most published social research and im starting to suspect you're complaining about me and not them because im publishing my results in blog posts and twitter instead of precisely formatted pdfs and don't have a degree
16
163
i'm not saying i don't have stuff to learn - i try to share my full process and data so people can critique and help me grow - but I am saying that the specific critique of sampling bias in the way it's usually leveled at me is poop-in-toilet-after-taco-bell quality
5
87
and unfortunately, this shitty critique is super common, like 25% of the replies to anything i post, and often given by people who claim to be in the field (which makes me skeptical of the competence of people working in the field). im starting to get real sensitive to it
3
99
it also like... makes me vaguely horrified? Like if the widespread understanding of how research works is so bad that most people are saying such misguided stuff with such confidence? like i knew it was bad but this is extremely basic/elementary
11
98
i wonder if another reason this is happening is that academic papers often slightly obscure where they get their sample from; it's deep in the pdf and in formal language and so ppl don't actually register the author just asked 30 college freshmen in exchange for a grade
5
209
whereas for me it's super clear/easily obvious that i am doing an internet survey, that individuals themselves can have access to, so it's easier for them to critique? the vulnerability of transparency.
13
123
Replying to
1) even if it's not repeated it still has value 2) I often repeat my own surveys, across time, with different places on the internet, to check if I'm getting the same results. I am currently redoing a survey I did five years ago with a different audience!