body { -ms-overflow-style: scrollbar; overflow-y: scroll; overscroll-behavior-y: none; } .errorContainer { background-color: #FFF; color: #0F1419; max-width: 600px; margin: 0 auto; padding: 10%; font-family: Helvetica, sans-serif; font-size: 16px; } .errorButton { margin: 3em 0; } .errorButton a { background: #1DA1F2; border-radius: 2.5em; color: white; padding: 1em 2em; text-decoration: none; } .errorButton a:hover, .errorButton a:focus { background: rgb(26, 145, 218); } .errorFooter { color: #657786; font-size: 80%; line-height: 1.5; padding: 1em 0; } .errorFooter a, .errorFooter a:visited { color: #657786; text-decoration: none; padding-right: 1em; } .errorFooter a:hover, .errorFooter a:active { text-decoration: underline; } #placeholder, #react-root { display: none !important; } body { background-color: #FFF !important; }

JavaScript is not available.

We’ve detected that JavaScript is disabled in this browser. Please enable JavaScript or switch to a supported browser to continue using twitter.com. You can see a list of supported browsers in our Help Center.

Terms of Service Privacy Policy Cookie Policy Imprint Ads info © 2023 Twitter, Inc.

Conversation

the amount of 'but your sample has bias' i get any time i faintly breathe about my data makes me wanna delete twitter. imo most people are wrong about when and why sampling bias is bad, and also assigning unfair scrutiny to me vs other published research because of credentialism

48

8

467

my samples are likely more reliable than most published social research and im starting to suspect you're complaining about me and not them because im publishing my results in blog posts and twitter instead of precisely formatted pdfs and don't have a degree

9:06 PM · Dec 26, 2022

Replying to

i'm not saying i don't have stuff to learn - i try to share my full process and data so people can critique and help me grow - but I am saying that the specific critique of sampling bias in the way it's usually leveled at me is poop-in-toilet-after-taco-bell quality

5

1

87

and unfortunately, this shitty critique is super common, like 25% of the replies to anything i post, and often given by people who claim to be in the field (which makes me skeptical of the competence of people working in the field). im starting to get real sensitive to it

3

1

99

it also like... makes me vaguely horrified? Like if the widespread understanding of how research works is so bad that most people are saying such misguided stuff with such confidence? like i knew it was bad but this is extremely basic/elementary

11

4

98

i wonder if another reason this is happening is that academic papers often slightly obscure where they get their sample from; it's deep in the pdf and in formal language and so ppl don't actually register the author just asked 30 college freshmen in exchange for a grade

5

6

209

whereas for me it's super clear/easily obvious that i am doing an internet survey, that individuals themselves can have access to, so it's easier for them to critique? the vulnerability of transparency.

13

3

123

hood adjacent Jack

Replying to

Not that I’m disagreeing but in what way are your samples more reliable?

1

2

Replying to

much, much bigger, and usually from more distributed/representative sources than what I'm under the impression they use

2

3

Show replies

@hannahmetzler1

Replying to

I do research with Twitter data as a proper academic & I get this as a question for every talk & review. It isn't about you, people have nothing more substantial to say, but feel smart saying sth. Also, they have no idea how bad normal psych study samples are.

2

1

Replying to

@hannahmetzler1

yeah I think it would feel less bad to me if it were a genuine question? the thing that really irks me are ppl who say they have authority, confidently informing me and everyone watching that actually my sample is worthless and I'm not doing science. happens SO much

1

Replying to

Completely agree.