Conversation

critiques i get from ppl who don't know stats: "Selection bias! Internet surveys aren't 'studies'! Just Aella's Audience!" from ppl who do know stats: "You should try [different new statistical method] instead, would account for [obscure math thing I need to go learn now]"

458

Replying to

The results are interesting and fun. But most readers make inferences about the broader population which makes selection bias is an issue. Saying it's only "Aella's audience" helps but doesn't solve it: it's some subset who choose to answer for diverse and unknown reasons.

Replying to

this would piss me off less if I saw the same critiques directed as much towards the majority of published, peer reviewed academic research that comes from surveys, which have *way* less representative samples than I do. The discrepancy in critique makes me think I'm targeted.

Replying to

and

And also, to clarify, the biggest survey I'm working on (500k people) is only around 5% my audience; the rest is from people who took the survey through tiktok and probably have never heard my name before.

Replying to

and

Do you see a lot of differences that depend on how people found the survey? I had thought that e.g. for the porn stuff you might see more porn enthusiasts given your love of shocking people on twitter.

Replying to

and

it's actually much more dependent on porn specific platforms; e.g. Twitter ppl are more similar to other normal platforms, but fetlife, a sex-specific website, is significantly different in how they answer questions.

5:10 AM · Feb 1, 2023

289

Views

Replying to

and

Huh that makes sense, Fetlife in my experience is a very… selected group. I guess I had in mind a sort of millennial zoomer divide between twitter and tictoc. Next technique to learn is traveling back in time to distinguish age and cohort effects.

Replying to

and

yeah my twitter ppl are older than tiktok, but the gap isn't huge, it's like 5-6 years