critiques i get from ppl who don't know stats: "Selection bias! Internet surveys aren't 'studies'! Just Aella's Audience!"
from ppl who do know stats:
"You should try [different new statistical method] instead, would account for [obscure math thing I need to go learn now]"
Conversation
Replying to
Critiques from ppl who are v familiar with stats are typically actionable - they're recommending I go learn a new method or technique, trying to explain why it's the appropriate move here.
Critiques from n00bs are always like "give up and stop trying, you're fundamentally flawed"
21
3
276
Replying to
The results are interesting and fun.
But most readers make inferences about the broader population which makes selection bias is an issue.
Saying it's only "Aella's audience" helps but doesn't solve it: it's some subset who choose to answer for diverse and unknown reasons.
4
28
Replying to
this would piss me off less if I saw the same critiques directed as much towards the majority of published, peer reviewed academic research that comes from surveys, which have *way* less representative samples than I do.
The discrepancy in critique makes me think I'm targeted.
21
2
76
Show replies
Replying to
I like your questionnaires. But aren't you afraid of what happens if let's say a politician with millions of followers retweets the survey? Isn't that an influence that's difficult to keep track off?
2
4
Replying to
you can keep track of it with a q asking where u found the survey. very easy
1
7
Replying to
I’ve always seen your stuff as a this is what aella’s audience thinks. Idk when you think about it in a different light, it makes it much easier to reconcile
1
3
Replying to
Both critiques are valuable in their own way, though a lot of it is just people wanting to be dismissive of you. Probably the most helpful tool for you if you really want to head-off the first group would be using something like the CASP checklist as you write about your findings
4
Replying to
100IQ people learning about sample size and selection bias has been a disaster for the internet.
4
8
99
Replying to
my main prob w your stuff is that almost all of your results indicate conditioning on a collider bc you’re trying to do causal analysis to suggest correlations in random non-causal phenomena. anyway i love survey methodology and internet quizzes so what can i say i’m stuck here
1
20