Conversation

Aella
@Aella_Girl
critiques i get from ppl who don't know stats: "Selection bias! Internet surveys aren't 'studies'! Just Aella's Audience!" from ppl who do know stats: "You should try [different new statistical method] instead, would account for [obscure math thing I need to go learn now]"
35
458
Erik Brynjolfsson
@erikbryn
Replying to
@Aella_Girl
The results are interesting and fun. But most readers make inferences about the broader population which makes selection bias is an issue. Saying it's only "Aella's audience" helps but doesn't solve it: it's some subset who choose to answer for diverse and unknown reasons.
4
28
Aella
@Aella_Girl
Replying to
@erikbryn
this would piss me off less if I saw the same critiques directed as much towards the majority of published, peer reviewed academic research that comes from surveys, which have *way* less representative samples than I do. The discrepancy in critique makes me think I'm targeted.
21
76
Keiran of woolly mammoths
@drenerbas
Replying to
@Aella_Girl
and
@erikbryn
How much do you know about your audience? As I’m sure you’re aware academic studies often go out of their way to characterise study populations, so they can do fancy stats things. Do you have more info than “follows aella”?
1
4
Aella
@Aella_Girl
Replying to
@drenerbas
and
@erikbryn
Yes! I both have been studying my own audience for many years (to the degree I am very good at predicting how to split questions in twitter polls for closer to 50-50 results), and I also include really comprehensive demographic questions in many of my surveys.
3
2
Aella
@Aella_Girl
Replying to
@Aella_Girl
@drenerbas
and
@erikbryn
I still am not very good at accounting for this in my data beyond "Here's the results for x subpopulation my data and here's the results for y", this is one of the things I'm trying to learn. I figured out how to do a linear regression but I need someone else to doublecheck it
2