lots of ppl seem very sure that my high-sample-size survey is bunk because the responses are anon and how do u verify?
Well, u can't verify for sure, but there's some ways to check/calibrate how accurate the responses are. 1/
Conversation
Replying to
for one, how well does the data line up with preexisting research? for example, I asked about "have u been assaulted as a child/adult"; does the correlation between these two questions roughly match the correlation in other published studies? we can check! 2/
1
62
another is i can check to see consistency of responses. i subtly included a few duplicate/similarly worded questions. we can see if ppl respond inconsistently to these, and remove their answers
1
60
i also straight up included two questions - 'is this your first time taking the survey', and 'how honest were you on this survey?' not foolproof, but some ppl who got through the consistency check answered 'no' and 'not honest,' and I removed those responses.
2
2
53
and lastly, I think the incentive for taking the survey is somewhat solid - ppl get a score at the end telling them how kinky they are. this incentive isn't perfect! But compared to incentives for lots of current research, (i'll pay u/for school), it's prob better
3
63
so far, my data seems to match up pretty well with other research done on other populations. Male/female, noncis, age differences, etc., all are exactly what I'd expect to see. if there *were* significant random/trolls on the responses they are highly numerous and sophisticated.
2
55
Replying to
They're bunk because there's a major sample bias. Not criticising just a reality of the data collection process you chose
2
5
Replying to
major sample bias does not mean bunk, this is a super common misconception
also: i can see the differences between how ppl answer questions when they come from different sources, so i can *check* the level of bias
1
20
Show replies
Replying to
I haven't taken the survey, so I don't know how much demographic info you take. But have you checked the sample distribution of demographic info against the population distribution?
For example, how closely does the age distribution reflect the general population?
1
1