lots of ppl seem very sure that my high-sample-size survey is bunk because the responses are anon and how do u verify?
Well, u can't verify for sure, but there's some ways to check/calibrate how accurate the responses are. 1/
10
4
135
Conversation
for one, how well does the data line up with preexisting research? for example, I asked about "have u been assaulted as a child/adult"; does the correlation between these two questions roughly match the correlation in other published studies? we can check! 2/
1
62
another is i can check to see consistency of responses. i subtly included a few duplicate/similarly worded questions. we can see if ppl respond inconsistently to these, and remove their answers
1
60
i also straight up included two questions - 'is this your first time taking the survey', and 'how honest were you on this survey?' not foolproof, but some ppl who got through the consistency check answered 'no' and 'not honest,' and I removed those responses.
Replying to
Survey length has a *huge* impact on finish rates; my survey is also *long*; it'd be pretty effortful to fill it with fake responses en masse. I would be much more concerned about troll responses if my survey were like 15 questions.
1
56
and lastly, I think the incentive for taking the survey is somewhat solid - ppl get a score at the end telling them how kinky they are. this incentive isn't perfect! But compared to incentives for lots of current research, (i'll pay u/for school), it's prob better
3
63
so far, my data seems to match up pretty well with other research done on other populations. Male/female, noncis, age differences, etc., all are exactly what I'd expect to see. if there *were* significant random/trolls on the responses they are highly numerous and sophisticated.
2
55
Replying to
Being Honest About Dishonesty: Correlating Self-Reports and Actual Lying
Tricky study design to show that liars are relatively honest about lying in surveys: