In total, out of the 91 children who started the trial, 8 dropped out That's actually pretty good, only 9%!
-
Show this thread
-
But then, a further 7 were excluded because of 'poor hydration compliance' What does this mean? Essentially, the researchers thought they were drinking more/less water than their experimental conditionpic.twitter.com/PJXeqv0IG5
1 reply 0 retweets 1 likeShow this thread -
Now, this makes the study what's known as a per-protocol analysis Without going into the details, per-protocol analyses are KNOWN for giving spurious/misleading resultspic.twitter.com/o2G7Yissgq
1 reply 0 retweets 4 likesShow this thread -
But there's more - after excluding participants entirely, the study excluded between 5 and 21 children from each analysis in the study That's up to 28% of the remaining sample (!)pic.twitter.com/j0KfldJaV2
1 reply 0 retweets 3 likesShow this thread -
This was due to "poor cognitive data" What does that mean? Well, according to the study, it means that these kids had results that were considered 'outliers' or 'poor performance'pic.twitter.com/70Z3ZHH8rl
1 reply 0 retweets 2 likesShow this thread -
Astute readers will note that this means that apparently 8/64 (13%!) of the children had results more than 3 standard deviations from the mean in the go/no-go task That seems...unlikely
2 replies 0 retweets 3 likesShow this thread -
To give you an idea of how unlikely, I ran the mean/standard deviations through SPRITE (thanks
@sTeamTraen@jamesheathers) for 100 repetitions, and didn't get a single distribution with more than 3% of results >3 SDs from the mean So...maybe possible? VERY weird2 replies 0 retweets 5 likesShow this thread -
Replying to @GidMK @jamesheathers
Well, we don't know the mean and SD before the 19 participants were removed, but I agree that 8/73=11% of 3-SD outliers is a lot, and SPRITE is going to struggle to find solutions (e.g., with 8 "fixed" values of 901ms, in the first go/no-go line with M=600.6 and SEM=11.7->SD=100)
1 reply 0 retweets 1 like -
Replying to @sTeamTraen @jamesheathers
It seems very odd that they had so many outliers to me, surely at that point they aren't really outliers any more!
1 reply 0 retweets 0 likes -
Replying to @GidMK @jamesheathers
Once you start selecting on the dependent variable, unless it's flagrantly bad, you are going to be in trouble. A number of their include/exclude decisions look very arbitrary to me.
1 reply 0 retweets 1 like
That's definitely true. Cannot see why you'd exclude children with lower than 40% accuracy, seems like an entirely arbitrary decision 
-
-
Also worrying because of the apparent lack of randomization - very likely that you'd see worse scores in early trials
1 reply 0 retweets 1 like -
Replying to @GidMK @jamesheathers
It's almost as if the researchers had a goal other than the dispassionate acquisition of knowledge.
1 reply 0 retweets 1 like - Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.