This is an issue because people get better at cognitive tests over time That means it's possible that any results of the experiment are simply differences in timing of the tests (i.e. given at the beginning or end of the total trial)
-
-
Also, if you look at the children that they excluded for "poor hydration compliance", they talk about an "a priori" defined criteria for exclusion This is something that you'd usually declare in your pre-registration. Let's check their's outpic.twitter.com/fcNm4YsOfG
Show this thread -
The pre-registration is...not great Firstly, no a priori exclusion criteria for urinary values. That's a problem, because it means that we have no way of knowing if the study authors decided to exclude these 7 children before the study or after looking at the results
pic.twitter.com/ipMm6CPCPJ
Show this thread -
But there's more! The pre-registration completely contradicts the study protocol in numerous ways
The number of participants, age brackets, intervention arm protocol, inclusion/exclusion criteria, testing schedule, and outcome measures are different!
That's worryingShow this thread -
It almost sounds like a completely different trial, but the authors and name/number are the same You can compare the pre-reg to the study here:https://clinicaltrials.gov/ct2/show/record/NCT02816450?term=NCT02816450&draw=2&rank=1 …
Show this thread -
So, after all of this, what did the study find? Hilariously, there was NO EFFECT of water intake (AL/Low/High) on any cognitive test!!!!pic.twitter.com/kC75Y16GdT
Show this thread -
In other words, the PRIMARY FINDINGS of this research project is that drinking more/less water DOESN'T IMPACT CHILDREN'S PERFORMANCE ON COGNITIVE TASKS That's the exact opposite of what the media reportedpic.twitter.com/uaa7gSlEpu
Show this thread -
What they DID find, and what was then converted into the headlines, was that if you look WITHIN GROUPS, there were some statistically significant trends, where better urine markers of hydration led to better accuracy on the testspic.twitter.com/8m5mwUGbVF
Show this thread -
I.e. if you look just at kids when they were in the AL condition (drinking as much water as they wanted), those who had more concentrated urine appeared to be slightly less accurate on some cognitive tests
Show this thread -
But remember - there was no difference between conditions In other words, perhaps dehydration affects cognition, but drinking more/less water doesn't appear to make any difference
Show this thread -
Here's the main table of results As you can see, there are really very few differences between the three groups on any test for reaction time or accuracypic.twitter.com/jdzJ4do8AH
Show this thread -
Moreover, it's very hard to tell if the statistical tests were controlled for multiple comparisons, which is concerning because many of the p-values were between 0.03-0.05pic.twitter.com/kCMktXnc3t
Show this thread -
And, reminder, this was an industry-funded trial, which makes the reporting of the apparently negative findings even more worrying
Show this thread -
Also worth noting that the study didn't really look at multitasking per se, this was based on an extrapolation from the within-group tests that looked at cognitive load!
Show this thread -
TO RECAP: - industry-funded trial - potential issues with study design - strange numbers - results NEGATIVE for primary outcome - didn't look at multitasking - somehow still reported as "water helps multitasking"
Show this thread -
P.S. I forgot to mention that I don't think you can call a trial measuring low water intake to high water intake a trial of water It's a trial of dehydrationpic.twitter.com/g6Hv4gyR0h
Show this thread -
I mean, imagine measuring cognitive ability in people who hadn't eaten in days vs those who'd eaten plenty It's a ridiculous thing to test in the first place!
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.