Now, there's a lot of criticism about the media reporting and content of both of these studies. @dailyzad's blog on the recent one is here https://lesslikely.com/nutrition/eggs-cholesterol/ … and I've written about the older one herehttps://medium.com/@gidmk/eggs-wont-stop-heart-attacks-and-fish-won-t-get-you-pregnant-4067bbeda48b …
-
-
Show this thread
-
And while there may be issues with the studies, what I wanted to focus on today was something that is often missed in the discussion of these epi studies: generalizabilitypic.twitter.com/GmEKlcip6h
Show this thread -
Let's kick off with a question: are studies with bigger sample sizes MORE or LESS generalizable?
Show this thread -
This is important because it's a discussion I have professionally ALL THE TIME It's also something that comes up on twitter constantly
Show this thread -
The question really comes down to whether increasing the sample size of a study reduces the biases in your data collection Let me explainpic.twitter.com/NZHBBEN1wu
Show this thread -
An example from my own work in diabetes is looking at prevalence. We want to know how many people have diabetes, but it's very hard to test enough people in a rigorous way to be sure of our estimates
Show this thread -
Instead, we look at a sample of people who have been tested. Let's say we take every person who has attended one of 10 GP clinics, and look at their diabetes test results That's a reasonable number of results, say 50,000
Show this thread -
So we've got data from a group of people who have attended some GPs, and we get a figure about how many of them have diabetes Is this figure generalizable to the population (i.e. can we use it to estimate the proportion of people who actually have diabetes?)
Show this thread -
What would happen if we then added another 100 GP clinics, or 500,000 patients, to the sample? Would it be more or less generalizable?
Show this thread -
Now this is a lot easier than that first question. Collecting data from people who visit their GP clinics introduces obvious biases into the equation
Show this thread -
They are more likely to be sick, are usually older, often richer than the general population Etc etc etc GETTING MORE RESULTS DOESN'T NECESSARILY REDUCE THE IMPACT OF THESE BIASES
Show this thread -
To put it another way, our bigger sample is just as problematic as our smaller sample in key ways We've got more people, which makes the estimate more precise, but it doesn't make it more generalizable to the population
Show this thread -
In other words, we have a very good figure for how many people have diabetes in the group of people who visit their GP clinics, but NOT for the general population because of the bias in our sample!pic.twitter.com/ccbNw9ez8C
Show this thread -
This brings us back to the original question The answer? Bigger sample size makes NO DIFFERENCE to generalizability. It's the QUALITY not QUANTITY of the sample (mostly)https://twitter.com/GidMK/status/1110729194680381440 …
Show this thread -
Note: it ~can~ make a difference in smaller sample sizes. We're talking here about the difference between 1,000 and 100,000, not 10 and 100
Show this thread -
So let's go back to our original two nutritional epidemiology studies. Both were huge (n=50,000 and 500,000) BUT they had very different populationspic.twitter.com/XP7HlGlHrc
Show this thread -
The recent study looked at a group of people living in North America. The older study was on adults living in China These are very different
Show this thread -
We can't expect the results of Chinese people, even if the sample size is HUGE, to directly generalize to people living in the USA
Show this thread -
These two regions of the world have vastly different food intakes, very different cultures, and this means that egg eating has different meanings and connotations in both placespic.twitter.com/kXSP5emQIC
Show this thread -
This was actually central to the hypothesis of the Chinese study. The researchers argued that Chinese eating habits were different, and so the egg consumption should be studied to see if it was beneficial despite previous studies in other populations showing harmpic.twitter.com/Weto2IJUv1
Show this thread -
In fact, while the two studies APPEAR contradictory, it's likely that this is, in part, context-based Chinese people who eat more eggs are healthier Americans who eat more eggs are less healthy BOTH OF THESE THINGS CAN BE TRUEpic.twitter.com/x5x57XLrkG
Show this thread -
TL:DR - two studies appear contradictory (eggs good or bad?) Realistically, it's likely just the differences between the settings for the studies Bottom line, bigger studies are not always better it's more complicated than that
Show this thread -
P.s. the REASON that egg eating is associated with good or bad health is, in my opinion, more complex than either of these studies suggests
Show this thread -
My personal argument is that it's ~almost certain~ that both of these studies are using egg eating as a marker for some sort of social influence that is really causing the heart disease
Show this thread -
So people in the US who eat more eggs are probably less well off in many ways, with the inverse being true in China
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.