The wonderful world of nutritional epidemiology Two studies, months apart. Both looked at egg consumption. Completely opposite results What's happening here?pic.twitter.com/TdIfwaGOK5
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
Let's kick off with a question: are studies with bigger sample sizes MORE or LESS generalizable?
This is important because it's a discussion I have professionally ALL THE TIME It's also something that comes up on twitter constantly
The question really comes down to whether increasing the sample size of a study reduces the biases in your data collection Let me explainpic.twitter.com/NZHBBEN1wu
An example from my own work in diabetes is looking at prevalence. We want to know how many people have diabetes, but it's very hard to test enough people in a rigorous way to be sure of our estimates
Instead, we look at a sample of people who have been tested. Let's say we take every person who has attended one of 10 GP clinics, and look at their diabetes test results That's a reasonable number of results, say 50,000
So we've got data from a group of people who have attended some GPs, and we get a figure about how many of them have diabetes Is this figure generalizable to the population (i.e. can we use it to estimate the proportion of people who actually have diabetes?)
What would happen if we then added another 100 GP clinics, or 500,000 patients, to the sample? Would it be more or less generalizable?
Now this is a lot easier than that first question. Collecting data from people who visit their GP clinics introduces obvious biases into the equation
They are more likely to be sick, are usually older, often richer than the general population Etc etc etc GETTING MORE RESULTS DOESN'T NECESSARILY REDUCE THE IMPACT OF THESE BIASES
To put it another way, our bigger sample is just as problematic as our smaller sample in key ways We've got more people, which makes the estimate more precise, but it doesn't make it more generalizable to the population
In other words, we have a very good figure for how many people have diabetes in the group of people who visit their GP clinics, but NOT for the general population because of the bias in our sample!pic.twitter.com/ccbNw9ez8C
This brings us back to the original question The answer? Bigger sample size makes NO DIFFERENCE to generalizability. It's the QUALITY not QUANTITY of the sample (mostly)https://twitter.com/GidMK/status/1110729194680381440 …
Note: it ~can~ make a difference in smaller sample sizes. We're talking here about the difference between 1,000 and 100,000, not 10 and 100
So let's go back to our original two nutritional epidemiology studies. Both were huge (n=50,000 and 500,000) BUT they had very different populationspic.twitter.com/XP7HlGlHrc
The recent study looked at a group of people living in North America. The older study was on adults living in China These are very different
We can't expect the results of Chinese people, even if the sample size is HUGE, to directly generalize to people living in the USA
These two regions of the world have vastly different food intakes, very different cultures, and this means that egg eating has different meanings and connotations in both placespic.twitter.com/kXSP5emQIC
This was actually central to the hypothesis of the Chinese study. The researchers argued that Chinese eating habits were different, and so the egg consumption should be studied to see if it was beneficial despite previous studies in other populations showing harmpic.twitter.com/Weto2IJUv1
In fact, while the two studies APPEAR contradictory, it's likely that this is, in part, context-based Chinese people who eat more eggs are healthier Americans who eat more eggs are less healthy BOTH OF THESE THINGS CAN BE TRUEpic.twitter.com/x5x57XLrkG
TL:DR - two studies appear contradictory (eggs good or bad?) Realistically, it's likely just the differences between the settings for the studies Bottom line, bigger studies are not always better it's more complicated than that
P.s. the REASON that egg eating is associated with good or bad health is, in my opinion, more complex than either of these studies suggests
My personal argument is that it's ~almost certain~ that both of these studies are using egg eating as a marker for some sort of social influence that is really causing the heart disease
So people in the US who eat more eggs are probably less well off in many ways, with the inverse being true in China
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.