Have another gander at those numbers The means are a ~tiny~ bit different, but the quartiles are virtually identical! The medians are exactly the same
-
-
Show this thread
-
And the test that the p-value comes from is actually a Wilcoxon signed-rank test, which compares DISTRIBUTIONS, not mean values In other words, it's VERY unlikely you'd see two samples with the same medians, virtually identical quartiles, but a p of 0.022pic.twitter.com/bTLVXJ6CsF
Show this thread -
How unlikely? Well, I thought I'd give it a quick test (using Stata 15) Here's the codepic.twitter.com/rwqwm2qo3H
Show this thread -
Basically, I'm generating two variables with the correct quartiles, where all numbers are random between those two I also skewed the numbers slightly up (making before higher and after lower) because...well, you'll see
Show this thread -
The code runs for 100,000 iterations, then dumps it all into the file I convert the z-scores into p-values, and voila! 100,000 random simulated distributions and their p-values for a Wilcoxon paired samples test
Show this thread -
Straight away it doesn't look good. There are very few tests where we get a significant p-value randomly (129/100000)pic.twitter.com/3kmDbe4glI
Show this thread -
Even worse, if I only look at the simulations where the means were congruent I can't find a single simulated example where the p-value is even close to 0.022!pic.twitter.com/TLJMvrI6Z6
Show this thread -
If I rejig the code to make the before values higher still, and the after values even lower, I can get a lot more significant results but ~none~ of the means are congruentpic.twitter.com/KAnRBe75pt
Show this thread -
I can't find a single example where those mean, median, p-value, and quartiles are all true That's...not ideal Please have a look at my code and tell me if I've made a mistake!
Show this thread
End of conversation
New conversation -
-
-
This may be SPRITEable... Ping
@jamesheathers@sTeamTraen -
It probably is, but I'm just not mentally equipped to deal with it right now. I think these observations are a problem all by themselves anyway.
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.