2. We start with what is known as a 2x2 contingency table in stats. Here is the table for Nov-Oct change in USDA corn yield estimates vs. Jan-Nov change in USDA estimates over 1988-2017.pic.twitter.com/K6oAcGyrao
You can add location information to your Tweets, such as your city or precise location, from the web and via third-party applications. You always have the option to delete your Tweet location history. Learn more
2. We start with what is known as a 2x2 contingency table in stats. Here is the table for Nov-Oct change in USDA corn yield estimates vs. Jan-Nov change in USDA estimates over 1988-2017.pic.twitter.com/K6oAcGyrao
3. 29 USDA crop reports 1988-2017 (missed Oct 2013). If distribution of changes was random, the numbers in each of the 2x2 cells would be 7 or 8 (no decimals as these are counts). Hopefully, it is obvious that the distribution of counts is not random.pic.twitter.com/5M6zmGKQKx
4. What you usually see in the trade is adding up the top left and bottom right cells (18) and dividing by the total count (29) to get a probability. This is 62.1% for corn. Nothing technically wrong with this but it is not really what we want to know.pic.twitter.com/NpM6sR2N6J
5. What we really want to know is a conditional probability. If Nov-Oct change is negative what is the probability that Jan-Nov will be negative? Table below computes exactly these conditional probabilities.pic.twitter.com/GG38cbCPHN
6. Now we are getting somewhere. Since 1988, given that Nov-Oct change in USDA corn yield estimate was negative there is a 70% chance that the Jan-Nov yield change is negative. So, the odds definitely favor the USDA dropping the US corn yield again in Jan.pic.twitter.com/nspvJjN418
7. Here is the same 2x2 contingency table counts for USDA soybean yield estimatespic.twitter.com/qbXZ68iAJb
8. And here is the conditional probability table for USDA soybean yield estimates. Since 1988, given that Nov-Oct change in USDA soy yield estimate was negative there is a 64% chance that the Jan-Nov yield change is negative. Odds favor USDA dropping the US soy yield in Jan.pic.twitter.com/bQrANZMuxR
9. Now, the really big question is why is there such a large degree of smoothing in changes to USDA corn and soybean yield estimates through time? Statistical forecasting theory says the changes from month-to-month should be unpredictable, or random.
10. I have been thinking about this for a long, long time. Here is the abstract to an AJAE article on the topic back in 2006. https://academic.oup.com/ajae/article/88/4/1091/78348 …pic.twitter.com/zWIoMRkJOv
11. Here is the abstract of my most recent paper (h/t on both papers to my amazing co-authors) from 2013 in the JAAE. http://ageconsearch.umn.edu/record/143639/files/jaae486.pdf …pic.twitter.com/9a7IkTyYhs
12. In short, I can't find any evidence that big crops get bigger and small crops get smaller in terms of USDA corn and soybean yield estimates. But, the USDA does tend to smooth yield estimates from month-to-month regardless of crop size. And it has been going on for decades.
13. Is the conservatism in changes in USDA corn and soybean yield estimates due to USDA processes or in the survey data they receive? I think it is some of both. Farmers probably slowly change yield estimates and USDA does not want to jerk the market around.
14. That is my two cents (well maybe ten bucks) on the topic of changes in USDA yield estimates. For some reason, just had to get that off my chest this morning.
15. I am going to add a definition of forecast smoothing courtesy of Peoples Grain @ad8871: "smoothing," the USDA spreads its forecast adjustments across multiple reporting periods instead of reporting the entire adjustment immediately
16. Also, Peoples Grain @ad8871 makes another point. When conditional probability is 70% you have a 30% change of being wrong. Always good to be reminded of that. Thanks!
As usual, great stuff in this thread from @ScottIrwinUI
Great info! You would think with all the data we have floating around in the cloud, the forecast would be more accurate. It would be nice to anonymously yield to a data base.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.