alright just for fun: AMA but only about math, will attempt to speed-explain stuff with as few symbols and equations as possible and see what happens
(esp happy to field questions about stuff that seems basic to you and that you feel like you should've gotten a long time ago!)
Conversation
let's start with entropy, why not !
1
1
(depends on how you're tailoring to your audience, ofc, but a good start would be why concentration/anticoncentration inequalities and entropy are intimately connected)
2
1
Replying to
oof i don't have this clear in my head yet either. but we can start with the simplest example: if you flip a jillion coins you get a binomial distribution, and large deviations for a binomial distribution is an exponential in the entropy
the large deviations implies, loosely, that the empirical distribution of a jillion coins is concentrated near "about 50% heads," with deviations from this controlled by the entropy (sorry, i'm writing this only for people who already know what large deviations are lol welp)
1
1
in particular we get a simple version of "maximum entropy": the most likely empirical distribution is the maximum-entropy distribution (50% heads) and everything else is exponentially vanishingly unlikely
1
1
Show replies