Will Crichton

Time to live-tweet the probabilistic graphical models course I'm taking. Prepare for a quarter of Hot Takes on how probabilistic programming languages are the future. cc @stephtwang

I absolutely despise almost all convention and notation in probability. Every time someone revisits the basics, I'm reminded how overloaded the "P" function is. You can take the probability of an outcome, an event, a random variable equal to a value (not an outcome), ...

... a joint distribution (comma separated), a joint distribution (\cap separated). You can write P(X) to represent the probability distribution over x. You can write P(X = x) to represent the probability of a single thing happening. You can write p(..), P(..), P[..].

My goal is to learn probability as a tool of thought. What is a general process for destructuring problems with incomplete information into programs/algorithms? I've always felt like these notational mishaps have been an impediment to me learning this process.

"Ok I don't know this piece of information... so is that an event? Err no a random variable. I know it's a categorical variable, so my prior should be drawn from a categorical distribu... hmm or multinomial... Dirichlet.......?"

From lecture: converting a Bayes net into a Markov net is called "moralization" because it involved "marrying the parents of a node". Perhaps the most unnecessary use of personal principles in the naming of math that I've ever seen.

Marginalization is such a powerful concept in probability. In decision-making, I think we humans tend to infer from context a few most likely possibilities for unobserved quantities. Marginalizing, by contrast, is like simultaneously imagining every possible state of the world.

While classic strategy algorithms like minimax employ similar kinds of search, they make assumptions like "I know how to play the game, and the opponent thinks like me." Marginalization (and expectation) allow inference of optimal strategy just based on previous observations.