tfw I've been describing all the self therapy I've been doing using gradient descent analogies for months
Conversation
i'm really hoping you or or somebody will one day explain these things to me like i'm five because let's be real here, i'm five
1
3
Gradient descent means "If you're looking to get to the bottom of the hill, head downwards".
This doesn't reliably work, because sometimes hills have dips on them and you get stuck in the bottom of a dip without reaching the bottom of the hill, but it's usually pretty good.
5
10
as a metaphor for navigating life "down" is supposed to be "better"; so you imagine that you're traversing a landscape (the "optimization landscape") and different points in the landscape are better or worse versions of your life, represented by height
so concretely gradient descent as a life strategy theoretically would mean "figure out a small movement you can make that is as good as possible and then make that movement, then repeat"
practically that "as good as possible" is rough so mostly you settle for "pretty good"
1
3
you can get stuck in a bowl where there are no small moves you can make to go further down / make your life better; this is what people are talking about with "local minima" (or "local maxima" if they flip the analogy so that "up" is "better")
1
3
Show replies
Yeah. I normally think of life optimisation problems as gradient ascent, but then the people I know who work on actual mathematical optimisation start sharpening knives as soon as I use that pair of words.
Quote Tweet
Replying to @DRMacIver
A maximiser :o Burn the heretic! Purge the unclean!
4