I recall a nice optimization method called SPSA: successive perturbation stochastic approximation. Trick was to half-ass it in early steps, and refine it only when you get close to answer. No reason to get intermediate answers perfect.
Excellent mediocrity engineering there.
Conversation
Replying to
Correction, it was *simulatenous* perturbation stochastic approximation
2
Replying to
This tweet sent me down a 4 hour rabbit hole about optimization problems, culminating in finding a paper that answers the question "can you generate solutions to TSP with an RNN?" - hint "yes" is the answer openreview.net/pdf?id=rJY3vK9
Thanks!
1



