The fact that evaluating ∇f(x) is as fast as f(x) is very important and often misunderstood http://timvieira.github.io/blog/post/2016/09/25/evaluating-fx-is-as-fast-as-fx/ …https://twitter.com/gabrielpeyre/status/1167663307668373504 …
If you are interested in some other applications of this technique, this is how we do the back-propagation for shape optimization ( == parameters) of PDE-systems http://www2.eng.cam.ac.uk/~mpj1001/papers/JFM_Kung_Jun_2019.pdf …
