michael_nielsen

We're moving up the stack a bit; Instead of writing explicit fully defined program we write a rough sketch "tube" of a program (tube parameterized by some \theta), and then if you have an evaluatable metric the best point in the tube gets selected via optimization.

Similar to the shift from writing assembly to writing a higher-level language* - with an order of magnitude more impact. *For most applications Software developers no longer need to understand the hardware architecture.

Gradient descent is actually a pretty exceptionally bad programmer (see e.g. section 5 here: https://arxiv.org/abs/1608.04428 ), but somehow you never hear people going around saying shit like "a SAT solver is a better programmer than you are".

I have said exactly that.

@jeremyphoward , @michael_nielsen ...sirs, is this similar to swift programming language then, or very different

#julialang is a different language and our goal is to make differentiable programming first class. The Swift TF team also has a similar goal - but these are different languages. Swift coming originally from Apple focussed on iOS, and #julialang from the numerical computing world.

Also excited at this direction, but perhaps a stretch to reiterate that gradient descent is a better programmer? The limitations of the best work in differentiable programming are still substantial.