François Chollet

In particular every single "innovation" that made Deep Learning was documented many years earlier.

There are no new ideas in neural networks, only new (and better) implementations. Any idea you can come up with was around in ~2000.

When I get excellent validation results, I double check, then triple check. Then I still don't believe it. I'm so used to false positives.

And Quickanswers.io is on track to be doing 25k unique visitors/year. Over 10k in the past 6 months.

Let's minimize some empirical risk today too

Of course the Deep Learning cargo-cultists are the same people that freak out about "evil self-aware AI taking over the world". Sad bunch.

Deep learning is following what “big data” started. Meaning everybody seems to be doing it, but not many know why they are doing it.

In other words, in a deep NN, your learning rate should be dependent on the layer's position.

To be learnable, a problem has to be stable. Therefore in a stacked ML architecture, the lower levels should evolve at slower scale.

Code that produces coherent results while having a flawed implementation can be most insidious.

The things when you implement complex NN architectures by hand, is that it's hard to be sure what you have is correct. It will work anyway.