François CholletVerified account

Implementing fully connected nets, convnets, RNNs, backprop and SGD from scratch (using pure python, numpy, or even JS) and training these models on small datasets is a great way to learn how neural nets work. Invest time to gain valuable intuition before jumping onto frameworks.https://twitter.com/dennybritz/status/961829329985400839 …

Implementing neural nets teaches you how to implement neural nets. It gives you an algorithmic understanding of how they work. It doesn't teach you what they do, or what they can or can't achieve. To learn that, you should apply them on a range of real problems (not XOR/MNIST)

Knowing the algorithm != Knowing how/why it works. My recommendation: do Kaggle competitions, and use more than neural nets. Use many different ML models. Make it visual. Plot your features.

It's perhaps clearer with a simpler example: implementing SVD teaches you how to implement SVD. Once you're done, you still don't know what it does. Using SVD on various datasets and plotting & using the results is what gives you intuition about SVD. Also, learning math

Grad students knew how to implement neural nets in C in 2000. And they didn't have good intuition about them. A high school student playing with NN frameworks in 2018 can develop stronger understanding of NNs in a matter of days -- just thanks to a better application context