François CholletVerified account

The best first step towards building a complex end-to-end system is to build a basic end-to-end system -- not to build a submodule of what you think the complex system should look like

This is also how babies learn, of course: the first step towards performing an advanced interaction with a thing is to perform a much simpler (potentially very different) interaction with the thing, not to master the first part of the complex interaction

I wonder if this is because you can only get gradient with an end-to-end system - it’s hard to learn more complex behavior if you only have a submodule that doesn’t do the final task, and thus can’t get feedback on performance.