Tensorflow 2.0: models migration and new design https://buff.ly/2zvL9oI Great overview of upcoming TF changes. Personally I agree with the author and I'm not a huge fan of eager mode either. Is it just because I'm used to thinking in terms of a graph? Is eager really superior?
I think graphs are the correct mental model and API for deep learning -- but not graph of ops like a TF graphdef, instead, graph of layers. Recursive graphs of high-level building blocks. Which is also how deep NNs are visualized in pretty much every paper or textbook ever...
-
-
Then again, when you're actually developing a new such building block from scratch (like a custom layer), you're probably going to want to do so in an imperative style, because that's what most people are used to (Numpy, Python itself, etc). That's when eager execution comes in.
-
I understand that it’s “easier” using an imperative style, but that IMO comes at a cost. My experience is that an imperative style leads throwing stuff at the wall (e.g. notebooks) + iterate/debug. Op graphs force you to think hard about what the computation should look like.
- Show replies
New conversation -
-
-
Could part of the middle ground be something akin to a type system for layers/blocks, to both aid composition and automatically check for "correct" compositions of high level abstractions?
-
We have that, and it's a lot more advanced than static type checking. Any Keras model that compiles will run. If it's only made of built-in layers.
- Show replies
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.