Here is the same dynamic RNN implemented in 4 different frameworks (TensorFlow/Keras, MXNet/Gluon, Chainer, PyTorch). Can you tell which is which?pic.twitter.com/nsfuTULlKS
Show this thread
-
Define-by-run is supported by all 4 APIs above. But unless you're doing advanced NLP research, you likely don't need it, and the drawbacks are significant compared to compiled graphs (performance, etc). "Dynamic is better than static" is just an empty talking point.
The smarter thing to do is to blend imperative and symbolic paradigms like tf.Keras and Gluon do. You retain the core advantages of compiled graphs in most situations, and you still have access to the flexibility of dynamic graphs when you need it.
-
-
It should be noted that chainer also supports graph and sub-graph caching for performance optimisation purposes.
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
And pytorch has a jit compiler as of 1.0
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
I'd add to this that dynamic "imperative" style execution is easier to debug. When you can trivially change what mode you're in (as in e.g., Mxnet Gluon) it's really convenient to debug in imperative, and then flip the "static" switch for regular use (.hyrbidize() in Gluon)
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.