ছেড়ে সামগ্রীতে যান

Tensorflow eager execution in 12 tweets

২৩ মে, ২০১৮
Write your model in Tensorflow eager mode, debug it in any Python debugger.

1/ Tensorflow eager mode in 12 tweets. You are going to love tweet #12 😁 To begin, import and enable eager mode:

৫ replies ৩০ টি পুনঃটুইট ৯৩টি পছন্দ

2/ In classic Tensorflow, this just works: y = tf.layers.dense(x, 10) It creates a 10-neuron dense layer with appropriate weights and biases. Where do these weights live? In a global somewhere. Convenient but hair-curling for a software developer !

১ reply ৩ টি পুনঃটুইট ১১টি পছন্দ

3/ In Eager mode, you know where your weights are because you have to put them somewhere yourself. My preferred pattern: a basic class. Define your layers in the constructor, line them up according to your preferred architecture in a "predict" function.

১ reply ২ টি পুনঃটুইট ৫টি পছন্দ

4/ You need a loss function, comparing what your model makes of the features against a target answer "yt"

১ reply ২ টি পুনঃটুইট ৪টি পছন্দ

5/ Tensorflow eager knows how to compute the gradient for this loss, relatively to the implicit weights and biases of your layers. "grads" is now a function of the same parameters as your loss function.

১ reply ৩ টি পুনঃটুইট ৭টি পছন্দ

6/ instantiate your model, pick an optimizer.

১ reply ২ টি পুনঃটুইট ৬টি পছন্দ

7/ And finally the training loop. You are passing your model as a parameter to grads so it is pretty obvious what weights and biases are being modified by the training.

১ reply ২ টি পুনঃটুইট ৪টি পছন্দ

8/ To pipe data into your training loop, my_next_batch() can be implemented with vanilla Python or use the .Dataset API which allows training on out of memory datasets. In eager mode, it is very natural to use:

১ reply ২ টি পুনঃটুইট ৪টি পছন্দ

9/ For power-users, there are alternative functions for computing gradients in eager mode: implicit_value_and_gradients gets the value of the loss at the same time as gradients. The code above can be made slightly more efficient:

১ reply ২ টি পুনঃটুইট ৫টি পছন্দ

10/ Also, tfe.GradientTape for differentiating any piece of code against any set of variables. Handy if you do not want to wrap your loss into a function and know the list of trainable variables explicitly, as in Keras Models where the list is in model.variables.

১ reply ২ টি পুনঃটুইট ৩টি পছন্দ
১ reply ৫ টি পুনঃটুইট ২২টি পছন্দ

12/ And now the really cool part. Here is my model paused during training in a vanilla Python debugger: the values of weights, biases, activations, you name it are now visible. Yay! Eager rocks!

১ reply ৮ টি পুনঃটুইট ৪৩টি পছন্দ