If you're using Colab and you feel like training your model on GPU is slow, switch to the TPU runtime and tune the "steps_per_execution" parameter in compile(). Seeing a 5-10x speedup is pretty common.
Show this thread
("steps per execution" sets the number of training batches to process sequentially in a single execution: increase the number until you reach full accelerator utilization, which, for a TPU, is a lot of flops)
-
-
Wouldn't it be possible to have an 'auto' / self tuning mode for this?
-
Yes, we are working on it (it will also benefit GPU users)
End of conversation
New conversation -
-
-
How to train model with tf gradienttape on tpu with keras?
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.