Use "mixed_float16" as the policy value on GPU, and "mixed_bfloat16" on TPU. You can also configure the dtype policy on a per-layer basis using the `dtype` layer constructor argument. Useful to keep some layers running in float32 when necessary!
-
-
Show this thread
-
In our example, we kept the data augmentation stage in float32 (created before setting the global policy) since it's meant to be run on CPU as part of the TF data pipeline.pic.twitter.com/2nRbBleVaZ
Show this thread -
Using mixed precision can improve training performance by over 3x on modern NVIDIA GPUs, and 60% on TPUs, at no loss of accuracy.
Show this thread
End of conversation
New conversation -
-
-
wait keras hosts imagenet in a bucket now? how long has this been going on?
-
You don't have access to the actual data -- you'd have to edit the data location to your own hosting.
- Show replies
New conversation -
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.