These Dataset objects are streaming data from disk and support prefetching, shuffling, and in-memory caching (for small data). They're an extremely efficient data loading solution for accelerators like TPUs
Show this thread
Tweetorial: if you have a bunch of text files or image files in subdirectories (one directory per class), you can use the Keras utilities text_dataset_from_directory and image_dataset_from_directory to generate labelled http://tf.data .Dataset objectspic.twitter.com/GpmdtOjkV2
-
-
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Tweetorial is a great name! lol
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
That was a sweetorial, thank you !
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
Tweetorial, love it!
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
-
-
What’s the best starting point if I wanted to build my own TF dataset like that but for audio, not necessarily from a directory but some metadata manifests (eg JSON)? I’d like to add TF data support in Lhotse eventually…https://github.com/lhotse-speech/lhotse …
-
You'd start by listing filenames (in Python) then you'd use TF IO ops to load them and turn them into tensors. You can check out the source code of image_dataset_from_directory to get an idea
- Show replies
New conversation -
-
-
I love this concept Tweetorial . Are they efficient as loading from http://tf.data API? Which one would be preferred in production? Please share from your experience
Thanks. Twitter will use this to make your timeline better. UndoUndo
-
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.