If the compression is the bottleneck than parallel implementations of popular algorithms do exist (and tar can use them)
> tar itself is kinda single-thread-limited due to the need to write the data sequentially. Interesting! -- Do you by any chance know if it writes sequentially, and then compresses? Or does it do both on the fly? In the case of the former, you could parallel write using offsets
-
-
afaik `tar -z …` is just `tar … | gzip`, so sequential writing of the archive, then compression over everything fun fact: “The name is derived from (t)ape (ar)chive, as it was originally developed to write data to sequential I/O devices with no file system of their own.”
-
thinking about it: there might still be ways to make use of async file reading* to fill in buffers (and pipe it through a single end into the compression). * i’d assume this will also highly depend on the disk you’re reading from plus actually having async file i/o
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.