I would love to learn to read very large files at a decent speed (ex: genomic files, in the tens of gbs, where I'm looking for "close" matches to some gene sequence). What things like memory alignment, threads, parallelism, mmap, SIMD, or other things be used to help? @cmuratori
-
-
Since you know the size of the file ahead of time, you should be able to partition the file into n chunks where n is the number of cores in the machine, so each core can process a completely independent section of the file, so you have no cross-thread communication.
-
Thank for answering! I am going to look all these up. Let me take the chance to thank you for your inspiring work! I look forward to Star Code Galaxy.
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.