I wonder when this "databases are nearly always bottlenecked on IO" perception finally is going to die. I think it's been false for > 50% of instances for at least 15 years. And it's just plainly wrong when we can have small-ish servers with >16 internal NVMe drives.
Kinda curious now whether the crc32c instruction can benefit from pipelining to make it worthwhile to compute checksums for several blocks at once.
-
-
TIL that CRC instructions are now a thing once again. This was one of the reasons that the VAX architecture was ridiculed (behind only the instruction that evaluated polynomials for you using Horner's Rule). This is what led to the RISC revolution.https://twitter.com/AndresFreundTec/status/1232026184327712768 …
-
Well, it's like an order of magnitude faster than doing it in software, even with very optimized software implementations. And given the widespread use in various protocols, we'll not fully replace it with checksums that are faster on modern CPUs anytime soon.
- 4 more replies
New conversation -
-
-
Agner lists a latency of three for r/r on skylake, and a reciprocal throughput of 1. Most database are probably using r/m, and there's multiple ports for loads. So I assume it'd help. PG only uses crc for the WAL however, and FNV1a for data. Data already uses SIMD for speed.
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.