I wonder when this "databases are nearly always bottlenecked on IO" perception finally is going to die. I think it's been false for > 50% of instances for at least 15 years. And it's just plainly wrong when we can have small-ish servers with >16 internal NVMe drives.
-
Show this thread
-
Replying to @AndresFreundTec
Even more CPU used if DBMS does checksum and/or decompress after each storage read
1 reply 0 retweets 2 likes -
Replying to @MarkCallaghanDB
True. It's impressive though what kind of decompression bandwidth e.g. lz4 can have. It's feasible to reach >2GB/s even for some database uses (where typically the number of blocks decompressed at once is small-ish).
2 replies 0 retweets 1 like -
Replying to @AndresFreundTec @MarkCallaghanDB
Similar for checksums. Decent non-crypto hashes can reach > 4GB/s. And with hardware assist more is possible. The difficulty really is to have large enough runs to be checksummed/decompressed at once.
1 reply 0 retweets 1 like -
Replying to @AndresFreundTec @MarkCallaghanDB
Kinda curious now whether the crc32c instruction can benefit from pipelining to make it worthwhile to compute checksums for several blocks at once.
2 replies 0 retweets 0 likes -
Replying to @AndresFreundTec @MarkCallaghanDB
Jason Bucata - Tech Retweeted Andres Freund (Tech)
TIL that CRC instructions are now a thing once again. This was one of the reasons that the VAX architecture was ridiculed (behind only the instruction that evaluated polynomials for you using Horner's Rule). This is what led to the RISC revolution.https://twitter.com/AndresFreundTec/status/1232026184327712768 …
Jason Bucata - Tech added,
1 reply 1 retweet 0 likes -
Replying to @tech31842 @MarkCallaghanDB
Well, it's like an order of magnitude faster than doing it in software, even with very optimized software implementations. And given the widespread use in various protocols, we'll not fully replace it with checksums that are faster on modern CPUs anytime soon.
1 reply 0 retweets 1 like -
There's no RISC solutions to that. CRC is pretty cheap to compute in hardware, but expensive in software due to dependencies. At least PowerPC, ARM have a CRC instruction too, it's not just intel.
1 reply 0 retweets 2 likes -
Replying to @AndresFreundTec @MarkCallaghanDB
Oh I'm not (seriously) arguing that it's not useful nor a good idea... So is the feeling today that RISC in general was just an overreaction to some poor design choices? Or that RISC is a good idea with a few tactical exceptions allowed? (If you have an opinion)
2 replies 0 retweets 0 likes -
Replying to @tech31842 @MarkCallaghanDB
A bit of both, I'd guess? I mean back then there were architectures proliferating instructions like there's no tomorrow, without clear needs. But since then the separation between the decoded instructions and the internal representation also has become a lot stronger.
1 reply 0 retweets 0 likes
So supporting new instructions doesn't necessarily mean supporting it through most of the pipeline. And instructions can be microcoded (both initially, and later to reduce hardware cost if it doesn't turn out to be as important). The uops in a modern x86 CPU are a pretty RISCy.
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.