Because these problems involve schema changes (e.g. creating tables), they may not impact users frequently. YugaByte doesn't think they're relevant to the core transactional mechanism in YugaByte DB, which is why they're not discussing them when they say "Jepsen tests passed".
-
-
Show this thread
-
There are other issues that we found in our work, like slowly spawning an ever-increasing number of backend worker processes which eventually consume all resources and kill the machine. This one's still open too.https://github.com/YugaByte/yugabyte-db/issues/2075 …
Show this thread -
Crashes, unavailability, and performance problems aren't usually reported by Jepsen as "failing" results, because we're primarily concerned with checking safety, rather than liveness problems. It's hard to say how slow is too slow. We file and discuss these issues qualitatively.
Show this thread -
In more general terms, almost every database we tested with Jepsen fails, sometimes in dozens of ways, before its test suite passes. That's how we know Jepsen is *working*! The vendor headline that comes out of that process is usually "X passes Jepsen".pic.twitter.com/bJBT3MDw74
Show this thread -
Most vendors are telling the truth here: by the time we conclude our collaboration, the safety issues we found have usually been addressed, and the test suite often passes. Many vendors also follow up "passes" with a description of the issues we found, which I think is honest.
Show this thread -
But still, some vendors do misrepresent the results of our work together, and this bugs me. I expected vendors to call each other out for this sort of thing, because they're more than willing to write take-downs over interpretation of benchmarks, but so far that hasn't happened.
Show this thread -
I'm committed to giving everyone the most accurate, rigorous reporting on database correctness that I can, and I encourage vendors to do the same. Be open, honest, and nuanced in your writing. That honesty is good for users, and it builds trust in your team.
Show this thread
End of conversation
New conversation -
Loading seems to be taking a while.
Twitter may be over capacity or experiencing a momentary hiccup. Try again or visit Twitter Status for more information.