Except that published everything: whitepaper, config, source code, 100% automation to reproduce the results and a bucket with the raw data. And just unproven statements.
Stay tuned, will publish a response blog post in the coming days.
Conversation
The OnGres trials ignored published best practices for MongoDB, claiming falsely that MongoDB does not require or benefit from tuning, used an experimental driver without connection pooling, and ignored indexes. Those aren’t unproven statements, they’re in the code and report.
2
5
There’s no need for MongoDB to publish anything — the OnGres repo is itself sufficient for any tester familiar with `explain` to see the problems with it.
3
1
3
I'm sincerely interested in how you got the numbers you claim.
Responding that anyone can see for themselves - if they do an explain - doesn't cut it IMHO;
I'd like to know what you did explicitly so anyone can corroborate/falsify your results.
1
4
The article I linked explains what we did. For example, we allowed MongoDB to use indexes!
1
Corroboration/falsification does not mean "run the code handed to you and see the same numbers." It means understanding the systems under test and scrutinizing the test methodology.
3
Wow, srsly!? I'm really amazed that a company like would ever claim such a thing :o
This is the exact opposite of how a benchmark is done. You should be able to repro it and get numbers in the same ballpark. Otherwise, the benchmark (or the claims) are snake oil.
2
1
Giving you the benefit of the doubt here. Consider my hypothetical benchmark code:
echo “Postgres: 86673ms\nMongoDB: 56ms”
It’s 100% reproducible. Is it real? Of course not. How can you tell? You read the code and decided it doesn’t model an accurate test.
3
2
The OnGres benchmark code is the same, only obfuscated by setting up some databases and making some queries. Not valid. If you’re not qualified to make that evaluation of the benchmark in question, then you’re certainly not qualified to decide if a MongoDB patch is valid.
1
If a team funded by the fossil fuel industry published a report, littered with falsehoods and methodological errors, denying climate change, NASA and NOAA aren't obligated to issue full a correction of that report along with their response calling shenanigans.
3
But, A.) You chose to do so anyway, while being very light on details, B.) MongoDB, inc is not NASA, C.) NASA/NOAA has published plenty of information about climate change in general, and do not claim to be singular authorities, D.) You're the one making extraordinary claims.