Not necessarily. A great example was for a proteomics data format. Thermo Scientific used SQLite for their Proteome Discoverer file format, which included both the spectral data (for mass spectrometry) and the metadata. /1
Conversation
Major competitors include MIME (yes, MIME), which aimed to be emailed across various vendors (from Matrix, for MASCOT). Another competitor included plain text files, and open-source XML files. However, the plain text files were gigabytes and painfully slow. So, SQLite or HDF5.
1
Would you rather people roll their own database format, or use a single tool that aims to be used for that purpose? And, considering the number of bugs I've encountered with HDF5, I'll gladly sign onto the SQLite train.
This Tweet was deleted by the Tweet author. Learn more
For the explicit use-cases I'm listing though, that is an expectation. People post proteomics files online for public use, so randomly uploading and downloading files for people to run on a local application is an expected use, much like it is with Word documents.
1
This Tweet was deleted by the Tweet author. Learn more
Sharing image editing documents / word processing documents, etc. is irresponsible? What about media files? It's certainly a design objective of SQLite to handle untrusted files properly and they put far more effort into that than most alternative file format implementations.
2
1
They are also far more successful at avoiding vulnerabilities in practice. Still, there are occasional memory corruption bugs. It doesn't make sense to blame on them rather than the tooling. I would certainly rather open an untrusted SQLite database than an MP4 file with FFmpeg.
1
1
I would expect more servers have been hacked due to exploits in ImageMagick than SQLite. Should we stop opening untrusted images on the web as well?
1
This Tweet was deleted by the Tweet author. Learn more
Except that we know how to write software that's decently secure and has these capabilities. Many people are doing it. There are implementations of formats like mp4 in memory safe languages without dynamic code exec. They're perfectly usable already today too.
Telling users not to exchange files, visit untrusted sites, etc. is not viable. If the software is exploited, that's the fault of the software. That's never actually the fault of the user. The software industry can do much better and needs to do much better to keep users safe.
1
1
When the software is actually *exploited* that's clearly 100% a software issue. Even in cases where a user does something like granting the attacker's code the ability to run, with no exploit, OSes can and should be designed to remain secure and protect data.
1
Show replies