So, considering the current revocation list, you haven't found a single OEM who shipped devices without hardware secure boot in three years?
Conversation
We don't test devices; the device makers run the tests and certify compliance. We effectively rely on the security research community to identify cheaters. Note that there have been a few revocations. I expect there are far more that need to be revoked, but don't know which.
4
1
11
Sorry, your answer isn't clear to me. In your answer, checking SoC's secure boot is done by "the tests"? If yes, which ones?
1
Of course tests can't check if the implementation is secure, only that it exhibits the expected functional behavior. There are non-functional requirements, too. Device makers effectively self-certify that they pass the tests and meet the untestable requirements.
1
5
It's not possible to write a test suite or list of requirements to impose upon other companies to force their software and hardware to be reasonably secure. They need to care about security themselves and put resources into it, beyond just complying with bare minimum standards.
2
4
Most OEMs are pretty stupid. But in a good way!
If CTS gets automatic testing of default bootrom for Qualcomm, Mediatek, Unisoc's OTPs (including detecting SoC default keys...), they'll probably check it. Issue is that it must be done not just in CTS, but also in production
1
2
CTS runs tests via apps and adb shell within the security model of the OS. It runs on a production build of the OS with the full security model intact and is very limited in how deeply it can probe and test. It can't peak behind the curtain and enforce implementation details.
1
3
CTS is very limited in what it can test. That's why there's the VTS for testing the kernel and vendor via a special system image, rather than on the production OS. However, that's still testing functionality via public APIs, not peaking into the details of the implementation.
2
4
Google doesn't require that vendors use a specific implementation of hardware, firmware or software. In theory, they could make a test suite specific to each common SoC, etc. but it's hard to see how that would really work and it's still up to the vendor to run it and comply.
1
1
If a vendor skips the step of signing their firmware and burning the keys into the fuses, it seems highly likely that they would skip testing that it works. It's already written that they need to do this, so if they're running through the list it's something they need to check.