Hector Martin

But when the entire purpose of your code falls apart as soon as people *understand* it, using well-documented algorithms is a hindrance. Nobody cares if your integrity hash algorithm is cryptographically secure if it means someone can just replace the hash.

As far as I know nobody ever broke the integrity detection of the current version of The Homebrew Channel. If it had used SHA-1 or something, everyone's findcrypt plugin would've shouted immediately upon loading the binary. Want to know what hash function it uses? Fletcher-32.

Sure, *now* that you know that, you could probably just look for all instances of "65535" in the binary and eventually find it. Or you could preimage attack it trivially. But would you have ever guessed that without just finding the code?

Would the "hash" being crap have ever helped you if you were trying to defeat it? Almost certainly not. Instead, it being a trivial algorithm means it is almost impossible to find, buried where it is among all the other code.

Of course, when you're doing integrity detection, it doesn't help if you hide your algorithm well statically, if when it goes off at runtime the whole program stops and you can just trace back to what caused it. Which is why the only thing the hash failure does is set a flag.

The flag is in a debug register only accessible from the PPC (no IOS snoopery!), and all addresses (what gets hashed, the hash value it is compared to, and the debug register) are mangled so you get no xrefs in IDA/Ghidra/etc.

Then a completely *different* piece of code checks the debug register (addressed in a *different* way), and if it is set, it introduces a buffer overflow. That buffer overflow is carefully crafted to corrupt heap structures just enough.

HBC won't crash immediately, but with high likelihood will later. This was based on a real bug I had, and which took me a month to debug. With a remote gdb stub and a debug build and logging. I ended up teaching myself to read swizzled glyph textures from hex dumps.