Hector Martin

Sometimes you can't do better than obscurity (DRM, "tamper detection", and other forms of "protect the software from the user"). Sometimes there are valid reasons to do that. I've been there and done that myself.

But when the entire purpose of your code falls apart as soon as people *understand* it, using well-documented algorithms is a hindrance. Nobody cares if your integrity hash algorithm is cryptographically secure if it means someone can just replace the hash.

As far as I know nobody ever broke the integrity detection of the current version of The Homebrew Channel. If it had used SHA-1 or something, everyone's findcrypt plugin would've shouted immediately upon loading the binary. Want to know what hash function it uses? Fletcher-32.

Sure, *now* that you know that, you could probably just look for all instances of "65535" in the binary and eventually find it. Or you could preimage attack it trivially. But would you have ever guessed that without just finding the code?

Would the "hash" being crap have ever helped you if you were trying to defeat it? Almost certainly not. Instead, it being a trivial algorithm means it is almost impossible to find, buried where it is among all the other code.

Of course, when you're doing integrity detection, it doesn't help if you hide your algorithm well statically, if when it goes off at runtime the whole program stops and you can just trace back to what caused it. Which is why the only thing the hash failure does is set a flag.

The flag is in a debug register only accessible from the PPC (no IOS snoopery!), and all addresses (what gets hashed, the hash value it is compared to, and the debug register) are mangled so you get no xrefs in IDA/Ghidra/etc.

Then a completely *different* piece of code checks the debug register (addressed in a *different* way), and if it is set, it introduces a buffer overflow. That buffer overflow is carefully crafted to corrupt heap structures just enough.

HBC won't crash immediately, but with high likelihood will later. This was based on a real bug I had, and which took me a month to debug. With a remote gdb stub and a debug build and logging. I ended up teaching myself to read swizzled glyph textures from hex dumps.

So yeah, when you're doing obfuscation, you're doing it wrong if you think using Real Crypto is going to help you. For integrity checking, make it as inconspicuous as possible, so it blends in. For code obfuscation, just go nuts with bizarre DIY stuff.

(For those new to this saga: HBC has integrity protection because I had to add "reverse DRM" because scammers were *selling* it by literally using game piracy tools; the rev-DRM means you can only install it with the official installer, which includes a warning that it is free).

And also: the reason why the heap overflow was carefully tuned to make it crashy, but not *always*, was our policy that any security we add can *never* make the app completely unusable, because users have in the past put themselves in situations where (...)

(...) HBC is the only thing that works on their console, and the only way they can fix the rest of it, and I absolutely *never* want some reverse-DRM bullshit to be the final nail in the coffin of a Wii brick (even if the user got themselves in this situation another way).