The boot ROM typically gets out of your way pretty quickly, though. At worst it means you have to deal with some firmware-signing nonsense before chaining into a Linux kernel (or U-Boot); it isn't active in a running system.
I should add that the code we saw may not have been the code that was actually run. It had reset vectors and whatnot, but that's no guarantee there were no hidden ROMs that ran code before the code we reviewed got run. And it's certainly no guarantee there are no hidden hardware-level state machines that unlock . . . things. Things like ignoring X bits in pages, or being able to do some low-bandwidth computation with code embedded at the stenographic level.