Having spent a while working in embedded and learning that this is not a lesson that's been internalised: this is why you never sign any executable that can boot on shipped hardware unless you'd be ok with everyone running it on shipped hardware. You can not promise it will not leak. You can not promise all copies will be destroyed. If it needs to run on production hardware then you should have some per-device mechanism for one-off signatures, and if it doesn't then it should either be unsigned (if fusing secure boot happens late) or have the signature invalidated as the last thing that happens before the device is put in the box.
A lot of companies do not appear to understand this. A lot of devices with silicon-level secure boot can be circumvented with signed images that have just never (officially) been distributed to the public, and anyone relying on their security is actually relying on vendors never accidentally trashing a drive containing one. In this case Nintendo (or a contractor) utterly failed to destroy media in the way they were presumably supposed to, but it would have been better to have never existed in this form in the first place.
I think they _might_ have thought a little farther than this; as far as I can tell this tool was _supposed_ to only boot images with the same security checks as the actual fused state of the console, and the issue was that the section header parsing code was vulnerable to a trivial attack which allowed arbitrary execution, which of course could then bypass the lifecycle state checks.
I'd extend your thesis to "you need to audit your recovery tools with the _exact same_ level of scrutiny with which you audit your production secondary bootloader, because they're effectively the same thing," which is the same concept but not _quite_ as boneheaded as you suggest.
Recently, I see this class of exploit more commonly, too: stuff like "there's a development bootloader signed with production keys" has gone away a little, replaced with "there's a recovery bootloader with signature checking that's broken in some obvious way." Baby steps, I guess...
I don't like this advice because it seems like it's only useful to people who want to do tivoization in the first place. I hope people who try to do that keep failing at it, because "success" is bad for the rest of us.
At a social level we should know how to do this well because there are cases where it needs to be done well. Some hardware is operating in incredibly safety critical scenarios where you do want to have strong confidence that it's running the correct software[1].
Should this be shipped to consumers as a default? Fuck no. This technology needs to exist for safety, but that doesn't mean it should be used to prop up business models. Unfortunately there's no good technical mechanism to prevent technology being used in user-hostile ways, and we're left with social pressure. We should be organising around that social pressure rather than refusing to talk about the tech.
[1] and let's not even focus on the "Someone hacked it" situation - what if it accidentally shipped with an uncertified debug build? This seems implausible, but when Apple investigated the firmware they'd shipped on laptops they found that some machines had been pulled off the production line, had a debug build installed to validate something, and had then been put back on the production line without a legitimate build being installed - and if Apple can get this wrong, everyone can get this wrong
Great point, in general I find that the story for security is always hackers but the result is that far more commonly you hack yourself with manufacturing process variation.
I think Apple is time and again proof that Tivoization is highly effective, and that if we want to fight it, the fight needs to be legal, not technical, as much as that may dismay the technically inclined.
Agreed. I'm rooting for the continued failure of everyone who locks down hardware (and software) to prevent its users from modifying or fully controlling it.
Indeed, having done the whole developement/test/manufacturing workflow using hardware based secure boot , I realized likely very few people ever do it properly.
We had developer keys and production keys. Burning one-time fuses with the production key meant developer code would be rejected.
It took a high amount of discipline and a lot of work in the build process (separate developer/production builds of components and corresponding signing).
Very few people had access to the production signing mechanism and I avoided signing root enabled builds, even though such would be extremely convenient. Other teams… freely published production signed internal use developer firmware internally (to the whole company).
Sadly, nobody gets an award for doing it right, and rarely face consequences for doing it wrong.
> never sign any executable that can boot on shipped hardware unless you'd be ok with everyone running it on shipped hardware.
How about if, when the lead engineers are on holiday, you ship the first batch of production units with a root a key that’s on everyone’s laptop and has been pushed to bitbucket, and been used to sign all sorts of things for dev units? Then, when confronted with that, you say “oh right, well… can we delete it from those places and import the key to the HSM? We’ll use it as the prod key going forwards?”
I was sad when that payment terminal never made it to market, but in the end perhaps it was for the best.
This reminds me a lot of the PSP Pandora's Battery: a special factory "boot from external flash" system with exploitable vulnerabilities - on PSP, the special Pandora's Battery "JigKick" serial number 0xFFFFFFFF or the factory battery challenge/response "Baryon Sweeper" on newer consoles, followed by a rather complicated exploit in the "ipl.bin" signature checking process on the external hardware. On the Wii U, the "unstable power" battery jig followed by a simple overflow in SDBoot1.
That was super interesting! Are there any details on how/where they found the sd and memory cards? It seems like you’d have to be incredibly lucky to find something like that.
Haha, I’m not though. I’m actually a professional game developer and have worked on the Wii and wiiu. But there’s no way I can express curiosity about this without sounding like a lawyer. Have an upvote though haha.
Left two in the first image and two at the bottom in the second image, possibly top right too, contain Chinese characters. The rest is Japanese. Chinese characters cannot be typed on Japanese systems in normal means and vice versa, so, Chinese factory dumpster leak?
I think you've mistook Kanji characters(Japanese) as Chinese characters, some chars are the same even in writing order, even though not shared in the same char codespace.
Unicode Hanzi/Kanji map is a huge political mess, but writings in CJK are divergent enough that anyone fluent in one can often just tell which one a character is from. Mutual intelligibility never existed and very little content is shared across the language spheres, which lead to each ones having its own self-emergent mannerisms with regularized shapes of characters and which ones to use.
In this instance, "稱" and "號" used on some of printed labels in place of "称" and "号" are outside of current Japanese common use(though "號" wasn't uncommon until very late in 20th century) and I can tell that the system used to print those labels must have been configured for Traditional Chinese(HK/TW). As for the handwriting, it just looks Chinese to me.
As I understand it, the author has an established reputation, to the point where, when someone finds one of these, they send it to someone like the author.
Some of the best gaming time in my life has been on handheld consoles, even when the games were available on PC or TV.
I wish there was a modern platform (not just a hobbyist Raspberry Pi kit or something) in the Switch or DS form factor, that boots straight into a coding environment like the legendary Commodore 64 and other "computer-consoles" of that era, with a central app store for indie devs to publish to for free. Add in dedicated support from a game engine like Godot, and I think something like that could spark a renaissance of solo devs/buddy teams experimenting with new game ideas and stuff.
What was magical about that coding environment is that you could go to the store and pick up a computing magazine and type in a game. Then you could play it and tweak it as you wanted. I have no idea what the equivalent would be today; the cost analogue I can think of is watching Mario maker or Minecraft videos and then implementing what you learn in your own world or level.
> I wish there was a modern platform (not just a hobbyist Raspberry Pi kit or something) in the Switch or DS form factor, that boots straight into a coding environment like the legendary Commodore 64 and other "computer-consoles" of that era, with a central app store for indie devs to publish to for free.
I’m not sure if this will do what you want, but it is Linux on a DS! No active developers at the moment. They have instructions to build your own images as well as some software built for it.
I think the golden years for this were around the PSP/DS homebrew scene circa 2006. I have some really fond memories of following the latest developments in the community, experimenting with toolchains, and general learning about programming and hardware.I was kinda playing around with C before this, but the scene really sparked a lifelong interest.
Having spent a while working in embedded and learning that this is not a lesson that's been internalised: this is why you never sign any executable that can boot on shipped hardware unless you'd be ok with everyone running it on shipped hardware. You can not promise it will not leak. You can not promise all copies will be destroyed. If it needs to run on production hardware then you should have some per-device mechanism for one-off signatures, and if it doesn't then it should either be unsigned (if fusing secure boot happens late) or have the signature invalidated as the last thing that happens before the device is put in the box.
A lot of companies do not appear to understand this. A lot of devices with silicon-level secure boot can be circumvented with signed images that have just never (officially) been distributed to the public, and anyone relying on their security is actually relying on vendors never accidentally trashing a drive containing one. In this case Nintendo (or a contractor) utterly failed to destroy media in the way they were presumably supposed to, but it would have been better to have never existed in this form in the first place.
I think they _might_ have thought a little farther than this; as far as I can tell this tool was _supposed_ to only boot images with the same security checks as the actual fused state of the console, and the issue was that the section header parsing code was vulnerable to a trivial attack which allowed arbitrary execution, which of course could then bypass the lifecycle state checks.
I'd extend your thesis to "you need to audit your recovery tools with the _exact same_ level of scrutiny with which you audit your production secondary bootloader, because they're effectively the same thing," which is the same concept but not _quite_ as boneheaded as you suggest.
Recently, I see this class of exploit more commonly, too: stuff like "there's a development bootloader signed with production keys" has gone away a little, replaced with "there's a recovery bootloader with signature checking that's broken in some obvious way." Baby steps, I guess...
I don't like this advice because it seems like it's only useful to people who want to do tivoization in the first place. I hope people who try to do that keep failing at it, because "success" is bad for the rest of us.
At a social level we should know how to do this well because there are cases where it needs to be done well. Some hardware is operating in incredibly safety critical scenarios where you do want to have strong confidence that it's running the correct software[1].
Should this be shipped to consumers as a default? Fuck no. This technology needs to exist for safety, but that doesn't mean it should be used to prop up business models. Unfortunately there's no good technical mechanism to prevent technology being used in user-hostile ways, and we're left with social pressure. We should be organising around that social pressure rather than refusing to talk about the tech.
[1] and let's not even focus on the "Someone hacked it" situation - what if it accidentally shipped with an uncertified debug build? This seems implausible, but when Apple investigated the firmware they'd shipped on laptops they found that some machines had been pulled off the production line, had a debug build installed to validate something, and had then been put back on the production line without a legitimate build being installed - and if Apple can get this wrong, everyone can get this wrong
Alas, it will virtually exclusively "be shipped to consumers as a default".
Great point, in general I find that the story for security is always hackers but the result is that far more commonly you hack yourself with manufacturing process variation.
I think Apple is time and again proof that Tivoization is highly effective, and that if we want to fight it, the fight needs to be legal, not technical, as much as that may dismay the technically inclined.
Agreed. I'm rooting for the continued failure of everyone who locks down hardware (and software) to prevent its users from modifying or fully controlling it.
Indeed, having done the whole developement/test/manufacturing workflow using hardware based secure boot , I realized likely very few people ever do it properly.
We had developer keys and production keys. Burning one-time fuses with the production key meant developer code would be rejected.
It took a high amount of discipline and a lot of work in the build process (separate developer/production builds of components and corresponding signing).
Very few people had access to the production signing mechanism and I avoided signing root enabled builds, even though such would be extremely convenient. Other teams… freely published production signed internal use developer firmware internally (to the whole company).
Sadly, nobody gets an award for doing it right, and rarely face consequences for doing it wrong.
> never sign any executable that can boot on shipped hardware unless you'd be ok with everyone running it on shipped hardware.
How about if, when the lead engineers are on holiday, you ship the first batch of production units with a root a key that’s on everyone’s laptop and has been pushed to bitbucket, and been used to sign all sorts of things for dev units? Then, when confronted with that, you say “oh right, well… can we delete it from those places and import the key to the HSM? We’ll use it as the prod key going forwards?”
I was sad when that payment terminal never made it to market, but in the end perhaps it was for the best.
This reminds me a lot of the PSP Pandora's Battery: a special factory "boot from external flash" system with exploitable vulnerabilities - on PSP, the special Pandora's Battery "JigKick" serial number 0xFFFFFFFF or the factory battery challenge/response "Baryon Sweeper" on newer consoles, followed by a rather complicated exploit in the "ipl.bin" signature checking process on the external hardware. On the Wii U, the "unstable power" battery jig followed by a simple overflow in SDBoot1.
https://www.psdevwiki.com/psp/Pandora
https://github.com/khubik2/pysweeper
Oh. TIL they found the pins to trigger Manufacturing Mode in the last ~4 years on the final few 'unbrickable' PSP models... neat!
That was super interesting! Are there any details on how/where they found the sd and memory cards? It seems like you’d have to be incredibly lucky to find something like that.
Nice try, Nintendo lawyer.
Haha, I’m not though. I’m actually a professional game developer and have worked on the Wii and wiiu. But there’s no way I can express curiosity about this without sounding like a lawyer. Have an upvote though haha.
Left two in the first image and two at the bottom in the second image, possibly top right too, contain Chinese characters. The rest is Japanese. Chinese characters cannot be typed on Japanese systems in normal means and vice versa, so, Chinese factory dumpster leak?
I think you've mistook Kanji characters(Japanese) as Chinese characters, some chars are the same even in writing order, even though not shared in the same char codespace.
Unicode Hanzi/Kanji map is a huge political mess, but writings in CJK are divergent enough that anyone fluent in one can often just tell which one a character is from. Mutual intelligibility never existed and very little content is shared across the language spheres, which lead to each ones having its own self-emergent mannerisms with regularized shapes of characters and which ones to use.
In this instance, "稱" and "號" used on some of printed labels in place of "称" and "号" are outside of current Japanese common use(though "號" wasn't uncommon until very late in 20th century) and I can tell that the system used to print those labels must have been configured for Traditional Chinese(HK/TW). As for the handwriting, it just looks Chinese to me.
As I understand it, the author has an established reputation, to the point where, when someone finds one of these, they send it to someone like the author.
I've seen people exploit hardware by messing with the power supply before. I've never seen it be the intended manufacturer maintenance key.
Sort of a related tangent:
Some of the best gaming time in my life has been on handheld consoles, even when the games were available on PC or TV.
I wish there was a modern platform (not just a hobbyist Raspberry Pi kit or something) in the Switch or DS form factor, that boots straight into a coding environment like the legendary Commodore 64 and other "computer-consoles" of that era, with a central app store for indie devs to publish to for free. Add in dedicated support from a game engine like Godot, and I think something like that could spark a renaissance of solo devs/buddy teams experimenting with new game ideas and stuff.
Even if you had the machine, it is not enough.
What was magical about that coding environment is that you could go to the store and pick up a computing magazine and type in a game. Then you could play it and tweak it as you wanted. I have no idea what the equivalent would be today; the cost analogue I can think of is watching Mario maker or Minecraft videos and then implementing what you learn in your own world or level.
> I wish there was a modern platform (not just a hobbyist Raspberry Pi kit or something) in the Switch or DS form factor, that boots straight into a coding environment like the legendary Commodore 64 and other "computer-consoles" of that era, with a central app store for indie devs to publish to for free.
I’m not sure if this will do what you want, but it is Linux on a DS! No active developers at the moment. They have instructions to build your own images as well as some software built for it.
https://www.dslinux.org/
There is the commander x16 from the 8-bit guy https://www.commanderx16.com/ although it’s not in a handheld form factor.
I think the golden years for this were around the PSP/DS homebrew scene circa 2006. I have some really fond memories of following the latest developments in the community, experimenting with toolchains, and general learning about programming and hardware.I was kinda playing around with C before this, but the scene really sparked a lifelong interest.
Probably the closest thing is a Steam Deck with a custom Linux distro.
Some SBC handhelds do support Pico-8 platform apparently, worth a try:https://youtu.be/R5jZRV2D-rM
Sorry to double reply, but I wanted to make sure you also saw this: Linux (edit: and Android!) on a Switch.
https://switchroot.org/
This was an amazing read!
Mirror (site seems down) https://archive.is/92OIx