Email or username:

Password:

Forgot your password?
Hector Martin

I just reminded myself of the extra fun shim shenanigans going on in Asahi Fedora. I've previously described the Asahi Linux boot chain as:

SecureROM -> iBoot1 -> iBoot2 -> m1n1 stage 1 -> m1n1 stage 2 -> u-boot -> GRUB - > Linux

Which is already amusing enough, but Fedora throws in another twist. To support other platforms with "interesting" secure boot requirements (cough Microsoft-controlled certificates cough), Fedora ships with shim to handle handoff to GRUB and allow users to control their UEFI secure boot keys. But it's even more fun than that, because our installs don't support UEFI variables and instead have a 1:1 mapping between EFI system partitions and OSes, relying on the default removable media boot path.

On an installed Asahi Fedora system, you get this:

EFI
EFI/BOOT
EFI/BOOT/BOOTAA64.EFI
EFI/BOOT/fbaa64.efi
EFI/BOOT/mmaa64.efi
EFI/fedora
EFI/fedora/BOOTAA64.CSV
EFI/fedora/grub.cfg
EFI/fedora/grubaa64.efi
EFI/fedora/mmaa64.efi
EFI/fedora/shim.efi
EFI/fedora/shimaa64.efi
EFI/fedora/gcdaa64.efi

Here, U-Boot runs BOOT/BOOTAA64.EFI, which is a copy of shim. shim is designed to boot grubaa64.efi from the same directory. But since it can't find it there (since it's in fedora), it goes ahead and loads fbaa64.efi instead. That's a fallback app that then goes off searching for bootable OSes, looking for CSV files in every subfolder of EFI. It finds fedora/BOOTAA64.CSV, which is a UTF-16 CSV file that points it to shimaa64.efi. That is, itself, another identical copy of shim, but this time it's booted from the right fedora directory. The fallback app tries to configure this as a boot image in the EFI variables, but since we don't support those, that fails and continues. Then this second fedora/shimaa64.efi runs and finally finds fedora/grubaa64.efi which is the GRUB core image which continues booting.

So, on Fedora Asahi, the boot chain is actually:

SecureROM -> iBoot1 -> iBoot2 -> m1n1 stage 1 -> m1n1 stage 2 -> u-boot -> shim (first copy) -> fallback -> shim (second copy) -> GRUB - > Linux

Every. Time. (Thankfully, these extra steps go fast so it doesn't materially affect boot time; the major bootloader time contributors are kernel/initramfs load time and U-Boot USB init time right now).

The shim stuff is, of course, completely useless for Asahi Linux, since there are no built-in platform keys or any of that UEFI secure boot nonsense; once we do support secure boot, the distro update handoff will be at the m1n1 stage 1/2 boundary, well before any of this stuff, and the UEFI layer might as well use a fixed distro key/cert that will be known to always work, plus the m1n1 stage 2 signing key (which is not going to use UEFI Secure boot, it will be its own simpler thing) would be set at initial install time to whatever the distro needs. But since this mechanism exists to support other platforms, we didn't want to diverge and attempt to "clean this up" further, since that just sets us up for more weird breakage in the future. Blame Microsoft for this extra mess...

But it's even sillier, because this whole UEFI secure boot mechanism isn't supported in Fedora aarch64 at all (the shim isn't actually signed, and GRUB/fallback are signed with a test certificate), so this whole mechanism is actually completely useless on all aarch64 platforms, it only gets built and signed properly on x86, and aarch64 just inherits it 🙃​

In the early Fedora Asahi days I noticed our Kiwi build stuff was dumping the GRUB core image into EFI/BOOT too, which bypassed this fallback mechanism... but also meant that the GRUB core image didn't get updated when GRUB got updated, which is a ticking time bomb. Thankfully we noticed that and got rid of it. So now there's a silly boot chain, but it should be safe for normal distro updates.

(As for the other stuff? mmaa64.efi is the MokManager that is useless to us, and shim.efi is just a copy of shim for some reason, and gcdaa64.efi is just a copy of grub for some reason. No idea why those two exist.)

18 comments
Man2Dev :idle:

@marcan I was just wondering since your fedora spin is just design for Apple hardware isn't using systemd-boot a better option? Of course if you don't want to diverged from fedora defaults to reduce bugs and, maintenance efforts thats understandable too.

Hector Martin

@Man2Dev The latter. Any divergence from Fedora has to be reconciled somehow. The goal here is that this should be Fedora. Diverging gratuitously is just creating more work for ourselves further down the line.

Daimar Stein :tux: :comunista:

@marcan @Man2Dev Upstream first always, right? Is there even any hope that this mess could be cleaned (at least on ARM versions of the distro)?

Raul

@marcan I think you described it that initial way in the Ubuntu Asahi talk, which I recently saw on Youtube.

You also mentioned there something quite interesting: that Asahi Linux is not really a distro, but works with distros. Yet Asahi Linux was a thing before the recent Fedora and Ubuntu flavors. So what was that back then based on, was it still Fedora-based, or suse, debian, arch, gentoo, sth else...?

Eric Engestrom 💙

@raulinbonn It was based on ArchLinux ARM, which is a fork of ArchLinux to add support for a couple of ARM architectures (see archlinuxarm.org if you're interested). It's maintained by a single person, who's doing a good job considering, but it's not maintained and tested enough to be relied upon, so when Fedora (which already supported aarch64 officially) integrated the Asahi bits, they became the recommended distro.

@marcan

CountZero

@raulinbonn @marcan it was Arch Linux ARM (ALARM) based. Which, if I'm not mistaken, looks like arch linux but is actually a separate, kinda poorly maintained project (that's why asahi decided to move to fedora)

Raul

@czero @1ace @viraptor @marcan Thanks a lot to all! Gee I should have remembered it was Arch. I see the inflexion point / switch was about March 2023, but I follow Asahi Lina since some time in second half of 2022, when she was still sort of with the preliminaries of the Rust driver. So I should have seen here and there on her videos that it was Arch-based.

Sobex

@raulinbonn @marcan The initial (alpha) Asahi Linux release installed Arch Linux ARM + the asahi extra packages (kernel etc).

And now the Asahi Linux release installs a Fedora with the few required packages.

(As far as I know, hopefully I just spared macron some work, and otherwise I shall stand corrected)

Jay Lanagan

@marcan and this is why on my personal Slackware install I wanted to concatenate the m1n1 boot process, because this entire boot process loses me. I can’t even begin to comprehend this madness. It’s also why I threw in the towel on it. Someone else stepped up and things will soon see the light of day at least, but I don’t ever want to manage this chaos.

Hector Martin

@jloc0 But when someone else already is managing the chaos, it's still easier to let them than to come up with your own simpler version 🙃​

Akihiko Odaki

@marcan Even worse, shim breaks VMs that carries the latest EDK2 firmware 🤦‍♂️
This is especially problematic for Arm since there is no legacy x86 BIOS and the "SystemReady" thing solely relies on UEFI.
https://bugzilla.redhat.com/show_bug.cgi?id=2113005

mlyxshi

@marcan
I don't understand the complex shim stuff. Anyway, is this the reason that uboot outputs( Failed to load '/') ?

elly
@marcan I wonder: why so many layers though?
I would expect that you can do BROM -> iB1 -> iB2 -> ST3 from storage (U-Boot).

We managed to do the same thing with ARM64 Chromebooks (although porting drivers for different platforms from Linux is a PITA). It looks something like that:

BootROM -> BL2 (Coreboot) -> BL31 (TF-A) -> Coreboot (drops execution level to EL2) -> DepthCharge -> ELF from storage (U-Boot/LinuxBoot).

I also wonder if you could get NVRAM working. U-Boot supports it in OP-TEE, but no idea if/how apple implemented it.
@marcan I wonder: why so many layers though?
I would expect that you can do BROM -> iB1 -> iB2 -> ST3 from storage (U-Boot).

We managed to do the same thing with ARM64 Chromebooks (although porting drivers for different platforms from Linux is a PITA). It looks something like that:
Demi Marie Obenour

@marcan Would it be possible to directly boot a kernel from m1n1, or even include the kernel image in m1n1? If Qubes OS ever gets Apple silicon support, I want to keep the secure boot chain as short as possible. Ideally, it would be no longer than Apple’s chain.

Demi Marie Obenour

@marcan Also, would it be possible to disable USB until the OS is running? That’s also something Qubes OS needs, as we isolate the USB stack in a VM for security. Outside of Qubes OS, it would allow USBGuard to be used.

joey

@alwayscurious yes you can do that. u-boot is embedded in the m1n1 binary currently (on the ESP), but it's not treated specially and you can just replace it with a Linux image

Neal Gompa (ニール・ゴンパ) :fedora:

@marcan `gcdaa64.efi` is a copy of GRUB with cdboot drivers built in. I'm not actually sure why they're split, maybe @vathpela knows?

Go Up