Extend `generate-update-metadata` to read from `/usr` (Unification of boot loader updates, phase 1)

[travier]

Extend the generate-update-metadata logic to not query the RPM db but instead pick the versions from the new paths where the grub2 & shim binaries will be installed in /usr/lib/(shim|grub)/%{version}-%{release}/EFI/... when we land https://fedoraproject.org/wiki/Changes/BootLoaderUpdatesPhase1.

See:

• https://fedoraproject.org/wiki/Changes/BootLoaderUpdatesPhase1
• https://gitlab.com/fedora/bootc/tracker/-/issues/61
• https://gitlab.com/fedora/bootc/base-images/-/merge_requests/90

Draft PRs for GRUB & shim that will use the new paths:

• https://src.fedoraproject.org/rpms/grub2/pull-request/136
• https://src.fedoraproject.org/rpms/grub2/pull-request/140
• https://src.fedoraproject.org/rpms/shim/pull-request/4

[HuijingHei]

Here are some concerns about the change of /usr/lib/(shim|grub)/%{version}-%{release}

• Can not get Epoch for the shim / grub, this might be inconsistent from before like grub2-efi-x64-1:2.12-34.fc43.x86_64

• We get version from the new path /usr/lib/(shim|grub)/%{version}-%{release}:
  shim/15.8-4 -> shim-15.8-4
grub2/2.12-34.fc43 -> grub2-2.12-34.fc43
  Before we use rpm, the result is shim-x64-15.8-4.x86_64, grub2-efi-x64-1:2.12-34.fc43.x86_64.

• Can not get buildtime of the package that included in EFI.json

In this case, for comparison, need to make them in the same format, maybe just compare the %{version}-%{release}?

For the build time, maybe can use the generate time instead?

For silverblue we install both shim-ia32 and shim-x64, in this case /usr/lib/shim would include both, right?

# cat EFI.json | jq
{
  "timestamp": "2025-03-27T10:27:15Z",
  "version": "grub2-efi-x64-1:2.12-28.fc42.x86_64,shim-x64-15.8-3.x86_64"
}

See the output

bash-5.2# rpm -q --queryformat "%{nevra},%{buildtime}" shim-x64
shim-x64-15.8-4.x86_64,1747848849 

bash-5.2# rpm -ql shim-x64
/boot/efi/EFI/BOOT/*X64.EFI
/boot/efi/EFI/BOOT/*x64.efi
/boot/efi/EFI/fedora/*x64*.efi
/boot/efi/EFI/fedora/BOOTX64.CSV
/boot/efi/EFI/fedora/shim.efi
/etc/dnf/protected.d/shim.conf
/usr/lib/bootloader/install_bootloader
/usr/lib/shim/15.8-4/EFI/BOOT/BOOTX64.EFI
/usr/lib/shim/15.8-4/EFI/BOOT/fbx64.efi
/usr/lib/shim/15.8-4/EFI/fedora/BOOTX64.CSV
/usr/lib/shim/15.8-4/EFI/fedora/mmx64.efi
/usr/lib/shim/15.8-4/EFI/fedora/shim.efi
/usr/lib/shim/15.8-4/EFI/fedora/shimx64.efi

bash-5.2# rpm -q --queryformat "%{nevra},%{buildtime}" grub2-efi-x64
grub2-efi-x64-1:2.12-34.fc43.x86_64,1747848998

bash-5.2# rpm -ql grub2-efi-x64
/boot/efi/EFI/fedora/grub.cfg
/boot/efi/EFI/fedora/grubx64.efi
/boot/grub2/fonts
/boot/grub2/fonts/unicode.pf2
/boot/grub2/grub.cfg
/boot/grub2/grubenv
/boot/loader/entries
/etc/dnf/protected.d/grub2-efi-x64.conf
/etc/grub2-efi.cfg
/etc/grub2.cfg
/usr/lib/bootloader/install_bootloader
/usr/lib/grub2/2.12-34.fc43/EFI/fedora/grubx64.efi

CC @travier

[travier]

Thanks for testing that out. Looks like we indeed can not stop relying on the RPM db for now. Maybe let's do something hybrid and query the RPM db but pick things from the /usr/lib/ paths instead.

The paths are indeed inconsistent when comparing the grub & shim packages so that should be fixed in the packages.

[travier]

Ah, there is no dist for shim as it's the same build for all releases

[travier]

So this makes installing the shim & grub binaries under versioned directories less interesting.

[HuijingHei]

Looks like we indeed can not stop relying on the RPM db for now. Maybe let's do something hybrid and query the RPM db but pick things from the /usr/lib/ paths instead.

When call generate-update-metadata, now we move dir usr/lib/ostree-boot/efi/EFI to usr/lib/bootupd/updates/, do you mean that we will move /usr/lib/(shim|grub)/%{version}-%{release}/EFI/ to usr/lib/bootupd/updates/ instead?

The paths are indeed inconsistent when comparing the grub & shim packages so that should be fixed in the packages.

How about like shim-x64/15.8-4, grub2-efi-x64/1:2.12-34.fc43? But the directories might be different on different arches, for example shim-aa64 on aarch64, and we need to update to the related dir.

[travier]

Looks like we indeed can not stop relying on the RPM db for now. Maybe let's do something hybrid and query the RPM db but pick things from the /usr/lib/ paths instead.

When call generate-update-metadata, now we move dir usr/lib/ostree-boot/efi/EFI to usr/lib/bootupd/updates/, do you mean that we will move /usr/lib/(shim|grub)/%{version}-%{release}/EFI/ to usr/lib/bootupd/updates/ instead?

I think that's probably what we should start with, yes.

The paths are indeed inconsistent when comparing the grub & shim packages so that should be fixed in the packages.

How about like shim-x64/15.8-4, grub2-efi-x64/1:2.12-34.fc43? But the directories might be different on different arches, for example shim-aa64 on aarch64, and we need to update to the related dir.

It's indeed not great to have those under different names than their subpackage. I'm also wondering if we shouldn't put them inside a single /usr/lib/efi/(shim*|grub*) directory instead as well so that we can stop looking at /boot and look at t he content of this directory instead. Otherwise we would have to look at /boot, get the packages owning files there, then look at the files in /usr/lib. Or instead have the generate-metadata command specifically list the package to take into account?

I'm not sure.

[lsandov1]

I have reworked MRs [1,2] (no scratch builds this time) based on your comments and the only piece different is new efi folder on top of /usr/lib/, where grub and shim will place their files on top

# rpm -ql grub2-efi-x64 shim-x64 | grep /usr/lib/efi
/usr/lib/efi/grub2/2.12-34.fc42/EFI/fedora/grubx64.efi
/usr/lib/efi/shim/15.8-4/EFI/BOOT/BOOTX64.EFI
/usr/lib/efi/shim/15.8-4/EFI/BOOT/fbx64.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/BOOTX64.CSV
/usr/lib/efi/shim/15.8-4/EFI/fedora/mmx64.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/shim.efi
/usr/lib/efi/shim/15.8-4/EFI/fedora/shimx64.efi

but maybe this does not solve the issue.

@travier @HuijingHei propose any directory hierarchy and I follow it, whatever if fits better for your scripting.

[1] https://src.fedoraproject.org/rpms/grub2/pull-request/170
[2] https://src.fedoraproject.org/rpms/shim/pull-request/5

[HuijingHei]

Thank you @lsandov1 for the update, will try /usr/lib/efi/(shim*|grub*).