Switch to upstream gnu-efi (v4.0.4)

[andreabolognani]

This is part of an attempt to bring riscv64 support into shim.

See #420 for the initial attempt and #641 for a revived one; the latter PR contains a lot of discussion, including requests from the maintainers to consolidate the work floating around in various branches into a single PR that could be looked at.

Mandatory disclaimer: I know basically nothing about the shim code base, so what I'm doing here consists in 1) taking code other people have written followed by 2) doing whatever is needed to make the compiler stop shouting at me. I am under no delusion that any of this would clear the bar for merging as is, hence the draft status. It's intended to be primarily a conversation starter. With the maintainers' guidance, hopefully we can refine it into something suitable for upstream.

I will have a second PR building on this one soon, which will take care of actually adding riscv64 support.

[andreabolognani]

Tagging @gmbr3 and @jmontleon since some of the changes come from them.

[andreabolognani]

The follow-up PR adding riscv64 support is #778.

[gmbr3]

I'll try and rework some of my commits (which obviously were purely test and look badly formatted early next week) - I should be less busy now since I've left university

[andreabolognani]

@gmbr3 that sounds fantastic, thank you in advance!

[gmbr3]

In progress
gmbr3/gnu-efi@
ncroxon/gnu-efi#82

[gmbr3]

Self reminder: Check shim's LDS

[andreabolognani]

Hey @gmbr3, I was going to ask you for updates about ncroxon/gnu-efi#82 since it had been sitting approved but unmerged for a few weeks, but I see that a couple of days ago you've tweaked it further. Hopefully it will be merged soon.

I have confirmed that, with that PR merged, we could drop the revert commit from this PR, since AsciiSPrint() would then be available in upstream gnu-efi. That would remove the most obvious hurdle to considering this PR, though of course we'd still need to do a bunch of tidying up as well.

@vathpela can you please take a look at what's here and provide some early feedback? Thanks!

[andreabolognani]

Updating now that ncroxon/gnu-efi#82 has been merged and gnu-efi 4.0.4 has been tagged.

With all that in place, I was able to undo the one remaining revert, meaning that a shim binary built from this branch will have feature parity with one built from the main branch.

The git history is still a bit of a mess and clearly unfit for merging as is, but I feel that this is the closest we've ever been to finally making this critical step towards riscv64 support!

@gmbr3 can you please go through your changes again and polish them up as needed?

@vathpela it would be great if you could take a look too. Even in its current state, it should be possible to do at least a first-pass review of the code and provide some high-level feedback that can guide further work.

Thank you both in advance!

[andreabolognani]

Rebased to match the current contents of the main branch.

[vathpela]

I don't understand what's going on with the MS_-prefixed string functions here. It seems to me that either we're getting compiled with -mabi=ms_abi or -mabi=sysv_abi, and the exposed API should be the same either way. What's the point of these wrappers?

/cc @gmbr3

[vathpela]

And please post a version with the DCI checks fixed - the main CI doesn't run without those, and that is important to have done before evaluating patches.

[andreabolognani]

Picked up a couple additional commits from @gmbr3's gnu_efi branch, plus some other minor tweaks. With these changes, both the native test cases and the cross builds should work fine. At least that's the case when trying stuff out locally, let's see what the actual CI thinks about it :)

Callum, there's still a question from @vathpela above for you to answer, and I assume you will also want to polish some of the commit messages. I hope you don't mind that I grabbed stuff from your WIP branch, I'm just trying to get things to a place where they can undergo at least some preliminary review. I'm happy to work with you to integrate things better, just let me know how you want to go about it!

[andreabolognani]

Rebased on top of main and adopted the modern CompareGuid API/ABI. The mkosi tests should pass now.

[andreabolognani]

Picked another commit off @gmbr3's gnu_efi branch, with some tweaks so that it applies. The arm cross-build now succeeds locally.

[andreabolognani]

Marking as ready to review since CI passes now.

@gmbr3 I hope that you will be able to spend some time on this. There are some questions from @vathpela up above that I am unfortunately unable to answer myself, and possibly you'll want to extend and polish the commit messages a bit.

We are incredibly close to finally getting this (and riscv64 support) merged. Let's get it over the finish line!

[gmbr3]

I don't understand what's going on with the MS_-prefixed string functions here. It seems to me that either we're getting compiled with -mabi=ms_abi or -mabi=sysv_abi, and the exposed API should be the same either way. What's the point of these wrappers?

/cc @gmbr3

gnu-efi by default compiles using SysV va_list ABI (we don't specify any -mabi), shim in some places needs MS va_list ABI.

These prefixed functions were created to allow building both in one library

The exposed API differs in the internal types va_list on SysV and char* on MS which causes a build failure