Hi Marvin -- Sorry, nothing I can share. Thanks,
Tim -----Original Message----- From: Marvin Häuser <mhaeu...@posteo.de> Sent: Thursday, August 26, 2021 9:07 AM To: tim.le...@insyde.com; devel@edk2.groups.io; michael.d.kin...@intel.com Cc: 'Andrew Fish' <af...@apple.com>; l...@nuviainc.com; 'Ni, Ray' <ray...@intel.com>; 'Gao, Zhichao' <zhichao....@intel.com>; 'Wang, Jian J' <jian.j.w...@intel.com>; 'Wu, Hao A' <hao.a...@intel.com>; 'Bi, Dandan' <dandan...@intel.com>; 'Dong, Eric' <eric.d...@intel.com>; 'Bret Barkelew' <bret.barke...@microsoft.com>; 'Vitaly Cheptsov' <vit9...@protonmail.com> Subject: Re: [edk2-devel] [RFC] Expose HII package list via C variables Hey Tim, Interesting, do you happen to have some tool or code that performs such edits at hand to take a look at? Seems like most modules already use variables and thus cannot be modified in such a way even right now? That's the kind of information I am looking for, thanks a lot! Best regards, Marvin On 26/08/2021 18:01, tim.le...@insyde.com wrote: > Hi Marvin -- > > I would like to add some historical perspective on this. One of the design > requirements back when HII was first introduced into the UEFI specification > after Intel's initial contribution was that of binary editability. In order > to be able to reliably find, extract and then re-insert the HII data into the > binary, it needed to be discoverable and not affect the offsets of the code > and data in the binary. > > While it was possible to put some sort of signature in the read-only data > sections of the binary and leave padding for possible future edits, it was > felt that using a PE/COFF section similar to the resource sections was > better. Resource sections are commonly in use for PE/COFF files (in Windows) > and the similar idea existed in the then-extant Mac binary format (resource > fork?). > > Thanks, > > Tim Lewis > CTO, Insyde Software > www.insyde.com > > -----Original Message----- > From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Michael > D Kinney > Sent: Thursday, August 26, 2021 7:37 AM > To: devel@edk2.groups.io; mhaeu...@posteo.de; Kinney, Michael D > <michael.d.kin...@intel.com> > Cc: Andrew Fish <af...@apple.com>; l...@nuviainc.com; Ni, Ray > <ray...@intel.com>; Gao, Zhichao <zhichao....@intel.com>; Wang, Jian J > <jian.j.w...@intel.com>; Wu, Hao A <hao.a...@intel.com>; Bi, Dandan > <dandan...@intel.com>; Dong, Eric <eric.d...@intel.com>; Bret Barkelew > <bret.barke...@microsoft.com>; Vitaly Cheptsov > <vit9...@protonmail.com> > Subject: Re: [edk2-devel] [RFC] Expose HII package list via C > variables > > Marvin, > > One constraint in this discussion is that the HII content in a PE/COFF > resource section is defined in the UEFI Specification, Which means UEFI Apps > and UEFI Drivers from media and option ROMs that are not part of the system > FW image are allowed to use this feature, This means the system FW PE/COFF > loader must support loading HII content from this PE/COFF resource section to > be UEFI conformant. So we cannot remove this feature from the PE/COFF loader > without changes to the UEFI Specification. Even if changes to the UEFI > Specification we made, we would have to continue to support this feature for > backward compatibility with existing UEFI Apps/Drivers that may be using this > feature. > > Thanks, > > Mike > >> -----Original Message----- >> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of Marvin >> Häuser >> Sent: Thursday, August 26, 2021 1:51 AM >> To: devel@edk2.groups.io; Kinney, Michael D >> <michael.d.kin...@intel.com> >> Cc: devel@edk2.groups.io; Andrew Fish <af...@apple.com>; >> l...@nuviainc.com; Ni, Ray <ray...@intel.com>; Gao, Zhichao >> <zhichao....@intel.com>; Wang, Jian J <jian.j.w...@intel.com>; Wu, >> Hao A <hao.a...@intel.com>; Bi, Dandan <dandan...@intel.com>; Dong, >> Eric <eric.d...@intel.com>; Bret Barkelew >> <bret.barke...@microsoft.com>; Vitaly Cheptsov >> <vit9...@protonmail.com> >> Subject: Re: [edk2-devel] [RFC] Expose HII package list via C >> variables >> >> Hey Mike, >> >> Thanks for your reply! >> >> Well, this switch is not well-documented. Looking now at both the >> generation code and the emitted code, it does not generate a package >> list like my code does, but separate data variables (strings and >> images) that cannot easily be passed to HiiDatabase as-is. However >> apparently there are drivers that use this functionality successfully >> by composing the package list at runtime [1]. >> >> Looking with this information now at the pattern of using HII C >> variables (which I did not know existed before) vs the PE/COFF HII >> section, most that use latter are Shell applications, which I guess >> means the section has actually been introduced to resolve D.? There >> are exceptions such as LogoDxe [2], which use the PE/COFF section while D. >> is not a problem, hence I got confused, sorry. I think these modules >> should be updated in any case. Do you agree? >> >> So, for modules that use C variables already, my patch would save >> some runtime generation code and dynamic memory allocation for the >> HII package list. This was not my goal (as I said, I didn't realise >> HII C variables already were a thing in the first place), but the >> changes are small enough that it might be worth considering anyway, in my >> opinion. >> If a HII package list is generated for both Shell and non-Shell apps, >> this also means code paths can be unified. For example, there could >> be a library class with constructor and destructor to add/remove >> packages from the HII database for all modules that use such, Shell >> or not. For BaseTools it means that there is no real need for >> separate Python and C paths as ideally they just generate the exact same >> data. >> >> Now to D., the only usage for this seems to be that Shell can locate >> the help text in the executable without executing it, yet it is fully >> loaded anyway [3]. To be honest, I find it hard to justify loading an >> executable (PE/COFF loading, memory permission application, the full >> process) to retrieve a help text and then unloading it again, >> especially with the HII code being on a core dispatcher level. 1. to >> 7. still hold true in my opinion. Was there any discussion I could >> read through why Shell apps cannot simply support a "--help" or "-?" >> command and output the string themselves? Pushing the burden to the >> Shell apps does preserve the "drawback" that a full loading is >> required (which honestly does not matter for a debugging application >> like Shell), however it does relieve the burden of PE/COFF HII >> parsing from the core dispatcher (which matters a lot in my opinion >> to keep the core simple). It would simply be a normal Shell app >> execution as any other however. If someone wants to avoid the PE/COFF >> burden altogether, they can still provide a .man file. >> >> As for my points 6. and 7., maybe I should provide some context. Due >> to many issues with TE files, platforms started abandoning them and >> returned to PE/COFF Images. I think a big reason for this is that TE >> is not really a sound and complete format, but a stripped version of >> PE/COFF with none of the necessary fixups applied. I'm currently >> sketching a possible alternative [4], and I would really like to not >> having to specify a HII section type, while still preserving >> compatibility with all of the UEFI Image types and use-cases [4]. >> >> Thanks again! >> >> Best regards, >> Marvin >> >> >> [1] >> https://github.com/tianocore/edk2/blob/7b4a99be8a39c12d3a7fc4b8db9f0e >> a b4ac688d5/MdeModulePkg/Application/BootManagerMenuAp >> p/BootManagerMenu.c#L929-L934 >> >> [2] >> https://github.com/tianocore/edk2/blob/7b4a99be8a39c12d3a7fc4b8db9f0e >> a >> b4ac688d5/MdeModulePkg/Logo/LogoDxe.inf#L23 >> >> [3] >> https://github.com/tianocore/edk2/blob/7b4a99be8a39c12d3a7fc4b8db9f0eab4ac688d5/ShellPkg/Application/Shell/ShellManParser. >> c#L646-L671 >> >> [4] >> https://github.com/mhaeuser/edk2/blob/ue_poc/MdePkg/Include/IndustryS >> t >> andard/UeImage.h >> >> 26.08.2021 00:34:12 Michael D Kinney <michael.d.kin...@intel.com>: >> >>> Hi Marvin, >>> >>> I think this feature is already there and supported. >>> >>> HII can either be in a global variable or in a PE/COFF resource section. >>> The default is a global variable because HII was implemented before >>> the PE/COFF resource section feature was added to the UEFI Specification. >>> >>> There is an INF [Defines] section statement to enable the PE/COFF >>> section. See UefiHiiResource in the following link. >>> >>> https://tianocore-docs.github.io/edk2-InfSpecification/draft/3_edk_i >>> i_inf_file_format/34_[defines]_section.html#34- >> defines-section >>> How is your proposal different than this existing capability? >>> >>> Thanks, >>> >>> Mike >>> >>>> -----Original Message----- >>>> From: devel@edk2.groups.io <devel@edk2.groups.io> On Behalf Of >>>> Marvin Häuser >>>> Sent: Wednesday, August 25, 2021 2:21 PM >>>> To: devel@edk2.groups.io >>>> Cc: Andrew Fish <af...@apple.com>; l...@nuviainc.com; Kinney, >>>> Michael D <michael.d.kin...@intel.com>; Ni, Ray <ray...@intel.com>; >>>> Gao, Zhichao <zhichao....@intel.com>; Wang, Jian J >>>> <jian.j.w...@intel.com>; Wu, Hao A <hao.a...@intel.com>; Bi, Dandan >>>> <dandan...@intel.com>; Dong, Eric <eric.d...@intel.com>; Bret >>>> Barkelew <bret.barke...@microsoft.com>; Vitaly Cheptsov >>>> <vit9...@protonmail.com> >>>> Subject: [edk2-devel] [RFC] Expose HII package list via C variables >>>> >>>> Good day everyone, >>>> >>>> Currently, the HII package list is stored in a PE/COFF resource >>>> section [1]. I propose to store it in a C variable (byte array with >>>> a pointer to it and its size exposed) instead. DxeCore would have a >>>> guard to toggle the deprecated support for the automatic protocol >>>> installation. This has the following advantages: >>>> >>>> 1. Fixes BZ (incl. future toolchains): >>>> https://bugzilla.tianocore.org/show_bug.cgi?id=557 >>>> 2. Universal method across all toolchains and output file formats >>>> 3. Saves error-prone parsing work 4. Saves protocol install/locate >>>> work, the data is available right away 5. The omission of a >>>> dedicated section can save space 6. Terse file formats can support >>>> this and remain terse :) 7. Removes a dependency on the PE/COFF >>>> format specifically >>>> >>>> A *very rough* PoC diff can be found here: >>>> https://github.com/mhaeuser/edk2/compare/master...wip_hii_cvar >>>> If the feedback is positive, I will clean it up of course. OVMF >>>> boots with everything working fine. >>>> >>>> I'd explicitly like feedback on the following: >>>> A. Is this an acceptable solution to BZ 557 (Andrew?)? >>>> B. Is this an acceptable solution for the "HII workflow" (MdeModule >>>> maintainers?)? >>>> C. Is it acceptable to make support UEFI-side support for the old >>>> mechanism optional (Stewards?)? >>>> D. Can an acceptable alternative be found for the removed ShellPkg >>>> code (Shell maintainers?)? >>>> >>>> As you can see the BaseTools part also is rough, but that is more a >>>> question of "how" rather than "whether", so I'll postpone asking about it. >>>> >>>> Thanks for your time and feedback! >>>> >>>> Best regards, >>>> Marvin >>>> >>>> >>>> [1] "Once the image is loaded, LoadImage() installs >>>> EFI_HII_PACKAGE_LIST_PROTOCOL on the handle if the image contains a >>>> custom PE/COFF resource with the type 'HII'." >>>> - UEFI 2.9, 7.4, "EFI_BOOT_SERVICES.LoadImage()" >>>> >>>> >>>> >>>> >>> >>> >>> >> >> >> > > > > > > -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#79856): https://edk2.groups.io/g/devel/message/79856 Mute This Topic: https://groups.io/mt/85147044/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
