Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, Raphael Hertzog wrote: > I believe that syslinux-efi works for less cases than grub-efi Discussions on syslinux mailing lists indicate that syslinux-efi does not work with optical drives. (And i know of no bootable ISO 9660 image which would have syslinux-efi in its El Torito boot image which serves as EFI System Partition.) Have a nice day :) Thomas
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, Raphael Hertzog wrote: > Let the work continue! Are there any result ISOs published yet ? I'd like to include them in my xorriso regression tests. Have a nice day :) Thomas
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, adrian15 wrote: > * What is it a secondary bootloader? > It's what happens when you request mkisofs that your bootloader to be > boot in second place or as a second partition. I don't know how it > actually works. An ISO may contain several lures for boot firmware. If it is booted from CD/DVD/BD, then the lures are in the El Torito boot catalog. In case of debian-cd it contains two entries which point to boot images. One entry is marked as suitable for BIOS and one marked as suitable for EFI. The boot firmware then picks what it deems right and starts it, or offers it to the user for starting. If the ISO is presented on USB stick or alike, then BIOS looks for the magic number of an MBR and if so, mindlessly executes the x86 machine code that starts at byte 0 of the ISO. This code then hops onto the same boot code that is advertised by El Torito (because xorriso or isohybrid patched its block address into the MBR code). EFI looks on USB stick for an MBR partition of type 0xef or for a single partition of type 0xee. In the latter case it assumes the presence of GPT and looks for a GPT partition with type GUID 28732ac11ff8d211ba4b00a0c93ec93b. Inside the found partition it expects a FAT filesystem and particular binaries for each supported processor archictecture. E.g. \EFI\BOOT\BOOTX64.EFI The El Torito EFI boot image has the same content specification as the EFI System Partition. Therefore both boot paths can share the same data. My cheat sheet is at http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/view/head:/doc/boot_sectors.txt - > So, I guess the -eltorito-alt-boot does the magic but I'm not sure. > Maybe someone else can explain it better than me. -eltorito-alt-boot is simply a separator between the options of El Torito boot images. I assume your first boot image is announced by -b and gets the usual options -no-emul-boot -boot-load-size 4 -boot-info-table Before the announcement of the EFI boot image begins, option -eltorito-alt-boot is needed to protect your -b and its helpers from being overwritten. > So in grub-efi we just add to the xorriso options: > -eltorito-alt-boot \ > -e boot/grub/efi.img \ > -no-emul-boot \ > -isohybrid-gpt-basdat \ > -isohybrid-apm-hfsplus Here a second boot image gets announced by -e and the next three options apply to it. > And in syslinux-efi (currently) we add: > -eltorito-alt-boot \ > --efi-boot boot/efi.img \ Option --efi-boot is a shortcut for -eltorito-alt-boot -e ... -no-emul-boot -eltorito-alt-boot normally used by grub-mkrescue. The SYSLINUX community prefers -e. > -append_partition 2 0x01 \ > binary/boot/efi.img I wonder what EFI firmware would hop on an MBR partition of type 0x01. An EFI System Partition in MBR should have type 0xef to be recognized. It looks like this variant will not get GPT. That's fully compliant to UEFI 2.4. Just the MBR partition type should be 0xef. (Does "syslinux-efi" mean that you can boot from ISO via EFI and SYSLINUX stuff to an operating system ? That would be new.) --- Above options do not produce HFS+ with blessing. Option -isohybrid-apm-hfsplus causes an Apple Partition Map entry which points to the data content of boot/grub/efi.img. There are two ways known to me how to get a boot path via HFS+: Fedora Live CD as of Matthew Garrett or grub-mkrescue as of Vladimir Serbinenko. --- Fedora Live, e.g. Fedora-Live-Xfce-x86_64-rawhide-20150704.iso is the ancestor of debian-cd's layout. It contains a small HFS+ filesystem image as ISO data file /isolinux/macboot.img debian-cd does not have such an HFS+ image file. Image content and production is out of my scope. xorriso would only bond it to El Torito, MBR, GPT, and APM. I understand that APM leads some Macs to that HFS+ filesystem. Whether there are customers for GPT is unclear. The lures in El Torito and MBR are probably unused. The essential xorriso -as mkisofs options are a superset of those of debian-cd for amd64: -isohybrid-mbr /path/to/isohdpfx.bin -apm-block-size 2048 -c '/isolinux/boot.cat' -b '/isolinux/isolinux.bin' -no-emul-boot -boot-load-size 4 -boot-info-table -eltorito-alt-boot -e '/isolinux/efiboot.img' -no-emul-boot -isohybrid-gpt-basdat -isohybrid-apm-hfsplus -eltorito-alt-boot -e '/isolinux/macboot.img' -no-emul-boot -isohybrid-gpt-hfsplus -isohybrid-apm-hfsplus (Options picked out of proposal of xorriso -hfsplus on \ -indev Fedora-Live-Xfce-x86_64-rawhide-20150704.iso \ -report_el_torito as_mkisofs ) No UEFI compliant firmware should use the GPT in there, because the MBR partition table contains a partition of type other than 0x00 and 0xee. EFI firmware should rather follow the MBR partition of type 0xef if the ISO is presented on USB st
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, Michal Suchanek wrote: > -eltorito-alt-boot is not documented option of xorriso. You need to look into man xorrisofs for the options of the -as mkisofs emulation. -eltorito-alt-boot Finalize the current El Torito boot catalog entry and begin a new one. A boot image file and all its necessary options shall be specified before option -eltorito-alt-boot. All further El Torito boot options apply to the new catalog entry. Up to 32 catalog entries are possible. > So any bootloader is made primary by leaving out -eltorito-alt-boot. There is no "primary" or "secondary" on the level of boot images and loaders. (Of course you may call them this way in your project.) There are first, second, third ... El Torito boot images or MBR, GPT, APM partition entries. The sequence is just needed because not all can be first in the list. The boot firmware inspects the medium for its known lures, picks a tasty one, and follows it to the boot loader or kernel. > You can probably use only one legacy bootloader but syslinux-efi and > grub-efi use different files so it should be possible to install both. > I am not sure how selecting one or the other would work, though. You can offer several EL Torito boot images for BIOS, indeed. It will depend on your BIOS what it does in this case. There can be only one MBR, though. So you would have to hop by MBR x86 code to a standalone program which chooses among the BIOS suitable El Torito boot images. With EFI it makes few sense to offer more than one El Torito boot image or EFI System Partition. UEFI 2.4 rather prescribes to handle all alternatives inside the FAT filesystem. The firmware will start the \EFI\BOOT\BOOT*.EFI binary which it deems suitable for its processor type. This binary is quite free in its further proceedings. Have a nice day :) Thomas
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi,
For the nomenclature: "USB" alone is misleading because there are
also optical drives attachable to USB. Better distinguish the boot
media families CDROM (CD, DVD, BD) and HDD (hard disk, USB stick,
memory card, ...).
adrian15 wrote:
> Grub-pc would be the one installed to be boot but syslinux files would be
> there for Multi-USB tools to know how to understand the iso and put it into
> an USB.
You mean the capability to boot the ISO via BIOS from USB stick ?
(Known with SYSLINUX as "isohybrid".)
grub-pc is supposed to be bootable that way too. Have a look at the
result of grub-mkrescue.
When running xorriso -as mkisofs, the decisive trick of grub-mkrescue
for BIOS bootability from HDD is:
--grub2-mbr /usr/lib/grub/i386-pc/boot_hybrid.img
(It is obvious that only one bootloader can put its MBR at the
start of the ISO. So additional -isohybrid-mbr for SYSLINUX
would conflict.)
CDROM bootability via BIOS is caused by
-b boot/grub/i386-pc/eltorito.img \
-no-emul-boot -boot-load-size 4 -boot-info-table \
--grub2-boot-info
(Options learned from grub-mkrescue tests on Debian Sid.)
> --bootloaders="syslinux,grub-efi,syslinux-efi"
Here i wonder whether (or how) your syslinux-efi boots out of an
ISO 9660 filesystem. My current knowledge is that SYSLINUX cannot
do this.
> Now primary means: "First lure" and secondary means "Second lure" by your
> definition.
There are normally two lures per firmware-hardware combination.
Depending on the medium, the lures are recognized in El Torito,
or in MBR, or in partition tables.
In general we have theses dimensions
{Medium: CDROM, HDD} x
{Firmware: BIOS, EFI, ... exotic others ...} x
{Hardware: i386, amd64, ... exotic others ...}
Not all tuples chosen from these sets are valid and not all
valid tuples can be combined in one ISO filesystem.
But the 8 main combinations for PC hardware are valid and
combinable.
I would avoid ranking terms like "first" or "primary".
Job descriptions for bootloaders could rather look like
(CDROM + HDD, BIOS, i386 + amd64)
(HDD, EFI, i386)
(HDD, EFI, amd64)
Some of them can hardly be separated from each other.
E.g. (HDD, BIOS, i386) and (HDD, BIOS, amd64) have identical
technical properties.
> Given the way that both grub-efi and syslinux-efi use:
> /efi/boot/bootia32.efi
> and
> /efi/boot/bootx64.efi
> the last one to be run would overwrite the first one efi files.
You could combine bootia32.efi from the one boot loader and
bootx64.efi from the other boot loader, if you find a use case
for that stunt.
Without such a use case, i would decide for one boot loader per
firmware. Just to keep things simple.
> So... How would you go about for a:
> --bootloaders="syslinux,grub-efi,syslinux-efi"
I'd state that you don't need syslinux-efi if you have grub-efi
which is much better exercised with ISO 9660.
As said, nobody ever reported about success of SYSLINUX from
CDROM via EFI. I dimly remember that hpa, the author of SYSLINUX,
once stated on sysli...@zytor.com that the capability was missing
in SYSLINUX EFI to hop from the FAT filesystem into the ISO filesystem.
> they would overwrite their /efi/boot/boot*efi files ?
The EFI startup binaries of grub-efi and syslinux-efi would conflict
inside your FAT filesystem, indeed. There are supposed to be ways to
solve this conflict and offer the choice at boot time. For that you'd
need to have own EFI startup binaries which then hop on the programs
of one of the offered boot loaders.
But, well, what would be the use case for such a choice ?
Have a nice day :)
Thomas
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, i wrote: > http://bazaar.launchpad.net/~libburnia-team/libisofs/scdbackup/view/head:/doc/boot_sectors.txt adrian15 wrote: > I want to document the different ways of how BIOS boot, UEFI boot and > Secure Boot work and that might be helpful. Secure Boot is not covered. (I am not sure whether it belongs there. Actually i don't know enough about it.) adrian15 wrote: > Do you mean if you have: > xorriso bunch-of-options-1 -eltorito-alt-boot bunch-of-options-2 > you could just re-arrange them as: > xorriso bunch-of-options-2 -eltorito-alt-boot bunch-of-options-1 > and it would be fine? >From the view of xorriso and El Torito specs this is ok. You have to keep the modifier options (e.g. -boot-info-table) in the same -eltorito-alt-boot department as their main options (e.g. -b). The program isohybrid of SYSLINUX expects BIOS in El Torito catalog entry 1, EFI in entry 2, and HFS+ in entry 3. So with genisoimage + isohybrid, you'd have to keep the sequence. One never knows what firmware does with such changes. Normally it should accept any sequence. But tradition is: BIOS first, EFI second. > > I wonder what EFI firmware would hop on an MBR partition of type 0x01. > > An EFI System Partition in MBR should have type 0xef to be recognized. > That needs to be asked to Hertzog. I just tried to use his original work. Did you try whether it boots via EFI if no BIOS boot equipment is in the ISO ? Raphaël ? Can you shed light on this ? Does the 0x01 partition stem from the "Firmware Partition" of mini.iso ? (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776317 reports about quite unintentional boot success via that partition.) > > (Does "syslinux-efi" mean that you can boot from ISO via EFI > > and SYSLINUX stuff to an operating system ? That would be new.) > I don't know what you mean by 'SYLINUX stuff to an operating system. Afaik, SYSLINUX begins to boot from an EFI System Partition inside or after the ISO filesystem. It does not get to booting the operating system files in the ISO filesystem, though. > The EFI image needs to include the configuration file Given the reports on sysli...@zytor.com i'd expect that enough of operating system has to reside in the FAT filesystem so it is able to boot to the capability to find and read the ISO filesystem. But in practice, all EFI bootable ISOs known to me use GRUB for the job. (One may boot a kernel directly from EFI, without intermediate boot loader. Examples are rare but seem to work.) > If I'm not mistaken Hertzog removed that option because you suggested to > remove it on: > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=731709#66 > Are you requesting to put this option back ? Only if you augment the boot equipment by a third boot image that contains a small HFS+ filesystem. (See Fedora Live CD.) As long as there is no HFS+, the option -isohybrid-apm-hfsplus is inappropriate but seems to do no harm. (See Debian amd64 netinst.) > > There are two ways known to me how to get a boot path via HFS+: > The debian-cd method is one of them? Or is it not? debian-cd has no HFS+ boot image. This makes it different from Fedora Live CD. The xorriso run could be easily adapted to include a HFS+ image. But i do not know how Matthew Garrett produced this image. One would have to investigate its content and find some Mac filesystem tool to produce such content. > The current debian-cd ( debian-8.2.0-amd64-i386-netinst.iso ) xorriso > options are: [well known from file /.disk/mkisofs in the ISO] > where you can see they use: > -isohybrid-apm-hfsplus > So... this is not useful for creating an HFS+ image file? No. It just announces the EFI boot image in an Apple Partition Map. This might be of use for GRUB2 if no other partition table points to that image. But debian-cd has an MBR partition 0xef and a GPT partition which point to that EFI boot image. (The GPT is to be ignored by all sane EFI implementations.) > What does this option do instead then? It is surplus, currently. > Maybe I'm just repeating > what you are going to saying later in this email about how Grub2 handles > boot in Mac systems. Regreattably i can only forward the rumor that Some (TM) Macs do not boot without HFS+ and its blessed files. They are in the time window between Apple Inc. giving up PowerPC and Apple Inc. adopting EFI. > > (Options picked out of proposal of > > xorriso -hfsplus on \ > > -indev Fedora-Live-Xfce-x86_64-rawhide-20150704.iso \ > > -report_el_torito as_mkisofs > Interesting. I did not know that you could request what an ISO > 'layout/configuration' was. This is a xorriso feature which is present in the Sid package xorriso-1.4.2 but not in xorriso-1.3.2 as of Debian 8. You can request a listing of the boot equipment xorriso -hfsplus on -indev ...iso \ -report_el_torito plain -report_system_area plain and you can request option proposals by above -report_el_torito as_mkisofs > A) Are you complaining ab
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, adrian15 wrote: > It seems you are proposing to add like three tags: Medium?, Firmware?, > Architecture? but I don't get how that would transform into options that the > final user can use. For now i only state that this is a model by which one can describe the purpose of boot loaders in an ISO filesystem. > And, well, how would you go into enabling or not a given bootloader given a > Medium?, Firmware?, Architecture? combination. I am pondering a better language for describing the boot equipment to xorriso. It would have to be less tricky than the zoo of inherited and self-invented mkisofs options. Not easy. > I'm not sure I understand why we need this. Mainly take care not to create a terminology which contradicts the correct model. (As far as it is identified yet.) > Your proposal begins to make sense although I'm not 100% convinced ;). I am not convinced either. The model matches the facts, so far. But its straightforward implementation is not yet clear to me. There is a fourth dimension to be expressed: Bootloader. Then there is the dimension of ISO filesystem objects. A user wish would contain at least ISO-Object, Bootloader, Medium, Firmware, Architecture E.g. Appended Partition, with GRUB2 content, for CDROM and HDD, via EFI, on i386 ISO Data File, with SYSLINUX content, for HDD, via BIOS, on i386 and amd64 I am not sure whether this list of more or less combinable dimensions is complete yet. Further one will want to express whether the gaps between partitions should be filled, which kinds of partition tables shall emerge, ... These properties are global to the ISO, not specific to a single boot image. Some are combinable, some are mutally exclusive. (Maybe it is time to break out an UML editor.) > > [SYSLINUX] > > Did you try whether it boots via EFI if no BIOS boot equipment > > is in the ISO ? > Well, I tried: kvm -bios /usr/share/ovmf/OVMF.fd -boot d -cdrom file.iso . More we cannot expect. :) The fact that there is a minimal operating system in the EFI System Partition explains why it works. > > debian-cd has an MBR partition 0xef and a GPT > > partition which point to that EFI boot image. (The GPT is to be > > ignored by all sane EFI implementations.) > Ok. Did you mean "NOT to be ignored"? No. The GPT is surplus, according to UEFI 2.4 specs. Either you have an MBR partition 0xef or you have GPT. Not both. But Matthew Garrett had reason to produce it. Re-reading his blog http://mjg59.dreamwidth.org/11285.html i get the suspicion, that he first tried GPT alone, then added MBR partition 0xef, and so invalidated his original GPT approach. > Maybe that's why the syslinux-efi does not boot. I could give it a try > changing the partition type and see what happens. Rather not. The presence of MBR partition 0xef should cause EFI to simply not look for GPT partitions. > > I recently tested on my Sid VM with various combinations of grub-pc, > > grub-efi-ia32-bin, and grub-efi-amd64-bin. All together, pair-wise, alone. > > They all booted with qemu via its default BIOS and via OVMF (as EFI). > Did you try to boot them either as a cdrom or as a hard disk? I think that > EFI + hard disk was the one failed for me. Hopefully i tested all reasonable combinations. (But i begin to doubt that i tried an i386 VM. Will have to check.) > That's interesting. So we have Fedora, debian-cd, grub-mkimage ways of > dealing with UEFI. Probably Ubuntu has also its own way of handling it too. Fedora, Debian, Ubuntu use the Matthew Garrett layout. Debian and Ubuntu omit the HFS+ aspect of that layout, though. OpenSuSE is different. They have a mountable ISO partition after the EFI System Partition. For that stunt they cut off the head of a first ISO and prepend it to a second, reproducibly built ISO, which lacks the EFI partition data file. (In american length units, Frankenstein castle is not very far from Nuremberg.) grub-mkrescue layout is rarely used for distro ISOs. I blame this on the existing SYSLINUX based production software for distro ISOs. In principle, a pure GRUB2 or a pure SYSLINUX equipment appears to be the more reasonable choice. > So you don't like the primary or secondary terms. > How does xorriso name them? They have no partitcular name in the docs. Joerg Schilling obviously considered them "alternative" when he invented option name -eltorito-alt-boot. We should not put too much weight on that options naming tradition. It grew slowly and in part cluelessly. (Joerg's and mine alike.) > Maybe we can just name them as: > "Main bootloader" > and > "Alternate bootloader". The model does not imply any ranking. "First", "Second", "Third" could be justified, because there are lists in El Torito and partition tables where the boot entries have to line up in sequence. For my taste, "Main" or "Primary" too much implies a rank. > So that we can force a given bootloader to be used only as a "Main eltorito entry" ? Currently this would be done by mentionin
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, after checking El Torito specs, i have to correct my statements about ranking of El Torito boot images. The first entry in the boot catalog not only has the title "Initial" but "Initial/Default". "4.4 Boot Entry Selection If the CD has several boot entries, a default entry which boots a election program may be provided as the Default/Initial catalog entry. This image will usually be a floppy image which loads a program that selects the proper boot image by examining the system configuration or questioning the user." Insofar, the first boot image in the catalog is something special. It is to be preferred over the other boot images with the same Platform Id. (Nevertheless, it is not obliged to really offer booting of the others of the same Platform Id.) I would plan for only one El Torito boot image per platform: 0 = 80x86 BIOS , 0xef = EFI. But if there is a use case for offering BIOS boot images from GRUB2 and ISOLINUX in the same El Torito catalog, then the sequence matters if one of them occupies the first entry in the catalog. If you put the EFI boot image first, then both BIOS boot images get equal rank. In the light of this, it seems wise to stay with the sequence that is tested since a few years: First the BIOS boot image, then the EFI boot image, no further BIOS boot images. This makes it clear to the firmware: BIOS will hop on the first entry, because it has the right Platform Id and is Default. BIOS will most probably not enforce that the first boot image is a floppy image, because there is no other BIOS boot image. (You always have to expect that the BIOS programmers read the specs with tired eyes.) EFI will not hop on the Default entry, because the Platform Id does not match. The MBR x86 BIOS code, too, would have to decide on which BIOS boot image binary to hop. If there is only one, the decision is easy. Have a nice day :) Thomas
Bug#731709: grub-efi UEFI support based on debian-cd work complete (repos)
Hi, adrian15 wrote: > I propose you to send these concerns to live-wrapper project which has just > begun and it's advertised as highly modular by Iain. New customers ? Welcome ! > I think we should focus on making UEFI to boot and not re-thinking all the > bootloader handling. I well understand that each project has its own goals and priorities. My remarks shall serve as background info. > I once asked in a Debconf why the people were so stubborn to use syslinux > instead of using grub2 [...] It would > seem that the same people that maintain syslinux happen to maintain the > kernel boot stack The kernels on an Installation-ISO or Live-ISO should not be adventurous enough to make this important. The devices on which the ISO gets presented are supposed to be oldfashioned. (A 4K disk might cause failure. But who puts an ISO on such a disk ?) As provider of xorriso i stay neutral towards the boot loaders. Michal Suchanek wrote: > I was wondering that also. The GRUB is so cool and can do everything, right? It seems to be more apt with EFI and ISO 9660. debian-cd needs a few hundred KB to hop out of FAT into ISO. SYSLINUX needs another operating system to finally get out of FAT. > I mean if you install grub on a LVM soft-RAID array Another unlikely device type for presenting ISO 9660, too our luck. > Could have saved hours of debugging by going with the stupid limited > Syslinux from the start, right? Never change a working system. I assume that this is the true reason why nearly everybody else still uses ISOLINUX for BIOS booting from ISO 9660. A well maintained iLive CD system based entirely on GRUB2 would help to find out whether this is supersticion or deep wisdom. Have a nice day :) Thomas
