Re: [RFC] split floppy support off biosdisk

2008-06-19 Thread Pavel Roskin
On Thu, 2008-06-19 at 14:20 +0200, Robert Millan wrote: > On Wed, Jun 18, 2008 at 10:37:03PM -0400, Pavel Roskin wrote: > > On Thu, 2008-06-19 at 04:14 +0200, Yoshinori K. Okuji wrote: > > > > > I took a look at lzma 4.32.6. All code in liblzmadec is licensed with > > > LGPL 2.1 > > > or any lat

Re: [RFC] split floppy support off biosdisk

2008-06-19 Thread Robert Millan
On Wed, Jun 18, 2008 at 10:37:03PM -0400, Pavel Roskin wrote: > On Thu, 2008-06-19 at 04:14 +0200, Yoshinori K. Okuji wrote: > > > I took a look at lzma 4.32.6. All code in liblzmadec is licensed with LGPL > > 2.1 > > or any later version. So where is the problem? > > Indeed, the "any later ver

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Pavel Roskin
On Thu, 2008-06-19 at 04:14 +0200, Yoshinori K. Okuji wrote: > I took a look at lzma 4.32.6. All code in liblzmadec is licensed with LGPL > 2.1 > or any later version. So where is the problem? Indeed, the "any later version" provision is there. It wasn't mentioned in the original discussion.

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Yoshinori K. Okuji
On Thursday 19 June 2008 01:56:52 Pavel Roskin wrote: > On Thu, 2008-06-19 at 00:15 +0200, Robert Millan wrote: > > > Which implementation of LZMA do you refer to? I think LZMA SDK is > > > available under LGPL, and most of LZMA Utils is also under LGPL. So > > > there should be no problem. > > > >

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Pavel Roskin
On Thu, 2008-06-19 at 00:15 +0200, Robert Millan wrote: > > Which implementation of LZMA do you refer to? I think LZMA SDK is available > > under LGPL, and most of LZMA Utils is also under LGPL. So there should be > > no > > problem. > > I expected that LGPL 2.1 wouldn't be compatible with GPL

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Robert Millan
On Wed, Jun 18, 2008 at 07:52:26PM +0200, Yoshinori K. Okuji wrote: > On Tuesday 17 June 2008 14:54:23 Robert Millan wrote: > > On Tue, Jun 17, 2008 at 02:33:48PM +0200, Robert Millan wrote: > > > On Tue, Jun 17, 2008 at 02:22:15PM +0200, Robert Millan wrote: > > > > A problem I see here is that LZ

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Yoshinori K. Okuji
On Tuesday 17 June 2008 17:54:27 Pavel Roskin wrote: > On Tue, 2008-06-17 at 23:02 +0800, Bean wrote: > > I'm not against other compression algorithm, but lzma seems to be the > > best, for example, for the previous c2.img: > > > > bzip2 c2.img && du -b c2.img.bz2 > > 29247 c2.img.bz2 > > > > gzi

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Yoshinori K. Okuji
On Tuesday 17 June 2008 14:54:23 Robert Millan wrote: > On Tue, Jun 17, 2008 at 02:33:48PM +0200, Robert Millan wrote: > > On Tue, Jun 17, 2008 at 02:22:15PM +0200, Robert Millan wrote: > > > A problem I see here is that LZMA is licensed under GPLv2-only, so it'd > > > probably take a rewrite of th

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Robert Millan
On Wed, Jun 18, 2008 at 01:32:57PM -0400, Isaac Dupree wrote: > if it's not too much work to maintain multiple decompression algorithms, > maybe we can have grub-install pick the weakest compression that still > shrinks the image enough to fit? Then only "complicated" setups will > need LZMA's

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Robert Millan
On Wed, Jun 18, 2008 at 02:55:44PM +0800, Bean wrote: > > You can try Robert's case: > > grub-mkimage -d . -o core.img pc ext2 lvm raid If you're into benchmarking, I'd suggest trying all permutations of "biosdisk (pc|gpt) (ext2|fat|reiserfs|xfs) lvm raid". (notice biosdisk too, I forgot to add

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Isaac Dupree
if it's not too much work to maintain multiple decompression algorithms, maybe we can have grub-install pick the weakest compression that still shrinks the image enough to fit? Then only "complicated" setups will need LZMA's memory excesses. (Might not be worth the effort though.) -Isaac _

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Javier Martín
El mié, 18-06-2008 a las 21:12 +0800, Bean escribió: > If you rerun the test using the above modules, you will notice that > besides lzma, others are close to the upper limit. Test rerun: Modules cat'ed together: kernel.img biosdisk pc ext2 fshelp lvm raid Uncompressed file size: 49644 bytes grub-m

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Bean
On Wed, Jun 18, 2008 at 9:12 PM, Bean <[EMAIL PROTECTED]> wrote: > Hi, > > We also need to consider how close it's to the upper limit, 32256, > which is the available free space in the mbr. To access partition with > lvm and raid enabled, we need to use: > > biosdisk pc ext2 fshelp lvm raid > > If

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Bean
On Wed, Jun 18, 2008 at 7:40 PM, Javier Martín <[EMAIL PROTECTED]> wrote: > El mar, 17-06-2008 a las 23:49 -0400, Pavel Roskin escribió: >> On Wed, 2008-06-18 at 05:14 +0200, Javier Martín wrote: >> >> > ORIGINAL FILE: /boot/grub/core.img (28449 bytes in Ubuntu Hardy default) >> >> That's an alread

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Javier Martín
El mar, 17-06-2008 a las 23:49 -0400, Pavel Roskin escribió: > On Wed, 2008-06-18 at 05:14 +0200, Javier Martín wrote: > > > ORIGINAL FILE: /boot/grub/core.img (28449 bytes in Ubuntu Hardy default) > > That's an already compressed file! Please see my posts in the thread. > We should be testing c

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Bean
On Wed, Jun 18, 2008 at 3:36 PM, Pavel Roskin <[EMAIL PROTECTED]> wrote: > On Wed, 2008-06-18 at 14:55 +0800, Bean wrote: > >> You can try Robert's case: >> >> grub-mkimage -d . -o core.img pc ext2 lvm raid > > 50764 image > 35090 image.lzo > 27308 image.gz > 27012 image.bz2 > 24470 image.lzma

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Bean
On Wed, Jun 18, 2008 at 3:36 PM, Pavel Roskin <[EMAIL PROTECTED]> wrote: > If I run string on the image to be compressed, I see all function names, > such as grub_biosdisk_get_diskinfo_int13_extensions. I don't see why we > need function names there. Function names starting with "grub_" take > 45

Re: [RFC] split floppy support off biosdisk

2008-06-18 Thread Pavel Roskin
On Wed, 2008-06-18 at 14:55 +0800, Bean wrote: > You can try Robert's case: > > grub-mkimage -d . -o core.img pc ext2 lvm raid 50764 image 35090 image.lzo 27308 image.gz 27012 image.bz2 24470 image.lzma > The uncompressed part need to be added as well, which is 1280 byte. It's actually go

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Bean
On Wed, Jun 18, 2008 at 11:49 AM, Pavel Roskin <[EMAIL PROTECTED]> wrote: > On Wed, 2008-06-18 at 05:14 +0200, Javier Martín wrote: > >> ORIGINAL FILE: /boot/grub/core.img (28449 bytes in Ubuntu Hardy default) > > That's an already compressed file! Please see my posts in the thread. > We should be

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Pavel Roskin
On Wed, 2008-06-18 at 05:14 +0200, Javier Martín wrote: > ORIGINAL FILE: /boot/grub/core.img (28449 bytes in Ubuntu Hardy default) That's an already compressed file! Please see my posts in the thread. We should be testing compression on the data we are actually compressing. That's what I'm gett

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Javier Martín
Huge Oops!! Please forget my last, grandiloquent post about my "investigation": it did not cross my mind that core.img was _already_ compressed with LZO, so the results are not fair to it. I will repeat the test tomorrow with what could be a pseudo pre-compression core image consisting of kernel.im

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Javier Martín
El mié, 18-06-2008 a las 01:35 +0800, Bean escribió: > On Wed, Jun 18, 2008 at 1:23 AM, Javier Martín <[EMAIL PROTECTED]> > wrote: > > I'm currently checking the memory requisites of LZMA decompression > > depending on the -1..9 compression setting, and will report back when > > finished > > Hi, >

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Bean
On Wed, Jun 18, 2008 at 1:23 AM, Javier Martín <[EMAIL PROTECTED]> wrote: > El mar, 17-06-2008 a las 22:52 +0800, Bean escribió: >> Hi, >> >> I think memory is not a problem here, as grub2 can use upper memory: >> >> src = raw data >> dest = 0x10 >> buf = dest + kernel_size >> > We could do tha

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Javier Martín
El mar, 17-06-2008 a las 22:52 +0800, Bean escribió: > Hi, > > I think memory is not a problem here, as grub2 can use upper memory: > > src = raw data > dest = 0x10 > buf = dest + kernel_size > We could do that, but since the code would execute at the very first stages, where we still don't

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Pavel Roskin
On Tue, 2008-06-17 at 23:02 +0800, Bean wrote: > I'm not against other compression algorithm, but lzma seems to be the > best, for example, for the previous c2.img: > > bzip2 c2.img && du -b c2.img.bz2 > 29247 c2.img.bz2 > > gzip c2.img && du -b c2.img.gz > 29825 c2.img.gz We need numbers f

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Bean
On Tue, Jun 17, 2008 at 10:39 PM, Javier Martín <[EMAIL PROTECTED]> wrote: > PS: have you tested the performance of other compression algorithms like > gzip-deflate (much simpler) or bzip2 (near LZMA for small files but with > less memory requisites)? Hi, I'm not against other compression algorit

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Bean
On Tue, Jun 17, 2008 at 10:39 PM, Javier Martín <[EMAIL PROTECTED]> wrote: > El mar, 17-06-2008 a las 11:06 +0800, Bean escribió: >> AFAIK, lzma decompresser don't need intermediate buffers, it operates >> directly on the input and output buffer, for example, the lzmadecoder >> in coreboot: > I thi

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Javier Martín
El mar, 17-06-2008 a las 11:06 +0800, Bean escribió: > AFAIK, lzma decompresser don't need intermediate buffers, it operates > directly on the input and output buffer, for example, the lzmadecoder > in coreboot: I think you didn't see the intermediate buffer because it is hidden in the Probs field

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Robert Millan
On Tue, Jun 17, 2008 at 02:33:48PM +0200, Robert Millan wrote: > On Tue, Jun 17, 2008 at 02:22:15PM +0200, Robert Millan wrote: > > > > A problem I see here is that LZMA is licensed under GPLv2-only, so it'd > > probably take a rewrite of the decompressor for us to use it, or ask the > > author to

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Robert Millan
On Tue, Jun 17, 2008 at 02:22:15PM +0200, Robert Millan wrote: > > A problem I see here is that LZMA is licensed under GPLv2-only, so it'd > probably take a rewrite of the decompressor for us to use it, or ask the > author to make it "or later". > > By the looks (no explicit license header in ind

Re: [RFC] split floppy support off biosdisk

2008-06-17 Thread Robert Millan
On Tue, Jun 17, 2008 at 10:03:15AM +0800, Bean wrote: > >> I think lzo is working fine, we > >> should avoid switching unless there is a good reason to do so. > > > > ISTR Okuji didn't like it; can't remember why. > > He is concerned about compression ratio, but doesn't mind in the end: > > http

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Tue, Jun 17, 2008 at 10:47 AM, Javier Martín <[EMAIL PROTECTED]> wrote: > El mar, 17-06-2008 a las 10:03 +0800, Bean escribió: >> The decompresser is about 2K, we should also exclude the uncompressed >> part at the beginning of kernel, but there is still plenty room left. > The problem is not ju

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Javier Martín
El mar, 17-06-2008 a las 10:03 +0800, Bean escribió: > The decompresser is about 2K, we should also exclude the uncompressed > part at the beginning of kernel, but there is still plenty room left. The problem is not just the decompresser size itself, but how much memory it uses. LZO is pretty speci

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Tue, Jun 17, 2008 at 5:31 AM, Robert Millan <[EMAIL PROTECTED]> wrote: > On Mon, Jun 16, 2008 at 08:37:43PM +0800, Bean wrote: >> >> But do we really need those space ? > > The typical use case where space is a PITA is boot an LVM system without a > separate /boot partition. There we don't have

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Robert Millan
On Mon, Jun 16, 2008 at 08:37:43PM +0800, Bean wrote: > > But do we really need those space ? The typical use case where space is a PITA is boot an LVM system without a separate /boot partition. There we don't have the ressort of using a blocklist (which is already bad), so embedding core.img is

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Robert Millan
On Mon, Jun 16, 2008 at 01:20:27PM +0200, Javier Martín wrote: > Hmm.. that could be done without too much fuss (except the part of > implementing the decompressor in assembly for each platform...), For other platforms than i386-pc, size is usually not a problem (well, for coreboot it is, but ther

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Robert Millan
On Mon, Jun 16, 2008 at 07:03:50PM +0800, Bean wrote: > On Mon, Jun 16, 2008 at 5:15 PM, Robert Millan <[EMAIL PROTECTED]> wrote: > > On Mon, Jun 16, 2008 at 02:30:38PM +0800, Bean wrote: > >> > >> The fd and hd device share many code, separating them is not a good > >> idea. > > > > If code duplic

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Pavel Roskin
Quoting Bean <[EMAIL PROTECTED]>: On Mon, Jun 16, 2008 at 9:06 PM, Pavel Roskin <[EMAIL PROTECTED]> wrote: No, it was uncompressed. GRUB 0.97 doesn't use LZO compression. Yes, grub 0.97 don't use lzo, but I use external lzo tool to get the result. It illustrates the result of lzo and lzma a

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Mon, Jun 16, 2008 at 9:06 PM, Pavel Roskin <[EMAIL PROTECTED]> wrote: > On Mon, 2008-06-16 at 14:09 +0200, Javier Martín wrote: >> El lun, 16-06-2008 a las 19:47 +0800, Bean escribió: >> > Well, I did some testing a while ago, the result is in: >> > >> > http://lists.gnu.org/archive/html/grub-de

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Pavel Roskin
On Mon, 2008-06-16 at 14:09 +0200, Javier Martín wrote: > El lun, 16-06-2008 a las 19:47 +0800, Bean escribió: > > Well, I did some testing a while ago, the result is in: > > > > http://lists.gnu.org/archive/html/grub-devel/2007-12/msg00114.html > > > > Even with lzo compressed image core.img, lz

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Mon, Jun 16, 2008 at 8:09 PM, Javier Martín <[EMAIL PROTECTED]> wrote: > El lun, 16-06-2008 a las 19:47 +0800, Bean escribió: >> Well, I did some testing a while ago, the result is in: >> >> http://lists.gnu.org/archive/html/grub-devel/2007-12/msg00114.html >> >> Even with lzo compressed image c

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Javier Martín
El lun, 16-06-2008 a las 19:47 +0800, Bean escribió: > Well, I did some testing a while ago, the result is in: > > http://lists.gnu.org/archive/html/grub-devel/2007-12/msg00114.html > > Even with lzo compressed image core.img, lzma can save up to 5K. > You mean that even with an already compress

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Mon, Jun 16, 2008 at 7:20 PM, Javier Martín <[EMAIL PROTECTED]> wrote: > El lun, 16-06-2008 a las 19:03 +0800, Bean escribió: >> In that case, I suggest more dramatic >> action, like replacing lzo with a strong compression algorithm, like >> lzma, which would save a few thousand bytes. >> > Hmm.

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Javier Martín
El lun, 16-06-2008 a las 19:03 +0800, Bean escribió: > In that case, I suggest more dramatic > action, like replacing lzo with a strong compression algorithm, like > lzma, which would save a few thousand bytes. > Hmm.. that could be done without too much fuss (except the part of implementing the d

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Bean
On Mon, Jun 16, 2008 at 5:15 PM, Robert Millan <[EMAIL PROTECTED]> wrote: > On Mon, Jun 16, 2008 at 02:30:38PM +0800, Bean wrote: >> >> The fd and hd device share many code, separating them is not a good >> idea. > > If code duplication is a problem, we could link the same code staticaly for > each

Re: [RFC] split floppy support off biosdisk

2008-06-16 Thread Robert Millan
On Mon, Jun 16, 2008 at 02:30:38PM +0800, Bean wrote: > > The fd and hd device share many code, separating them is not a good > idea. If code duplication is a problem, we could link the same code staticaly for each module. Size of biosdisk module is an issue, because it's always used in core.img

Re: [RFC] split floppy support off biosdisk

2008-06-15 Thread Bean
On Mon, Jun 16, 2008 at 8:32 AM, Pavel Roskin <[EMAIL PROTECTED]> wrote: > On Sun, 2008-06-15 at 19:48 +0200, Robert Millan wrote: >> Floppy support seems to be a constant source of trouble because it's the >> only disk type that can't be probed for reliably without causing unacceptable >> delays o

Re: [RFC] split floppy support off biosdisk

2008-06-15 Thread Pavel Roskin
On Sun, 2008-06-15 at 19:48 +0200, Robert Millan wrote: > Floppy support seems to be a constant source of trouble because it's the > only disk type that can't be probed for reliably without causing unacceptable > delays on some systems. > > How would you think about splitting it off biosdisk into

[RFC] split floppy support off biosdisk

2008-06-15 Thread Robert Millan
Floppy support seems to be a constant source of trouble because it's the only disk type that can't be probed for reliably without causing unacceptable delays on some systems. How would you think about splitting it off biosdisk into a separate module? This will probably add some code redundancy,