Re: CD sizes again (and BoF reminder!)

* Adam Borowski , 2012-07-22, 23:51: having an option to disable fsync in dpkg without unreliable LD_PRELOAD tricks would be great. http://bugs.debian.org/613428 -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact

xz (Re: CD sizes again (and BoF reminder!)

Hi, On Sat, 21 Jul 2012 19:13:45 -0400 Joey Hess wrote: > Which is why I asked for actual, real-world benchmarks... As I said in my presentation, I just tested a few case, fonts-horai-umefont, poppler-data and openclipart-png. Install fonts- and poppler-data is almost same time, but opencli

Re: CD sizes again (and BoF reminder!)

On Mon, 23 Jul 2012, Adam Borowski wrote: > You tested ext4. On btrfs, dpkg is around an order of magnitude slower, > making using it without eatmydata a laughable idea. > > And that's on a filesystem whose features include: > * transactions (so all dpkg processing could be done without a single

Re: CD sizes again (and BoF reminder!)

On Sun, Jul 22, 2012 at 07:58:42PM +0300, Uoti Urpala wrote: > Simon Paillard wrote: > > , I understand debian-installer ask dpkg not to fsync: > > > - Run dpkg with --force-unsafe-io during installation; syncing is > > This only affects one particular instance of syncing (which I think may

Re: CD sizes again (and BoF reminder!)

On Sun, Jul 22, 2012 at 01:58:59AM +0200, Adam Borowski wrote: > > BTW, when we switched to building udebx with xz, Philipp Kern benchmarked > > it using little or no additional CPU to decompress xz produced with > > -Zxz -z1 -Sextreme http://lists.debian.org/debian-boot/2011/10/msg00247.html > Per

Re: CD sizes again (and BoF reminder!)

Simon Paillard wrote: > On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: > > So at least in this case the biggest performance problem by far is the > > inappropriate use of fsync() or other disk synchronization primitives, > > and CPU use for unpacking is pretty much irrelevant. > > Th

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 21, 2012 at 11:42:03AM -0400, Joey Hess wrote: > Hideki Yamane wrote: > > On Sun, 8 Jul 2012 17:58:16 +0200 > > Adam Borowski wrote: > > > • xz -6 (the default) is a lot slower when compressing, fast when > > > decompressing, needs only 10MB memory, 58% size > > > • xz -9 has v

Re: CD sizes again (and BoF reminder!)

Mike Hommey wrote: > Note that slower decompression doesn't necessarily mean longer > installation time. I/O is still more time consuming than CPU. Which is why I asked for actual, real-world benchmarks... -- see shy jo signature.asc Description: Digital signature

Re: CD sizes again (and BoF reminder!)

Hi, On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: > Joey Hess wrote: > > Hideki Yamane wrote: > > > I tested as well, and sometimes decompression with xz is so slw, > > > it takes 6-8 times than default gz. > > > > I was just watching your DebConf presentation "Lets shrink De

Re: CD sizes again (and BoF reminder!)

brian m. carlson wrote: > On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: > > So at least in this case the biggest performance problem by far is the > > inappropriate use of fsync() or other disk synchronization primitives, > > and CPU use for unpacking is pretty much irrelevant. > >

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 21, 2012 at 09:25:50PM +0300, Uoti Urpala wrote: > Most of the time taken by cdebootstrap is wasted by dpkg on doing > useless file syncs: > > cdebootstrap --arch=amd64 unstable debian-tree/ > > from local package cache on ext4: 138 seconds > > on tmpfs where dpkg can't waste time on

Re: CD sizes again (and BoF reminder!)

Joey Hess wrote: > Hideki Yamane wrote: > > I tested as well, and sometimes decompression with xz is so slw, > > it takes 6-8 times than default gz. > > I was just watching your DebConf presentation "Lets shrink Debian > package archive" and I think there you said decompression with xz was >

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 21, 2012 at 11:42:03AM -0400, Joey Hess wrote: > Hideki Yamane wrote: > > On Sun, 8 Jul 2012 17:58:16 +0200 > > Adam Borowski wrote: > > > • xz -6 (the default) is a lot slower when compressing, fast when > > > decompressing, needs only 10MB memory, 58% size > > > • xz -9 has v

Re: CD sizes again (and BoF reminder!)

Hideki Yamane wrote: > On Sun, 8 Jul 2012 17:58:16 +0200 > Adam Borowski wrote: > > • xz -6 (the default) is a lot slower when compressing, fast when > > decompressing, needs only 10MB memory, 58% size > > • xz -9 has very slow compression, takes gobs of memory, 56% size > > (Obviously

Re: CD sizes again (and BoF reminder!)

Hi, On Sun, 8 Jul 2012 17:58:16 +0200 Adam Borowski wrote: > • xz -6 (the default) is a lot slower when compressing, fast when > decompressing, needs only 10MB memory, 58% size > • xz -9 has very slow compression, takes gobs of memory, 56% size > (Obviously, the "size" numbers are dra

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 04:22:58PM -0600, Stefano Zacchiroli wrote: > Ansgar has been experimenting with .deb sizes to make the packages > needed for a minimal desktop installation fit in the first CD. It looks > like that's doable by switching to xz compression for the involved > binaries. Would

Re: CD sizes again (and BoF reminder!)

Le Sun, Jul 08, 2012 at 02:12:44PM -0600, Joey Hess a écrit : > > grub-legacy is still used for multipath and sataraid. > Something was going to be done to make grub2 support those, but > I don't know the status. Hi, Grub-legacy is also useful for booting virtual machine images with pv-grub (suc

Re: CD sizes again (and BoF reminder!)

> Ben Hutchings writes: [...] > - twm: no-one should have to suffer this And, exactly, why not? Before I've switched to Openbox, it was one of the two WM's I've used, along with FVWM. And they say [1] that it still can be handy at times. The “obscure” lab

Re: CD sizes again (and BoF reminder!)

On Sun, Jul 08, 2012 at 02:12:44PM -0600, Joey Hess wrote: > Cyril Brulebois wrote: > > It looks to me like a current debian-installer build installs grub2, > > with no option for grub-legacy, even in expert mode. > > grub-legacy is still used for multipath and sataraid. > Something was going to b

Re: CD sizes again (and BoF reminder!)

Cyril Brulebois wrote: > It looks to me like a current debian-installer build installs grub2, > with no option for grub-legacy, even in expert mode. grub-legacy is still used for multipath and sataraid. Something was going to be done to make grub2 support those, but I don't know the status. -- s

Re: CD sizes again (and BoF reminder!)

Wouter Verhelst (08/07/2012): > On Sun, Jul 08, 2012 at 01:17:32AM +0200, Bastian Blank wrote: > > Even grub-legacy? > > Yes; d-i in expert mode still has the ability to explicitly choose for > grub legacy, if you really want to. It looks to me like a current debian-installer build installs grub

Re: CD sizes again (and BoF reminder!)

Hi Stefano Zacchiroli writes: > Ansgar has been experimenting with .deb sizes to make the packages > needed for a minimal desktop installation fit in the first CD. It looks > like that's doable by switching to xz compression for the involved > binaries. Would you grant freeze exceptions for pac

Re: CD sizes again (and BoF reminder!)

On Sun, Jul 08, 2012 at 01:17:32AM +0200, Bastian Blank wrote: > On Sat, Jul 07, 2012 at 04:36:34PM -0600, Joey Hess wrote: [...] > > > - grub-legacy > > These are all installed by d-i in various situations. > > Even grub-legacy? Yes; d-i in expert mode still has the ability to explicitly choose

Re: CD sizes again (and BoF reminder!)

Hi, On Sat, 7 Jul 2012 04:47:35 +0100 Steve McIntyre wrote: > So, yes - looks like xz will make a difference here for the wheezy > release, for amd64 at least. It's enough that we'd probably even have > space for the installation manual and release notes to fit \o/. BTW, I'll talk about using x

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 08:42:07PM +0100, Ben Hutchings wrote: > On Sat, 2012-07-07 at 16:14 +0200, Bastian Blank wrote: > - ftp, telnet: mostly redundant with wget and nc, unless you really like > cleartext authentication I don't get why anyone would talk about "authentication" in the context of

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 04:22:58PM -0600, Stefano Zacchiroli wrote: > On Sun, Jul 08, 2012 at 12:03:44AM +0200, Cyril Brulebois wrote: > > Ansgar Burchardt (07/07/2012): > > > Another question is if the release team would consider unblocking > > > updated packages (with just the switch to xz for b

Re: CD sizes again (and BoF reminder!)

Quoting Ben Hutchings (b...@decadent.org.uk): > > partman-crypto still installs this. > > Seems like a bug...? The package is orphaned and dm-crypt has support > for a compatible encryption mode. Given the level of attention which partman-crypto got during the squeeze->wheezy release, I'd bet

Re: CD sizes again (and BoF reminder!)

On Sat, 2012-07-07 at 16:36 -0600, Joey Hess wrote: > > - ftp, telnet: mostly redundant with wget and nc, unless you really like > > cleartext authentication > > - procmail: server > > These are priority standard. Which is not the same as being important to include in a desktop installation. > >

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 04:36:34PM -0600, Joey Hess wrote: > > - ftp, telnet: mostly redundant with wget and nc, unless you really like > > cleartext authentication > > - procmail: server > These are priority standard. This is fixeable. > > - pcmciautils: PCMCIA is dead > > - jfsutils, reiserfspr

Re: CD sizes again (and BoF reminder!)

> - ftp, telnet: mostly redundant with wget and nc, unless you really like > cleartext authentication > - procmail: server These are priority standard. > - pcmciautils: PCMCIA is dead > - jfsutils, reiserfsprogs, ufsutils: obscure > - discover > - openssh-server: server, not desktop > - grub-lega

Re: CD sizes again (and BoF reminder!)

On Sun, Jul 08, 2012 at 12:03:44AM +0200, Cyril Brulebois wrote: > Ansgar Burchardt (07/07/2012): > > Another question is if the release team would consider unblocking > > updated packages (with just the switch to xz for binaries). I think > > it would be nice to be able to get a useful desktop s

Re: CD sizes again (and BoF reminder!)

Ansgar Burchardt (07/07/2012): > Another question is if the release team would consider unblocking > updated packages (with just the switch to xz for binaries). I think > it would be nice to be able to get a useful desktop system using just > the first CD, but I'm not sure if they would make an e

Re: CD sizes again (and BoF reminder!)

On Sat, 2012-07-07 at 16:14 +0200, Bastian Blank wrote: > On Sat, Jul 07, 2012 at 02:34:44PM +0100, Steve McIntyre wrote: > > The list posted there is the full sorted list of *all* packages, as > > applied to the full set of CDs. The last one on CD#1 is > > gnome-packagekit-data, as I said, and I d

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 02:34:44PM +0100, Steve McIntyre wrote: > The list posted there is the full sorted list of *all* packages, as > applied to the full set of CDs. The last one on CD#1 is > gnome-packagekit-data, as I said, and I don't see any debug packages > above that in the list. Ups, I di

Re: CD sizes again (and BoF reminder!)

On Sat, Jul 07, 2012 at 03:10:15PM +0200, Bastian Blank wrote: >On Fri, Jul 06, 2012 at 10:10:04PM +0100, Steve McIntyre wrote: >> http://cdimage.debian.org/cdimage/tmp/new-tasks/gnome-cd.list.gz > >Why does this contain debug packages? The list posted there is the full sorted list of *all* pack

Re: CD sizes again (and BoF reminder!)

On Fri, Jul 06, 2012 at 10:10:04PM +0100, Steve McIntyre wrote: > http://cdimage.debian.org/cdimage/tmp/new-tasks/gnome-cd.list.gz Why does this contain debug packages? Bastian -- I've already got a female to worry about. Her name is the Enterprise. -- Kirk, "The Corbomite Ma

Re: CD sizes again (and BoF reminder!)

Steve McIntyre writes: > On Fri, Jul 06, 2012 at 09:00:32PM -0600, Ansgar Burchardt wrote: >>I tried recompressing all packages in wheezy with xz. The total size >>for all amd64+all packages was reduced from 42GB to 33 GB (about 20%). >>A per-package listing is available from [1] >> >> [1]

Re: CD sizes again (and BoF reminder!)

On Fri, Jul 06, 2012 at 09:00:32PM -0600, Ansgar Burchardt wrote: >Hi, > >Steve McIntyre writes: >> Back in May I warned about CD sizes[1] for the Wheezy release, >> pointing out that CD#1 isn't big enough any more to provide usable >> Gnome or KDE installations. There was some discussion about wh

Re: CD sizes again (and BoF reminder!)

Hi, Steve McIntyre writes: > Back in May I warned about CD sizes[1] for the Wheezy release, > pointing out that CD#1 isn't big enough any more to provide usable > Gnome or KDE installations. There was some discussion about what to do > about that (change compression to xz, switch to the lighter d

Re: CD sizes again (and BoF reminder!)

As suggested by Ansgar IRL, here's a summary of what compression types are showing up on each CD (by looking at data.tar.$EXT for all the .debs and .udebs): >Gnome >= > >The last package on amd64 CD#1 is gnome-packagekit-data. task-desktop >fits on CD#1, but task-gnome-desktop is ~110 packages

Re: CD sizes again (and BoF reminder!)

Steve McIntyre writes: > Back in May I warned about CD sizes[1] for the Wheezy release, > pointing out that CD#1 isn't big enough any more to provide usable > Gnome or KDE installations. Indeed. CD1 was really problematic in squeeze too: http://lists.debian.org/debian-boot/2011/08/msg00172.html

CD sizes again (and BoF reminder!)

Hey people, Following up on this again... Back in May I warned about CD sizes[1] for the Wheezy release, pointing out that CD#1 isn't big enough any more to provide usable Gnome or KDE installations. There was some discussion about what to do about that (change compression to xz, switch to the li