Re: Using a better compression than .gz for one's CPAN modules

* David Golden [2010-11-28 22:45]: > On the other hand, if someone wants to recompress all of CPAN > into XYZ compression scheme and release their own client that > deals with it (or patch cpanm or whatever), then they can have > the benefits (and any resulting hassles) themselves. And note that

Re: Using a better compression than .gz for one's CPAN modules

On Sun, Nov 28, 2010 at 4:22 PM, Aristotle Pagaltzis wrote: > I agree with the notion. But let me ask how much pressure changing > the compression format on CPAN would exert on the world to adapt > itself to it. Note too the quote is written from the perspective > of the world: no mention goes to

Re: Using a better compression than .gz for one's CPAN modules

* Shlomi Fish [2010-11-26 22:05]: > In any case, regardless of its age, xz does tend to compress > better than bz2 and should also be faster. I know. I heard of it quite early and switched from bzip2 to xz for my database dumps and mail archives. That’s not the point of the quote though. New thi

Re: Using a better compression than .gz for one's CPAN modules

On Fri, Nov 26, 2010 at 3:59 PM, Shlomi Fish wrote: >>     There are two kinds of fool. One says, >>     “This is old, and therefore good.” And one says, >>     “This is new, and therefore better.” > > That put aside sticking with an older solution may be preferable due to the > better adoption ra

Re: Using a better compression than .gz for one's CPAN modules

* Shlomi Fish [2010-11-24 21:05]: > Welcome to 2010. There are two kinds of fool. One says, “This is old, and therefore good.” And one says, “This is new, and therefore better.” —John Brunner Regards, -- Aristotle Pagaltzis //

Re: Using a better compression than .gz for one's CPAN modules

On Wed, Nov 24, 2010 at 09:59:59PM +0200, Shlomi Fish wrote: > On Friday 19 November 2010 22:02:48 David Cantrell wrote: > > Even if it does, there's not much point. bzip2 support is nowhere near > > universal, and preventing lots of users from using your code would seem > > to be a poor trade-off

Re: Reducing rsync cost (was: Re: Using a better compression than .gz for one's CPAN modules)

On Mon, Nov 22, 2010 at 4:37 AM, David Landgren wrote: > Yeah, this is the killer. In an ideal world, we would kill the symlinks such > as authors/id/*, modules/by-category/*, modules/by-module/* and so on. These > could be recreated via shell scripts locally on mirrors for people who wish > to ma

Reducing rsync cost (was: Re: Using a better compression than .gz for one's CPAN modules)

On 19/11/2010 20:57, dhu...@hudes.org wrote: source code, even 100KLOC? Once you go to .gz you're already at better than 2:1. What are you going to save by going to even 3:1, 10Kbytes? compared to the nuisance inflicted, it's nothing. Over the entire CPAN archive, it'd be significant... I agre

Re: Using a better compression than .gz for one's CPAN modules

* Andreas J. Koenig [2010-11-22 09:20]: > Agreed, but since bzip2 support is already done we can welcome > it when people actually use it. I am unwilling to encourage it but I won’t argue if someone wants to use it. And it can be a win for distributions with very large bundled data files so one

Re: Using a better compression than .gz for one's CPAN modules

> On Sat, 20 Nov 2010 23:22:52 +0100, Aristotle Pagaltzis > said: > It’s gonna be a lot of work to iron out the entire tool chain to > support the newer formats; then it will take a lot of time until > the work trickles out far enough that people could start relying > on it. In

Re: Using a better compression than .gz for one's CPAN modules

While I completely agree with Aristotle I wish to clarify that Solaris 10 and 11 ship with bzip2. I can't recall about Solaris 9 and I am recalling this was not the case with 8 and earlier. Sent from my BlackBerry® smartphone with Nextel Direct Connect

Re: Using a better compression than .gz for one's CPAN modules

* Shlomi Fish [2010-11-19 19:55]: > here is a report on compressing Graph-Easy-0.70.tar with various > compression methods: > > {{{ > shlomif:~/progs/perl/cpan/Graph/Easy/trunk/Graph-Easy/TEMP$ ls -l > total 3420 > -rw-r--r-- 1 shlomif shlomif 2160640 Nov 14 22:20 Graph-Easy-0.70.tar > -rw-r--r--

Re: Using a better compression than .gz for one's CPAN modules

On Fri, November 19, 2010 2:57 pm, dhu...@hudes.org wrote: > Disk space is cheap. Bandwidth is cheap. What's rough is the rsync between > mirrors. Compressing to .bz2 won't help that: the stress is doing a stat > on every single file in CPAN not the transfer. Work toward optimizing the > mirror d

Re: Using a better compression than .gz for one's CPAN modules

On Fri, 19 Nov 2010 11:57 -0800, dhu...@hudes.org wrote: > Disk space is cheap. Bandwidth is cheap. What's rough is the rsync > between > mirrors. Compressing to .bz2 won't help that: the stress is doing a stat > on every single file in CPAN not the transfer. Work toward optimizing the > mirror d

Re: Using a better compression than .gz for one's CPAN modules

On Fri, Nov 19, 2010 at 08:53:12PM +0200, Shlomi Fish wrote: > here is a report on compressing Graph-Easy-0.70.tar with various > compression methods: > > -rw-r--r-- 1 shlomif shlomif 416916 Nov 14 22:23 Graph-Easy-0.70.tar.gz > > -rw-r--r-- 1 shlomif shlomif 329197 Nov 5 12:24 Graph-Easy-0.70

Re: Using a better compression than .gz for one's CPAN modules

>> source code, even 100KLOC? Once you go to .gz you're already at better >> than 2:1. What are you going to save by going to even 3:1, 10Kbytes? >> compared to the nuisance inflicted, it's nothing. > > Over the entire CPAN archive, it'd be significant... > > I agree on the individual case it's pro

Re: Using a better compression than .gz for one's CPAN modules

On Fri, November 19, 2010 2:18 pm, dhu...@hudes.org wrote: > The savings for going to .bz2 over .gz for source code are fairly > insignificant. We're talking about source code for a perl module. Is > your stuff tens of megabytes in size? That's a lot of code if so. I could > understand if you we

Re: Using a better compression than .gz for one's CPAN modules

The savings for going to .bz2 over .gz for source code are fairly insignificant. We're talking about source code for a perl module. Is your stuff tens of megabytes in size? That's a lot of code if so. I could understand if you were distributing a sizable database with your code but source code, e

Re: Using a better compression than .gz for one's CPAN modules

On Fri, Nov 19, 2010 at 1:53 PM, Shlomi Fish wrote: > 1. Will the CPAN testing and downloading toolchian will handle modules > uploaded as .tar.bz2?  (Allow to install them, unpack them, etc.)  How > about tar.xz. .bz2, yes. .xz, possibly, but not reliably. CPANPLUS uses Archive::Extract, which

Using a better compression than .gz for one's CPAN modules

Hi all, here is a report on compressing Graph-Easy-0.70.tar with various compression methods: {{{ shlomif:~/progs/perl/cpan/Graph/Easy/trunk/Graph-Easy/TEMP$ ls -l total 3420 -rw-r--r-- 1 shlomif shlomif 2160640 Nov 14 22:20 Graph-Easy-0.70.tar -rw-r--r-- 1 shlomif shlomif 329197 Nov 5 12:24 Gr