I did this exercise over many iterations during Docker container packing and found that as long as the data is under 1gb, it's going to compress really well with xz. Over 1gb and lrzip looks more attractive (but only on high memory systems). In reality, we're looking at log footprints from OpenStack environments on the order of 500mb to 2gb.
xz is very slow on single-core systems with 1.5gb of memory, but it's quite a bit faster if you run it on a more powerful system. I've found level 4 compression to be the best compromise that works well enough that it's still far better than gzip. If increasing compression time by 3-5x is too much for you guys, why not just go to bzip? You'll still improve compression but be able to cut back on time. Best Regards, Matthew Mosesohn On Mon, Nov 24, 2014 at 3:14 PM, Vladimir Kuklin <vkuk...@mirantis.com> wrote: > IMO, we should get a bunch of snapshots and decide which compression to use > according to the results of an experiment. XZ compression takes much longer, > so you will need to use parallel xz compression utility. > > On Fri, Nov 21, 2014 at 9:09 PM, Tomasz Napierala <tnapier...@mirantis.com> > wrote: >> >> >> > On 21 Nov 2014, at 16:55, Dmitry Pyzhov <dpyz...@mirantis.com> wrote: >> > >> > We have a request for change compression from gz to xz. This simple >> > change halfs our snapshots. Does anyone has any objections? Otherwise we >> > will include this change in 6.0 release. >> >> I agree with the change, but it shouldn’t be high >> >> Regards, >> -- >> Tomasz 'Zen' Napierala >> Sr. OpenStack Engineer >> tnapier...@mirantis.com >> >> >> >> >> >> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev@lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Yours Faithfully, > Vladimir Kuklin, > Fuel Library Tech Lead, > Mirantis, Inc. > +7 (495) 640-49-04 > +7 (926) 702-39-68 > Skype kuklinvv > 45bk3, Vorontsovskaya Str. > Moscow, Russia, > www.mirantis.com > www.mirantis.ru > vkuk...@mirantis.com > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
