On Mon, Feb 08, 2010 at 05:23:03AM +0000, Robin H. Johnson wrote: > On Sun, Feb 07, 2010 at 05:02:22PM -0800, Brian Harring wrote: > > On Sun, Jan 31, 2010 at 10:04:40AM +0000, Robin H. Johnson wrote: > > > Changes: > > > - This GLEP can stand independently of GLEP58. > > > - Add XZ to compression types list. > > > - Move cutoff to 32KiB. Provide size example w/ 32KiB+gzip. > > > - Split specification into generation and validation. > > One concern w/ this glep- the intention seems to be to reduce on disk > > space requirements but the addition of compression raises questions > > for rsync transferance of the manifests. > > > > Have you done any testing to quantify how much of an increase in rsync > > bandwidth this will add? Specifically thinking about the metamanifest > > on this one. > > I think the best course of action is to end up generating the compressed > MetaManifests when we start generated the MetaManifests themselves, but > not placing them into the tree yet. Instead simply use them to measure > rsync transfer size impact on the generation server and produce > statistics to see if the cutoff could benefit from being altered, or if > the disk space should be wasted in favour of smaller transfer size.
Works for me- I do strongly suspect that if we use compression we'll be purely trading disk space decrease for the cost of transfering each changed compressed manifest in full though... rsync + compression do not get along at all. Either way, stats would be useful when you've got time. ~brian
pgprlatgpHXDC.pgp
Description: PGP signature
