Public bug reported:
Just found the following in the log:
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz...
(12162KiB)
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz...
(8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it
to acng they will stay there basically forever and their information
being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in
terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore,
I assume that serving the uncompressed file to local clients as a
fallback should always work.
Would it be possible that as a matter of policy acng tries to download
the best compressed file first and if that is 404 goes through xz, bz2,
gz and then uncompressed in that order and stores only that Packages
file? If a client in the future requests Packages.bz2 and that is
indeed available, then this is served, otherwise, fallback to Packages
(which acng can get from the compressed file without any issues,
recompressing with a different algorithm would possibly result in a
different hashsum so I do not propose this). Any reason this would not
work?
PS: if this is workable, acng might want to cache for a few weeks the
information that xz is not but bz2 is available for a certain Packages file.
This is unlikely to change frequently and going straight to downloading bz2
next time around will lessen the number of connections to open.
** Affects: apt-cacher-ng (Ubuntu)
Importance: Undecided
Status: New
** Description changed:
Just found the following in the log:
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz...
(12162KiB)
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz...
(8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it
to acng they will stay there basically forever and their information
being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in
terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore,
I assume that serving the uncompressed file to local clients as a
fallback should always work.
Would it be possible that as a matter of policy acng tries to download
- the best compressed first and if that is 404 goes tries xz, bz2, gz and
- then uncompressed in that order and stores only that Packages file? If
- a client in the future requests Packages.bz2 and that is indeed
- available, then this is served, otherwise, fallback to Packages (which
- acng can get from the compressed file without any issues, recompressing
- with a different algorithm would possibly result in a different hashsum
- so I do not propose this). Any reason this would not work?
+ the best compressed file first and if that is 404 goes through xz, bz2,
+ gz and then uncompressed in that order and stores only that Packages
+ file? If a client in the future requests Packages.bz2 and that is
+ indeed available, then this is served, otherwise, fallback to Packages
+ (which acng can get from the compressed file without any issues,
+ recompressing with a different algorithm would possibly result in a
+ different hashsum so I do not propose this). Any reason this would not
+ work?
** Description changed:
Just found the following in the log:
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz...
(12162KiB)
Checking/Updating
mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz...
(8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it
to acng they will stay there basically forever and their information
being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in
terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore,
I assume that serving the uncompressed file to local clients as a
fallback should always work.
Would it be possible that as a matter of policy acng tries to download
the best compressed file first and if that is 404 goes through xz, bz2,
gz and then uncompressed in that order and stores only that Packages
file? If a client in the future requests Packages.bz2 and that is
indeed available, then this is served, otherwise, fallback to Packages
(which acng can get from the compressed file without any issues,
recompressing with a different algorithm would possibly result in a
different hashsum so I do not propose this). Any reason this would not
work?
+
+
+ PS: if this is workable, acng might want to cache for a few weeks the
information that xz is not but bz2 is available for a certain Packages file.
This is unlikely to change frequently and going straight to downloading bz2
next time around will lessen the number of connections to open.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1589231
Title:
RFD: better handling of the Packages files
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apt-cacher-ng/+bug/1589231/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
