Actually, I've just realised that wget preserves the mtime of the remote file by default, and furthermore our Puppet infrastruction runs 'puppet agent' at minimum 30 minute intervals. Since we wget the file to two different locations, the most it could have gotten in any one day would be ~6GB. So while I can no longer claim to know when these downloads occured, I can confidently state that we were probably downloading up to 96 times a day for the best part of two months at some point *after* Nov 25th 2013. I wouldn't be surprised if we are being explicitly throttled by domain name at this point. :-/
On Thursday, January 22, 2015 at 5:01:15 PM UTC, sweeney wrote: > > When I started at my current employer I found thousands of copies of > jenkins.war in /root on on of our Jenkins masters, mostly dated Nov 25 > 2013. I have just found and fixed our broken Puppet configuration that > must have caused this. I'm trying to download now (I'm building a test > Jenkins instance to test upgrades) and I'm currently seeing about 1KB/s > from archives.jenkins-ci.org; I guess I'm paying the price for my idiot > predecessor downloading the same file over and over to the tune of 350GB in > one day :( > > On Friday, January 16, 2015 at 4:13:38 PM UTC, Christopher Orr wrote: >> >> On 16/01/15 16:16, Aaron Johnson wrote: >> > Can someone please verify this: >> > >> > I am unable to download RPMs from archives.jenkins-ci.org, example: >> > >> http://archives.jenkins-ci.org/redhat-stable/jenkins-1.424.2-1.1.noarch.rpm >> > >> > The download starts, and goes very slow (less than 5Kbps) and never >> > finishes... >> >> Yup, I can reproduce this from Germany; but Rackspace seems to be doing >> fine, and other nearby IP addresses seem speedy. >> >> Since you're downloading a three-year-old package, you're being >> redirected from mirrors.j.o to archives.j.o. >> I'm just speculating, but maybe there's a data transfer limit on the >> archives server which gets activated if there has been too much traffic, >> to avoid excess data transfer costs (as has happened in the past). >> >> >> > Because of this I am unable to create an internal mirror of RPM >> packages >> > for Jenkins. >> >> Out of interest, why do you need to create a mirror of long-outdated >> Jenkins packages? >> >> Regards, >> Chris >> > -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/9edd181e-f90f-4c59-a596-87909892fa67%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
