Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote: > Mattias Wadenstein wrote: > > Well, that thing has been more or less recommended on this list as I > > understand it for sites without the restrictions that apply to the non-us > > images. I just carry both versions though, less hassle that way. > > If it's accepted I can make the script stop checking sizes (or hack in > some kind of symlink detection). It strikes me as a very bad idea to > make the user randomly get the non-us version with the name that > indicates it is the us version though. That can make for very confusing > bug reports if the non-us version breaks something. Well, the non-us one is considered the real one, it is supposed to be that image, just that some sites in the US might want to carry a different version. > > Well, looks good. Perhaps extrating the number of the latest version > > instead of just the binary check of "has 3.0_r0"? And perhaps a check of > > incomplete sets (just i386). Or perhaps those are the RFEs for version > > 2.0. :) > > I might improve it for 3.0_r1, since the difference between r0 and r1 > will not be a big deal for many people when getting cd's. I'm only > checking the first cd, but the code can have the names and sizes of all > the other cd's added to it in the configuration section. I suspect most > people want only one cd. I may add a check for source cd's. Yeah, that might be nice. It is probably more important to get this up and running though. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Too few up-to-date CD image mirrors
On Fri, 29 Nov 2002, Joey Hess wrote: > Mattias Wadenstein wrote: > > Where do you get those sizes from? My ftp and rsync clients shows > > 612171776 for debian-30r0-i386-binary-1.iso on those mirrors. > > > > lftp ftp.funet.fi:/pub/Linux/mirrors/debian-cdimage/3.0_r0/i386> ls >debian-30r0-i386-binary-1.iso > > -rw-r--r-- 1 mirrormirror 612171776 Jul 20 16:31 >debian-30r0-i386-binary-1.iso > > > > This is also the size of the downloaded file, which has a correct md5sum. > > Hmm, my script gets the size with the ftp SIZE command. For example, on > ftp.tu-clausthal.de: > > Net::FTP=GLOB(0x8327c38)>>> SIZE debian-30r0-i386-binary-1.iso > Net::FTP=GLOB(0x8327c38)<<< 213 614780752 > > It too shows up as 612171776 in the ls command, so I am not sure what is > going on there. Ok. If the SIZE command says that on ftp.funet.fi, it is lying. That is the image I downloaded and checked the size and md5sum on. > > I also noticed that I have this structure: > > debian-cd/3.0_r0/images/i386 > > > > Should I remove the "images" dir? I am fairly certain that it ended up > > there during the discussions on jigdo-mirror and a standard structure for > > the release, but since there is no official master site, I can't be sure. > > > > Other mirrors seems to not have the "images" dir. > > I only find one or two with the images directory. IMHO it should be > removed for consistency. My script shall not default to looking for it. Ok. I'll wait a day or two to see if someone shouts that it should be left as it is, then I'll move it. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
On Fri, 29 Nov 2002, Joey Hess wrote: > Joey Hess wrote: > > Here is the current state of the cdrom mirror network. I decided not to > > try to track yada files since the web site only links to 2 mirrors for > > yada stuff. I added source cd checking. It only checks the first image > > for US and non-US. > > Correction, it checks for the first image binary and source. It does not > check for non-us stuff at all, though I could add it. A check for i386 (and perhaps source) or a full set could also be interesting for those users that aren't on i386. I think non-us is less important. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
On Sat, 30 Nov 2002, jason andrade wrote: > On Fri, 29 Nov 2002, Joey Hess wrote: > > > Many sites have old isos, or no isos, or a nonstandard directory > > structure or filenames, earning a 0 in the "binary" column. And of > > course a lot of mirrors don't have source. > > hmm, i have been skipping most of this thread. what new isos ? i > was under the impression (as of the last few years) that isos are > generated once for a release and unless there are some exceptional > circumstances are never regenerated without an increment of some > kind. New isos as in 3.0. Many sites just carry 2.2_r[67]. > at ftp.au.debian.org, we generated the images using jigdo and then > did a final sync against the "authoritative" images at cdimage.debian.org. > > we never check after that for various reasons > > o images are never supposed to change > o there could be some problem at cdimage which would wipe out our > local archive also Yes, this is the good method of handling cdimage mirroring. > so has there been some fundamental shift in the way debian is producing > iso images - as evidenced by the very high number of sites below which > get a 0 in the binary column ? Just that cdimage.debian.org does not carry the images, just the jigdo files. That is enough to get most sites not updated. /Mattias Wadenstein -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
Joey Hess <[EMAIL PROTECTED]> writes:
> A few places have question marks; this is where the file exists, but
> seems to have the wrong size. I think some/all of those are really ok,
> and are due to weirdness in the ftp SIZE command on some servers.
It may be that those servers are functioning correctly. Does you
client do something like $ftp->type("I"); to ask for non-translated
files?
ftp.fi.debian.org was one of the mirrors reporting weird size, so I
did some testing. Here is a transcript:
ftp> ls
229 Entering Extended Passive Mode (|||60002|)
150 Opening ASCII mode data connection for '/bin/ls'.
total 4907684
-rwxr-xr-x 1 deb deb797 Jul 20 19:41 MD5SUMS
-rwxr-xr-x 1 deb deb 612171776 Jul 20 19:31 debian-30r0-i386-binary-1.iso
ftp> size debian-30r0-i386-binary-1.iso
debian-30r0-i386-binary-1.iso 614780752
ftp> bin
200 Type set to I.
ftp> size debian-30r0-i386-binary-1.iso
debian-30r0-i386-binary-1.iso 612171776
When the type was set to binary, the response came immediately and was
correct. Before that, the command took a long time to complete, and I
noticed that the ftpd was taking all the CPU time it could get. I did
not check the sources, but I bet it was doing the line end translation
for ascii mode.
--
Heikki Vatiainen * [EMAIL PROTECTED]
Tampere University of Technology * Tampere, Finland
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
jason andrade wrote: > for us, the non us *is* the first binary image and source. we explicitly > only mirror the non us image and symlink to it as we are not a US site and > it doesn't seem to make any sense to carry two "image1" iso images where > one is a superset of the other. I still think that symlinking the non-us image to the filename of the us image is a bad idea for reproducability and bug reporting purposes (yes I can think of many scenarios where a non-us image would exhibit bugs that the us image doesn't have, or vice-versa). But I can defer to the list here, if that's really not a serious enough concern. -- see shy jo msg04888/pgp0.pgp Description: PGP signature
Re: state of the cdrom mirrors report
Heikki Vatiainen wrote:
> It may be that those servers are functioning correctly. Does you
> client do something like $ftp->type("I"); to ask for non-translated
> files?
I think you're right, I'm getting far fewer question marks in the report
after calling $ftp->binary. The remaining ones are probably non-us
symlinked images.
--
see shy jo
msg04889/pgp0.pgp
Description: PGP signature
Re: state of the cdrom mirrors report
On Sat, Nov 30, 2002 at 03:24:43PM +1000, jason andrade wrote: > > Many sites have old isos, or no isos, or a nonstandard directory ~ > > structure or filenames, earning a 0 in the "binary" column. And of ~ > > course a lot of mirrors don't have source. > > so has there been some fundamental shift in the way debian is producing > iso images - as evidenced by the very high number of sites below which > get a 0 in the binary column ? See above. One of my sites, iso.linux.hr, got a 0 in the binary column even though it does have the i386 3.0r0 set. (That's a LUG site which holds images for a bunch of distros. I didn't feel like creating a directory structure for something like that because it's only those few i386 binaries that are in there (no space or interest for other stuff) and because it would make Debian stand out, as other images are simply placed directly in the foo/ directory.) -- Joy -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
tools/boot/sarge/boot-i386 fails because of missing files
When making sarge images, tools/boot/sarge/boot-i386 fails because of missing files. The missing files are : cdrom-initrd.gz cdrom cdrom144-initrd.gz net-initrd.gz and the script tries to down load them from http://people.debian.org/~tfheen/d-i/images/daily/ . and, unfortunately they are no such files at that location. Question: 1.) will http://people.debian.org/~tfheen/d-i/images/daily/ be updated anytime soon ? 2.) should debian-cd depend on files that are not part of the main debian archive ? -- Jim Westveer :q :w :w! :quit :quit! :help help helpquit quit quithelp :quitnow :leave :shit ^X^C ^C ^D ^Z ^Q QUITDAMMIT ^[ :q! [EMAIL PROTECTED] +425-591-3002 pgp-key 0x36129171gpg-key 0x9823336C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
Josip Rodin wrote: > On Sat, Nov 30, 2002 at 03:24:43PM +1000, jason andrade wrote: > > > Many sites have old isos, or no isos, or a nonstandard directory >~ > > > structure or filenames, earning a 0 in the "binary" column. And of > ~ > > See above. One of my sites, iso.linux.hr, got a 0 in the binary column even > though it does have the i386 3.0r0 set. Yes, it does not have the images in the directory structure most mirrors use. It's impossible to check all the off-the-cuff structures out there, and I'm not gonna try. We also need a consistent structure to make a mirror multiplexor cgi work. I'll be doing that next.. > (That's a LUG site which holds images for a bunch of distros. I didn't feel > like creating a directory structure for something like that because it's > only those few i386 binaries that are in there (no space or interest for > other stuff) and because it would make Debian stand out, as other images are > simply placed directly in the foo/ directory.) mkdir -p 3.0_r0/i386 mv * 3.0_r0/i386 -- see shy jo msg04892/pgp0.pgp Description: PGP signature
Bug#171297: please add the various other OS binaries to the jigdo uploads via byhand files, or something of the sort
Package: jigdo Hi, Extra jigdo binaries should really be distributed via the tools/ directory on all Debian mirrors. This can be accomplished by having a source package upload include the necessary files as "byhand", i.e. with something like this: dpkg-distaddfile jigdo-bin-0.6.8.tar.bz2 byhand - dpkg-distaddfile jigdo-file_0.6.8-1_i386.deb byhand - dpkg-distaddfile jigdo-win-0.6.8.zip byhand - dpkg-distaddfile jigdo-bin-0.6.8-solaris.tar.bz2 byhand - That way, we could simply point to all the official mirrors directly from the www.d.o web pages, and the links would be a bit more straightforward. -- 2. That which causes joy or happiness. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: tools/boot/sarge/boot-i386 fails because of missing files
* Jim Westveer | 1.) will http://people.debian.org/~tfheen/d-i/images/daily/ | be updated anytime soon ? yes, in roughly 20 hours 30 minutes. Unless something is broken again, that is. | 2.) should debian-cd depend on files that are not part | of the main debian archive ? They will be part of the main debian archive, eventually. The procedure for handling that is Not Ready Yet. -- Tollef Fog Heen,''`. UNIX is user friendly, it's just picky about who its friends are : :' : `. `' `- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: state of the cdrom mirrors report
I've more or less funished up the program to check the status of our ftp
cdrom mirrors. My program generates a Mirrors.masterlist.status file,
with some fields to indicate the state of the mirrors. The fields are
chosen so we can run this program every few days, and stop pointing to a
mirror if it has been down for a given time period.
And I've modified mirror_list.pl to use the new fields to generate a list
that has only known-good mirrors on it. I took the approach of assuming
that a site's http mirror is good if its ftp mirror checks out ok.
As an example, I've attached a version of the cd image mirror list web
page generated with the above.
If all this looks good, I'll need help checking it into cvs, as I have
not had access to the website cvs since I stopped authoring DWN. I have
attached a diff against cvs for my changes, and a current copy of
check_cd_mirrors.pl and of Mirrors.masterlist.status (which should both
be put into webwml/english/mirror/).
I think we should set up a cron to run the check_cd_mirrors.pl every day
(or every 3 days, but it's not very expensive to run, though it does
take about an hour to complete) on gluck, and commit its changes to the
Mirrors.masterlist.status (which can be done on gluck best since the cvs
repository is there).
--
see shy jo
? english/mirror/Mirrors.masterlist.status
? english/mirror/check_cd_mirrors.pl
Index: english/CD/http-ftp/index.wml
===
RCS file: /cvs/webwml/webwml/english/CD/http-ftp/index.wml,v
retrieving revision 1.15
diff -u -r1.15 index.wml
--- english/CD/http-ftp/index.wml 16 Oct 2002 20:30:45 - 1.15
+++ english/CD/http-ftp/index.wml 30 Nov 2002 23:55:29 -
@@ -41,10 +41,12 @@
Official CD images of the "stable" releases
-
+
+
#use wml::debian::countries
#include "$(ENGLISHDIR)/CD/http-ftp/cdimage_mirrors.list"
Index: english/distrib/netinst.wml
===
RCS file: /cvs/webwml/webwml/english/distrib/netinst.wml,v
retrieving revision 1.6
diff -u -r1.6 netinst.wml
--- english/distrib/netinst.wml 25 Aug 2002 20:37:58 - 1.6
+++ english/distrib/netinst.wml 30 Nov 2002 23:55:30 -
@@ -14,9 +14,9 @@
There are two options for installs over the network:
- Minimal CD:
- Instead of getting a full 650MB CD image, you just download a CD image
- file which contains the bare essentials necessary to install the rest.
+ CD:
+ You download a CD image file. Minimal CD images are available, as is
+ a fast and efficient means to download a full CD from a mirror near you.
For the moment, it's necessary to have access to a CD recorder in order
to use this.
Floppy disks:
Index: english/mirror/mirror_list.pl
===
RCS file: /cvs/webwml/webwml/english/mirror/mirror_list.pl,v
retrieving revision 1.44
diff -u -r1.44 mirror_list.pl
--- english/mirror/mirror_list.pl 24 Nov 2002 18:50:19 - 1.44
+++ english/mirror/mirror_list.pl 30 Nov 2002 23:55:37 -
@@ -477,8 +477,9 @@
foreach $site (sort @{ $countries{$country} }) {
(my $countrycode = $country) =~ s/^(..) .*/$1/;
if ($which eq "httpftp") {
-if (defined $mirror{$site}{method}{'cdimage-ftp'} ||
-defined $mirror{$site}{method}{'cdimage-http'}) {
+if (defined $mirror{$site}{method}{'cdimage-ftp'} &&
+ (! defined $mirror{$site}{status}{'cdimage-ftp-status'} ||
+$mirror{$site}{status}{'cdimage-ftp-status'} eq 'good')) {
print "<${countrycode}c>: $site: ";
if (defined $mirror{$site}{method}{'cdimage-ftp'}) {
print <) {
+if (/Site:\s+(.*)/i) {
+my $site=$1;
+foreach my $line (split("\n", $_)) {
+my ($key, $value) = $line =~ /^(.*): (.*)/;
+ $key=lc($key);
+ $mirror{$site}{status}{$key}=$value;
+ }
+ }
+ }
}
# count the number of mirrors
Site: antirix.softnet.tuc.gr
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038694000
Site: carroll.aset.psu.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691559
Site: debian-cd.rutgers.edu
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038690879
Site: debian.ankara.edu.tr
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038690826
CDImage-ftp-lastgood: 1038690826
Site: debian.attica.net.nz
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038691778
Site: debian.balt.net
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038693633
Site: debian.blueyonder.co.uk
CDImage-ftp-status: bad
CDImage-ftp-lastchecked: 1038692014
Site: debian.efis.ucr.ac.cr
CDImage-ftp-status: down
CDImage-ftp-lastchecked: 1038696001
Site: debian.ens-cachan.fr
CDImage-ftp-status: good
CDImage-ftp-lastchecked: 1038689472
CDImag
