Hello all Thanks to Nicholas hints, I have managed to make the script skip the packages that are producing errors; in particular, with these commits:
https://salsa.debian.org/l10n-team/dl10n/commit/ded6ee5874700ddeb5faec86cc16c52ff8f1e4f7 https://salsa.debian.org/l10n-team/dl10n/commit/1af15de8b55d0058d7904d91e7c3f3a14488539c and now the gen-material script go to the end, but fails when trying to write the data: Unable to write to /srv/i18n.debian.org//dl10n/git/../data/gen-material/data/unstable.gz: Cannot allocate memory Same happens with the gen-material call for testing: Unable to write to /srv/i18n.debian.org//dl10n/git/../data/gen-material/data/testing.gz: Cannot allocate memory I've had a look at the code and I cannot find any place where the scripts are called with a limitation for memory. If I am not wrong, this is the stack of calls: 1.- cron: 25 14 * * * run-parts --report /srv/i18n.debian.org/etc/cron.d/ 2.- cron.d/10gen-material-unstable #!/bin/bash /srv/i18n.debian.org/dl10n/git/cron/gen-material unstable 3.- gen-material unstable gzip -dc $MIRRORDIR/dists/$d/main/source/Sources.gz \ $MIRRORDIR/dists/$d/contrib/source/Sources.gz \ $MIRRORDIR/dists/$d/non-free/source/Sources.gz \ | grep-dctrl -n -s Directory,Files '' \ | sed -n -e '/^pool\//{N;N;s/\n.* /\//g;p;}' \ | sed -e "s,^,$MIRRORDIR/," \ | grep -Ev "/${IGNMATERIAL}/" \ | $DL10NDIR/dl10n-check \ --remove-unused \ --files-from=- \ --tmp=$TMPDIR \ --db=$OUTDIR/data/$d.gz \ --po=$OUTDIR/po/$d \ --templates=$OUTDIR/templates/$d \ --menu=$OUTDIR/menu/$d $* >> $LOGPREFIX.log 2>> $LOGPREFIX.err 4.- dl10n-check L326: $data->write($DB_FILE); 5.- lib/Debian/L10n/Db.pm L259 (in sub write): if ($file =~ m/\.gz$/) { open (DB,"| gzip -c > $file") || die "Unable to write to $file: $!\n"; And this is when the program stops with the error "Cannot allocate memory". What can we do? I've looked at tye.debian.org $ free -h total used free shared buff/cache available Mem: 2,0G 695M 91M 5,9M 1,2G 1,1G Swap: 511M 151M 360M (But I don't know how to monitor memory while the script is running). The resultant (written) files shouldn't be very big (the current, outdated ones are): [ ] testing.gz 2018-04-09 14:33 17K [ ] unstable.gz 2017-11-23 14:33 3.9M So I don't know if there is a problem in the resources in tye, or a problem in our scripts. I have manually run the gen-material unstable with --verbose and --debug, and those produced huge log/err files, but couldn't find any more detail about the current problem. Any hint? Should I file an RT ticket for DSA to help us? -- Laura Arjona Reina https://wiki.debian.org/LauraArjona
