On Sat, Jan 3, 2009 at 4:27 PM, Ed Ahlsen-Girard <[email protected]> wrote:
>
> But my real problem was getting the download to work inside a script,
> and none of the presented ideas so far have helped that.
>
A simple shell alternative maybe?
---------------------------------------------------------
#!/bin/sh
# -- retrieve snapshot files with a ftp pipeline
DESTDIR=SNAP
mkdir -p ${DESTDIR}
SITE=ftp.caly.nl
SITE=ftp.eu.openbsd.org
DIR=/pub/OpenBSD/snapshots/i386
FILES="
MD5
pxeboot
"
for THIS in $FILES ; do
echo "Getting file: ${THIS}..."
echo "get ${DIR}/${THIS} ${DESTDIR}/${THIS}" | ftp -4ai ${SITE}
# remote_filename local_filename
done
ls -l ${DESTDIR}
---------------------------------------------------------
Disadvantage of this type of solution is the repeating ftp logins. If
you are "user 99 out of 100 allowed connections" for the first file,
you could be locked out haflway if you happen
to become user 101 ;)
ftp(1) is rather scriptable if you use a ".netrc" file, unfortunately
rather unknown, and thus not beloved..
The following shell script creates a ".netrc" to do the actual work in
a single ftp session.
---------------------------------------------------------
d
#!/bin/sh
# $Id: snapget,v 1.2 2009/01/04 00:38:09 j65nko Exp $
# get snapshot files with a on the fly generated '.netrc' for ftp(1)
# variables used in 'netrc' template
# ------------------------------------------------------------------
ARCH='i386' # architecture name used as ftp dir
RN=44 # used in "base${RN}.tgz" etc
#VERSION='4.4' # ftp dir for releases
VERSION='snapshots' # ftp dir for snapshots
DIR="/pub/OpenBSD/${VERSION}/${ARCH}"
SITE='ftp.calyx.nl'
SITE='ftp.eu.openbsd.org'
# customizable file sets
# ------------------------------------------------------------------
#SETS_START
base="
MD5
base${RN}.tgz
bsd
bsd.mp
bsd.rd
comp${RN}.tgz
etc${RN}.tgz
man${RN}.tgz
misc${RN}.tgz
"
x="
xbase${RN}.tgz
xetc${RN}.tgz
xfont${RN}.tgz
xserv${RN}.tgz
xshare${RN}.tgz
"
mini="
MD5
base${RN}.tgz
bsd
bsd.rd
etc${RN}.tgz
"
pxe="
pxeboot
"
floppy="
floppy${RN}.fs
floppyB${RN}.fs
floppyC${RN}.fs
"
ALLSETS="base x mini pxe floppy"
#SETS_END
# ------------------------------------------------------------------
if [ $# -eq 0 ] ; then
echo "$0: Please specify one or more sets"
echo "The sets are: ${ALLSETS}"
exit 1
fi
# -- accumulate all sets in FILES
for SET in "$@" ; do
eval FILES=\"\${FILES} \$${SET}\"
done
# echo ${FILES} | tr '[:blank:]' '\n'
# -- destination directory
# DESTDIR="SNAP$(date -u "+%Y-%m-%d_%H:%M_UTC")"
# mkdir ${DESTDIR} || exit 1
DESTDIR="SNAP"
mkdir -p SNAP
#echo "Destination directory: ${DESTDIR}"
# --- 'here document' as '.netrc' template
cat <<-TEMPLATE >${DESTDIR}/.netrc
machine ${SITE} login anonymous password [email protected]
macdef init
prompt off
epsv4 off
preserve on
progress on
$( for THIS in ${FILES} ; do
echo "get ${DIR}/${THIS} ${DESTDIR}/${THIS}"
done
)
quit
TEMPLATE
# -- An empty line is the terminator for a macdef macro
# -- So KEEP EMPTY LINE between 'quit' and here document marker 'TEMPLATE'
# -- to prevent 'Macro definition missing null line terminator' messages
# -- from 'ftp'
echo -------------- Created '.netrc'-----------------------
cat -n $DESTDIR/.netrc
echo ------------------------------------------------------
echo "Do you want to start 'ftp' with this '.netrc' ? (Y/N)" ; read answer
case "$answer" in
[Yy] ) ;;
* ) echo "$0: I take this for a 'NO', aborting ..."
exit 2
;;
esac
# -- ftp uses HOME to locate '.netrc'
env HOME=${DESTDIR} ftp -4 ${SITE} 2>&1 | tee ${DESTDIR}/Logfile
# -- EOF
-----------------------------------------------------------------------------------
Example of usage:
$ snapget
./snapget: Please specify one or more sets
The sets are: base x mini pxe floppy
$ snapget base x
-------------- Created .netrc-----------------------
1
2 machine ftp.eu.openbsd.org login anonymous password [email protected]
3
4 macdef init
5 prompt off
6 epsv4 off
7 preserve on
8 progress on
9 get /pub/OpenBSD/snapshots/i386/MD5 SNAP/MD5
10 get /pub/OpenBSD/snapshots/i386/base44.tgz SNAP/base44.tgz
11 get /pub/OpenBSD/snapshots/i386/bsd SNAP/bsd
12 get /pub/OpenBSD/snapshots/i386/bsd.mp SNAP/bsd.mp
13 get /pub/OpenBSD/snapshots/i386/bsd.rd SNAP/bsd.rd
14 get /pub/OpenBSD/snapshots/i386/comp44.tgz SNAP/comp44.tgz
15 get /pub/OpenBSD/snapshots/i386/etc44.tgz SNAP/etc44.tgz
16 get /pub/OpenBSD/snapshots/i386/man44.tgz SNAP/man44.tgz
17 get /pub/OpenBSD/snapshots/i386/misc44.tgz SNAP/misc44.tgz
18 get /pub/OpenBSD/snapshots/i386/xbase44.tgz SNAP/xbase44.tgz
19 get /pub/OpenBSD/snapshots/i386/xetc44.tgz SNAP/xetc44.tgz
20 get /pub/OpenBSD/snapshots/i386/xfont44.tgz SNAP/xfont44.tgz
21 get /pub/OpenBSD/snapshots/i386/xserv44.tgz SNAP/xserv44.tgz
22 get /pub/OpenBSD/snapshots/i386/xshare44.tgz SNAP/xshare44.tgz
23 quit
24
------------------------------------------------------
Do you want to start 'ftp' with this '.netrc' ? (Y/N)
Y
=Adriaan=
