On Mon, 27 Jan 2025 14:02:17 +0000 Chris Billington wrote: > I am setting up net-booting of amd64 clients with an amd64 server > (HP Z400) running 7.6-release, using the diskless(8) manpage. > > I want to set up a shared /usr nfs mount for the clients as described > in the manpage, so that a single set of installed packages can be > managed centrally. > > tftpd, bootparams, dhcpd and nfs are set up as decribed in the mapage > > Server IP (em0) 192.168.0.254 > NFS rootfs /export is on a separate partition. > > Client IP (also em0) 192.168.0.2 as set statically by mac address in > dhcpd.conf > > /etc/exports on the server: > /export/client2 -maproot=root -alldirs 192.168.0.2 > /usr -ro -network=192.168.0.0 -netmask=255.255.255.0 > /var/db/pkg -ro -network=192.168.0.0 -netmask=255.255.255.0 > > showmount -e: > /var/db/pkg 192.168.0.0 > /usr 192.168.0.0 > /export/client2 192.168.0.2 > > /etc/fstab on client2: > 192.168.0.254:/export/client2 / nfs rw 0 0 > 192.168.0.254:/usr/ nfs ro 0 0 > swap /tmp mfs rw,-s=512M 0 0 > 192.168.0.254:/var/db/pkg /var/db/pkg nfs 0 0 > > When booting multi-user, all goes normally until the boot hangs at the > point of mounting /usr in the client's /etc/rc (line 489): > > mount -s /usr >/dev/null 2>&1 # if NFS, fstab must use IP address > > By removing the redirection temporarily, I can see the following error > on the client: > > mount_nfs: bad MNT RPC: RPC: Unable to send; errno = Permission denied > > This is repeated at intervals of 30 seconds or so. > > However, showmount -a on the server thinks /usr is mounted: > > showmount -a: > client2:/export/client2 > client2:/usr > client2:/var/db/pkg > > At this point if I interrupt the processing of /etc/rc the boot > continues, but fails miserably because /usr is not mounted. (verified > with mountd -d on the server) > > If I do 'boot -s', after going to a shell it is possible to > mount /usr, /tmp and /var/db/pkg without issue. > > If I add the bg (backgrouund the mount task) option to the client's > fstab for /usr (ro,bg) then boot proceeds but /usr never gets mounted. > > As a check, I tried booting with a non-shared /usr in > the /export/client2 directory. Booting then works without problems. But > that defeats the object of net booting, to have a shared set of > installed packages. > > One strange thing that may be relevant: > If I listen with 'tcpdump -nvi em0' on the server, I can see the rpc > request going to the server port 111 over udp each time the client > attempts to mount /usr : > > 192.168.0.2.xxx > 192.168.0.254.111: [udp sum ok] udp 56 (ttl 64, id > xxxxx, len 84) > > But the reply back to the client from the server from the same port has: > > 192.168.0.254.111 > 192.168.0.2.xxx: [bad udp csum 8682! -> zzzz]] udp > 28 (ttl 64, id xxxxx, len 56) > > (xxxx, yyyy, zzzz are the random values chosen by the networking stack) > > Is it still possible to boot diskless clients with a shared /usr? What > could be the cause of the 'bad UDP csum' errors, and the 'mount_nfs bad > MNT RPC' error? > It's particularly odd because single-user boot allows /usr to be > mounted read-only without issue. > > I'm running out of things to try! All assistance in resolving this > gratefully accepted.... > > Possibly unrelated: before the boot process of /bsd starts, I see the > following PXEboot error flash by: > pxe_netif_open : PXENV_UDP_OPEN failed: 0x60 > net_open: netif_open() failed > However, after this the booting of /bsd continues as normal until the > nfs mount hang described above. > > -- > Chris Billington
Further infomation: - the bad checksum errors are a 'red herring', they occur because the network card supports checksum offloading and tcpdump sees the packets before the checksum is added. - changing the mount options for /usr to 'ro,tcp' results in slightly different error messages on the client when booting multiuser and trying to mount /usr readonly: first error: Cannot MNT RPC: RPC: Remote system error: Permission denied subsequent errors: mount_nfs: Bad MNT RPC: RPC: Unable to send; errno = Bad file descriptor -- Chris Billington
