On Mon, 3 Apr 2023, Michael Schmitz wrote: > Am 02.04.2023 um 21:31 schrieb Finn Thain: > > > >> > >> Maybe an interaction between (multiple?) signals and syscall > >> return... > > > > When running dash from gdb in QEMU, there's only one signal (SIGCHLD) > > and it gets handled before __wait3() returns. (Of course, the "stack > > smashing detected" failure never shows up in QEMU.) > > Might be a clue that we need multiple signals to force the stack > smashing error. And we might not get that in QEMU, due to the faster > execution in emulating on a modern processor. >
Right -- being that the failure is intermittent on real hardware, it's not surprising that I can't make it show up in QEMU or Aranym. But no-one has reproduced it on Atari or Amiga hardware yet so I guess it could be a driver issue... I wonder whether anyone else is actually running recent Debian/SID with sysvinit and without a Debian initrd on a Motorola 68030 system. > Thinking a bit more about interactions between signal delivery and > syscall return, it turns out that we don't check for pending signals > when returning from a syscall. That's OK on SMP systems, because we > don't have another process running while we execute the syscall (and we > _do_ run signal handling when scheduling, i.e. when wait4 sleeps or is > woken up)? > > Seems we can forget about that interaction then. > > > > >> depends on how long we sleep in wait4, and whether a signal happens > >> just during that time. > >> > > > > I agree, there seems to be a race condition there. (And dash's > > waitproc() seems to take pains to reap the child and handle the signal > > in any order.) > > Yes, it makes sure the SIGCHLD is seen no matter in what order the > signals are delivered ... > > > I wouldn't be surprised if this race somehow makes the failure rare. > > > > I don't want to recompile any userland binaries at this stage, so it > > would be nice if we could modify the kernel to keep track of exactly > > how that race gets won and lost. Or perhaps there's an easy way to rig > > the outcome one way or the other. > > A race between syscall return due to child exit and signal delivery > seems unlikely, but maybe there is a race between syscall return due to > a timer firing and signal delivery. Are there any timers set to > periodically interrupt wait3? > I searched the source code and SIGALRM appears to be unused by dash. And 'timeout' is not a dash builtin. But that doesn't mean we don't get multiple signals. One crashy script looks like this: TMPFS_SIZE="$(tmpfs_size_vm "$TMPFS_SIZE")" RUN_SIZE="$(tmpfs_size_vm "$RUN_SIZE")" LOCK_SIZE="$(tmpfs_size_vm "$LOCK_SIZE")" SHM_SIZE="$(tmpfs_size_vm "$SHM_SIZE")" TMP_SIZE="$(tmpfs_size_vm "$TMP_SIZE")" Is it possible that the SIGCHLD from the first sub-shell got delayed? > > Still no nearer to a solution - something smashes the stack near %sp, > causes the %a3 register restore after __GI___wait4_time64 to return a > wrong pointer to the stack canary, and triggers a stack smashing warning > in this indirect way. But what?? > I've no idea. The actual corruption might offer a clue here. I believe the saved %a3 was clobbered with the value 0xefee1068 which seems to be a pointer into some stack frame that would have come into existence shortly after __GI___wait4_time64 was called. That stack frame is gone by the time the core dump was made. Was it dash's signal handler, onsig(), or some libc subroutine called by __GI___wait4_time64(), or was it something that the kernel put there? Dash's SIGCHLD handler looks safe enough -- I don't see how it could corrupt the saved registers in the __GI___wait4_time64 stack frame (it's not like 1 was stored in the wrong place). https://sources.debian.org/src/dash/0.5.12-2/src/trap.c/?hl=285#L285
