To wrap up my stack smashing tests, I ran a few tests on a Mac IIfx (128 MiB, external SCSI2SD). Serial console logs are attached.
1) Default Debian kernel (vmlinux-6.1.0-4-m68k, initrd-6.1.0-4-m68k) Default Debian sysvinit scripts Boot time (ABC... to login prompt): 15 min 3 sec NIC not detected, no stack smashing 2) Custom kernel (6.1.11-mac, no initrd) Default Debian sysvinit scripts Boot time (ABC... to login prompt): 9 min 29 sec NIC detected, stack smashing 3) Custom kernel (6.1.11-mac, no initrd) Custom sysvinit scripts Boot time (ABC... to login prompt): 3 min 59 sec NIC detected, no stack smashing Notes: a) Boot times Going from 15 min to about 4 min seems worth the effort on old hardware. As always, YMMV. Developers who use QEMU or other emulators likely don't always realize how long it takes to boot real hardware. I could have shaved off another 30 sec or so from each test by not starting sshd. The longest delays happened while waiting for systemd-udevd to populate /dev (which isn't needed anyway for a Mac IIfx, and this timed out on the IIci -- I didn't see an option for setting the timeout anywhere). b) NIC detection I'm not sure why the NIC isn't detected in test 1. It seems to be included in the config files. It was also not detected in the IIci using Debian's defaults. # fgrep 8390 config-6.1.0-4-m68k CONFIG_NET_VENDOR_8390=y CONFIG_MAC8390=m CONFIG_ZORRO8390=m # fgrep 8390 config-6.1.11-mac CONFIG_NET_VENDOR_8390=y CONFIG_MAC8390=y # grep 8390 *1.txt # grep 8390 *2.txt mac8390 slot.D: Memory length resource not found, probing mac8390 slot.D: MacCon Ethernet (type asante) mac8390 slot.D: MAC 00:00:94:07:e7:9f, IRQ 60, 32 KB shared memory at 0xfddd0000, 16-bit access. c) stack smashing For now, I've concluded that the stack smashing is not a kernel issue but is instead most likely caused by parts of one or more of the sysvinit scripts that are triggered by my config file options. I noticed earlier today (before upgrading the IIfx rootfs), that there was no stack smashing with my custom kernel and an older Debian SID, which included the following: orphan-sysvinit-scripts 0.11 sysvinit-core 3.01-2 sysvinit-utils 3.01-2 After upgrading, I see: orphan-sysvinit-scripts 0.13 sysvinit-core 3.06-2 sysvinit-utils 3.06-2 I could re-install the previous sysvinit over the version in the current Debian SID and see if the stack smashing is still gone. I don't know how to do that, but if someone has instructions, I'll try. I'm guessing I need to download the previous .deb binaries and use dpkg to install the older versions over the newer versions, while the newer init is still running; maybe rename /sbin/init to /sbin/init.tmp and boot with init=/sbin/init.tmp to get it out of the way (?). Or I could download the source for sysvinit-core 3.06-2 and sysvinit-utils 3.06-2 and compile using all of Debian's options plus -fstack-protector-all (and perhaps other options?) to see whether there might be a bounds issue on an array somewhere. But I also don't know where to find the source or the options that Debian uses for compilation. Otherwise, I'm done looking into the stack smashing for now. If anyone is interested in developing "68030-lean" kernel config options or custom sysvinit scripts, I'll be happy to contribute. thanks -Stan
maciifx_02152023-1.txt.xz
Description: Binary data
maciifx_02152023-2.txt.xz
Description: Binary data
maciifx_02152023-3.txt.xz
Description: Binary data
