On 11/12/17 14:13, Otto Moerbeek wrote:
> On Sun, Nov 12, 2017 at 01:28:39PM -0500, Nick Holland wrote:
>
>> Help.
>>
>> I was upgrading a few very similar machines to -current today.
>> ONE of the three decided to be unpleasant. The thing has a
>> serial console, and but it is about 370km from me. :-/
>>
>> Upgrade from Sep 9 current to today's current via bsd.rd, just
>> like the other two.
>>
>> Upon reboot, it does this (from /boot) :
>>
>> booting hd0a:/bsd: 8484712+2429968+244048+0+667648 [636809heap full
>> (0x9d304+65536)
>>
>> And then reboots the system, as if from power-down/power-up.
>> (already something I haven't seen before)
>>
>> Reboot from "bsd.rd" and "bsd.sp", same results. reboot from "obsd"
>> (Sept 9), same results. Not a kernel problem, it seems. About this
>> point, I'm starting to think how the serial console has let me down.
>>
>> I remember how to bring up a DRAC remote CD image via ssh tunnels
>> to the drac and how to run java in a windows browser, and
>> reboot off the remote CD image, do another upgrade, all goes fine
>> (again), but upon reboot, same results... "heap full" and reboot.
>>
>> Boot from remote CD, at the boot> prompt, enter "boot hd0a:/bsd",
>> and it boots Just Fine from the local hard disk (only boot pulled
>> from the remote CD). Boot loader! Reinstalled boot:
>>
>> # installboot -v sd0
>> Using / as root
>> installing bootstrap on /dev/rsd0c
>> using first-stage /usr/mdec/biosboot, second-stage /usr/mdec/boot
>> copying /usr/mdec/boot to /boot
>> /boot is 3 blocks x 32768 bytes
>> fs block shift 3; part offset 64; inode block 24, offset 2088
>> master boot record (MBR) at sector 0
>> partition 3: type 0xA6 offset 64 size 2000397671
>> /usr/mdec/biosboot will be written at sector 64
>>
>> good, right?
>>
>> Reboot off local hard disk, boom. problem is still there. maybe
>> not the boot loader. :-/
>>
>> Verified /boot on trouble system and good system are the same.
>>
>> I'm not going to cry "bug", since there are two nearly identical
>> systems working just fine. But I can't think of what I did wrong
>> or what to do to fix it.
>>
>> Suggestions?
>
> You are hitting -DHEAP_LIMIT=0xA0000 in /boot. The code is in libsa/alloa.c
>
> No idea why. But something in that system is different.
>
> You do have one weird line in your disklabel output: a filesystem
> mounted on swap?
that's an mfs. This application has one directory which has a HUGE
benefit to an MFS for tmp files. Though the reboot happens long before
the mfs is created.
$ more /etc/fstab
cde728ba2c9bbe7.b none swap sw
ccde728ba2c9bbe7.a / ffs rw,noatime 1 1
ccde728ba2c9bbe7.h /home ffs rw,noatime,nodev,nosuid 1 2
ccde728ba2c9bbe7.e /tmp ffs rw,noatime,nodev,nosuid 1 2
ccde728ba2c9bbe7.d /usr ffs rw,noatime,nodev 1 2
ccde728ba2c9bbe7.f /var ffs rw,noatime,nodev,nosuid 1 2
ccde728ba2c9bbe7.g /repo ffs rw,noatime,nodev 1 2
ccde728ba2c9bbe7.i /repo/anoncvs/dev ffs rw,noatime,nosuid 1 2
/dev/sd0b /repo/anoncvs/tmp mfs rw,nodev,nosuid,-m=1,-s=3072000,-i=2048 0 0
> Can you boot into single user mode?
nope. Considering how fast the reboot happens, I wouldn't have expected
it to, unless something is very different very early in the boot process.
This is what happened:
On the console:
Using drive 0, partion 3.
Loading...
probing: pc0 com0 com1 mem[631K 3038M 2M 68K 72K 176k 64K 13312M a20=on]
disk: fd0 hd0+
>> OpenBSD/amd64 BOOT 3.33
switching console to com0
and then on the serial console:
>> OpenBSD/amd64 BOOT 3.33
boot> boot -s
booting hd0a:/bsd: 8484304+2429960+244080+0+667648 [643739heap full
(0x9d4fc+65536)
(boom. reboot)
here's a dmesg diff between the "good" and "bad" machines...
$ diff -u dmesg.good dmesg.bad
--- dmesg.good Sun Nov 12 14:51:30 2017
+++ dmesg.bad Sun Nov 12 14:51:21 2017
@@ -1,7 +1,7 @@
OpenBSD 6.2-current (GENERIC.MP) #203: Sat Nov 11 19:01:19 MST 2017
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 17131339776 (16337MB)
-avail mem = 16605302784 (15836MB)
+avail mem = 16605294592 (15836MB)
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
@@ -16,46 +16,46 @@
acpihpet0 at acpi0: 14318179 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
-cpu0: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3193.18 MHz
+cpu0: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3193.22 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu0: 256KB 64b/line 8-way L2 cache
-acpihpet0: recalibrated TSC frequency 3192750214 Hz
+acpihpet0: recalibrated TSC frequency 3192750287 Hz
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 99MHz
cpu0: mwait min=64, max=64, C-substates=0.2.1.1, IBE
cpu1 at mainbus0: apid 1 (application processor)
-cpu1: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu1: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu1: 256KB 64b/line 8-way L2 cache
cpu1: smt 1, core 0, package 0
cpu2 at mainbus0: apid 2 (application processor)
-cpu2: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu2: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu2: 256KB 64b/line 8-way L2 cache
cpu2: smt 0, core 1, package 0
cpu3 at mainbus0: apid 3 (application processor)
-cpu3: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu3: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu3: 256KB 64b/line 8-way L2 cache
cpu3: smt 1, core 1, package 0
cpu4 at mainbus0: apid 4 (application processor)
-cpu4: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu4: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu4: 256KB 64b/line 8-way L2 cache
cpu4: smt 0, core 2, package 0
cpu5 at mainbus0: apid 5 (application processor)
-cpu5: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu5: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu5: 256KB 64b/line 8-way L2 cache
cpu5: smt 1, core 2, package 0
cpu6 at mainbus0: apid 6 (application processor)
-cpu6: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu6: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu6:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu6: 256KB 64b/line 8-way L2 cache
cpu6: smt 0, core 3, package 0
cpu7 at mainbus0: apid 7 (application processor)
-cpu7: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.75 MHz
+cpu7: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz, 3192.76 MHz
cpu7:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,DEADLINE,AES,XSAVE,AVX,NXE,RDTSCP,LONG,LAHF,PERF,ITSC,SENSOR,ARAT
cpu7: 256KB 64b/line 8-way L2 cache
cpu7: smt 1, core 3, package 0
@@ -121,16 +121,15 @@
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 31 function 0 "Intel C202 LPC" rev 0x04
ahci0 at pci0 dev 31 function 2 "Intel 6 Series AHCI" rev 0x04: msi, AHCI 1.3
-ahci0: port 2: 3.0Gb/s
-ahci0: port 3: 3.0Gb/s
+ahci0: port 0: 3.0Gb/s
+ahci0: port 1: 1.5Gb/s
scsibus1 at ahci0: 32 targets
-sd0 at scsibus1 targ 2 lun 0: <ATA, Samsung SSD 850, EXM0> SCSI3 0/direct
fixed naa.50025388400562d4
+sd0 at scsibus1 targ 0 lun 0: <ATA, Samsung SSD 850, EXM0> SCSI3 0/direct
fixed naa.50025388400563fe
sd0: 976762MB, 512 bytes/sector, 2000409264 sectors, thin
-sd1 at scsibus1 targ 3 lun 0: <ATA, Samsung SSD 850, EXM0> SCSI3 0/direct
fixed naa.5002538c70007b02
-sd1: 1953514MB, 512 bytes/sector, 4000797360 sectors, thin
+cd0 at scsibus1 targ 1 lun 0: <PLDS, DVD+-RW DS-8A8SH, KD51> ATAPI 5/cdrom
removable
ichiic0 at pci0 dev 31 function 3 "Intel 6 Series SMBus" rev 0x04: apic 0 int
19
iic0 at ichiic0
-sdtemp0 at iic0 addr 0x18: mcp98243
+sdtemp0 at iic0 addr 0x18: stts2002
sdtemp1 at iic0 addr 0x19: mcp98243
sdtemp2 at iic0 addr 0x1a: mcp98243
sdtemp3 at iic0 addr 0x1b: stts2002
@@ -141,7 +140,6 @@
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
-com0: console
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5 irq 1 irq 12
pcppi0 at isa0 port 0x61
@@ -155,14 +153,23 @@
uhidev1: iclass 3/1
ums0 at uhidev1: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
+umass0 at uhub2 port 2 configuration 1 interface 0 "Avocent USB Composite
Device-1" rev 2.00/0.00 addr 4
+umass0: using SCSI over Bulk-Only
+scsibus2 at umass0: 2 targets, initiator 0
+sd1 at scsibus2 targ 1 lun 0: <iDRAC, LCDRIVE, 0323> SCSI0 0/direct removable
+umass1 at uhub2 port 2 configuration 1 interface 1 "Avocent USB Composite
Device-1" rev 2.00/0.00 addr 4
+umass1: using SCSI over Bulk-Only
+scsibus3 at umass1: 2 targets, initiator 0
+cd1 at scsibus3 targ 1 lun 0: <iDRAC, Virtual CD, 0323> SCSI0 5/cdrom removable
+sd2 at scsibus3 targ 1 lun 1: <iDRAC, Virtual Floppy, 0323> SCSI0 0/direct
removable
uhub3 at uhub1 port 1 configuration 1 interface 0 "Intel Rate Matching Hub"
rev 2.00/0.00 addr 2
uhub4 at uhub3 port 5 configuration 1 interface 0 "Standard Microsystems
product 0x2514" rev 2.00/0.00 addr 3
vscsi0 at root
-scsibus2 at vscsi0: 256 targets
+scsibus4 at vscsi0: 256 targets
softraid0 at root
-scsibus3 at softraid0: 256 targets
-root on sd0a (ff6add5e908e72c7.a) swap on sd0b dump on sd0b
-bnx0: address d4:ae:52:b9:6a:80
+scsibus5 at softraid0: 256 targets
+root on sd0a (ccde728ba2c9bbe7.a) swap on sd0b dump on sd0b
+bnx0: address d4:ae:52:b9:6a:10
brgphy0 at bnx0 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
-bnx1: address d4:ae:52:b9:6a:81
+bnx1: address d4:ae:52:b9:6a:11
brgphy1 at bnx1 phy 1: BCM5709 10/100/1000baseT PHY, rev. 8
Nick.
