Today I faced a issue which blowed my mind because if left no traces at
the affected system.
The OS: OpenBSD 4.5-STABLE, SMP
A process died and became a zombie process in a screen-session.
The process was irssi and thus not critical.
I tried to kill the zmbie-process without luck.
Thus I re-loged in and killed each ksh I ran.
Then I did a screen-x and my shell freezed.
I tried again to reconnect to the server but had no luck.
I got not even a ssh-fingerprint from the sshd anymore (removed
known_host to check) but connections where accapted by sshd (nmap scan
and others showed "open"9 and the ssh-connections made to it NEVER
pinged out.
Another server process using OpenSSL (silc) became NON-responsive too.
A field technican in the US went to the box and made a photo on which I
can see very strange symbol lines.
He was able to enter a username but the "password:"-field never
appeared.
I triggered the bug as I pressed "ctrl-c" during the reconnection because
the connection was slow and I wanted to kill it (client-side).
Could this be a indication for a OpenSSL bug and if so: How could I
might trace or -trigger it again?
there are NO core-files and nothing except "last" shows that the box
crashed:
root ttyp0 x.x.x.x Tue Sep 15 20:23 - 20:25
(00:02) reboot ~ Tue Sep 15 20:20
gandalf ttyp2 x.x.x.x Tue Sep 15 16:59 - crash (03:20)
The box came up normal after the fsck and I as able to log in as root.
Other services NOT using any OpenSSL-related stuff (like the
http-Server) where still fully functional.
Just:
- NO login worked; neither remotely or localy via keyboard
- SILC showed similiar sympthomes like SSH
-> accapted connections but keept them open and "empty". No full
handshake
I would appreciate any recommendation.
The only faulty thing I saw was the PSU not delivering straight 5V on a
5V line but that might has nothing to do with it. NO kernelpanic was
triggered and there is absolutely NOTHING in the logfiles.
Did anybody out there had such sympthomes before?
The box did NOT overheated nor has any HW problems I am aware off.
Best regards,
Gandalf
dmesg -without MAC addresses-:
OpenBSD 4.5-stable (GENERIC.MP) #0: Wed Aug 5 03:06:31 CEST 2009
r...@openbsd.local:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel(R) Pentium(R) 4 CPU 2.80GHz ("GenuineIntel" 686-class) 2.82 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
real mem = 527986688 (503MB)
avail mem = 502214656 (478MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 08/17/05, BIOS32 rev. 0 @ 0xfae40, SMBIOS
rev. 2.3 @ 0xf0100 (40 entries)
bios0: vendor Award Software International, Inc. version "FI" date 08/17/2005
bios0: Gigabyte Technology Co., Ltd. 8IG1000MK
acpi0 at bios0: rev 0
acpi0: tables DSDT FACP APIC
acpi0: wakeup devices SLPB(S5) HUB0(S4) USB0(S1) USB1(S1) USB2(S1) USB3(S1)
USBE(S1) PCI0(S4)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 200MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: Intel(R) Pentium(R) 4 CPU 2.80GHz ("GenuineIntel" 686-class) 2.82 GHz
cpu1:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID,xTPR
ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins
ioapic0: misconfigured as apic 0, remapped to apid 2
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (HUB0)
acpicpu0 at acpi0
acpicpu1 at acpi0
acpitz0 at acpi0: critical temperature 75 degC
acpibtn0 at acpi0: PWRB
acpibtn1 at acpi0: SLPB
bios0: ROM list: 0xc0000/0xa200! 0xcc000/0x8000! 0xd4000/0x1800
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "Intel 82865G Host" rev 0x02
vga1 at pci0 dev 2 function 0 "Intel 82865G Video" rev 0x02
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
intagp0 at vga1
agp0 at intagp0: aperture at 0xf0000000, size 0x8000000
inteldrm0 at vga1: apic 2 int 16 (irq 5)
drm0 at inteldrm0
uhci0 at pci0 dev 29 function 0 "Intel 82801EB/ER USB" rev 0x02: apic 2 int 16
(irq 5)
uhci1 at pci0 dev 29 function 1 "Intel 82801EB/ER USB" rev 0x02: apic 2 int 19
(irq 4)
uhci2 at pci0 dev 29 function 2 "Intel 82801EB/ER USB" rev 0x02: apic 2 int 18
(irq 11)
uhci3 at pci0 dev 29 function 3 "Intel 82801EB/ER USB" rev 0x02: apic 2 int 16
(irq 5)
ehci0 at pci0 dev 29 function 7 "Intel 82801EB/ER USB2" rev 0x02: apic 2 int 23
(irq 3)
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb0 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xc2
pci1 at ppb0 bus 1
fxp0 at pci1 dev 8 function 0 "Intel PRO/100 VE" rev 0x02, i82562: apic 2 int
20 (irq 10), address *cut*
inphy0 at fxp0 phy 1: i82562ET 10/100 PHY, rev. 0
ichpcib0 at pci0 dev 31 function 0 "Intel 82801EB/ER LPC" rev 0x02
pciide0 at pci0 dev 31 function 2 "Intel 82801EB SATA" rev 0x02: DMA, channel 0
configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 2 int 18 (irq 11) for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <ST380013AS>
wd0: 16-sector PIO, LBA48, 76318MB, 156299375 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5
ichiic0 at pci0 dev 31 function 3 "Intel 82801EB/ER SMBus" rev 0x02: apic 2 int
17 (irq 7)
iic0 at ichiic0
spdmem0 at iic0 addr 0x50: 512MB DDR SDRAM non-parity PC2100CL2.5
usb1 at uhci0: USB revision 1.0
uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb2 at uhci1: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci2: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci3: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at ichpcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
it0 at isa0 port 0x2e/2: IT8712F rev 5, EC port 0x290
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
mtrr: Pentium Pro MTRR support
softraid0 at root
root on wd0a swap on wd0b dump on wd0b
WARNING: / was not properly unmounted
