I have several routers that have been running great for many months. (even better since I upgraded to 4.1 on them oround May 4th)
OpenBSD 4.1-stable (GENERIC.MP) #0: Fri May 4 21:56:51 MST 2007 This morning, one of them went down and nagios paged me. Getting to work, I just thought it was odd, looked at the trace and restarted it and went home. About half an hour later, it happened again. I again thought it weird and restarted it. Then, about an hour and a half later it did it again and I captured a ps and trace and within a few minutes, while I was still logging and watching, it ddb'ed again and I got another ps and trace. (see the end of the email for the log with the dmesg in between) Now it appears to be a real problem, but I am not sure what would be causing it. I haven't changed anything on the routers in several days and that was only adding a script to ping some hosts regularly to crontab. That is the only change since upgrading. The other router which is the same hardware (but connected to different lines) has been fine. It sounds similar to these messages, but they were not enough to give me ideas of what could be causing it. http://marc.info/?l=openbsd-bugs&m=117855291725222&w=2 http://marc.info/?l=openbsd-misc&m=117204993326219&w=2 I see "san2: LCP keepalive timeout" output to the console and the below in /var/log/messages right around when it locked up but the traces don't seem to have anything to do with the network. Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: T1 YELLOW ON Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: T1 disconnected! Jun 21 06:05:05 rrlhcrtr0200 /bsd: san3: Link connecting... Jun 21 06:40:56 rrlhcrtr0200 syslogd: start Jun 21 07:15:44 rrlhcrtr0200 /bsd: san3: T1 LB activation code received. Jun 21 07:15:44 rrlhcrtr0200 /bsd: san3: Unknown signal (15). Jun 21 07:15:59 rrlhcrtr0200 /bsd: san3: T1 LB deactivation code received. Jun 21 07:16:04 rrlhcrtr0200 /bsd: san3: T1 LB deactivation code received. Jun 21 07:16:08 rrlhcrtr0200 /bsd: san3: Unknown signal (09). Jun 21 07:16:08 rrlhcrtr0200 /bsd: san3: Unknown signal (15). Jun 21 07:16:15 rrlhcrtr0200 /bsd: san2: T1 LB activation code received. Jun 21 07:16:15 rrlhcrtr0200 /bsd: san2: Unknown signal (15). Jun 21 07:16:39 rrlhcrtr0200 /bsd: san2: T1 LB deactivation code received. Jun 21 08:08:49 rrlhcrtr0200 syslogd: start Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: T1 AIS ON Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: T1 disconnected! Jun 21 09:26:53 rrlhcrtr0200 /bsd: san2: Link connecting... Jun 21 09:26:54 rrlhcrtr0200 /bsd: san2: T1 RED ON Jun 21 09:30:51 rrlhcrtr0200 syslogd: start Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: T1 AIS ON Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: T1 disconnected! Jun 21 09:39:51 rrlhcrtr0200 /bsd: san2: Link connecting... Jun 21 09:39:52 rrlhcrtr0200 /bsd: san2: T1 RED ON Jun 21 09:50:21 rrlhcrtr0200 syslogd: start Jun 21 09:50:21 rrlhcrtr0200 syslogd: start Here is the output from ifconfig if it could prove useful: $ ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33224 groups: lo inet 127.0.0.1 netmask 0xff000000 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0xa fxp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:b0:d0:20:8a:b1 groups: internal media: Ethernet autoselect (100baseTX full-duplex) status: active inet 66.185.224.2 netmask 0xffffffe0 broadcast 66.185.224.31 inet6 fe80::2b0:d0ff:fe20:8ab1%fxp0 prefixlen 64 scopeid 0x1 fxp1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:50:8b:5e:e7:ac media: Ethernet autoselect (none) status: no carrier fxp2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 lladdr 00:50:8b:5e:e7:ad media: Ethernet autoselect (100baseTX full-duplex) status: active inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255 inet6 fe80::250:8bff:fe5e:e7ad%fxp2 prefixlen 64 scopeid 0x3 san0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500 description: T1 to Sprint groups: sprint external media: TDM t1 inet 144.228.193.62 --> 144.228.193.61 netmask 0xfffffffc inet6 fe80::2b0:d0ff:fe20:8ab1%san0 -> prefixlen 64 scopeid 0x4 san1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500 description: T1 to Sprint groups: sprint external media: TDM t1 inet 144.228.193.74 --> 144.228.193.73 netmask 0xfffffffc inet6 fe80::2b0:d0ff:fe20:8ab1%san1 -> prefixlen 64 scopeid 0x5 san2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500 description: T1 to AT&T sppp: phase network groups: att external media: TDM t1 ppp inet 12.124.16.22 --> 12.124.16.21 netmask 0xfffffffc inet6 fe80::2b0:d0ff:fe20:8ab1%san2 -> prefixlen 64 scopeid 0x6 san3: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500 description: T1 to AT&T sppp: phase network groups: att external media: TDM t1 ppp inet 12.124.17.122 --> 12.124.17.121 netmask 0xfffffffc inet6 fe80::2b0:d0ff:fe20:8ab1%san3 -> prefixlen 64 scopeid 0x7 pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33224 enc0: flags=0<> mtu 1536 lo1: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33224 description: Interface for BGP with Sprint (AS1239) groups: lo sprint inet 66.185.239.55 netmask 0xffffffff lo2: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33224 description: Interface for BGP with AT&T (AS7018) groups: lo att inet 12.105.248.57 netmask 0xffffffff pfsync0: flags=0<> mtu 1460 pfsync: syncdev: fxp2 syncpeer: 224.0.0.240 maxupd: 128 groups: carp pfsync =~=~=~=~=~=~=~=~=~=~=~= PuTTY log 2007.06.21 08:23:31 =~=~=~=~=~=~=~=~=~=~=~= san2: LCP keepalive timeout kernel: page fault trap, code=0 Stopped at 0: kernel: page fault trap, code=0 Stopped at db_read_bytes+0x14: movb 0(%edx),%al ddb{0}> sh panic the kernel did not panic ddb{0}> ps PID PPID PGRP UID S FLAGS WAIT COMMAND 17044 1 17044 0 3 0x2004082 ttyin getty 13851 1 13851 0 3 0x2004082 ttyin getty 28948 1 28948 0 3 0x2004082 ttyin getty 25392 1 25392 0 3 0x2004082 ttyin getty 19205 1 19205 0 3 0x2004082 ttyin getty 16033 1 16033 0 3 0x2004082 ttyin getty 3601 1 3601 0 3 0x2000080 select cron 22773 1 18569 0 3 0x2000080 select snmpd 14640 1 14640 0 3 0x2040180 select sendmail 9088 1 9088 0 3 0x2000080 select sshd 7879 1 7879 0 3 0x2000180 select inetd 1374 28790 28790 75 3 0x2000180 poll bgpd 19622 28790 28790 75 3 0x2000180 poll bgpd 28790 1 28790 0 2 0x2000080 bgpd 25120 7271 7271 83 3 0x2000180 poll ntpd 7271 1 7271 0 3 0x2000080 poll ntpd 27666 13983 13983 74 3 0x2000180 bpf pflogd 13983 1 13983 0 3 0x2000080 netio pflogd 31209 24737 24737 73 2 0x2000180 syslogd 24737 1 24737 0 3 0x2000088 netio syslogd 14 0 0 0 3 0x2100200 crypto_wa crypto 13 0 0 0 3 0x2100200 aiodoned aiodoned 12 0 0 0 3 0x2100200 syncer update 11 0 0 0 3 0x2100200 cleaner cleaner 10 0 0 0 3 0x100200 reaper reaper 9 0 0 0 3 0x2100200 pgdaemon pagedaemon 8 0 0 0 3 0x2100200 pftm pfpurge 7 0 0 0 3 0x2100200 wait wskbd_hotkey 6 0 0 0 3 0x2100200 usbtsk usbtask 5 0 0 0 3 0x2100200 usbevt usb0 4 0 0 0 3 0x2100200 timeout sensors 3 0 0 0 3 0x2100200 slacking scsi 2 0 0 0 3 0x2100200 kmalloc kmthread 1 0 1 0 3 0x2004080 wait init 0 -1 0 0 3 0x2080200 scheduler swapper ddb{0}> trace db_read_bytes(0,1,e7f2fd5c,2,0) at db_read_bytes+0x14 db_get_value(0,1,0,d067dbc3,0) at db_get_value+0x19 db_disasm(0,0,d033f310,0,50) at db_disasm+0x1d db_print_loc_and_inst(0,e7f2fe14,e7f2fe2c,d0473534,0) at db_print_loc_and_inst+ 0x2d db_trap(6,0,e7f2fe4c,d04642dd,1) at db_trap+0x75 kdb_trap(6,0,e7f2fe94,50) at kdb_trap+0xe8 trap() at trap+0xa1 --- trap (number 6) --- (null)(0,d1229240,0,e7f2e000,0) at 0 softclock(e7f20058,e7f20010,10,e7f20010,e7f2e000) at softclock+0x22c Bad frame pointer: 0xe7f2ff20 ddb{0}> machine ddb 1 Stopped at Debugger+0x4: leave ddb{1}> trace Debugger(d122cc00,0,0,d6c8ed14,0) at Debugger+0x4 i386_ipi_handler(b0,58,10,d0790010,e7f30010) at i386_ipi_handler+0x57 Xintripi() at Xintripi+0x47 --- interrupt --- i386_softintlock(0,58,10,d0200010,10) at i386_softintlock+0x65 Xintrltimer() at Xintrltimer+0x47 --- interrupt --- apm_cpu_idle(0,0,0,0,0) at apm_cpu_idle+0x4a ddb{1}> boot sync >> OpenBSD/i386 BOOT 2.10 boot> booting hd0a:/bsd: \|/-\|/-5611032\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/+882424 [52+286400-\|/-\|/-\|/-\|/-+266500\|/-\|/-\|/-\|/-\]=0x6b867c entry point at 0x200120* [ using 553324 bytes of bsd ELF symbol table ] Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. Copyright (c) 1995-2007 OpenBSD. All rights reserved. http://www.OpenBSD.org OpenBSD 4.1-stable (GENERIC.MP) #0: Fri May 4 21:56:51 MST 2007 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP cpu0: Intel Pentium III ("GenuineIntel" 686-class) 732 MHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 536436736 (523864K) avail mem = 481710080 (470420K) using 4278 buffers containing 26943488 bytes (26312K) of memory mainbus0 (root) bios0 at mainbus0: AT/286+ BIOS, date 08/04/03, BIOS32 rev. 0 @ 0xffe90, SMBIOS rev. 2.3 @ 0xfafc0 (51 entries) bios0: Dell Computer Corporation PowerEdge 2450 pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000 pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfc2c0/144 (7 entries) pcibios0: PCI Interrupt Router at 000:15:0 ("ServerWorks OSB4" rev 0x00) pcibios0: PCI bus #1 is the last bus bios0: ROM list: 0xc0000/0x8000 0xc8000/0x6000 0xec000/0x4000! acpi at mainbus0 not configured mainbus0: Intel MP Specification (Version 1.4) cpu0 at mainbus0: apid 1 (boot processor) cpu0: apic clock running at 132 MHz cpu1 at mainbus0: apid 0 (application processor) cpu1: Intel Pentium III ("GenuineIntel" 686-class) 732 MHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE mainbus0: bus 0 is type PCI mainbus0: bus 1 is type PCI mainbus0: bus 2 is type PCI mainbus0: bus 3 is type PCI mainbus0: bus 4 is type ISA ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 16 pins ioapic0: misconfigured as apic 0, remapped to apid 2 ioapic1 at mainbus0: apid 3 pa 0xfec01000, version 11, 16 pins ioapic1: misconfigured as apic 0, remapped to apid 3 esm0 at mainbus0 esm0: PowerEdge 2450 Embedded Server Management 5.24 esm0: Primary System Backplane 1.16 pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 "ServerWorks CNB20LE Host" rev 0x05 pchb1 at pci0 dev 0 function 1 "ServerWorks CNB20LE Host" rev 0x05 pci1 at pchb1 bus 2 ppb0 at pci1 dev 2 function 0 "Intel i960 RM PCI-PCI" rev 0x01 pci2 at ppb0 bus 3 ahc0 at pci2 dev 4 function 0 "Adaptec AIC-7899 U160" rev 0x01: apic 3 int 15 (irq 10) scsibus0 at ahc0: 16 targets sd0 at scsibus0 targ 0 lun 0: <QUANTUM, ATLAS 10K 9SCA, UCIE> SCSI3 0/direct fixed sd0: 8683MB, 10042 cyl, 6 head, 295 sec, 512 bytes/sec, 17783249 sec total sd1 at scsibus0 targ 1 lun 0: <QUANTUM, ATLAS 10K 9SCA, UCIE> SCSI3 0/direct fixed sd1: 8683MB, 10042 cyl, 6 head, 295 sec, 512 bytes/sec, 17783249 sec total safte0 at scsibus0 targ 6 lun 0: <DELL, 1x4 U2W SCSI BP, 1.16> SCSI2 3/processor fixed ahc1 at pci2 dev 4 function 1 "Adaptec AIC-7899 U160" rev 0x01: apic 3 int 14 (irq 11) scsibus1 at ahc1: 16 targets fxp0 at pci1 dev 8 function 0 "Intel 8255x" rev 0x08, i82559: apic 3 int 0 (irq 11), address 00:b0:d0:20:8a:b1 inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4 ppb1 at pci0 dev 2 function 0 "DEC 21152 PCI-PCI" rev 0x03 pci3 at ppb1 bus 1 fxp1 at pci3 dev 4 function 0 "Intel 8255x" rev 0x05, i82558: apic 3 int 4 (irq 11), address 00:50:8b:5e:e7:ac inphy1 at fxp1 phy 1: i82555 10/100 PHY, rev. 0 fxp2 at pci3 dev 5 function 0 "Intel 8255x" rev 0x05, i82558: apic 3 int 5 (irq 10), address 00:50:8b:5e:e7:ad inphy2 at fxp2 phy 1: i82555 10/100 PHY, rev. 0 san0 at pci0 dev 4 function 0 "Sangoma A10x" rev 0x01 apic 3 int 1 (irq 11) san1 at pci0 dev 8 function 0 "Sangoma A10x" rev 0x01 apic 3 int 6 (irq 10) vga1 at pci0 dev 14 function 0 "ATI Mach64 GY" rev 0x7a wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) piixpm0 at pci0 dev 15 function 0 "ServerWorks OSB4" rev 0x4f: SMI iic0 at piixpm0 pciide0 at pci0 dev 15 function 1 "ServerWorks OSB4 IDE" rev 0x00: DMA atapiscsi0 at pciide0 channel 0 drive 0 scsibus2 at atapiscsi0: 2 targets cd0 at scsibus2 targ 0 lun 0: <SAMSUNG, CD-ROM SN-124, S003> SCSI0 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 2 ohci0 at pci0 dev 15 function 2 "ServerWorks OSB4/CSB5 USB" rev 0x04: apic 2 int 5 (irq 5), version 1.0, legacy support usb0 at ohci0: USB revision 1.0 uhub0 at usb0 uhub0: ServerWorks OHCI root hub, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered isa0 at mainbus0 isadma0 at isa0 pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 midi0 at pcppi0: <PC speaker> spkr0 at pcppi0 lpt0 at isa0 port 0x378/4 irq 7 npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16 pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo pccom0: console pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec pctr: 686-class user-level performance counters enabled mtrr: Pentium Pro MTRR support ahc0: target 0 using 16bit transfers ahc0: target 0 synchronous at 80.0MHz DT, offset = 0x1f dkcsum: sd0 matches BIOS drive 0x80 ahc0: target 1 using 16bit transfers ahc0: target 1 synchronous at 80.0MHz DT, offset = 0x1f dkcsum: sd1 matches BIOS drive 0x81 root on sd0a rootdev=0x400 rrootdev=0xd00 rawdev=0xd02 WARNING: / was not properly unmounted Automatic boot in progress: starting file system checks. /dev/rsd0a: 1780 files, 21087 used, 43258 free (362 frags, 5362 blocks, 0.6% fragmentation) /dev/rsd0a: MARKING FILE SYSTEM CLEAN /dev/rsd0h: 995 files, 32540 used, 31821 free (69 frags, 3969 blocks, 0.1% fragmentation) /dev/rsd0h: MARKING FILE SYSTEM CLEAN /dev/rsd0d: 1 files, 1 used, 128730 free (18 frags, 16089 blocks, 0.0% fragmentation) /dev/rsd0d: MARKING FILE SYSTEM CLEAN /dev/rsd0g: 8844 files, 90868 used, 166155 free (2259 frags, 20487 blocks, 0.9% fragmentation) /dev/rsd0g: MARKING FILE SYSTEM CLEAN /dev/rsd0e: 727 files, 3359 used, 61002 free (34 frags, 7621 blocks, 0.1% fragmentation) /dev/rsd0e: MARKING FILE SYSTEM CLEAN /dev/rsd0f: 68 files, 2252 used, 126479 free (127 frags, 15794 blocks, 0.1% fragmentation) /dev/rsd0f: MARKING FILE SYSTEM CLEAN setting tty flags pf enabled net.inet.ip.forwarding: 0 -> 1 starting network add host 207.173.230.131: gateway 66.185.224.3 add net 216.190.36.132/30: gateway 66.185.224.3 add net 216.190.36.136/30: gateway 66.185.224.3 add net 216.19san0: Bringing interface up. san0: Configuring A101 PMC T1/E1/J1 Front End san0: Link connecting... san0: Bringing interface up. 0.36.140/30: gateway 66.185.224.3 add net 216.190.36.144/30: gateway 66.185.224.3 add net 66.185.224.0/20: gateway 66.185.224.1 add net 10.0.s0.0/8: gateway 6a6.185.224.1 addn net 192.168.0.00/16: gateway 66.:185.224.1 add host 144.228U.242.172: gatewany 144.228.193.61k nown signal (00). san0: T1 connected! san0: Link connected! san0: T1 YELLOW ON san0: T1 disconnected! san0: Link connecting... san1: Bringing interface up. san1: Configuring A101 PMC T1/E1/J1 Front End san1: Link connecting... san1: Bringing interface up. san1: Unknown signal (00). san1: T1 connected! san1: Link connected! san1: T1 YELLOW ON san1: T1 disconnected! san1: Link connecting... san2: Bringing interface up. san2: Configuring A101 PMC T1/E1/J1 Front End san2: Link connecting... san2: Bringing interface up. add host 12.123.145.122: gateway 12.124.16.21 add net 135.89.154.144/28: gateway 12.124.16.21 add net 135.89.152.48/28: gateway 12.124.16.21 san2: T1 connected! san2: Link connected! san3: Bringing interface up. san3: Configuring A101 PMC T1/E1/J1 Front End san3: Link connecting... san3: Bringing interface up. san3: Unknown signal (00). san3: T1 connected! san3: Link connected! san3: T1 YELLOW ON san3: T1 disconnected! san3: Link connecting... san0: T1 YELLOW OFF san0: T1 connected! san0: Link connected! starting system logger starting initial daemons: ntpd. savecore: no core dump checking quotas: done. building ps databases: kvm dev. clearing /tmp starting pre-securelevel daemons:. setting kernel security level: kern.securelevel: 0 -> 1 creating runtime link editor directory cache. preserving editor files starting network daemons: bgpd sendmail inetd sshd. starting local daemons: snmpd. standard daemons: cron. Thu Jun 21 09:30:55 MST 2007 OpenBSD/i386 (rrlhcrtr0200.lhc.redrivernet.com) (tty00) login: san2: LCP keepalive timeout kernel: page fault trap, code=0 Stopped at 0: kernel: page fault trap, code=0 Stopped at db_read_bytes+0x14: movb 0(%edx),%al ddb{0}> sh panic the kernel did not panic ddb{0}> ps PID PPID PGRP UID S FLAGS WAIT COMMAND 22386 1 27708 0 3 0x2040180 select sendmail 995 1 995 0 3 0x2004082 ttyin getty 8671 1 8671 0 3 0x2004082 ttyin getty 20388 1 20388 0 3 0x2004082 ttyin getty 17825 1 17825 0 3 0x2004082 ttyin getty 1200 1 1200 0 3 0x2004082 ttyin getty 9074 1 9074 0 3 0x2004082 ttyin getty 27708 1 27708 0 3 0x2040180 select sendmail 1106 1 1106 0 3 0x2000080 select cron 7629 1 23351 0 3 0x2000080 select snmpd 11951 1 11951 0 3 0x2000080 select sshd 25778 1 25778 0 3 0x2000180 select inetd 18827 13692 13692 75 3 0x2000180 poll bgpd 30940 13692 13692 75 3 0x2000180 poll bgpd 13692 1 13692 0 2 0x2000080 bgpd 15778 26717 26717 83 3 0x2000180 poll ntpd 26717 1 26717 0 3 0x2000080 poll ntpd 7143 1030 1030 74 3 0x2000180 bpf pflogd 1030 1 1030 0 3 0x2000080 netio pflogd 29385 10578 10578 73 2 0x2000180 syslogd 10578 1 10578 0 3 0x2000088 netio syslogd 14 0 0 0 3 0x2100200 crypto_wa crypto 13 0 0 0 3 0x2100200 aiodoned aiodoned 12 0 0 0 3 0x2100200 syncer update 11 0 0 0 3 0x2100200 cleaner cleaner 10 0 0 0 3 0x100200 reaper reaper 9 0 0 0 3 0x2100200 pgdaemon pagedaemon 8 0 0 0 3 0x2100200 pftm pfpurge 7 0 0 0 3 0x2100200 wait wskbd_hotkey 6 0 0 0 3 0x2100200 usbtsk usbtask 5 0 0 0 3 0x2100200 usbevt usb0 4 0 0 0 3 0x2100200 timeout sensors 3 0 0 0 3 0x2100200 slacking scsi 2 0 0 0 3 0x2100200 kmalloc kmthread 1 0 1 0 3 0x2004080 wait init 0 -1 0 0 3 0x2080200 scheduler swapper ddb{0}> trace db_read_bytes(0,1,e7f2fd5c,2,0) at db_read_bytes+0x14 db_get_value(0,1,0,d067dbc3,0) at db_get_value+0x19 db_disasm(0,0,d033f310,0,50) at db_disasm+0x1d db_print_loc_and_inst(0,e7f2fe14,e7f2fe2c,d0473534,0) at db_print_loc_and_inst+ 0x2d db_trap(6,0,e7f2fe4c,d04642dd,1) at db_trap+0x75 kdb_trap(6,0,e7f2fe94,50) at kdb_trap+0xe8 trap() at trap+0xa1 --- trap (number 6) --- (null)(0,0,0,d128cc40,0) at 0 softclock(58,e7f20010,10,e7f20010,e7f2e000) at softclock+0x22c Bad frame pointer: 0xe7f2ff20 ddb{0}> machine ddb 1 Stopped at Debugger+0x4: leave ddb{1}> trace Debugger(d122cc00,d079f480,0,e7f31ecc,d05a9007) at Debugger+0x4 i386_ipi_handler(b0,58,10,d0790010,e7f30010) at i386_ipi_handler+0x57 Xintripi() at Xintripi+0x47 --- interrupt --- i386_softintlock(0,d0460058,e7f30010,10,10) at i386_softintlock+0x65 Xintrltimer() at Xintrltimer+0x47 --- interrupt --- apm_cpu_idle(0,0,0,0,0) at apm_cpu_idle+0x4a ddb{1}> boot sync l8rZ, -- andrew - ICQ# 253198 - Jabber: [EMAIL PROTECTED] BOFH excuse of the day: Cow-tippers tipped a cow onto the server.
