Hello everybody, I have a very disturbing problem with one of our FreeBSD 5.5-stable machines. It is a box on which ~10 jail systems run, each with small to moderate network traffic.
Now from time to time - sometimes after a few days, sometimes after a couple of weeks - the network interface rl0 (which is the main interface on the maschine, rl1 is for backups/internal use only) stops working. Each jailed system has its own firewall ruleset, permitting only traffic for the services in that specific jail. The packet filter used is ipfw. Some of the rules are stateful (keep-state). When rl0 stops working ipfw loggs lots of denied packets so that it seems that the dynamic (keep-state) rules don't work any longer. We checked and increased the buffers for the dynamic rules to no avail - I doubt they are part of the problem. I'm not even sure ipfw is part of the problem. After the stop on the interface occurs there is no other way to get the interface up and running again than rebooting the whole machine. Restarting /etc/rc.d/netif, the jails or ipfw doesn't help anything. The bad thing is I haven't found any way to trigger this problem so that I can only check and change things and wait if the situation improves or not. For example I've already set debug.mpsafenet="0" but this doesn't help, in contrast it seems to worsen the problem a little bit. Find attached the dmesg output of the machine. If any other information is needed to hunt down the cause of this problem please let me know. I checked various list archives but haven't found a clue yet. -------------------------[ dmesg ]------------------------- Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.5-STABLE #5: Tue May 30 13:51:55 CEST 2006 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/SHAWSHANK WARNING: MPSAFE network stack disabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2411.60-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf34 Stepping = 4 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> real memory = 2147418112 (2047 MB) avail memory = 2096037888 (1998 MB) ACPI APIC Table: <GBT AWRDACPI> ioapic0 <Version 2.0> irqs 0-23 on motherboard npx0: <math processor> on motherboard npx0: INT 16 interface acpi0: <GBT AWRDACPI> on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_button0: <Power Button> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0x1000-0x10bf,0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 agp0: <Intel 82865 host to AGP bridge> mem 0xe8000000-0xefffffff at device 0.0 on pci0 pcib1: <PCI-PCI bridge> at device 1.0 on pci0 pci1: <PCI bus> on pcib1 pcib2: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci2: <ACPI PCI bus> on pcib2 pci2: <display, VGA> at device 0.0 (no driver attached) rl0: <RealTek 8139 10/100BaseTX> port 0x9000-0x90ff mem 0xf5000000-0xf50000ff irq 21 at device 1.0 on pci2 miibus0: <MII bus> on rl0 rlphy0: <RealTek internal media interface> on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl0: Ethernet address: 00:02:2a:d5:39:74 rl1: <RealTek 8139 10/100BaseTX> port 0x9400-0x94ff mem 0xf5001000-0xf50010ff irq 22 at device 2.0 on pci2 miibus1: <MII bus> on rl1 rlphy1: <RealTek internal media interface> on miibus1 rlphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto rl1: Ethernet address: 00:02:2a:d5:39:53 isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel ICH5 UDMA100 controller> port 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 pci0: <serial bus, SMBus> at device 31.3 (no driver attached) acpi_tz0: <Thermal Zone> on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A pmtimer0 on isa0 orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x100> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x64,0x60 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 ppc0: parallel port not found. Timecounter "TSC" frequency 2411601876 Hz quality 800 Timecounters tick every 10.000 msec ipfw2 initialized, divert disabled, rule-based forwarding disabled, default to deny, logging disabled ad0: 114497MB <STARDOM SohoRaid Mirror Rev:B2.7/Rev 2.7> [232629/16/63] at ata0-master UDMA100 acd0: DVDROM <TOSHIBA DVD-ROM SD-M1912/TM01> at ata1-master PIO4 Mounting root from ufs:/dev/ad0s1a -------------------------[ dmesg ]------------------------- Best regards, Hank
pgpesF2HPryqd.pgp
Description: PGP signature