On 2013/09/27 10:09, Thomas Greer wrote: > Hi All > > I'm seeing high CPU usage with bgpd session engine, and this was knocking out > all my routing. The only way to get routing back is to pill the bgpd and then > start it again. sthen suggested I ktraced it and the output is below.
To clarify from an IRC discussion (tgreer please correct me if I'm wrong), AIUI this was running OK until announcements are made on a particular peer session, just bringing the session up without making announcements doesn't trigger it. Same happens when either announcing just a default route or when announcing full table. > This is on 5.3 however same issues experienced on 5.2. Different hardware > also. > > Peer is running bird. Versions: 1.3.8 and 1.3.11 > > Kdump: > > 6326 bgpd EMUL "native" > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) EAGAIN - send(2) says "The socket is marked non-blocking and the requested operation would block." Anyone have clues? > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > 6326 bgpd CALL sendmsg(0x9,0x7f7ffffe7080,0) > 6326 bgpd RET sendmsg -1 errno 35 Resource temporarily unavailable > > Dmesg: > > OpenBSD 5.3 (GENERIC.MP) #0: Wed Jul 10 20:29:08 BST 2013 > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > real mem = 2136473600 (2037MB) > avail mem = 2057166848 (1961MB) > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.6 @ 0x9f000 (19 entries) > bios0: vendor American Megatrends Inc. version "1.2a" date 02/21/12 > bios0: Supermicro X7SPA-H > acpi0 at bios0: rev 2 > acpi0: sleep states S0 S1 S4 S5 > acpi0: tables DSDT FACP APIC MCFG OEMB HPET GSCI EINJ BERT ERST HEST > acpi0: wakeup devices P0P1(S4) PS2K(S4) PS2M(S4) USB0(S4) USB1(S4) USB2(S4) > USB5(S4) EUSB(S4) USB3(S4) USB4(S4) USB6(S4) USBE(S4) P0P4(S4) P0P5(S4) > P0P6(S4) P0P7(S4) P0P8(S4) P0P9(S4) GBE_(S4) SLPB(S4) > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.23 MHz > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS > H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3, > CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF > cpu0: 512KB 64b/line 8-way L2 cache > cpu0: apic clock running at 199MHz > cpu1 at mainbus0: apid 2 (application processor) > cpu1: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.00 MHz > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS > H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3, > CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF > cpu1: 512KB 64b/line 8-way L2 cache > cpu2 at mainbus0: apid 1 (application processor) > cpu2: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.00 MHz > cpu2: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS > H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3, > CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF > cpu2: 512KB 64b/line 8-way L2 cache > cpu3 at mainbus0: apid 3 (application processor) > cpu3: Intel(R) Atom(TM) CPU D525 @ 1.80GHz, 1800.00 MHz > cpu3: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS > H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,DTES64,MWAIT,DS-CPL,TM2,SSSE3, > CX16,xTPR,PDCM,MOVBE,NXE,LONG,LAHF,PERF > cpu3: 512KB 64b/line 8-way L2 cache > ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 20, 24 pins > ioapic0: misconfigured as apic 1, remapped to apid 4 > acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255 > acpihpet0 at acpi0: 14318179 Hz > acpiprt0 at acpi0: bus 0 (PCI0) > acpiprt1 at acpi0: bus 7 (P0P1) > acpiprt2 at acpi0: bus 1 (P0P4) > acpiprt3 at acpi0: bus 5 (P0P8) > acpiprt4 at acpi0: bus 6 (P0P9) > acpicpu0 at acpi0 > acpicpu1 at acpi0 > acpicpu2 at acpi0 > acpicpu3 at acpi0 > acpibtn0 at acpi0: SLPB > acpibtn1 at acpi0: PWRB > ipmi at mainbus0 not configured > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel Pineview DMI" rev 0x02 > vga1 at pci0 dev 2 function 0 "Intel Pineview Video" rev 0x02 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > intagp0 at vga1 > agp0 at intagp0: aperture at 0xd0000000, size 0x10000000 > inteldrm0 at vga1: apic 4 int 16 > drm0 at inteldrm0 > "Intel Pineview Video" rev 0x02 at pci0 dev 2 function 1 not configured > uhci0 at pci0 dev 26 function 0 "Intel 82801I USB" rev 0x02: apic 4 int 16 > uhci1 at pci0 dev 26 function 1 "Intel 82801I USB" rev 0x02: apic 4 int 21 > uhci2 at pci0 dev 26 function 2 "Intel 82801I USB" rev 0x02: apic 4 int 19 > ehci0 at pci0 dev 26 function 7 "Intel 82801I USB" rev 0x02: apic 4 int 18 > usb0 at ehci0: USB revision 2.0 > uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 > ppb0 at pci0 dev 28 function 0 "Intel 82801I PCIE" rev 0x02: msi > pci1 at ppb0 bus 1 > ppb1 at pci1 dev 0 function 0 "IDT 89HPES12N3A" rev 0x04 > pci2 at ppb1 bus 2 > ppb2 at pci2 dev 0 function 0 "IDT 89HPES12N3A" rev 0x04 > pci3 at ppb2 bus 3 > em0 at pci3 dev 0 function 0 "Intel PRO/1000 QP (82571EB)" rev 0x06: apic 4 > int 17, address 00:15:17:80:63:8d > em1 at pci3 dev 0 function 1 "Intel PRO/1000 QP (82571EB)" rev 0x06: apic 4 > int 16, address 00:15:17:80:63:8c > ppb3 at pci2 dev 1 function 0 "IDT 89HPES12N3A" rev 0x04 > pci4 at ppb3 bus 4 > em2 at pci4 dev 0 function 0 "Intel PRO/1000 QP (82571EB)" rev 0x06: apic 4 > int 18, address 00:15:17:80:63:8f > em3 at pci4 dev 0 function 1 "Intel PRO/1000 QP (82571EB)" rev 0x06: apic 4 > int 17, address 00:15:17:80:63:8e > ppb4 at pci0 dev 28 function 4 "Intel 82801I PCIE" rev 0x02: msi > pci5 at ppb4 bus 5 > em4 at pci5 dev 0 function 0 "Intel PRO/1000 MT (82574L)" rev 0x00: msi, > address 00:25:90:96:68:94 > ppb5 at pci0 dev 28 function 5 "Intel 82801I PCIE" rev 0x02: msi > pci6 at ppb5 bus 6 > em5 at pci6 dev 0 function 0 "Intel PRO/1000 MT (82574L)" rev 0x00: msi, > address 00:25:90:96:68:95 > uhci3 at pci0 dev 29 function 0 "Intel 82801I USB" rev 0x02: apic 4 int 23 > uhci4 at pci0 dev 29 function 1 "Intel 82801I USB" rev 0x02: apic 4 int 19 > uhci5 at pci0 dev 29 function 2 "Intel 82801I USB" rev 0x02: apic 4 int 18 > ehci1 at pci0 dev 29 function 7 "Intel 82801I USB" rev 0x02: apic 4 int 23 > usb1 at ehci1: USB revision 2.0 > uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1 > ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x92 > pci7 at ppb6 bus 7 > pcib0 at pci0 dev 31 function 0 "Intel 82801IR LPC" rev 0x02 > pciide0 at pci0 dev 31 function 2 "Intel 82801I SATA" rev 0x02: DMA, channel 0 > configured to native-PCI, channel 1 configured to native-PCI > pciide0: using apic 4 int 19 for native-PCI interrupt > ichiic0 at pci0 dev 31 function 3 "Intel 82801I SMBus" rev 0x02: apic 4 int > 18 > iic0 at ichiic0 > lm1 at iic0 addr 0x2d: W83627DHG > spdmem0 at iic0 addr 0x51: 2GB DDR3 SDRAM PC3-10600 SO-DIMM > pciide1 at pci0 dev 31 function 5 "Intel 82801I SATA" rev 0x02: DMA, channel 0 > wired to native-PCI, channel 1 wired to native-PCI > pciide1: using apic 4 int 19 for native-PCI interrupt > wd0 at pciide1 channel 1 drive 0: <M4-CT064M4SSD2> > wd0: 16-sector PIO, LBA48, 61053MB, 125037167 sectors > wd0(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5 > usb2 at uhci0: USB revision 1.0 > uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > usb3 at uhci1: USB revision 1.0 > uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > usb4 at uhci2: USB revision 1.0 > uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > usb5 at uhci3: USB revision 1.0 > uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > usb6 at uhci4: USB revision 1.0 > uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > usb7 at uhci5: USB revision 1.0 > uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > com0: console > com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > wbsio0 at isa0 port 0x2e/2: W83627DHG rev 0x25 > lm2 at wbsio0 port 0xca0/8: W83627DHG > mtrr: Pentium Pro MTRR support > lm1: disabling sensors due to alias with lm2 > vscsi0 at root > scsibus0 at vscsi0: 256 targets > softraid0 at root > scsibus1 at softraid0: 256 targets > root on wd0a (ae4edeff80316ed9.a) swap on wd0b dump on wd0b > > Bgpd.conf > > peer1="X.X.X..40" > peer1v4block="{ Y.Y.Y.0/21 }" > > AS 198236 > router-id X.X.X.3 > > neighbor $peer1 { > remote-as ##### > announce all > announce IPv6 none > } > > deny from $peer1 > allow from $peer1 prefix $peer1v4block
