Re: openbsd router performance (i know.. again)
Hi all,
I'm waiting for the 5.2 to reinstall my routers/firewalls and see if things
on my hardware improved.
I'll also disable MP; how about 386 vs amd64? Is there any difference in
terms of speed in managing interrupt and forwarding traffic? I've found a
post from Henning telling that 386 is much better for routing/firewalling
but it was 5-6 years ago and I'm sure things changed a lot
Thanks for your help, can't wait to see my 5.2 cd on my desk :)
Alessandro
On Wed, Sep 26, 2012 at 5:31 PM, noah pugsley wrote:
> What is your performance like with -current and no knob twisting?
>
> On Wed, Sep 26, 2012 at 4:45 AM, rik wrote:
>
>> Hi,
>> I'm a happy Openbsd "user"; we've beeing using it since 2001 as
>> router/firewall in our datacenter facility (we host as ONG some no profit
>> project and website).
>> At the moment we're using a couple of SuperMicro with the following specs:
>> OpenBSD 5.0 (GENERIC.MP) #59: Wed Aug 17 10:19:44 MDT 2011
>> dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
>> cpu0: Intel(R) Pentium(R) 4 CPU 3.00GHz ("GenuineIntel" 686-class) 2.98
>> GHz
>> cpu0:
>>
>> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,
>> MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,PDCM
>> real mem = 3890663424 (3710MB)
>> avail mem = 3816964096 (3640MB)
>> ppb3 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01: apic 2 int 17
>> em0 at pci4 dev 0 function 0 "Intel PRO/1000MT (82573E)" rev 0x03: msi,
>> address 00:30:xx:xx:xx:xx
>> ppb4 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01: apic 2 int 16
>> em1 at pci5 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: msi,
>> address 00:30:xx:xx:xx:xx
>>
>> the netcard are on-board.
>> Unfortunately we're a bit straggling with the performances as we have
>> almost 100% interrupt with 110Mbps and 12k pps
>> We've already increased net.inet.ip.ifq.maxlen to 500 in order to avoid
>> packet loss and also disabling pf has no influence.
>> Do you think these performances are fair and we have to upgrade to better
>> hardware to have higher pps and Mpbs?
>> Beside trying to upgrade to the last stable and not use MP we have no idea
>> how to procede
>> Thanks for your help
>> Alessandro
packet loss
Good day, I'm using 2 openbsd boxes as router firewall with carp in a colo-like setup. In the last few days we saw the packet loss percentuale increase up to 8-10% and it doesn't look like a problem for outside. If I ping from the master firewall one of the server inside I can see something like this: 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms ping: sendto: No route to host ping: wrote xx.xx.xx.12 64 chars, ret=-1 ping: sendto: No route to host ping: wrote xx.xx.xx.12 64 chars, ret=-1 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms No errors in syslog. Any idea? Thanks Alessandro
Re: packet loss
Hi, On Mon, Nov 28, 2011 at 5:59 PM, Peter N. M. Hansteen wrote: > rik writes: > > > I'm using 2 openbsd boxes as router firewall with carp in a colo-like > setup. > > In the last few days we saw the packet loss percentuale increase up to > > 8-10% and it doesn't look like a problem for outside. > > I take this to mean that the CARP setup provided the needed redundancy. > Yes exactly, we've 2 carp interfaces, one for the internal interface, the second for the external interface; the setup is working with no major issue for 3 years or so > > If I ping from the master firewall one of the server inside I can see > > something like this: > > > > 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms > > ping: sendto: No route to host > > ping: wrote xx.xx.xx.12 64 chars, ret=-1 > > ping: sendto: No route to host > > ping: wrote xx.xx.xx.12 64 chars, ret=-1 > > 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms > > > > No errors in syslog. > > Any idea? > > This is what it looks like when your link goes down, then comes back > again. I'd check with the upstream if they know of any specific incident > that matches your disruption. > The ping I've tried is from the master firewall to a server inside the network: firewall -> switch -> xx.xx.xx.12 The switch works ok, if I ping from one server to another one in the same subnet there's no packet lost so it looks like something on the firewall. The two machines are idle as 99,9% and no high interrupt or mbuf clusters number Thanks! Alessandro
Re: packet loss
Hi,
this is the dmesg:
cpu0: Intel Pentium III ("GenuineIntel" 686-class) 745 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,SER,MMX,FXSR,SSE
real mem = 536449024 (523876K)
avail mem = 482430976 (471124K)
using 4278 buffers containing 26927104 bytes (26296K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 03/17/00, BIOS32 rev. 0 @ 0xfd6b1
pcibios0 at bios0: rev 2.1 @ 0xf/0x
pcibios0: PCI BIOS has 10 Interrupt Routing table entries
pcibios0: PCI Interrupt Router at 000:07:0 ("Intel 82371AB PIIX4 ISA" rev
0x00)
pcibios0: PCI bus #2 is the last bus
bios0: ROM list: 0xc/0x9a00 0xc9a00/0xd800 0xd7200/0x4800
mainbus0: Intel MP Specification (Version 1.1) (IBM ENSW Kiowa SMP )
cpu0 at mainbus0: apid 1 (boot processor)
cpu0: apic clock running at 99 MHz
cpu1 at mainbus0: apid 0 (application processor)
cpu1: Intel Pentium III ("GenuineIntel" 686-class)
cpu1: FPU,CX8,APIC
mainbus0: bus 0 is type PCI
mainbus0: bus 1 is type PCI
mainbus0: bus 2 is type PCI
mainbus0: bus 3 is type ISA
ioapic0 at mainbus0: apid 14 pa 0xfec0, version 11, 24 pins
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82440BX AGP" rev 0x00
ppb0 at pci0 dev 1 function 0 "Intel 82440BX AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "Chips and Technologies 69000" rev 0x64
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 7 function 0 "Intel 82371AB PIIX4 ISA" rev 0x02
pciide0 at pci0 dev 7 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel
0 wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 ignored (disabled)
atapiscsi0 at pciide0 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: SCSI0 5/cdrom
removable
cd0(pciide0:1:0): using PIO mode 4, DMA mode 2
"Intel 82371AB USB" rev 0x01 at pci0 dev 7 function 2 not configured
piixpm0 at pci0 dev 7 function 3 "Intel 82371AB Power" rev 0x02: SMI
iic0 at piixpm0
admtemp0 at iic0 addr 0x18: max1617
admtemp1 at iic0 addr 0x1a: max1617
"unknown" at iic0 addr 0x2d not configured
admtemp2 at iic0 addr 0x4c: max1617
admtemp3 at iic0 addr 0x4e: max1617
fxp0 at pci0 dev 17 function 0 "Intel 8255x" rev 0x08, i82559: apic 14 int
18 (irq 10), address xx:xx:xx:xx:xx:xx
inphy0 at fxp0 phy 1: i82555 10/100 PHY, rev. 4
fxp1 at pci0 dev 18 function 0 "Intel 8255x" rev 0x08, i82559: apic 14 int
17 (irq 11), address xx:xx:xx:xx:xx:xx
inphy1 at fxp1 phy 1: i82555 10/100 PHY, rev. 4
ppb1 at pci0 dev 20 function 0 "DEC 21152 PCI-PCI" rev 0x03
pci2 at ppb1 bus 2
rl0 at pci2 dev 14 function 0 "D-Link Systems 530TX+" rev 0x10: apic 14 int
17 (irq 11), address xx:xx:xx:xx:xx:xx
rlphy0 at rl0 phy 0: RTL internal PHY
ahc0 at pci2 dev 15 function 0 "Adaptec AHA-2940U" rev 0x01: apic 14 int 16
(irq 9)
scsibus1 at ahc0: 16 targets
ahc0: target 0 using 8bit transfers
ahc0: target 0 using asynchronous transfers
sd0 at scsibus1 targ 0 lun 0: SCSI2 0/direct
fixed
sd0: 8678MB, 11721 cyl, 5 head, 303 sec, 512 bytes/sec, 17774160 sec total
ahc0: target 1 using 8bit transfers
ahc0: target 1 using asynchronous transfers
sd1 at scsibus1 targ 1 lun 0: SCSI2 0/direct
fixed
sd1: 8678MB, 11721 cyl, 5 head, 303 sec, 512 bytes/sec, 17774160 sec total
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
midi0 at pcppi0:
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
biomask 0 netmask 0 ttymask 0
pctr: 686-class user-level performance counters enabled
mtrr: Pentium Pro MTRR support
ahc0: target 0 using 16bit transfers
ahc0: target 0 synchronous at 10.0MHz, offset = 0x8
dkcsum: sd0 matches BIOS drive 0x80
ahc0: target 1 using 16bit transfers
ahc0: target 1 synchronous at 10.0MHz, offset = 0x8
dkcsum: sd1 matches BIOS drive 0x81
root on sd0a
rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
Thanks!
Alessandro
On Mon, Nov 28, 2011 at 7:53 PM, Stuart Henderson wrote:
> dmesg?
>
> On 2011-11-28, rik wrote:
> > Good day,
> > I'm using 2 openbsd boxes as router firewall with carp in a colo-like
> setup.
> > In the last few days we saw the packet loss percentuale increase up to
> > 8-10% and it doesn't look like a problem for outside. If I ping from the
> > master firewall one of the server inside I can see something like this:
> >
> > 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms
> > 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms
Re: packet loss
Hi James, both carp on the master firewall are in master status (one on the external side, one on the internal side), but as much as I know they've always been like this; on the backup firewall they both are in backup status (and the backup, using the phisical interface, can ping without any packet loss). Thanks Alessandro On Mon, Nov 28, 2011 at 8:08 PM, James Shupe wrote: > Run > > ifconfig carp | grep status > > on both machines... If they're pre 4.8, do: > > ifconfig carp | grep 'carp: ' > > . > > If both think they're masters, they'll do what you're seeing. > > Thank you, > James Shupe > > On 11/28/11 12:53 PM, Stuart Henderson wrote: > > dmesg? > > > > On 2011-11-28, rik wrote: > >> Good day, > >> I'm using 2 openbsd boxes as router firewall with carp in a colo-like > setup. > >> In the last few days we saw the packet loss percentuale increase up to > >> 8-10% and it doesn't look like a problem for outside. If I ping from > the > >> master firewall one of the server inside I can see something like this: > >> > >> 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms > >> 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms > >> 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms > >> ping: sendto: No route to host > >> ping: wrote xx.xx.xx.12 64 chars, ret=-1 > >> ping: sendto: No route to host > >> ping: wrote xx.xx.xx.12 64 chars, ret=-1 > >> 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms > >> 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms > >> > >> No errors in syslog. > >> Any idea? > >> Thanks > >> Alessandro > > > > > -- > James Shupe, OSRE > developer/ engineer > BSD/ Linux support & hosting > jsh...@osre.org | www.osre.org > O 9032530140 | F 9032530150 | M 9035223425
Re: packet loss
Sorry, I've mised the top 2 rows of the dmesg: OpenBSD 3.9 (FIREWALL) #0: Sun Sep 17 15:49:07 CEST 2006 r...@fw1.domain.com:/usr/src/sys/arch/i386/compile/FIREWALL Firewall is just the generic.mp with a device (cpu temp monitor) removed because not working. This is my netstat -i from the master NameMtu Network Address Ipkts IerrsOpkts Oerrs Colls lo0 33224 2170 0 2170 0 0 lo0 33224 loopbacklocalhost 2170 0 2170 0 0 lo0 33224 localhost.n ::1 2170 0 2170 0 0 fxp01500xx:xx:xx:xx:xx:xx 4080602979 5814 3643673264 0 0 fxp11500xx:xx:xx:xx:xx:xx 3990056491 256 4226316164 0 0 fxp11500 x.x.x.0 fw1 3990056491 256 4226316164 0 0 rl0 1500xx:xx:xx:xx:xx:xx 4757956 0 16291765 0 0 rl0 1500 10.1.0/24 10.1.0.3 4757956 0 16291765 0 0 pflog0 332240 00 0 0 pfsync0 1460 0 00 0 0 enc0* 1536 0 00 0 0 carp0 1500xx:xx:xx:xx:xx:xx 4077521045 0 4450639 0 0 carp0 1500 xx.xx.ww.2 xx.xx.ww.30 4077521045 0 4450639 0 0 carp1 1500xx:xx:xx:xx:xx:xx 397833709935 4450637 2 0 carp1 1500 xx.xx.xx.0 xx.xx.xx.1 397833709935 4450637 2 0 carp1 1500 xx.xx.xx.1 xx.xx.xx.17 397833709935 4450637 2 0 carp1 1500 xx.xx.xx.3 xx.xx.xx.33 397833709935 4450637 2 0 carp1 1500 xx.xx.xx.4 xx.xx.xx.49 397833709935 4450637 2 0 carp1 1500 xx.xx.zz.1 xx.xx.zz.129 397833709935 4450637 2 0 carp1 1500 xx.xx.zz.1 xx.xx.zz.145 397833709935 4450637 2 0 carp1 1500 xx.xx.zz.1 xx.xx.zz.161 397833709935 4450637 2 0 carp1 1500 xx.xx.zz.1 xx.xx.zz.177 397833709935 4450637 2 0 carp1 1500 xx.xx.yy.1 xx.xx.yy.129 397833709935 4450637 2 0 I've tried to switch on the basckup with no difference. It has also been changed the cable and the port on the switch Thanks! alessandro On Mon, Nov 28, 2011 at 8:58 PM, James Shupe wrote: > Your dmesg doesn't show the version you're running. Can you provide > that, along with ifconfig output from both machines? You may want to > check the physical connectivity (cable/ NIC/ switch) for the internal > interface of the carp master... Or just fail over to the secondary box > to see if the issue goes away. > > Also, provide the netstat -i output. > > On 11/28/11 1:37 PM, rik wrote: > > Hi James, > > both carp on the master firewall are in master status (one on the > external > > side, one on the internal side), but as much as I know they've always > been > > like this; on the backup firewall they both are in backup status (and the > > backup, using the phisical interface, can ping without any packet loss). > > Thanks > > Alessandro > > > > > > On Mon, Nov 28, 2011 at 8:08 PM, James Shupe wrote: > > > >> Run > >> > >> ifconfig carp | grep status > >> > >> on both machines... If they're pre 4.8, do: > >> > >> ifconfig carp | grep 'carp: ' > >> > >> . > >> > >> If both think they're masters, they'll do what you're seeing. > >> > >> Thank you, > >> James Shupe > >> > >> On 11/28/11 12:53 PM, Stuart Henderson wrote: > >>> dmesg? > >>> > >>> On 2011-11-28, rik wrote: > >>>> Good day, > >>>> I'm using 2 openbsd boxes as router firewall with carp in a colo-like > >> setup. > >>>> In the last few days we saw the packet loss percentuale increase up to > >>>> 8-10% and it doesn't look like a problem for outside. If I ping from > >> the > >>>> master firewall one of the server inside I can see something like > this: > >>>> > >>>> 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms > >>>> 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms > >>>> 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms > >>>> ping: sendto: No route to host > >>>> ping: wrote xx.xx.xx.12 64 chars, ret=-1 > >>>> ping: sendto: No route to host > >>>> ping: wrote xx.xx.xx.12 64 chars, ret=-1 > >>>> 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms > >>>> 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms > >>>> > >>>> No errors in syslog. > >>>> Any idea? > >>>> Thanks > >>>> Alessandro > >>> > >> > >> > >> -- > >> James Shupe, OSRE > >> developer/ engineer > >> BSD/ Linux support & hosting > >> jsh...@osre.org | www.osre.org > >> O 9032530140 | F 9032530150 | M 9035223425 > > > > > -- > James Shupe, OSRE > developer/ engineer > BSD/ Linux support & hosting > jsh...@osre.org | www.osre.org > O 9032530140 | F 9032530150 | M 9035223425
Re: packet loss
Thanks for the suggestion, I'll try with the GENERIC kernel Is that possibile that this problem is due to hardware limitation (it's quite an old server)? Apparently when the traffic decrease the packet loss decrease as well and disappear just like the odd ping's result Thanks! Alessandro On Tue, Nov 29, 2011 at 12:15 AM, Stuart Henderson wrote: > On 2011-11-28, James Shupe wrote: > > Your dmesg doesn't show the version you're running. Can you provide > > that, > > Yep, seconded. If people ask for a dmesg, they mean a complete one. > I would also try a GENERIC kernel (not GENERIC.MP). > > > along with ifconfig output from both machines? You may want to > > check the physical connectivity (cable/ NIC/ switch) for the internal > > interface of the carp master... Or just fail over to the secondary box > > to see if the issue goes away. > > Well there appears to be something very odd going on with timers there > so who knows what else might follow from that. > > > 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms > > ping: sendto: No route to host > > ping: wrote xx.xx.xx.12 64 chars, ret=-1 > > ping: sendto: No route to host > > ping: wrote xx.xx.xx.12 64 chars, ret=-1 > > 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms > > 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms
Re: packet loss
We've solved the problem increasing net.inet.ip.ifq.maxlen from the default of our version (50) to the default of the more recent versions (250). Does it make sens to you? How far do you think we can go with that value considering that we've 3 physical interfaces (int 100mbit, ext 100mbit and pfsync 10mbit) and that the servers have only 512Mb of RAM? Something like "Henning's rule" with 256*3 (number of physical interfaces) would be a good and safe choice with our hardware (of course we're planning an upgrade of both servers and openbsd version)? Thanks for your help Alessandro On Tue, Nov 29, 2011 at 7:49 PM, rik wrote: > Thanks for the suggestion, I'll try with the GENERIC kernel > Is that possibile that this problem is due to hardware limitation (it's > quite an old server)? Apparently when the traffic decrease the packet loss > decrease as well and disappear just like the odd ping's result > Thanks! > Alessandro > > > > On Tue, Nov 29, 2011 at 12:15 AM, Stuart Henderson > wrote: > >> On 2011-11-28, James Shupe wrote: >> > Your dmesg doesn't show the version you're running. Can you provide >> > that, >> >> Yep, seconded. If people ask for a dmesg, they mean a complete one. >> I would also try a GENERIC kernel (not GENERIC.MP). >> >> > along with ifconfig output from both machines? You may want to >> > check the physical connectivity (cable/ NIC/ switch) for the internal >> > interface of the carp master... Or just fail over to the secondary box >> > to see if the issue goes away. >> >> Well there appears to be something very odd going on with timers there >> so who knows what else might follow from that. >> >> >>>>> 64 bytes from xx.xx.xx.12: icmp_seq=4 ttl=64 time=-3.-656 ms >> >>>>> 64 bytes from xx.xx.xx.12: icmp_seq=5 ttl=64 time=0.794 ms >> >>>>> 64 bytes from xx.xx.xx.12: icmp_seq=6 ttl=64 time=0.-491 ms >> >>>>> ping: sendto: No route to host >> >>>>> ping: wrote xx.xx.xx.12 64 chars, ret=-1 >> >>>>> ping: sendto: No route to host >> >>>>> ping: wrote xx.xx.xx.12 64 chars, ret=-1 >> >>>>> 64 bytes from xx.xx.xx.12: icmp_seq=9 ttl=64 time=0.526 ms >> >>>>> 64 bytes from xx.xx.xx.12: icmp_seq=10 ttl=64 time=1.415 ms
carp with different versions of OpenBSD
Hi all, is it possibile to have a dual firewall setup with carp using (temporarly) 2 different versions of OpenBSD? I've to setup some new firewalls and upgrade old one and I'd like to keep redudancy while upgrading but during the process some firewalls will run the 5.0, some still the old version. Thanks! Alessandro
Re: carp with different versions of OpenBSD
Hi all, thanks for your replies and your help. I did try yesterday and today on some test boxes and it looks working pretty well between a very old version (3.9) and the most recent one (5.0). I just had for few minutes problems with states (increasing up to 10k until I flushed them, but it could be a problem with my pf.conf due to the big differences between the two versions of pf). My setup is not that complex and so the pf rules (approx 300 rows); I think I'll run the upgrade in the production env creating a simple pf.conf on purpose that doesn't use states. Thanks again for your support and the great work (you definitely didn't screw it up :) ) Alessandro On Thu, Dec 8, 2011 at 6:01 PM, Henning Brauer wrote: > * rik [2011-12-06 21:40]: > > is it possibile to have a dual firewall setup with carp using > (temporarly) > > 2 different versions of OpenBSD? I've to setup some new firewalls and > > upgrade old one and I'd like to keep redudancy while upgrading but during > > the process some firewalls will run the 5.0, some still the old version. > > in general that works as long as all of these are true: > 1) the two are just one release apart, all bets off if more > 2) the upgradeXX.html doesn't mention an incompatibility > 3) we didn't screw up > > that is the pfsync centric view. carp's on-the-wire format hasn't > changed in ages. > > -- > Henning Brauer, h...@bsws.de, henn...@openbsd.org > BS Web Services, http://bsws.de, Full-Service ISP > Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully > Managed > Henning Brauer Consulting, http://henningbrauer.com/
Re: carp with different versions of OpenBSD
On Thu, Dec 8, 2011 at 6:49 PM, rik wrote: > Hi all, > thanks for your replies and your help. I did try yesterday and today on > some test boxes and it looks working pretty well between a very old version > (3.9) and the most recent one (5.0). I just had for few minutes problems > with states (increasing up to 10k until I flushed them, but it could be a > problem with my pf.conf due to the big differences between the two versions > of pf). My setup is not that complex and so the pf rules (approx 300 > rows); I think I'll run the upgrade in the production env creating a > simple pf.conf on purpose that doesn't use states. > Thanks again for your support and the great work (you definitely didn't > screw it up :) ) > Alessandro > > > > > On Thu, Dec 8, 2011 at 6:01 PM, Henning Brauer wrote: > >> * rik [2011-12-06 21:40]: >> > is it possibile to have a dual firewall setup with carp using >> (temporarly) >> > 2 different versions of OpenBSD? I've to setup some new firewalls and >> > upgrade old one and I'd like to keep redudancy while upgrading but >> during >> > the process some firewalls will run the 5.0, some still the old version. >> >> in general that works as long as all of these are true: >> 1) the two are just one release apart, all bets off if more >> 2) the upgradeXX.html doesn't mention an incompatibility >> 3) we didn't screw up >> >> that is the pfsync centric view. carp's on-the-wire format hasn't >> changed in ages. >> >> -- >> Henning Brauer, h...@bsws.de, henn...@openbsd.org >> BS Web Services, http://bsws.de, Full-Service ISP >> Secure Hosting, Mail and DNS Services. Dedicated Servers, Root to Fully >> Managed >> Henning Brauer Consulting, http://henningbrauer.com/
Re: CARP health check ?
Just an idea, but you might consider giving private ip to the phydev and
using nrpe plugin for nagios so you'll be able to ping them from the inside
and report everything to your external nagios monitor
Alex
On Fri, Jan 13, 2012 at 5:12 AM, PP;Q Q P(P8P?P8QP8P=
wrote:
> sounds nice.
>
> I came to somewhat similar. Just ssh to external address and ping both carp
> peers (via internal addresses), if there're less than 2 answers, we are in
> trouble.
>
> your idea is also good.
>
> 2012/1/13 Nick Holland
>
> > ok, let's try this idea...
> >
> > Your systems have ONE external address, but they can have as many
> > internal addresses as desired, right?
> >
> > SO...let's say you have two CARP'd firewalls, FW1 and FW2. They share
> > external address of x.x.x.x.
> >
> > FW1: FW2:
> > Externalx.x.x.xx.x.x.x (same)
> > Internal real 10.0.0.2 10.0.0.3
> > internal CARP 10.0.0.1 10.0.0.1 (same)
> >
> > port 22 gets you ssh on the active firewall...but which is that?
> >
> > How about a PF ruleset that redirects port 2202 to 10.0.0.2 port 22 and
> > port 2203 to 10.0.0.3? Now you can find out anything you wish about
> > either box ON DEMAND by selecting the port you ssh to? If 2202 doesn't
> > answer, you've lost fw1, if 2203 doesn't answer, you have lost fw2
> >
> > In addition to checking to see that the box is up, it's good to check
> > for a sane CARP status -- i.e., all "MASTER" on one box, "SLAVE" on the
> > other, plus other overall health issues.
> >
> > Nick.
> >
> > On 01/12/12 13:48, iLXQ {IPICIN wrote:
> > > well, it's usually not possible.
> > > we use OpenBSD, because it supports "carpdev" option (FreeBSD does not
> > > support it)
> > >
> > > most of our carp clusters run on single address. no spare IP space.
> > >
> > > we could do ssh and ping carp peer (some trouble with preemption), but
> we
> > > do not want to stick with certain IP addresses. we would like to
> monitor
> > > "in general"
> > >
> > > 1) define new carp cluster for monitoring
> > > 2) ssh to it and monitorcarp peer in general without specifying it's
> > address
> > >
> > > 2012/1/13 Simon Perreault
> > >
> > >> On 01/12/2012 01:18 PM, P P;Q Q P(P8P?P8Q P8P= wrote:
> > >>
> > >>> we are using nagios for monitoring and it is running on separate
> > server.
> > >>> we
> > >>> do not want to monitor server from inside.
> > >>> we want to run run something via ssh and see whether carp peer is
> dead
> > or
> > >>> not.
> > >>>
> > >>
> > >> Give each server it's unique IP address.
> > >> Use a third IP address for carp.
> > >> Monitor all three addresses.
> > >>
> > >> Simon
> > >> --
> > >> DTN made easy, lean, and smart --> http://postellation.viagenie.**ca<
> > http://postellation.viagenie.ca>
> > >> NAT64/DNS64 open-source--> http://ecdysis.viagenie.ca
> > >> STUN/TURN server --> http://numb.viagenie.ca
Re: openbsd router performance (i know.. again)
Hi,
at the moment we have the default configuration, beside the increase of
net.inet.ip.ifq.maxlen to 500 (I trust OpenBSD programmers more than I
trust myself about tuning).
I haven't had yet the opportunity to upgrade it to -current, I'll do in the
next few days.
Just a small punctualization about the pps, I have 22-24k in total, 10-12k
per card (internet facing and LAN facing)
Thanks again
Alessandro
On Wed, Sep 26, 2012 at 5:31 PM, noah pugsley wrote:
> What is your performance like with -current and no knob twisting?
>
> On Wed, Sep 26, 2012 at 4:45 AM, rik wrote:
>
>> Hi,
>> I'm a happy Openbsd "user"; we've beeing using it since 2001 as
>> router/firewall in our datacenter facility (we host as ONG some no profit
>> project and website).
>> At the moment we're using a couple of SuperMicro with the following specs:
>> OpenBSD 5.0 (GENERIC.MP) #59: Wed Aug 17 10:19:44 MDT 2011
>> dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC.MP
>> cpu0: Intel(R) Pentium(R) 4 CPU 3.00GHz ("GenuineIntel" 686-class) 2.98
>> GHz
>> cpu0:
>>
>> FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,
>> MWAIT,DS-CPL,CNXT-ID,CX16,xTPR,PDCM
>> real mem = 3890663424 (3710MB)
>> avail mem = 3816964096 (3640MB)
>> ppb3 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01: apic 2 int 17
>> em0 at pci4 dev 0 function 0 "Intel PRO/1000MT (82573E)" rev 0x03: msi,
>> address 00:30:xx:xx:xx:xx
>> ppb4 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01: apic 2 int 16
>> em1 at pci5 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: msi,
>> address 00:30:xx:xx:xx:xx
>>
>> the netcard are on-board.
>> Unfortunately we're a bit straggling with the performances as we have
>> almost 100% interrupt with 110Mbps and 12k pps
>> We've already increased net.inet.ip.ifq.maxlen to 500 in order to avoid
>> packet loss and also disabling pf has no influence.
>> Do you think these performances are fair and we have to upgrade to better
>> hardware to have higher pps and Mpbs?
>> Beside trying to upgrade to the last stable and not use MP we have no idea
>> how to procede
>> Thanks for your help
>> Alessandro
limiting mbuf cluster
Hi there, we've 2 openbsd boxes used as firewall/router with pf and carp to host some websites and application for a students and researchers lab. Sometimes the boxes reboot because they reach the mbuf cluster limit. Unfortunately not all the application hosted in our lab work always correctly so I'm wondering if there's any way to limit the mbuf cluster with a pf rule on ip basis; so if one application has problem, it doesn't create problem to all the network and doesn't make the firewall crash. Thanks for your help! Rick sample output of our netstat: # netstat -m 134 mbufs in use: 130 mbufs allocated to data 1 mbuf allocated to packet headers 3 mbufs allocated to socket names and addresses 131/6089/6144 mbuf clusters in use (current/peak/max) 6512 Kbytes allocated to network (4% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines
Re: Can I use carp with just one public IP?
Hi Stefan, i'm not doing load balance, just active/passive router/firewall configuration, but we're using only one ip on carp, with no ip address on the phisical interfaces. Our configuration is like this: # cat /etc/hostname.fxp0 up # cat /etc/hostname.carp0 inet 81.xx.xx.xx 255.255.255.252 81.xx.xx.xx vhid 1 carpdev fxp0 same thing on the other machine (with advskew 100) Hope this helps Rick On Sun, Oct 9, 2011 at 7:01 PM, Stefan Midjich wrote: > Everything I read about CARP, including my Book on PF 2nd edition, > says you're supposed to have two different ip-addresses set for each > carp device, for two hosts that is. And one third ip-address in the > same network on the psuedo carp0 interface you create. > > Since I'm aiming to load balance on the first hop of a network this > means I need to allocate three external static IPs for my system of > two OpenBSD gateway hosts. > > Is there a less wasteful way of doing load balancing with carp using IPv4? > > -- > > > Med vdnliga hdlsningar / With kind regards > > Stefan Midjich > http://swehack.se
limit mbuf clusters
Hi there, we've 2 openbsd boxes used as firewall/router with pf and carp to host some websites and applications for a students and researchers lab. Sometimes the boxes reboot because they reach the mbuf cluster limit. Unfortunately not all the applications hosted in our lab work always correctly (and we cannot just put them offline) so I'm wondering if there's any way to limit the mbuf cluster with a pf rule on ip basis; so if one application has problem, it doesn't create problem to all the network and doesn't make the firewall crash. Thanks for your help! Rick sample output of our netstat: # netstat -m 134 mbufs in use: 130 mbufs allocated to data 1 mbuf allocated to packet headers 3 mbufs allocated to socket names and addresses 131/6089/6144 mbuf clusters in use (current/peak/max) 6512 Kbytes allocated to network (4% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines
More surprises
I gotta say I was pleasantly surprised again by the OpenBSD OS. I had been dual booting to another OS almost strictly for working with my BlackBerry. This other OS isn't based on FreeBSD and is distributed only in binary form. Well it finally happened that I had left my BB plugged into it's USB cable while I booted into OpenBSD. I was awestruck when I came back to the computer and noticed that the BB was happily charging. Holy f** shit I said to myself, about 30 seconds of digging and I found out why. You guys have had the uberry driver installed since 4.1. That is f** awesome now I can charge my BB and load my ipod without ever having to boot up to the virus trap. FREEEDOM. -- George Carlin - "Weather forecast for tonight: dark."
ftp-proxy and packetfilter + vlans
hey all, first of all: i'm not subscribed to the mailinglist, so please send replies to me personally too. thanks! :) what's this mail about? i'll explain how the network is setup. internet (0.0.0.0/0) | openbsdcluster -- windows management (192.168.2.0/24) | management network (192.168.0.0/24) | linux packet forwarding cluster | internal network (192.168.1.0/24) now, the external interface of the openbsd cluster is on bge0 the management interface is on em0, vlan 1 windows management interface is on em0, vlan 2 internal network is another vlan... but not defined on an interface of the openbsd cluster (du'uh ;)) those are the 3 networks the openbsd cluster is on now, i want ftp for my internal computers and for my windows management and other management network, so i thought: nat-anchor "ftp-proxy/*" rdr-anchor "ftp-proxy/*" rdr on vlan1 inet proto tcp from vlan2:network to any port 21 -> 127.0.0.1 port 8021 rdr on vlan1 inet proto tcp from 192.168.1.0/24 to any port 21 -> 127.0.0.1 port 8021 rdr on vlan2 inet proto tcp from vlan2:network to any port 21 -> 127.0.0.1 port 8021 but... that doesnt seem to work. what i did now as a workaround: rdr on em0 inet proto tcp from any to any port 21 -> 127.0.0.1 port 8021 this one seems to work perfectly... can anyone tell me what's wrong with defining vlans for traffic? if it's impossible, why? and what's the best/correct way of allowing traffic to the ftp proxy? should i do it like this?: rdr on em0 inet proto tcp from vlan2:network to any port 21 -> 127.0.0.1 port 8021 rdr on em0 inet proto tcp from 192.168.1.0/24 to any port 21 -> 127.0.0.1 port 8021 rdr on em0 inet proto tcp from vlan2:network to any port 21 -> 127.0.0.1 port 8021 thanks in advance! ps if there are more questions, don't hesitate to ask! -- harry aka Rik Bobbaers K.U.Leuven - LUDIT -=- Tel: +32 485 52 71 50 [EMAIL PROTECTED] -=- http://people.linux-vserver.org/~harry thinking always leads to conclusions... and those can be extremely dangerous -- me ;) Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
iic problem
I have an older Intel P3 board that uses the piixm sensor driver. For some reason it is still not working after all the fantastic work that has gone on. Here is the dmesg output of my problem: piixpm0 at pci0 dev 7 function 3 "Intel 82371AB Power" rev 0x02: SMI iic0 at piixpm0 iic0: addr 0x28 00=01 01=7f 02=2d 05=3c 06=80 07=80 08=80 09=80 0a=80 0b=80 0c=bf 0d=b6 0e=1b 0f=00 10=01 11=00 12=00 15=3c 16=80 17=3c 18=3c 19=3c 1a=3c 1b=3c 1c=bf 1d=b6 1e=1b 1f=00 20=c3 21=92 22=c2 23=88 24=c3 25=c2 26=c1 27=27 29=20 2a=04 2b=00 2c=00 2d=00 2e=00 2f=00 30=00 31=00 32=00 33=00 34=00 35=00 36=00 37=00 38=00 39=00 3a=00 3b=04 3c=10 3d=00 3e=00 3f=40 40=01 41=00 42=00 45=3c 46=80 47=80 48=00 49=80 4a=00 4b=80 4c=bf 4d=b6 4e=1b 4f=00 50=01 51=00 52=00 55=3c 56=80 57=80 58=80 59=80 5a=80 5b=80 5c=bf 5d=b6 5e=1b 5f=00 60=c3 61=92 62=c2 63=88 64=c3 65=c2 66=c1 67=27 69=20 6a=04 6b=00 6c=00 6d=00 6e=00 6f=00 70=00 71=00 72=00 73=00 74=00 75=00 76=00 77=00 78=00 79=00 7a=00 7b=04 7c=10 7d=00 7e=00 7f=40 80=01 81=00 82=00 85=3c 86=80 87=80 88=80 89=80 8a=80 8b=80 8c=bf 8d=b6 8e=1b 8f=00 90=01 91=00 92=00 95=3c 96=80 97=80 98=80 99=80 9a=80 9b=80 9c=bf 9d=b6 9e=1b 9f=00 a0=c3 a1=92 a2=c2 a3=88 a4=c3 a5=c2 a6=c1 a7=27 a9=20 aa=04 ab=00 ac=00 ad=00 ae=00 af=00 b0=00 b1=00 b2=00 b3=00 b4=00 b5=00 b6=00 b7=00 b8=00 b9=00 ba=00 bb=04 bc=10 bd=00 be=00 bf=40 c0=01 c1=00 c2=00 c5=3c c6=80 c7=80 c8=80 c9=80 ca=80 cb=80 cc=bf cd=b6 ce=1b cf=00 d0=01 d1=00 d2=00 d5=3c d6=80 d7=80 d8=80 d9=80 da=80 db=80 dc=bf dd=b6 de=1b df=00 e0=c3 e1=92 e2=c2 e3=88 e4=c3 e5=c2 e6=c1 e7=27 e9=20 ea=04 eb=00 ec=00 ed=00 ee=00 ef=00 f0=00 f1=00 f2=00 f3=00 f4=00 f5=00 f6=00 f7=00 f8=00 f9=00 fa=00 fb=04 fc=10 fd=00 fe=00 ff=40 I used to see this kind of 'dump' on my other machine until the asb100 fixes were put in. The box is running -current as of January 29. Rik
Re: mplayer-port - No picture but sound works well?
Jacob Meuser wrote: On Sun, May 01, 2005 at 12:43:13AM +0200, [EMAIL PROTECTED] wrote: I'm sorry maybe somebody else noticed that "problem" already but I noticed that mplayer displays nothing if I wanna watch a movie. I can hear the sound but there nothing visual (realy nothing, just sound output). "a movie" isn't very descriptive. it could mean lots an lots of things. I did the same things like on 3.6 and installed mplayer from the ports but seams it's brocken in 3.7. not sure what 'brocken' means. probably not broken, because I watch "movies" with mplayer all the time. I tested it on AMD64 and i386 (diffrent computers) and would be happy if somebody could tell me what I missed this time. how about reading the information mplayer prints when it starts? I get if you install the win32-codecs port your problem will be solved. Rik
