Ok im not at the office now. But tomorrow we could do more test. Regards and thank you !!! El nov 14, 2013 8:01 p.m., "Andy Lemin" <a...@brandwatch.com> escribió:
> In fact thinking about it if think that is a relayd issue somewhere and > not pf at all.. > > Sent from my iPhone > > On 14 Nov 2013, at 19:37, Leonardo Santagostini <lsantagost...@gmail.com> > wrote: > > Well well well there is one thing its ocurring that i cant figure out. > > im getting some relay site3 session 3370 (502 active), 0, 190.179.249.128 > -> :0, buffer event timeout > > And after a couple a minutes (i couldnt take note exactly how many) relayd > get restarted > > Is there any clue where to look into? > > Thanks in advance > > > Saludos.- > Leonardo Santagostini > > <http://ar.linkedin.com/in/santagostini> > > > > > > 2013/11/14 Leonardo Santagostini <lsantagost...@gmail.com> > >> Ok, just added my second website to both servers like your recommendation. >> >> I will post my config before the end of the day just to share it with you. >> >> Thank you so much !!! >> >> Regards >> >> Saludos.- >> Leonardo Santagostini >> >> <http://ar.linkedin.com/in/santagostini> >> >> >> >> >> >> 2013/11/14 Leonardo Santagostini <lsantagost...@gmail.com> >> >>> Thanks a lot to all, i will give it a try and gives tou you feedback as >>> soon as it get implemented. >>> >>> Saludos.- >>> Leonardo Santagostini >>> >>> <http://ar.linkedin.com/in/santagostini> >>> >>> >>> >>> >>> >>> 2013/11/14 Andy <a...@brandwatch.com> >>> >>>> On 14/11/13 15:21, Leonardo Santagostini wrote: >>>> >>>> Hello misc, >>>> >>>> Im doing my final approach to put a production system with >>>> carp+pfsync+relayd on production. >>>> >>>> The point is that im facing some trouble setting more than one ip alias >>>> address with different vhid and different passwd. >>>> >>>> So, this is the scenario. >>>> >>>> Im trying to relayd more or less 15 sites so i have conceptual doubts. >>>> >>>> 1) is it nesessary to create one carp interface for each one of my >>>> internals VIP address >>>> 2) my understanding is that i have to work with pf on my carp interfaces. >>>> >>>> I have tried to put two different VIP's on my carp, but whitout lucky. >>>> >>>> Here is the homework. >>>> >>>> [root@server ~]# uname -a >>>> OpenBSD server.internaldomain.com 5.4 GENERIC#37 amd64 >>>> [root@server ~]# >>>> >>>> [root@server ~]# cat /etc/hostname.em0 >>>> inet 172.19.224.180 255.255.255.0 >>>> >>>> [root@server ~]# cat /etc/hostname.em1 >>>> inet 172.19.226.231 255.255.255.0 172.19.226.255 >>>> >>>> [root@server ~]# cat /etc/hostname.carp0 >>>> # inet alias 172.19.224.16 255.255.255.255 172.19.224.255 vhid 1 advskew 10 >>>> carpdev em0 pass Ahsooqu3 >>>> inet alias 172.19.224.131 255.255.255.0 172.19.224.255 vhid 2 advskew 10 >>>> carpdev em0 pass Meixo9oe >>>> # inet alias 172.19.224.41 255.255.255.255 172.19.224.255 vhid 3 advskew 10 >>>> carpdev em0 pass av5eG9Gi >>>> # inet alias 172.19.224.40 255.255.255.255 172.19.224.255 vhid 4 advskew 10 >>>> carpdev em0 pass Rei6thai >>>> # inet alias 172.19.224.181 255.255.255.0 172.19.224.255 vhid 5 advskew 10 >>>> carpdev em0 pass Toobohz3 >>>> # inet alias 172.19.224.182 255.255.255.255 172.19.224.255 vhid 6 adskew 10 >>>> carpdev em0 pass Quahng6U >>>> >>>> CARP should look like this (master); >>>> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >>>> Ahsooqu3 advskew 0 >>>> inet alias 172.19.224.131 255.255.255.255 >>>> inet alias 172.19.224.41 255.255.255.255 >>>> inet alias 172.19.224.40 255.255.255.255 >>>> inet alias 172.19.224.181 255.255.255.255 >>>> inet alias 172.19.224.182 255.255.255.255 >>>> >>>> And (backup); >>>> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >>>> Ahsooqu3 advskew 200 >>>> inet alias 172.19.224.131 255.255.255.255 >>>> inet alias 172.19.224.41 255.255.255.255 >>>> inet alias 172.19.224.40 255.255.255.255 >>>> inet alias 172.19.224.181 255.255.255.255 >>>> inet alias 172.19.224.182 255.255.255.255 >>>> >>>> And yes the subnet masks for the alias' should be /32 and you will see >>>> a warning in the logs during fail-over. This is fine, the devs just haven't >>>> muted the check warning yet. >>>> >>>> You've done it right if 'netstat -rn' shows; >>>> >>>> 172.19.224.131 127.0.0.1 UGHS 0 0 33152 8 >>>> lo0 >>>> 172.19.224.131/32 172.19.224.131 U 0 0 - >>>> 4 carp0 >>>> >>>> >>>> [root@server ~]# cat /etc/hostname.pfsync0 >>>> up syncdev em1 >>>> >>>> [root@server ~]# cat /etc/pf.conf >>>> ext_if="carp0" >>>> >>>> You don't refer to CARP as an interface, it is simply a VRRP watchdog >>>> interface (for example you cannot set the MTU on a CARP interface as it is >>>> not really an interface. >>>> Use the physical.. >>>> >>>> ext_if="em0" >>>> >>>> >>>> set fingerprints "/etc/pf.os" >>>> set optimization aggressive >>>> set limit states 90000 >>>> >>>> Definitely needs to be higher! try 1 million.. >>>> >>>> >>>> set limit src-nodes 65000 >>>> >>>> table <bad_ip> persist >>>> table <internat_net> persist file "/etc/internal_net" >>>> table <admitted_net> persist file "/etc/admitted.txt" >>>> >>>> # vip1_address = "172.19.224.181" >>>> # vip2_address = "172.19.224.16" >>>> vip3_address = "172.19.224.131" >>>> # vip4_address = "172.19.224.41" >>>> # vip5_address = "172.19.224.40" >>>> >>>> Just to keep you sane remember these rules; >>>> # (SNAT) NATing is done before filtering, 'pass out on $if_ext from >>>> $external_carp_ip1' (public address as src for outbound). >>>> # (DNAT) RDRing is done before filtering, 'pass in on $if_ext from any >>>> to $internal_ip1' (private address as dst for inbound). >>>> >>>> <OpenBSD_PF_flow.png> >>>> >>>> >>>> # Dejo de procesar cuando se trata de las redes internas >>>> pass in quick from <internat_net> to any >>>> >>>> # Dejo pasar las ips desde las redes permitidas >>>> # pass in quick from <admitted_net> to $vip1_address >>>> pass in quick from <admitted_net> to $vip3_address >>>> >>>> # Genero el block >>>> block in quick from <bad_ip> >>>> >>>> Your 'block in quick's should be above your 'pass in quick's! >>>> quick means stop evaluating and do this action now.. >>>> >>>> >>>> block in log quick on $ext_if proto tcp from any os "NMAP" to any label >>>> ExtNMAPScan >>>> >>>> # Proteccion contra nmap y herramientas similares >>>> # block in quick on $ext_if proto tcp flags FUP/WEUAPRSF >>>> block in quick on $ext_if proto tcp flags WEUAPRSF/WEUAPRSF >>>> block in quick on $ext_if proto tcp flags SRAFU/WEUAPRSF >>>> block in quick on $ext_if proto tcp flags /WEUAPRSF >>>> block in quick on $ext_if proto tcp flags SR/SR >>>> block in quick on $ext_if proto tcp flags SF/SF >>>> block in quick from urpf-failed >>>> >>>> >>>> # Aplico reglas de DoS y Syn Flood en site1 >>>> # pass in log on $mob_if proto tcp to $vip1_address port www keep state >>>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 100, max-src-conn-rate >>>> 95/2, adaptive .start 6000, adaptive.end 12000, tcp.first >>>> 15, tcp.opening 5, tcp.established 3600, tcp.closing 5, tcp.finwait 15, >>>> tcp.closed 15, tcp.tsdiff 5) >>>> >>>> Be careful, Direct Server Return does require sloppy states but be >>>> aware that this totally undermines state security! >>>> You still need a firewall on the outside of the load balancer to >>>> sanitize the flows.. Catch 22 as you cannot have the load balancer (with >>>> DSR) on the same box as your PF filtering if you want load balancing (with >>>> DSR) and full security.. >>>> I.e. DSR and full state modulation/security are mutually exclusive.. >>>> >>>> # Aplico reglas de DoS y Syn Flood en site2 >>>> # pass in on $ext_if proto tcp to $vip2_address port www keep state >>>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>>> 150/3) >>>> >>>> # Aplico reglas para site3 >>>> pass in on $ext_if proto tcp to $vip3_address port www keep state (sloppy, >>>> max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate 100/3) >>>> >>>> # Aplico reglas de DoS y Syn Flood en site4 >>>> # pass in on $ext_if proto tcp to $vip4_address port www keep state >>>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>>> 100/3) >>>> >>>> # Aplico reglas de DoS y Syn Flood en site5 >>>> # pass in on $ext_if proto tcp to $vip5_address port www keep state >>>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>>> 100/3) >>>> >>>> # Anchor Para relayd >>>> anchor "relayd/*" >>>> >>>> >>>> [root@server ~]# cat /etc/relayd.conf >>>> # Archivo de configuracion de balanceo >>>> >>>> ## Opciones globales >>>> interval 5 >>>> timeout 500 >>>> prefork 15 >>>> log all >>>> >>>> ## Direcciones de las vip >>>> # address1="172.19.224.16" >>>> # address2="172.19.224.181" >>>> address3="172.19.224.131" >>>> # address4="172.19.224.41" >>>> # address5="172.19.224.40" >>>> >>>> ## Direcciones de los servidores >>>> wsapp1="172.19.224.200" >>>> wsapp2="172.19.224.201" >>>> webcache01="172.19.224.70" >>>> webcache02="172.19.224.71" >>>> webcache03="172.19.224.72" >>>> webcache04="172.19.224.73" >>>> >>>> ## Definicion de Tablas >>>> table <mobileweb> { $wsapp1 $wsapp2 } >>>> table <webcaches> { $webcache01 $webcache02 $webcache03 $webcache04 } >>>> table <webcaches1> { $webcache01 } >>>> >>>> ## Definicion de protocolos (Filtros) >>>> >>>> http protocol "httpSite1" { >>>> >>>> header change "Connection" to "close" >>>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>>> cookie hash "sessid" >>>> >>>> } >>>> >>>> http protocol "httpSite2" { >>>> >>>> header change "Connection" to "close" >>>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>>> cookie hash "sessid" >>>> >>>> } >>>> >>>> http protocol "httpSite3" { >>>> >>>> header change "Connection" to "close" >>>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>>> } >>>> >>>> http protocol "httpSite4" { >>>> >>>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>>> >>>> } >>>> >>>> http protocol "httpSite5" { >>>> >>>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>>> >>>> } >>>> >>>> ## Definicion de los relays >>>> >>>> #relay site1 { >>>> # listen on $address2 port 80 >>>> # protocol "httpSite1" >>>> # forward to <mobileweb> port 80 mode roundrobin check http >>>> "/relaycheck/index.php" code 200 >>>> #} >>>> >>>> relay site2 { >>>> listen on $address3 port 80 >>>> protocol "httpSite2" >>>> forward to <webcaches> port 80 mode roundrobin check http >>>> "/monitoreo/relayd.txt" code 200 >>>> } >>>> >>>> #relay site3 { >>>> # listen on $address1 port 80 >>>> # protocol "httpSite3" >>>> # forward to <webcaches> port 80 mode roundrobin check http >>>> "/monitoreo/relayd.txt" code 200 >>>> #} >>>> >>>> #relay site4 { >>>> # listen on $address4 port 80 >>>> # protocol "httpSite4" >>>> # forward to <webcaches> port 80 mode roundrobin check http >>>> "/monitoreo/relayd.txt" code 200 >>>> #} >>>> >>>> #relay site5 { >>>> # listen on $address5 port 80 >>>> # protocol "httpSite5" >>>> # forward to <webcaches> port 80 mode roundrobin check http >>>> "/monitoreo/relayd.txt" code 200 >>>> #} >>>> >>>> Haven't implemented this myself so can't comment! >>>> >>>> Good luck :) >>>> Andy. >>>> >>>> >>>> >>>> [root@server ~]# dmesg >>>> OpenBSD 5.4 (GENERIC) #37: Tue Jul 30 15:24:05 MDT 2013 >>>> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC >>>> real mem = 2130694144 (2031MB) >>>> avail mem = 2066341888 (1970MB) >>>> mainbus0 at root >>>> bios0 at mainbus0: SMBIOS rev. 2.4 @ 0x7ffffea0 (11 entries) >>>> bios0: vendor Seabios version "0.5.1" date 01/01/2007 >>>> bios0: Red Hat KVM >>>> acpi0 at bios0: rev 0 >>>> acpi0: sleep states S5 >>>> acpi0: tables DSDT FACP SSDT APIC >>>> acpi0: wakeup devices >>>> acpitimer0 at acpi0: 3579545 Hz, 24 bits >>>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >>>> acpiprt0 at acpi0: bus 0 (PCI0) >>>> acpicpu0 at acpi0 >>>> mpbios0 at bios0: Intel MP Specification 1.4 >>>> cpu0 at mainbus0: apid 0 (boot processor) >>>> cpu0: Opteron or Athlon 64, 2659.01 MHz >>>> cpu0: >>>> FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2,SSE3,POPCN T >>>> cpu0: smt 0, core 0, package 0 >>>> cpu0: apic clock running at 1000MHz >>>> cpu at mainbus0: not configured >>>> mpbios0: bus 0 is type PCI >>>> mpbios0: bus 1 is type ISA >>>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins >>>> ioapic0: misconfigured as apic 0, remapped to apid 2 >>>> pci0 at mainbus0 bus 0 >>>> pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02 >>>> pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00 >>>> pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel >>>> 0 wired to compatibility, channel 1 wired to compatibility >>>> wd0 at pciide0 channel 0 drive 0: <QEMU HARDDISK> >>>> wd0: 16-sector PIO, LBA48, 5000MB, 10240000 sectors >>>> wd0(pciide0:0:0): using PIO mode 0, DMA mode 2 >>>> atapiscsi0 at pciide0 channel 1 drive 0 >>>> scsibus0 at atapiscsi0: 2 targets >>>> cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 0.12> ATAPI 5/cdrom >>>> removable >>>> cd0(pciide0:1:0): using PIO mode 0 >>>> uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 2 int 11 >>>> piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 2 int >>>> 9 >>>> iic0 at piixpm0 >>>> iic0: addr 0x4c 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >>>> 06=0000 07=0000 >>>> iic0: addr 0x4e 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >>>> 06=0000 07=0000 >>>> vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00 >>>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >>>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >>>> em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x03: apic 2 int 11, >>>> address 52:54:00:4a:77:22 >>>> virtio0 at pci0 dev 4 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio >>>> Memory Balloon Device >>>> viomb0 at virtio0 >>>> virtio0: apic 2 int 11 >>>> em1 at pci0 dev 5 function 0 "Intel 82540EM" rev 0x03: apic 2 int 10, >>>> address 52:54:00:1e:57:27 >>>> isa0 at pcib0 >>>> isadma0 at isa0 >>>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >>>> com0: probed fifo depth: 0 bytes >>>> pckbc0 at isa0 port 0x60/5 >>>> pckbd0 at pckbc0 (kbd slot) >>>> pckbc0: using irq 1 for kbd slot >>>> wskbd0 at pckbd0: console keyboard, using wsdisplay0 >>>> pms0 at pckbc0 (aux slot) >>>> pckbc0: using irq 12 for aux slot >>>> wsmouse0 at pms0 mux 0 >>>> pcppi0 at isa0 port 0x61 >>>> spkr0 at pcppi0 >>>> fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 >>>> fd0 at fdc0 drive 0: density unknown >>>> fd1 at fdc0 drive 1: density unknown >>>> usb0 at uhci0: USB revision 1.0 >>>> uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>>> nvram: invalid checksum >>>> vscsi0 at root >>>> scsibus1 at vscsi0: 256 targets >>>> softraid0 at root >>>> scsibus2 at softraid0: 256 targets >>>> root on wd0a (bf96a0273abe52bf.a) swap on wd0b dump on wd0b >>>> WARNING: / was not properly unmounted >>>> clock: unknown CMOS layout >>>> carp: pfsync0 demoted group carp by 32 to 160 (pfsync init) >>>> carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) >>>> carp: pfsync0 demoted group carp by 1 to 161 (pfsync bulk start) >>>> carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) >>>> carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk fail) >>>> carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk fail) >>>> carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) >>>> carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) >>>> pfsync: failed to receive bulk update >>>> carp0: state transition: BACKUP -> MASTER >>>> carp0: state transition: BACKUP -> MASTER >>>> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >>>> carp0: state transition: BACKUP -> MASTER >>>> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >>>> >>>> >>>> Two more things >>>> >>>> 1) Sorry for my english, is not my mothers tongue >>>> 2) Thank you for doing this great operantig system >>>> >>>> Saludos / Regards >>>> Leonardo Santagostini
