Well well well there is one thing its ocurring that i cant figure out. im getting some relay site3 session 3370 (502 active), 0, 190.179.249.128 -> :0, buffer event timeout
And after a couple a minutes (i couldnt take note exactly how many) relayd get restarted Is there any clue where to look into? Thanks in advance Saludos.- Leonardo Santagostini <http://ar.linkedin.com/in/santagostini> 2013/11/14 Leonardo Santagostini <lsantagost...@gmail.com> > Ok, just added my second website to both servers like your recommendation. > > I will post my config before the end of the day just to share it with you. > > Thank you so much !!! > > Regards > > Saludos.- > Leonardo Santagostini > > <http://ar.linkedin.com/in/santagostini> > > > > > > 2013/11/14 Leonardo Santagostini <lsantagost...@gmail.com> > >> Thanks a lot to all, i will give it a try and gives tou you feedback as >> soon as it get implemented. >> >> Saludos.- >> Leonardo Santagostini >> >> <http://ar.linkedin.com/in/santagostini> >> >> >> >> >> >> 2013/11/14 Andy <a...@brandwatch.com> >> >>> On 14/11/13 15:21, Leonardo Santagostini wrote: >>> >>> Hello misc, >>> >>> Im doing my final approach to put a production system with >>> carp+pfsync+relayd on production. >>> >>> The point is that im facing some trouble setting more than one ip alias >>> address with different vhid and different passwd. >>> >>> So, this is the scenario. >>> >>> Im trying to relayd more or less 15 sites so i have conceptual doubts. >>> >>> 1) is it nesessary to create one carp interface for each one of my >>> internals VIP address >>> 2) my understanding is that i have to work with pf on my carp interfaces. >>> >>> I have tried to put two different VIP's on my carp, but whitout lucky. >>> >>> Here is the homework. >>> >>> [root@server ~]# uname -a >>> OpenBSD server.internaldomain.com 5.4 GENERIC#37 amd64 >>> [root@server ~]# >>> >>> [root@server ~]# cat /etc/hostname.em0 >>> inet 172.19.224.180 255.255.255.0 >>> >>> [root@server ~]# cat /etc/hostname.em1 >>> inet 172.19.226.231 255.255.255.0 172.19.226.255 >>> >>> [root@server ~]# cat /etc/hostname.carp0 >>> # inet alias 172.19.224.16 255.255.255.255 172.19.224.255 vhid 1 advskew 10 >>> carpdev em0 pass Ahsooqu3 >>> inet alias 172.19.224.131 255.255.255.0 172.19.224.255 vhid 2 advskew 10 >>> carpdev em0 pass Meixo9oe >>> # inet alias 172.19.224.41 255.255.255.255 172.19.224.255 vhid 3 advskew 10 >>> carpdev em0 pass av5eG9Gi >>> # inet alias 172.19.224.40 255.255.255.255 172.19.224.255 vhid 4 advskew 10 >>> carpdev em0 pass Rei6thai >>> # inet alias 172.19.224.181 255.255.255.0 172.19.224.255 vhid 5 advskew 10 >>> carpdev em0 pass Toobohz3 >>> # inet alias 172.19.224.182 255.255.255.255 172.19.224.255 vhid 6 adskew 10 >>> carpdev em0 pass Quahng6U >>> >>> CARP should look like this (master); >>> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >>> Ahsooqu3 advskew 0 >>> inet alias 172.19.224.131 255.255.255.255 >>> inet alias 172.19.224.41 255.255.255.255 >>> inet alias 172.19.224.40 255.255.255.255 >>> inet alias 172.19.224.181 255.255.255.255 >>> inet alias 172.19.224.182 255.255.255.255 >>> >>> And (backup); >>> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >>> Ahsooqu3 advskew 200 >>> inet alias 172.19.224.131 255.255.255.255 >>> inet alias 172.19.224.41 255.255.255.255 >>> inet alias 172.19.224.40 255.255.255.255 >>> inet alias 172.19.224.181 255.255.255.255 >>> inet alias 172.19.224.182 255.255.255.255 >>> >>> And yes the subnet masks for the alias' should be /32 and you will see a >>> warning in the logs during fail-over. This is fine, the devs just haven't >>> muted the check warning yet. >>> >>> You've done it right if 'netstat -rn' shows; >>> >>> 172.19.224.131 127.0.0.1 UGHS 0 0 33152 8 >>> lo0 >>> 172.19.224.131/32 172.19.224.131 U 0 0 - 4 >>> carp0 >>> >>> >>> [root@server ~]# cat /etc/hostname.pfsync0 >>> up syncdev em1 >>> >>> [root@server ~]# cat /etc/pf.conf >>> ext_if="carp0" >>> >>> You don't refer to CARP as an interface, it is simply a VRRP watchdog >>> interface (for example you cannot set the MTU on a CARP interface as it is >>> not really an interface. >>> Use the physical.. >>> >>> ext_if="em0" >>> >>> >>> set fingerprints "/etc/pf.os" >>> set optimization aggressive >>> set limit states 90000 >>> >>> Definitely needs to be higher! try 1 million.. >>> >>> >>> set limit src-nodes 65000 >>> >>> table <bad_ip> persist >>> table <internat_net> persist file "/etc/internal_net" >>> table <admitted_net> persist file "/etc/admitted.txt" >>> >>> # vip1_address = "172.19.224.181" >>> # vip2_address = "172.19.224.16" >>> vip3_address = "172.19.224.131" >>> # vip4_address = "172.19.224.41" >>> # vip5_address = "172.19.224.40" >>> >>> Just to keep you sane remember these rules; >>> # (SNAT) NATing is done before filtering, 'pass out on $if_ext from >>> $external_carp_ip1' (public address as src for outbound). >>> # (DNAT) RDRing is done before filtering, 'pass in on $if_ext from any >>> to $internal_ip1' (private address as dst for inbound). >>> >>> [image: OpenBSD_PF_flow] >>> >>> >>> # Dejo de procesar cuando se trata de las redes internas >>> pass in quick from <internat_net> to any >>> >>> # Dejo pasar las ips desde las redes permitidas >>> # pass in quick from <admitted_net> to $vip1_address >>> pass in quick from <admitted_net> to $vip3_address >>> >>> # Genero el block >>> block in quick from <bad_ip> >>> >>> Your 'block in quick's should be above your 'pass in quick's! >>> quick means stop evaluating and do this action now.. >>> >>> >>> block in log quick on $ext_if proto tcp from any os "NMAP" to any label >>> ExtNMAPScan >>> >>> # Proteccion contra nmap y herramientas similares >>> # block in quick on $ext_if proto tcp flags FUP/WEUAPRSF >>> block in quick on $ext_if proto tcp flags WEUAPRSF/WEUAPRSF >>> block in quick on $ext_if proto tcp flags SRAFU/WEUAPRSF >>> block in quick on $ext_if proto tcp flags /WEUAPRSF >>> block in quick on $ext_if proto tcp flags SR/SR >>> block in quick on $ext_if proto tcp flags SF/SF >>> block in quick from urpf-failed >>> >>> >>> # Aplico reglas de DoS y Syn Flood en site1 >>> # pass in log on $mob_if proto tcp to $vip1_address port www keep state >>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 100, max-src-conn-rate >>> 95/2, adaptive .start 6000, adaptive.end 12000, tcp.first >>> 15, tcp.opening 5, tcp.established 3600, tcp.closing 5, tcp.finwait 15, >>> tcp.closed 15, tcp.tsdiff 5) >>> >>> Be careful, Direct Server Return does require sloppy states but be >>> aware that this totally undermines state security! >>> You still need a firewall on the outside of the load balancer to >>> sanitize the flows.. Catch 22 as you cannot have the load balancer (with >>> DSR) on the same box as your PF filtering if you want load balancing (with >>> DSR) and full security.. >>> I.e. DSR and full state modulation/security are mutually exclusive.. >>> >>> # Aplico reglas de DoS y Syn Flood en site2 >>> # pass in on $ext_if proto tcp to $vip2_address port www keep state >>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>> 150/3) >>> >>> # Aplico reglas para site3 >>> pass in on $ext_if proto tcp to $vip3_address port www keep state (sloppy, >>> max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate 100/3) >>> >>> # Aplico reglas de DoS y Syn Flood en site4 >>> # pass in on $ext_if proto tcp to $vip4_address port www keep state >>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>> 100/3) >>> >>> # Aplico reglas de DoS y Syn Flood en site5 >>> # pass in on $ext_if proto tcp to $vip5_address port www keep state >>> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >>> 100/3) >>> >>> # Anchor Para relayd >>> anchor "relayd/*" >>> >>> >>> [root@server ~]# cat /etc/relayd.conf >>> # Archivo de configuracion de balanceo >>> >>> ## Opciones globales >>> interval 5 >>> timeout 500 >>> prefork 15 >>> log all >>> >>> ## Direcciones de las vip >>> # address1="172.19.224.16" >>> # address2="172.19.224.181" >>> address3="172.19.224.131" >>> # address4="172.19.224.41" >>> # address5="172.19.224.40" >>> >>> ## Direcciones de los servidores >>> wsapp1="172.19.224.200" >>> wsapp2="172.19.224.201" >>> webcache01="172.19.224.70" >>> webcache02="172.19.224.71" >>> webcache03="172.19.224.72" >>> webcache04="172.19.224.73" >>> >>> ## Definicion de Tablas >>> table <mobileweb> { $wsapp1 $wsapp2 } >>> table <webcaches> { $webcache01 $webcache02 $webcache03 $webcache04 } >>> table <webcaches1> { $webcache01 } >>> >>> ## Definicion de protocolos (Filtros) >>> >>> http protocol "httpSite1" { >>> >>> header change "Connection" to "close" >>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>> cookie hash "sessid" >>> >>> } >>> >>> http protocol "httpSite2" { >>> >>> header change "Connection" to "close" >>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>> cookie hash "sessid" >>> >>> } >>> >>> http protocol "httpSite3" { >>> >>> header change "Connection" to "close" >>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>> } >>> >>> http protocol "httpSite4" { >>> >>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>> >>> } >>> >>> http protocol "httpSite5" { >>> >>> header append "$REMOTE_ADDR" to "X-Forwarded-For" >>> >>> } >>> >>> ## Definicion de los relays >>> >>> #relay site1 { >>> # listen on $address2 port 80 >>> # protocol "httpSite1" >>> # forward to <mobileweb> port 80 mode roundrobin check http >>> "/relaycheck/index.php" code 200 >>> #} >>> >>> relay site2 { >>> listen on $address3 port 80 >>> protocol "httpSite2" >>> forward to <webcaches> port 80 mode roundrobin check http >>> "/monitoreo/relayd.txt" code 200 >>> } >>> >>> #relay site3 { >>> # listen on $address1 port 80 >>> # protocol "httpSite3" >>> # forward to <webcaches> port 80 mode roundrobin check http >>> "/monitoreo/relayd.txt" code 200 >>> #} >>> >>> #relay site4 { >>> # listen on $address4 port 80 >>> # protocol "httpSite4" >>> # forward to <webcaches> port 80 mode roundrobin check http >>> "/monitoreo/relayd.txt" code 200 >>> #} >>> >>> #relay site5 { >>> # listen on $address5 port 80 >>> # protocol "httpSite5" >>> # forward to <webcaches> port 80 mode roundrobin check http >>> "/monitoreo/relayd.txt" code 200 >>> #} >>> >>> Haven't implemented this myself so can't comment! >>> >>> Good luck :) >>> Andy. >>> >>> >>> >>> [root@server ~]# dmesg >>> OpenBSD 5.4 (GENERIC) #37: Tue Jul 30 15:24:05 MDT 2013 >>> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC >>> real mem = 2130694144 (2031MB) >>> avail mem = 2066341888 (1970MB) >>> mainbus0 at root >>> bios0 at mainbus0: SMBIOS rev. 2.4 @ 0x7ffffea0 (11 entries) >>> bios0: vendor Seabios version "0.5.1" date 01/01/2007 >>> bios0: Red Hat KVM >>> acpi0 at bios0: rev 0 >>> acpi0: sleep states S5 >>> acpi0: tables DSDT FACP SSDT APIC >>> acpi0: wakeup devices >>> acpitimer0 at acpi0: 3579545 Hz, 24 bits >>> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >>> acpiprt0 at acpi0: bus 0 (PCI0) >>> acpicpu0 at acpi0 >>> mpbios0 at bios0: Intel MP Specification 1.4 >>> cpu0 at mainbus0: apid 0 (boot processor) >>> cpu0: Opteron or Athlon 64, 2659.01 MHz >>> cpu0: >>> FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2,SSE3,POPCNT >>> cpu0: smt 0, core 0, package 0 >>> cpu0: apic clock running at 1000MHz >>> cpu at mainbus0: not configured >>> mpbios0: bus 0 is type PCI >>> mpbios0: bus 1 is type ISA >>> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins >>> ioapic0: misconfigured as apic 0, remapped to apid 2 >>> pci0 at mainbus0 bus 0 >>> pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02 >>> pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00 >>> pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel >>> 0 wired to compatibility, channel 1 wired to compatibility >>> wd0 at pciide0 channel 0 drive 0: <QEMU HARDDISK> >>> wd0: 16-sector PIO, LBA48, 5000MB, 10240000 sectors >>> wd0(pciide0:0:0): using PIO mode 0, DMA mode 2 >>> atapiscsi0 at pciide0 channel 1 drive 0 >>> scsibus0 at atapiscsi0: 2 targets >>> cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 0.12> ATAPI 5/cdrom >>> removable >>> cd0(pciide0:1:0): using PIO mode 0 >>> uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 2 int 11 >>> piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 2 int >>> 9 >>> iic0 at piixpm0 >>> iic0: addr 0x4c 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >>> 06=0000 07=0000 >>> iic0: addr 0x4e 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >>> 06=0000 07=0000 >>> vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00 >>> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >>> wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >>> em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x03: apic 2 int 11, >>> address 52:54:00:4a:77:22 >>> virtio0 at pci0 dev 4 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio >>> Memory Balloon Device >>> viomb0 at virtio0 >>> virtio0: apic 2 int 11 >>> em1 at pci0 dev 5 function 0 "Intel 82540EM" rev 0x03: apic 2 int 10, >>> address 52:54:00:1e:57:27 >>> isa0 at pcib0 >>> isadma0 at isa0 >>> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >>> com0: probed fifo depth: 0 bytes >>> pckbc0 at isa0 port 0x60/5 >>> pckbd0 at pckbc0 (kbd slot) >>> pckbc0: using irq 1 for kbd slot >>> wskbd0 at pckbd0: console keyboard, using wsdisplay0 >>> pms0 at pckbc0 (aux slot) >>> pckbc0: using irq 12 for aux slot >>> wsmouse0 at pms0 mux 0 >>> pcppi0 at isa0 port 0x61 >>> spkr0 at pcppi0 >>> fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 >>> fd0 at fdc0 drive 0: density unknown >>> fd1 at fdc0 drive 1: density unknown >>> usb0 at uhci0: USB revision 1.0 >>> uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >>> nvram: invalid checksum >>> vscsi0 at root >>> scsibus1 at vscsi0: 256 targets >>> softraid0 at root >>> scsibus2 at softraid0: 256 targets >>> root on wd0a (bf96a0273abe52bf.a) swap on wd0b dump on wd0b >>> WARNING: / was not properly unmounted >>> clock: unknown CMOS layout >>> carp: pfsync0 demoted group carp by 32 to 160 (pfsync init) >>> carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) >>> carp: pfsync0 demoted group carp by 1 to 161 (pfsync bulk start) >>> carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) >>> carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk fail) >>> carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk fail) >>> carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) >>> carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) >>> pfsync: failed to receive bulk update >>> carp0: state transition: BACKUP -> MASTER >>> carp0: state transition: BACKUP -> MASTER >>> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >>> carp0: state transition: BACKUP -> MASTER >>> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >>> >>> >>> Two more things >>> >>> 1) Sorry for my english, is not my mothers tongue >>> 2) Thank you for doing this great operantig system >>> >>> Saludos / Regards >>> Leonardo Santagostini [demime 1.01d removed an attachment of type image/png which had a name of OpenBSD_PF_flow.png]
