Thanks a lot to all, i will give it a try and gives tou you feedback as soon as it get implemented.
Saludos.- Leonardo Santagostini <http://ar.linkedin.com/in/santagostini> 2013/11/14 Andy <a...@brandwatch.com> > On 14/11/13 15:21, Leonardo Santagostini wrote: > > Hello misc, > > Im doing my final approach to put a production system with > carp+pfsync+relayd on production. > > The point is that im facing some trouble setting more than one ip alias > address with different vhid and different passwd. > > So, this is the scenario. > > Im trying to relayd more or less 15 sites so i have conceptual doubts. > > 1) is it nesessary to create one carp interface for each one of my > internals VIP address > 2) my understanding is that i have to work with pf on my carp interfaces. > > I have tried to put two different VIP's on my carp, but whitout lucky. > > Here is the homework. > > [root@server ~]# uname -a > OpenBSD server.internaldomain.com 5.4 GENERIC#37 amd64 > [root@server ~]# > > [root@server ~]# cat /etc/hostname.em0 > inet 172.19.224.180 255.255.255.0 > > [root@server ~]# cat /etc/hostname.em1 > inet 172.19.226.231 255.255.255.0 172.19.226.255 > > [root@server ~]# cat /etc/hostname.carp0 > # inet alias 172.19.224.16 255.255.255.255 172.19.224.255 vhid 1 advskew 10 > carpdev em0 pass Ahsooqu3 > inet alias 172.19.224.131 255.255.255.0 172.19.224.255 vhid 2 advskew 10 > carpdev em0 pass Meixo9oe > # inet alias 172.19.224.41 255.255.255.255 172.19.224.255 vhid 3 advskew 10 > carpdev em0 pass av5eG9Gi > # inet alias 172.19.224.40 255.255.255.255 172.19.224.255 vhid 4 advskew 10 > carpdev em0 pass Rei6thai > # inet alias 172.19.224.181 255.255.255.0 172.19.224.255 vhid 5 advskew 10 > carpdev em0 pass Toobohz3 > # inet alias 172.19.224.182 255.255.255.255 172.19.224.255 vhid 6 adskew 10 > carpdev em0 pass Quahng6U > > CARP should look like this (master); > inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass > Ahsooqu3 advskew 0 > inet alias 172.19.224.131 255.255.255.255 > inet alias 172.19.224.41 255.255.255.255 > inet alias 172.19.224.40 255.255.255.255 > inet alias 172.19.224.181 255.255.255.255 > inet alias 172.19.224.182 255.255.255.255 > > And (backup); > inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass > Ahsooqu3 advskew 200 > inet alias 172.19.224.131 255.255.255.255 > inet alias 172.19.224.41 255.255.255.255 > inet alias 172.19.224.40 255.255.255.255 > inet alias 172.19.224.181 255.255.255.255 > inet alias 172.19.224.182 255.255.255.255 > > And yes the subnet masks for the alias' should be /32 and you will see a > warning in the logs during fail-over. This is fine, the devs just haven't > muted the check warning yet. > > You've done it right if 'netstat -rn' shows; > > 172.19.224.131 127.0.0.1 UGHS 0 0 33152 8 > lo0 > 172.19.224.131/32 172.19.224.131 U 0 0 - 4 > carp0 > > > [root@server ~]# cat /etc/hostname.pfsync0 > up syncdev em1 > > [root@server ~]# cat /etc/pf.conf > ext_if="carp0" > > You don't refer to CARP as an interface, it is simply a VRRP watchdog > interface (for example you cannot set the MTU on a CARP interface as it is > not really an interface. > Use the physical.. > > ext_if="em0" > > > > set fingerprints "/etc/pf.os" > set optimization aggressive > set limit states 90000 > > Definitely needs to be higher! try 1 million.. > > > set limit src-nodes 65000 > > table <bad_ip> persist > table <internat_net> persist file "/etc/internal_net" > table <admitted_net> persist file "/etc/admitted.txt" > > # vip1_address = "172.19.224.181" > # vip2_address = "172.19.224.16" > vip3_address = "172.19.224.131" > # vip4_address = "172.19.224.41" > # vip5_address = "172.19.224.40" > > Just to keep you sane remember these rules; > # (SNAT) NATing is done before filtering, 'pass out on $if_ext from > $external_carp_ip1' (public address as src for outbound). > # (DNAT) RDRing is done before filtering, 'pass in on $if_ext from any to > $internal_ip1' (private address as dst for inbound). > > [image: OpenBSD_PF_flow] > > > > # Dejo de procesar cuando se trata de las redes internas > pass in quick from <internat_net> to any > > # Dejo pasar las ips desde las redes permitidas > # pass in quick from <admitted_net> to $vip1_address > pass in quick from <admitted_net> to $vip3_address > > # Genero el block > block in quick from <bad_ip> > > Your 'block in quick's should be above your 'pass in quick's! > quick means stop evaluating and do this action now.. > > > block in log quick on $ext_if proto tcp from any os "NMAP" to any label > ExtNMAPScan > > # Proteccion contra nmap y herramientas similares > # block in quick on $ext_if proto tcp flags FUP/WEUAPRSF > block in quick on $ext_if proto tcp flags WEUAPRSF/WEUAPRSF > block in quick on $ext_if proto tcp flags SRAFU/WEUAPRSF > block in quick on $ext_if proto tcp flags /WEUAPRSF > block in quick on $ext_if proto tcp flags SR/SR > block in quick on $ext_if proto tcp flags SF/SF > block in quick from urpf-failed > > > # Aplico reglas de DoS y Syn Flood en site1 > # pass in log on $mob_if proto tcp to $vip1_address port www keep state > (sloppy, max 10000, max-src-nodes 5000, max-src-conn 100, max-src-conn-rate > 95/2, adaptive .start 6000, adaptive.end 12000, tcp.first > 15, tcp.opening 5, tcp.established 3600, tcp.closing 5, tcp.finwait 15, > tcp.closed 15, tcp.tsdiff 5) > > Be careful, Direct Server Return does require sloppy states but be aware > that this totally undermines state security! > You still need a firewall on the outside of the load balancer to sanitize > the flows.. Catch 22 as you cannot have the load balancer (with DSR) on the > same box as your PF filtering if you want load balancing (with DSR) and > full security.. > I.e. DSR and full state modulation/security are mutually exclusive.. > > > # Aplico reglas de DoS y Syn Flood en site2 > # pass in on $ext_if proto tcp to $vip2_address port www keep state > (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate > 150/3) > > # Aplico reglas para site3 > pass in on $ext_if proto tcp to $vip3_address port www keep state (sloppy, > max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate 100/3) > > # Aplico reglas de DoS y Syn Flood en site4 > # pass in on $ext_if proto tcp to $vip4_address port www keep state > (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate > 100/3) > > # Aplico reglas de DoS y Syn Flood en site5 > # pass in on $ext_if proto tcp to $vip5_address port www keep state > (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate > 100/3) > > # Anchor Para relayd > anchor "relayd/*" > > > [root@server ~]# cat /etc/relayd.conf > # Archivo de configuracion de balanceo > > ## Opciones globales > interval 5 > timeout 500 > prefork 15 > log all > > ## Direcciones de las vip > # address1="172.19.224.16" > # address2="172.19.224.181" > address3="172.19.224.131" > # address4="172.19.224.41" > # address5="172.19.224.40" > > ## Direcciones de los servidores > wsapp1="172.19.224.200" > wsapp2="172.19.224.201" > webcache01="172.19.224.70" > webcache02="172.19.224.71" > webcache03="172.19.224.72" > webcache04="172.19.224.73" > > ## Definicion de Tablas > table <mobileweb> { $wsapp1 $wsapp2 } > table <webcaches> { $webcache01 $webcache02 $webcache03 $webcache04 } > table <webcaches1> { $webcache01 } > > ## Definicion de protocolos (Filtros) > > http protocol "httpSite1" { > > header change "Connection" to "close" > header append "$REMOTE_ADDR" to "X-Forwarded-For" > cookie hash "sessid" > > } > > http protocol "httpSite2" { > > header change "Connection" to "close" > header append "$REMOTE_ADDR" to "X-Forwarded-For" > cookie hash "sessid" > > } > > http protocol "httpSite3" { > > header change "Connection" to "close" > header append "$REMOTE_ADDR" to "X-Forwarded-For" > } > > http protocol "httpSite4" { > > header append "$REMOTE_ADDR" to "X-Forwarded-For" > > } > > http protocol "httpSite5" { > > header append "$REMOTE_ADDR" to "X-Forwarded-For" > > } > > ## Definicion de los relays > > #relay site1 { > # listen on $address2 port 80 > # protocol "httpSite1" > # forward to <mobileweb> port 80 mode roundrobin check http > "/relaycheck/index.php" code 200 > #} > > relay site2 { > listen on $address3 port 80 > protocol "httpSite2" > forward to <webcaches> port 80 mode roundrobin check http > "/monitoreo/relayd.txt" code 200 > } > > #relay site3 { > # listen on $address1 port 80 > # protocol "httpSite3" > # forward to <webcaches> port 80 mode roundrobin check http > "/monitoreo/relayd.txt" code 200 > #} > > #relay site4 { > # listen on $address4 port 80 > # protocol "httpSite4" > # forward to <webcaches> port 80 mode roundrobin check http > "/monitoreo/relayd.txt" code 200 > #} > > #relay site5 { > # listen on $address5 port 80 > # protocol "httpSite5" > # forward to <webcaches> port 80 mode roundrobin check http > "/monitoreo/relayd.txt" code 200 > #} > > Haven't implemented this myself so can't comment! > > Good luck :) > Andy. > > > > > [root@server ~]# dmesg > OpenBSD 5.4 (GENERIC) #37: Tue Jul 30 15:24:05 MDT 2013 > dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC > real mem = 2130694144 (2031MB) > avail mem = 2066341888 (1970MB) > mainbus0 at root > bios0 at mainbus0: SMBIOS rev. 2.4 @ 0x7ffffea0 (11 entries) > bios0: vendor Seabios version "0.5.1" date 01/01/2007 > bios0: Red Hat KVM > acpi0 at bios0: rev 0 > acpi0: sleep states S5 > acpi0: tables DSDT FACP SSDT APIC > acpi0: wakeup devices > acpitimer0 at acpi0: 3579545 Hz, 24 bits > acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat > acpiprt0 at acpi0: bus 0 (PCI0) > acpicpu0 at acpi0 > mpbios0 at bios0: Intel MP Specification 1.4 > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Opteron or Athlon 64, 2659.01 MHz > cpu0: > FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2,SSE3,POPCNT > cpu0: smt 0, core 0, package 0 > cpu0: apic clock running at 1000MHz > cpu at mainbus0: not configured > mpbios0: bus 0 is type PCI > mpbios0: bus 1 is type ISA > ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins > ioapic0: misconfigured as apic 0, remapped to apid 2 > pci0 at mainbus0 bus 0 > pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02 > pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00 > pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel > 0 wired to compatibility, channel 1 wired to compatibility > wd0 at pciide0 channel 0 drive 0: <QEMU HARDDISK> > wd0: 16-sector PIO, LBA48, 5000MB, 10240000 sectors > wd0(pciide0:0:0): using PIO mode 0, DMA mode 2 > atapiscsi0 at pciide0 channel 1 drive 0 > scsibus0 at atapiscsi0: 2 targets > cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 0.12> ATAPI 5/cdrom > removable > cd0(pciide0:1:0): using PIO mode 0 > uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 2 int 11 > piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 2 int > 9 > iic0 at piixpm0 > iic0: addr 0x4c 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 > 06=0000 07=0000 > iic0: addr 0x4e 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 > 06=0000 07=0000 > vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00 > wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) > wsdisplay0: screen 1-5 added (80x25, vt100 emulation) > em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x03: apic 2 int 11, > address 52:54:00:4a:77:22 > virtio0 at pci0 dev 4 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio > Memory Balloon Device > viomb0 at virtio0 > virtio0: apic 2 int 11 > em1 at pci0 dev 5 function 0 "Intel 82540EM" rev 0x03: apic 2 int 10, > address 52:54:00:1e:57:27 > isa0 at pcib0 > isadma0 at isa0 > com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo > com0: probed fifo depth: 0 bytes > pckbc0 at isa0 port 0x60/5 > pckbd0 at pckbc0 (kbd slot) > pckbc0: using irq 1 for kbd slot > wskbd0 at pckbd0: console keyboard, using wsdisplay0 > pms0 at pckbc0 (aux slot) > pckbc0: using irq 12 for aux slot > wsmouse0 at pms0 mux 0 > pcppi0 at isa0 port 0x61 > spkr0 at pcppi0 > fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 > fd0 at fdc0 drive 0: density unknown > fd1 at fdc0 drive 1: density unknown > usb0 at uhci0: USB revision 1.0 > uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 > nvram: invalid checksum > vscsi0 at root > scsibus1 at vscsi0: 256 targets > softraid0 at root > scsibus2 at softraid0: 256 targets > root on wd0a (bf96a0273abe52bf.a) swap on wd0b dump on wd0b > WARNING: / was not properly unmounted > clock: unknown CMOS layout > carp: pfsync0 demoted group carp by 32 to 160 (pfsync init) > carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) > carp: pfsync0 demoted group carp by 1 to 161 (pfsync bulk start) > carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) > carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk fail) > carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk fail) > carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) > carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) > pfsync: failed to receive bulk update > carp0: state transition: BACKUP -> MASTER > carp0: state transition: BACKUP -> MASTER > nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 > carp0: state transition: BACKUP -> MASTER > nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 > > > Two more things > > 1) Sorry for my english, is not my mothers tongue > 2) Thank you for doing this great operantig system > > Saludos / Regards > Leonardo Santagostini [demime 1.01d removed an attachment of type image/png which had a name of OpenBSD_PF_flow.png]
