Ok, just added my second website to both servers like your recommendation. I will post my config before the end of the day just to share it with you.
Thank you so much !!! Regards Saludos.- Leonardo Santagostini <http://ar.linkedin.com/in/santagostini> 2013/11/14 Leonardo Santagostini <lsantagost...@gmail.com> > Thanks a lot to all, i will give it a try and gives tou you feedback as > soon as it get implemented. > > Saludos.- > Leonardo Santagostini > > <http://ar.linkedin.com/in/santagostini> > > > > > > 2013/11/14 Andy <a...@brandwatch.com> > >> On 14/11/13 15:21, Leonardo Santagostini wrote: >> >> Hello misc, >> >> Im doing my final approach to put a production system with >> carp+pfsync+relayd on production. >> >> The point is that im facing some trouble setting more than one ip alias >> address with different vhid and different passwd. >> >> So, this is the scenario. >> >> Im trying to relayd more or less 15 sites so i have conceptual doubts. >> >> 1) is it nesessary to create one carp interface for each one of my >> internals VIP address >> 2) my understanding is that i have to work with pf on my carp interfaces. >> >> I have tried to put two different VIP's on my carp, but whitout lucky. >> >> Here is the homework. >> >> [root@server ~]# uname -a >> OpenBSD server.internaldomain.com 5.4 GENERIC#37 amd64 >> [root@server ~]# >> >> [root@server ~]# cat /etc/hostname.em0 >> inet 172.19.224.180 255.255.255.0 >> >> [root@server ~]# cat /etc/hostname.em1 >> inet 172.19.226.231 255.255.255.0 172.19.226.255 >> >> [root@server ~]# cat /etc/hostname.carp0 >> # inet alias 172.19.224.16 255.255.255.255 172.19.224.255 vhid 1 advskew 10 >> carpdev em0 pass Ahsooqu3 >> inet alias 172.19.224.131 255.255.255.0 172.19.224.255 vhid 2 advskew 10 >> carpdev em0 pass Meixo9oe >> # inet alias 172.19.224.41 255.255.255.255 172.19.224.255 vhid 3 advskew 10 >> carpdev em0 pass av5eG9Gi >> # inet alias 172.19.224.40 255.255.255.255 172.19.224.255 vhid 4 advskew 10 >> carpdev em0 pass Rei6thai >> # inet alias 172.19.224.181 255.255.255.0 172.19.224.255 vhid 5 advskew 10 >> carpdev em0 pass Toobohz3 >> # inet alias 172.19.224.182 255.255.255.255 172.19.224.255 vhid 6 adskew 10 >> carpdev em0 pass Quahng6U >> >> CARP should look like this (master); >> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >> Ahsooqu3 advskew 0 >> inet alias 172.19.224.131 255.255.255.255 >> inet alias 172.19.224.41 255.255.255.255 >> inet alias 172.19.224.40 255.255.255.255 >> inet alias 172.19.224.181 255.255.255.255 >> inet alias 172.19.224.182 255.255.255.255 >> >> And (backup); >> inet 172.19.224.16 255.255.255.0 172.19.224.255 vhid 1 carpdev em0 pass >> Ahsooqu3 advskew 200 >> inet alias 172.19.224.131 255.255.255.255 >> inet alias 172.19.224.41 255.255.255.255 >> inet alias 172.19.224.40 255.255.255.255 >> inet alias 172.19.224.181 255.255.255.255 >> inet alias 172.19.224.182 255.255.255.255 >> >> And yes the subnet masks for the alias' should be /32 and you will see a >> warning in the logs during fail-over. This is fine, the devs just haven't >> muted the check warning yet. >> >> You've done it right if 'netstat -rn' shows; >> >> 172.19.224.131 127.0.0.1 UGHS 0 0 33152 8 >> lo0 >> 172.19.224.131/32 172.19.224.131 U 0 0 - 4 >> carp0 >> >> >> [root@server ~]# cat /etc/hostname.pfsync0 >> up syncdev em1 >> >> [root@server ~]# cat /etc/pf.conf >> ext_if="carp0" >> >> You don't refer to CARP as an interface, it is simply a VRRP watchdog >> interface (for example you cannot set the MTU on a CARP interface as it is >> not really an interface. >> Use the physical.. >> >> ext_if="em0" >> >> >> set fingerprints "/etc/pf.os" >> set optimization aggressive >> set limit states 90000 >> >> Definitely needs to be higher! try 1 million.. >> >> >> set limit src-nodes 65000 >> >> table <bad_ip> persist >> table <internat_net> persist file "/etc/internal_net" >> table <admitted_net> persist file "/etc/admitted.txt" >> >> # vip1_address = "172.19.224.181" >> # vip2_address = "172.19.224.16" >> vip3_address = "172.19.224.131" >> # vip4_address = "172.19.224.41" >> # vip5_address = "172.19.224.40" >> >> Just to keep you sane remember these rules; >> # (SNAT) NATing is done before filtering, 'pass out on $if_ext from >> $external_carp_ip1' (public address as src for outbound). >> # (DNAT) RDRing is done before filtering, 'pass in on $if_ext from any to >> $internal_ip1' (private address as dst for inbound). >> >> [image: OpenBSD_PF_flow] >> >> >> # Dejo de procesar cuando se trata de las redes internas >> pass in quick from <internat_net> to any >> >> # Dejo pasar las ips desde las redes permitidas >> # pass in quick from <admitted_net> to $vip1_address >> pass in quick from <admitted_net> to $vip3_address >> >> # Genero el block >> block in quick from <bad_ip> >> >> Your 'block in quick's should be above your 'pass in quick's! >> quick means stop evaluating and do this action now.. >> >> >> block in log quick on $ext_if proto tcp from any os "NMAP" to any label >> ExtNMAPScan >> >> # Proteccion contra nmap y herramientas similares >> # block in quick on $ext_if proto tcp flags FUP/WEUAPRSF >> block in quick on $ext_if proto tcp flags WEUAPRSF/WEUAPRSF >> block in quick on $ext_if proto tcp flags SRAFU/WEUAPRSF >> block in quick on $ext_if proto tcp flags /WEUAPRSF >> block in quick on $ext_if proto tcp flags SR/SR >> block in quick on $ext_if proto tcp flags SF/SF >> block in quick from urpf-failed >> >> >> # Aplico reglas de DoS y Syn Flood en site1 >> # pass in log on $mob_if proto tcp to $vip1_address port www keep state >> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 100, max-src-conn-rate >> 95/2, adaptive .start 6000, adaptive.end 12000, tcp.first >> 15, tcp.opening 5, tcp.established 3600, tcp.closing 5, tcp.finwait 15, >> tcp.closed 15, tcp.tsdiff 5) >> >> Be careful, Direct Server Return does require sloppy states but be aware >> that this totally undermines state security! >> You still need a firewall on the outside of the load balancer to sanitize >> the flows.. Catch 22 as you cannot have the load balancer (with DSR) on the >> same box as your PF filtering if you want load balancing (with DSR) and >> full security.. >> I.e. DSR and full state modulation/security are mutually exclusive.. >> >> # Aplico reglas de DoS y Syn Flood en site2 >> # pass in on $ext_if proto tcp to $vip2_address port www keep state >> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >> 150/3) >> >> # Aplico reglas para site3 >> pass in on $ext_if proto tcp to $vip3_address port www keep state (sloppy, >> max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate 100/3) >> >> # Aplico reglas de DoS y Syn Flood en site4 >> # pass in on $ext_if proto tcp to $vip4_address port www keep state >> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >> 100/3) >> >> # Aplico reglas de DoS y Syn Flood en site5 >> # pass in on $ext_if proto tcp to $vip5_address port www keep state >> (sloppy, max 10000, max-src-nodes 5000, max-src-conn 150, max-src-conn-rate >> 100/3) >> >> # Anchor Para relayd >> anchor "relayd/*" >> >> >> [root@server ~]# cat /etc/relayd.conf >> # Archivo de configuracion de balanceo >> >> ## Opciones globales >> interval 5 >> timeout 500 >> prefork 15 >> log all >> >> ## Direcciones de las vip >> # address1="172.19.224.16" >> # address2="172.19.224.181" >> address3="172.19.224.131" >> # address4="172.19.224.41" >> # address5="172.19.224.40" >> >> ## Direcciones de los servidores >> wsapp1="172.19.224.200" >> wsapp2="172.19.224.201" >> webcache01="172.19.224.70" >> webcache02="172.19.224.71" >> webcache03="172.19.224.72" >> webcache04="172.19.224.73" >> >> ## Definicion de Tablas >> table <mobileweb> { $wsapp1 $wsapp2 } >> table <webcaches> { $webcache01 $webcache02 $webcache03 $webcache04 } >> table <webcaches1> { $webcache01 } >> >> ## Definicion de protocolos (Filtros) >> >> http protocol "httpSite1" { >> >> header change "Connection" to "close" >> header append "$REMOTE_ADDR" to "X-Forwarded-For" >> cookie hash "sessid" >> >> } >> >> http protocol "httpSite2" { >> >> header change "Connection" to "close" >> header append "$REMOTE_ADDR" to "X-Forwarded-For" >> cookie hash "sessid" >> >> } >> >> http protocol "httpSite3" { >> >> header change "Connection" to "close" >> header append "$REMOTE_ADDR" to "X-Forwarded-For" >> } >> >> http protocol "httpSite4" { >> >> header append "$REMOTE_ADDR" to "X-Forwarded-For" >> >> } >> >> http protocol "httpSite5" { >> >> header append "$REMOTE_ADDR" to "X-Forwarded-For" >> >> } >> >> ## Definicion de los relays >> >> #relay site1 { >> # listen on $address2 port 80 >> # protocol "httpSite1" >> # forward to <mobileweb> port 80 mode roundrobin check http >> "/relaycheck/index.php" code 200 >> #} >> >> relay site2 { >> listen on $address3 port 80 >> protocol "httpSite2" >> forward to <webcaches> port 80 mode roundrobin check http >> "/monitoreo/relayd.txt" code 200 >> } >> >> #relay site3 { >> # listen on $address1 port 80 >> # protocol "httpSite3" >> # forward to <webcaches> port 80 mode roundrobin check http >> "/monitoreo/relayd.txt" code 200 >> #} >> >> #relay site4 { >> # listen on $address4 port 80 >> # protocol "httpSite4" >> # forward to <webcaches> port 80 mode roundrobin check http >> "/monitoreo/relayd.txt" code 200 >> #} >> >> #relay site5 { >> # listen on $address5 port 80 >> # protocol "httpSite5" >> # forward to <webcaches> port 80 mode roundrobin check http >> "/monitoreo/relayd.txt" code 200 >> #} >> >> Haven't implemented this myself so can't comment! >> >> Good luck :) >> Andy. >> >> >> >> [root@server ~]# dmesg >> OpenBSD 5.4 (GENERIC) #37: Tue Jul 30 15:24:05 MDT 2013 >> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC >> real mem = 2130694144 (2031MB) >> avail mem = 2066341888 (1970MB) >> mainbus0 at root >> bios0 at mainbus0: SMBIOS rev. 2.4 @ 0x7ffffea0 (11 entries) >> bios0: vendor Seabios version "0.5.1" date 01/01/2007 >> bios0: Red Hat KVM >> acpi0 at bios0: rev 0 >> acpi0: sleep states S5 >> acpi0: tables DSDT FACP SSDT APIC >> acpi0: wakeup devices >> acpitimer0 at acpi0: 3579545 Hz, 24 bits >> acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat >> acpiprt0 at acpi0: bus 0 (PCI0) >> acpicpu0 at acpi0 >> mpbios0 at bios0: Intel MP Specification 1.4 >> cpu0 at mainbus0: apid 0 (boot processor) >> cpu0: Opteron or Athlon 64, 2659.01 MHz >> cpu0: >> FPU,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,PGE,CMOV,PAT,MMX,FXSR,SSE,SSE2,SSE3,POPCNT >> cpu0: smt 0, core 0, package 0 >> cpu0: apic clock running at 1000MHz >> cpu at mainbus0: not configured >> mpbios0: bus 0 is type PCI >> mpbios0: bus 1 is type ISA >> ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 11, 24 pins >> ioapic0: misconfigured as apic 0, remapped to apid 2 >> pci0 at mainbus0 bus 0 >> pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02 >> pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00 >> pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel >> 0 wired to compatibility, channel 1 wired to compatibility >> wd0 at pciide0 channel 0 drive 0: <QEMU HARDDISK> >> wd0: 16-sector PIO, LBA48, 5000MB, 10240000 sectors >> wd0(pciide0:0:0): using PIO mode 0, DMA mode 2 >> atapiscsi0 at pciide0 channel 1 drive 0 >> scsibus0 at atapiscsi0: 2 targets >> cd0 at scsibus0 targ 0 lun 0: <QEMU, QEMU DVD-ROM, 0.12> ATAPI 5/cdrom >> removable >> cd0(pciide0:1:0): using PIO mode 0 >> uhci0 at pci0 dev 1 function 2 "Intel 82371SB USB" rev 0x01: apic 2 int 11 >> piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 2 int >> 9 >> iic0 at piixpm0 >> iic0: addr 0x4c 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >> 06=0000 07=0000 >> iic0: addr 0x4e 48=00 words 00=0000 01=0000 02=0000 03=0000 04=0000 05=0000 >> 06=0000 07=0000 >> vga1 at pci0 dev 2 function 0 "Cirrus Logic CL-GD5446" rev 0x00 >> wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) >> wsdisplay0: screen 1-5 added (80x25, vt100 emulation) >> em0 at pci0 dev 3 function 0 "Intel 82540EM" rev 0x03: apic 2 int 11, >> address 52:54:00:4a:77:22 >> virtio0 at pci0 dev 4 function 0 "Qumranet Virtio Memory" rev 0x00: Virtio >> Memory Balloon Device >> viomb0 at virtio0 >> virtio0: apic 2 int 11 >> em1 at pci0 dev 5 function 0 "Intel 82540EM" rev 0x03: apic 2 int 10, >> address 52:54:00:1e:57:27 >> isa0 at pcib0 >> isadma0 at isa0 >> com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo >> com0: probed fifo depth: 0 bytes >> pckbc0 at isa0 port 0x60/5 >> pckbd0 at pckbc0 (kbd slot) >> pckbc0: using irq 1 for kbd slot >> wskbd0 at pckbd0: console keyboard, using wsdisplay0 >> pms0 at pckbc0 (aux slot) >> pckbc0: using irq 12 for aux slot >> wsmouse0 at pms0 mux 0 >> pcppi0 at isa0 port 0x61 >> spkr0 at pcppi0 >> fdc0 at isa0 port 0x3f0/6 irq 6 drq 2 >> fd0 at fdc0 drive 0: density unknown >> fd1 at fdc0 drive 1: density unknown >> usb0 at uhci0: USB revision 1.0 >> uhub0 at usb0 "Intel UHCI root hub" rev 1.00/1.00 addr 1 >> nvram: invalid checksum >> vscsi0 at root >> scsibus1 at vscsi0: 256 targets >> softraid0 at root >> scsibus2 at softraid0: 256 targets >> root on wd0a (bf96a0273abe52bf.a) swap on wd0b dump on wd0b >> WARNING: / was not properly unmounted >> clock: unknown CMOS layout >> carp: pfsync0 demoted group carp by 32 to 160 (pfsync init) >> carp: pfsync0 demoted group pfsync by 32 to 32 (pfsync init) >> carp: pfsync0 demoted group carp by 1 to 161 (pfsync bulk start) >> carp: pfsync0 demoted group pfsync by 1 to 33 (pfsync bulk start) >> carp: pfsync0 demoted group carp by -1 to 32 (pfsync bulk fail) >> carp: pfsync0 demoted group pfsync by -1 to 32 (pfsync bulk fail) >> carp: pfsync0 demoted group carp by -32 to 0 (pfsync init) >> carp: pfsync0 demoted group pfsync by -32 to 0 (pfsync init) >> pfsync: failed to receive bulk update >> carp0: state transition: BACKUP -> MASTER >> carp0: state transition: BACKUP -> MASTER >> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >> carp0: state transition: BACKUP -> MASTER >> nd6_na_input: duplicate IP6 address fe80:0005::0200:5eff:fe00:0102 >> >> >> Two more things >> >> 1) Sorry for my english, is not my mothers tongue >> 2) Thank you for doing this great operantig system >> >> Saludos / Regards >> Leonardo Santagostini [demime 1.01d removed an attachment of type image/png which had a name of OpenBSD_PF_flow.png]
