On Wed, May 29, 2024 at 12:48:41PM +0200, Radek wrote:
> Thank you, that explains everything.
> Does wireguard support replication? Will it work properly in my CARP setup?
>
No for both questions. However, wireguard allows to create complicated
connections where one wg(4) interface could have multiple associated
peers on "client" side too.
> Radek
>
> On Mon, 27 May 2024 21:00:40 +0300
> Vitaliy Makkoveev <o...@bsdbox.dev> wrote:
>
> > npppd does not support replication
> >
> > > On 27 May 2024, at 19:58, Radek <r...@int.pl> wrote:
> > >
> > > Hello,
> > > I have two redundant firewalls with CARP: [krz75-MAS]<->[krz75-SLA]. I'm
> > > trying to set up redundant IPSEC VPN on it.
> > >
> > > - CARP + pfsync is working as expected - ca 1-2 pings lost at switchover.
> > > - sasyncd seems to work as expected - flows and SADs are replicated
> > > between nodes
> > > - isakmpd is running with "-S -K" on both nodes
> > > - IPSEC/npppd is working as expected on [krz75-MAS] - client can connect
> > > to VPN node
> > > - IPSEC/npppd is working as expected on [krz75-SLA] (when running as
> > > master) - client can connect to VPN node
> > >
> > > Problem to solve:
> > > When I perform the switchover between nodes the "new master" doesn't pick
> > > up the VPN sessions. Clinet needs to disconnect, to wait several dozen
> > > seconds and then to reconnect to VPN at new master.
> > >
> > > Can anybody help me out with making it working?
> > > Thanks!
> > >
> > > Configs on both nodes are the same.
> > >
> > >
> > > May 27 17:37:22 krz75-SLA reorder_kernel: kernel relinking done
> > > May 27 17:37:28 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:38:00 krz75-SLA last message repeated 8 times
> > > May 27 17:40:03 krz75-SLA last message repeated 31 times
> > > May 27 17:42:46 krz75-SLA last message repeated 41 times
> > > May 27 17:42:49 krz75-SLA /bsd: carp100: state transition: BACKUP ->
> > > MASTER
> > > May 27 17:42:49 krz75-SLA /bsd: carp2: state transition: BACKUP -> MASTER
> > > May 27 17:42:50 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:42:52 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:42:52 krz75-SLA /bsd: carp0: state transition: BACKUP -> MASTER
> > > May 27 17:42:52 krz75-SLA isakmpd[98426]: conf_set_now: duplicate tag
> > > [peer-10.0.15.11]:Refcount, ignoring...
> > > May 27 17:42:52 krz75-SLA isakmpd[98426]: message_recv: cleartext phase 2
> > > message
> > > May 27 17:42:52 krz75-SLA isakmpd[98426]: dropped message from 10.0.15.11
> > > port 500 due to notification type INVALID_FLAGS
> > > May 27 17:42:56 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:42:58 krz75-SLA /bsd: carp100: state transition: MASTER ->
> > > BACKUP
> > > May 27 17:42:58 krz75-SLA /bsd: carp2: state transition: MASTER -> BACKUP
> > > May 27 17:42:59 krz75-SLA isakmpd[98426]: message_recv: invalid cookie(s)
> > > e0f66ed709fcf140 16c20619d6f11bf4
> > > May 27 17:42:59 krz75-SLA isakmpd[98426]: dropped message from 10.0.15.11
> > > port 500 due to notification type INVALID_COOKIE
> > > May 27 17:42:59 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:42:59 krz75-SLA /bsd: carp0: state transition: MASTER -> BACKUP
> > > May 27 17:43:03 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:43:07 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:43:08 krz75-SLA isakmpd[98426]: sendmsg (36, 0x73a6d3321e08,
> > > 0): Network is unreachable
> > > May 27 17:43:11 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:43:15 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > > May 27 17:43:19 krz75-SLA isakmpd[98426]: sendmsg (36, 0x73a6d3321e08,
> > > 0): Network is unreachable
> > > May 27 17:43:19 krz75-SLA isakmpd[98426]: transport_send_messages: giving
> > > up on exchange peer-10.0.15.11, no response from peer 10.0.15.11:500
> > > May 27 17:43:19 krz75-SLA sasyncd[38852]: m_priv_iked_imsg: connect: No
> > > such file or directory
> > >
> > > [root@@krz75-MAS~:]ipsecctl -sa
> > > FLOWS:
> > > flow esp in proto udp from 10.0.15.11 port l2tp to 10.0.15.216 port l2tp
> > > peer 10.0.15.11 srcid 10.0.15.216/32 dstid 10.0.15.11/32 type require
> > > flow esp out proto udp from 10.0.15.216 port l2tp to 10.0.15.11 port l2tp
> > > peer 10.0.15.11 srcid 10.0.15.216/32 dstid 10.0.15.11/32 type require
> > >
> > > SAD:
> > > esp transport from 10.0.15.11 to 10.0.15.216 spi 0x6df78c14 auth
> > > hmac-sha1 enc aes
> > > esp transport from 10.0.15.216 to 10.0.15.11 spi 0xabdcff29 auth
> > > hmac-sha1 enc aes
> > >
> > > [root@@krz75-SLA~:]ipsecctl -sa
> > > FLOWS:
> > > flow esp in proto udp from 10.0.15.11 port l2tp to 10.0.15.216 port l2tp
> > > peer 10.0.15.11 srcid 10.0.15.216/32 dstid 10.0.15.11/32 type require
> > > flow esp out proto udp from 10.0.15.216 port l2tp to 10.0.15.11 port l2tp
> > > peer 10.0.15.11 srcid 10.0.15.216/32 dstid 10.0.15.11/32 type require
> > >
> > > SAD:
> > > esp transport from 10.0.15.11 to 10.0.15.216 spi 0x6df78c14 auth
> > > hmac-sha1 enc aes
> > > esp transport from 10.0.15.216 to 10.0.15.11 spi 0xabdcff29 auth
> > > hmac-sha1 enc aes
> > >
> > >
> > > [root@@krz75-MAS~:]cat /etc/sysctl.conf
> > > net.inet.ip.forwarding=1
> > > net.inet.ipcomp.enable=1
> > > net.inet.esp.enable=1
> > > # CARP
> > > net.inet.carp.allow=1
> > > net.inet.carp.preempt=1
> > >
> > > [root@@krz75-SLA~:]cat /etc/sysctl.conf
> > > net.inet.ip.forwarding=1
> > > net.inet.ipcomp.enable=1
> > > net.inet.esp.enable=1
> > > # CARP
> > > net.inet.carp.allow=1
> > > net.inet.carp.preempt=1
> > >
> > > [root@@krz75-SLA~:]egrep -e ips -e sas -e isa /etc/rc.conf.local
> > > ipsec=YES
> > > ipsec_rules=/etc/ipsec.conf
> > > isakmpd_flags="-S -K"
> > > sasyncd_flags=
> > >
> > > [root@@krz75-MAS~:]egrep -e ips -e sas -e isa /etc/rc.conf.local
> > > ipsec=YES
> > > ipsec_rules=/etc/ipsec.conf
> > > isakmpd_flags="-S -K"
> > > sasyncd_flags=
> > >
> > > [root@@krz75-MAS~:]cat /etc/hostname.em3
> > > -inet
> > > inet 172.16.1.11 255.255.255.0 172.16.1.255 description "pfsync if to
> > > krz-slave"
> > >
> > > [root@@krz75-SLA~:]cat /etc/hostname.em3
> > > -inet
> > > inet 172.16.1.12 255.255.255.0 172.16.1.255 description "pfsync if to
> > > krz-master"
> > >
> > > [root@@krz75-MAS/etc:]cat /etc/hostname.pfsync0
> > > -inet
> > > syncdev em3
> > > up
> > > [root@@krz75-SLA~:]cat /etc/hostname.pfsync0
> > > -inet
> > > syncdev em3
> > > up
> > >
> > > [root@@krz75-MAS~:]cat /etc/hostname.em0
> > > -inet
> > > up
> > >
> > > [root@@krz75-SLA~:]cat /etc/hostname.em0
> > > -inet
> > > up
> > >
> > >
> > > [root@@krz75-MAS~:]cat /etc/hostname.carp0
> > > -inet
> > > inet 10.0.15.216 255.255.255.0 NONE description "WAN_KRZ" vhid 1 advbase
> > > 1 advskew 0 carpdev em0 pass test678
> > >
> > > [root@@krz75-SLA~:]cat /etc/hostname.carp0
> > > -inet
> > > inet 10.0.15.216 255.255.255.0 NONE description "WAN_KRZ" vhid 1 advbase
> > > 1 advskew 128 carpdev em0 pass test678
> > > up
> > >
> > >
> > > [root@@krz75-MAS~:]cat /etc/ipsec.conf
> > > wan_ipv4 = 10.0.15.216
> > > ike passive esp transport \
> > > proto udp from $wan_ipv4 to any port 1701 \
> > > main auth "hmac-sha1" enc "3des" group modp1024 \
> > > quick auth "hmac-sha1" enc "aes" group modp1024 \
> > > psk "c98743717aa5f7"
> > >
> > > [root@@krz75-SLA~:]cat /etc/ipsec.conf
> > > wan_ipv4 = 10.0.15.216
> > > ike passive esp transport \
> > > proto udp from $wan_ipv4 to any port 1701 \
> > > main auth "hmac-sha1" enc "3des" group modp1024 \
> > > quick auth "hmac-sha1" enc "aes" group modp1024 \
> > > psk "c98743717aa5f7"
> > >
> > > [root@@krz75-MAS~:]cat /etc/sasyncd.conf
> > > interface carp0
> > > group carp
> > > peer 172.16.1.12
> > > sharedkey
> > > 0x115c413529ba5ac96b208d83a50473b3e6ade60e66c59a10a944ad3d273148dd
> > >
> > >
> > > [root@@krz75-SLA~:]cat /etc/sasyncd.conf
> > > interface carp0
> > > group carp
> > > peer 172.16.1.11
> > > sharedkey
> > > 0x115c413529ba5ac96b208d83a50473b3e6ade60e66c59a10a944ad3d273148dd
> > >
> > >
> > >
> > > [root@@krz75-MAS~:]cat /etc/npppd/npppd.conf
> > > authentication LOCAL type local {
> > > users-file "/etc/npppd/npppd-users"
> > > }
> > > tunnel L2TP protocol l2tp {
> > > listen on 10.0.15.216
> > > #listen on 0.0.0.0
> > > }
> > > ipcp IPCP {
> > > pool-address 10.0.211.1-10.0.211.253
> > > dns-servers 1.1.1.1
> > > }
> > > interface pppx0 address 10.0.211.254 ipcp IPCP
> > > bind tunnel from L2TP authenticated by LOCAL to pppx0
> > >
> > >
> > >
> > > [root@@krz75-SLA~:]cat /etc/npppd/npppd.conf
> > > authentication LOCAL type local {
> > > users-file "/etc/npppd/npppd-users"
> > > }
> > > tunnel L2TP protocol l2tp {
> > > listen on 10.0.15.216
> > > #listen on 0.0.0.0
> > > }
> > > ipcp IPCP {
> > > pool-address 10.0.211.1-10.0.211.253
> > > dns-servers 1.1.1.1
> > > }
> > > interface pppx0 address 10.0.211.254 ipcp IPCP
> > > bind tunnel from L2TP authenticated by LOCAL to pppx0
> > >
> > >
> > >
> > > Radek
> > >
> >
>
