Hi Neale,
Thanks a lot for looking into this and providing a fix, much appreciated.
I have applied the fix, and that solved the issue on the sender side.
The packet successfully reached the other side. However, the issue is on
the other side now.
The packet trace looks like this: (I've put dummy local and remote IPv6 IPs)
Packet 12
02:06:48:110231: dpdk-input
wan0 rx queue 0
buffer 0x11e885: current data 0, length 74, buffer-pool 1, ref-count 1,
totlen-nifb 0, trace handle 0xb
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 1, nb_segs 1, pkt_len 74
buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr
0xe27a21c0
packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 1, nb_segs 1, pkt_len 190
buf_len 2176, data_len 190, ol_flags 0x181, data_off 128, phys_addr
0xe279a300
packet_type 0x41 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 67
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV6 (0x0040) IPv6 packet without extension headers
IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67
IPSEC_ESP: 2001::2 -> 2001::1
tos 0x28, flow label 0x0, hop limit 238, payload length 132
02:06:48:074127: ethernet-input
frame: flags 0x3, hw-if-index 2, sw-if-index 2
IP6: 5c:5e:ab:d0:29:f0 -> b4:96:91:18:eb:be 802.1q vlan 67
02:06:48:074128: ip6-input
IPSEC_ESP: 2001::2 -> 2001::1
tos 0x28, flow label 0x0, hop limit 238, payload length 132
02:06:48:074128: ip6-lookup
fib 0 dpo-idx 8 flow hash: 0x00000000
IPSEC_ESP: 2001::2 -> 2001::1
tos 0x28, flow label 0x0, hop limit 238, payload length 132
02:06:48:074129: ip6-local
IPSEC_ESP: 2001::2 -> 2001::1
tos 0x28, flow label 0x0, hop limit 238, payload length 132
02:06:48:074130: ip6-punt
IPSEC_ESP: 2001::2 -> 2001::1
tos 0x28, flow label 0x0, hop limit 238, payload length 132
02:06:48:074130: error-punt
rx:wan0.67
02:06:48:074131: punt
ip6-input: valid ip6 packets
Now, in my understanding, this is because the SPD action for decryption is
not triggered - probably because I set the SPD on the lan0.218 interface
and nothing on the wan0.67 interface, which is the one that has the IPv6 IP.
I looked at the script at
https://gerrit.fd.io/r/c/vpp/+/27019/4/src/scripts/vnet/ipsec_spd in your
patch.
Things I noticed there,
1. SPD is set on the pipe interface with IPv6 which makes sense.
2. The inbound action actually just mentions IPv6 local and remote IP,
which I guess also makes sense since no other information is available
until decryption.
The problem I am facing is, I am unable to set SPD on the wan0.67
interface. As soon as I set an SPD, I lose the ability to communicate with
the interface. This actually happened with the lan interface as well, but I
side stepped by adding a loopback interface and setting SPD there.
(I remember being able to ping the interface IP even after setting the SPD
on it in the past, but I am not completely sure)
This is how it was done:
loopback create
set int state loop0 up
set int ip address loop0 11.11.11.11/31
set ip neighbor loop0 11.11.11.10 24:6e:96:9c:e5:df
ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
<DST_IPv6>
ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
<DST_IPv6>
ipsec spd add 1
set interface ipsec spd loop0 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10
local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
10.6.255.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20
local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
10.6.255.255
ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218
In the inbound action, I had directly translated what I had seen in
documentations for IPv4.
I realise that this may not be the correct thing to do.
So I guess my doubts are on 2 things,
1. What is the right way to set inbound action to correctly decrypt the
incoming ESP packets? Would something like this work?
ipsec policy add spd 1 priority 10 inbound action protect sa 20
local-ip-range <local_IPv6> - <local_IPv6> remote-ip-range <remote_IPv6> -
<remote_IPv6>
2. How would I overcome the fact that setting SPD means I am no longer able
to reach that interface?
I am guessing another loopback interface, that gets all the packets
destined for wan0.67 (may be through an l2 bridge?) and SPD should be
there. Am I in the right path?
If yes, could you direct me to documentation regarding oneway L2 bridging
to a virtual / loopback interface?
Thanks so much for looking into this, once again.
Thanks,
Muthu
On Wed, May 13, 2020 at 2:54 PM Neale Ranns (nranns) <nra...@cisco.com>
wrote:
>
>
> Hi Muthu,
>
>
>
> I tried your 4over6 config, it doesn’t work… here’s the fix:
>
> https://gerrit.fd.io/r/c/vpp/+/27019
>
>
>
> /neale
>
>
>
> *From: *Muthu Raj <muthuraj.muth...@gmail.com>
> *Date: *Monday 11 May 2020 at 21:36
> *To: *"Neale Ranns (nranns)" <nra...@cisco.com>
> *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" <
> fteh...@cisco.com>, "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hi Neale,
>
>
>
> Thank you so much for patiently explaining. I initially added next-hop
> like this -
>
> ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218
>
>
> But it still got dropped with ip4-arp: ARP requests sent.
>
>
>
> Then I read your message again, and realised that the ARP response really
> doesn't matter. So I set a static ARP entry like so -
>
> set ip neighbor lan0.218 172.30.0.6 c8:5b:76:d9:7f:9c
>
>
>
> That thankfully, changed the nodes visited. Unfortunately, it still fails
> with an error like below - ipsec6-no-such-tunnel.
>
> I understand that it must be some mistake in setting up the tunnel, but
> once again, I am not sure how to proceed.
>
>
>
> The packet trace is at the end.
>
>
>
> Here is the way I set-up the tunnel:
>
>
>
> ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec spd add 1
> set interface ipsec spd lan0.218 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
> ipsec policy add spd 1 priority 10 outbound action protect sa 10
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ip route add 10.6.0.0/16 via 172.30.0.6 lan0.218
>
> set ip neighbor lan0.218 172.30.0.6 24:6e:96:9c:e5:df
>
>
>
> *Packet trace:*
>
>
> 00:01:35:388393: dpdk-input
> lan0 rx queue 0
> buffer 0x1389c3: current data 0, length 102, buffer-pool 1, ref-count 1,
> totlen-nifb 0, trace handle 0x10
> ext-hdr-valid
> l4-cksum-computed l4-cksum-correct
> PKT MBUF: port 0, nb_segs 1, pkt_len 102
> buf_len 2176, data_len 102, ol_flags 0x181, data_off 128, phys_addr
> 0xe2e27140
> packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
> rss 0x0 fdir.hi 0x0 fdir.lo 0x0
> Packet Offload Flags
> PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet
> PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
> PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218
> Packet Types
> RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
> RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers
> IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN
> fragment id 0x91cf, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x15bb
> 00:01:35:388395: ethernet-input
> frame: flags 0x3, hw-if-index 1, sw-if-index 1
> IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
> 00:01:35:388396: ip4-input
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN
> fragment id 0x91cf, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x15bb
> 00:01:35:388397: ip4-lookup
> fib 0 dpo-idx 6 flow hash: 0x00000000
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xe791 dscp CS0 ecn NON_ECN
> fragment id 0x91cf, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x15bb
> 00:01:35:388397: ip4-rewrite
> tx_sw_if_index 4 dpo-idx 6 : ipv4 via 172.30.0.6 lan0.218: mtu:9000
> 246e969ce5dfb4969118ebbc810000da0800 flow ha
> sh: 0x00000000
> 00000000:
> 246e969ce5dfb4969118ebbc810000da08004500005491cf40003f01e891ac1e
> 00000020: 00020a060b22080015bb4fa504f9a1a5b95e0000000067cf0c000000
> 00:01:35:388397: ipsec4-output-feature
> spd 1 policy 2
> 00:01:35:388398: esp4-encrypt
> esp: sa-index 0 spi 1000 (0x000003e8) seq 11 sa-seq-hi 0 crypto
> aes-cbc-128 integrity sha1-96
> 00:01:35:388401: punt-dispatch
> reason: [2] ipsec6-no-such-tunnel
> 00:01:35:388403: drop
> punt-dispatch: No registrations
>
> Thanks again,
>
> Muthu
>
>
>
> On Tue, May 12, 2020 at 12:24 AM Neale Ranns (nranns) <nra...@cisco.com>
> wrote:
>
>
>
> Hi Muthu,
>
>
>
> Your lan0.218 has address in the 172.16.0.5/16, set the next hop to the
> peer on that link in that subnet. If you don’t have one, in this case, you
> can use any address… the problem is that IPSec SPD runs as an output
> feature, but we don’t run features for packets that hit the glean (i.e. I
> need to ARP) adjacency. So the route needs to resolve via a neighbour
> adjacency. It doesn’t have to be a complete adj, hence you don’t need an
> ARP response, so any address will do.
>
>
>
> /neale
>
>
>
> *From: *<vpp-dev@lists.fd.io> on behalf of Muthu Raj <
> muthuraj.muth...@gmail.com>
> *Date: *Monday 11 May 2020 at 19:22
> *To: *"Neale Ranns (nranns)" <nra...@cisco.com>
> *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" <
> fteh...@cisco.com>, "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hi Neale,
>
>
>
> Thanks for responding so quickly.
>
> It might sound a little bit pedestrian, but what would be the next hop in
> this case? Would it be the IPv6 IP (over which the tunnel is established) ?
>
>
>
> I have added it through the lan0.218 interface with the assumption that
> the sa will kick in and send over the tunnel.
>
>
>
>
>
> On Mon, May 11, 2020, 10:26 PM Neale Ranns (nranns) <nra...@cisco.com>
> wrote:
>
>
>
>
>
> *From: *Muthu Raj <muthuraj.muth...@gmail.com>
> *Date: *Monday 11 May 2020 at 18:42
> *To: *"Neale Ranns (nranns)" <nra...@cisco.com>
> *Cc: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" <
> fteh...@cisco.com>, "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hi Neale,
>
>
>
> The 10.6.0.0/16 is actually on the remote side, and should flow over the
> IPv6 IPsec tunnel.
>
> I was hoping by adding route via the lan0.218 interface, the spd will kick
> in and send over the tunnel. I guess I'm missing something very glaring, so
> would appreciate pointers.
>
>
>
> You’re missing next-hop for the route.
>
>
>
> /neale
>
>
>
> The source of the traced packet is actually another node, that sends to
> the lan0.218 IP for the remote subnet destination.
>
>
>
>
>
> Thanks,
>
> Muthu
>
>
>
> On Mon, May 11, 2020, 9:41 PM Neale Ranns (nranns) <nra...@cisco.com>
> wrote:
>
>
>
> Hi Muthu,
>
>
>
> Your lan0 interface is not point-2-point, so your route needs a nexthop;
>
> ip route add 10.6.0.0/16 via 172.30.x.y lan0.218
>
>
>
> x and y can be anything, apart from x=0 and y=5.
>
>
>
> /neale
>
>
>
> *From: *<vpp-dev@lists.fd.io> on behalf of Muthu Raj <
> muthuraj.muth...@gmail.com>
> *Date: *Monday 11 May 2020 at 17:39
> *To: *"Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco)" <
> fteh...@cisco.com>
> *Cc: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> *Subject: *Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hi Filip,
>
>
>
> I have traced the packet.
>
>
>
> Here is how I setup the tunnel
>
>
>
> <home subnet=172.30.0.0/16> || <SRC_IPv6> <=======> <DST_IPv6> || <Remote
> Subnet=10.6.0.0/16>
>
>
>
> ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec spd add 1
> set interface ipsec spd lan0.218 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
> ipsec policy add spd 1 priority 10 outbound action protect sa 10
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
>
> ip route add 10.6.0.0/16 via lan0.218
>
>
>
> vpp# show int address
> lan0 (up):
> lan0.218 (up):
> L3 172.30.0.5/16
> local0 (up):
> wan0 (up):
> wan0.67 (up):
> L3 <SRC_IPv6>
>
>
>
> Corresponding config on remote side.
>
> Trace:
>
>
>
> 00:32:40:825694: dpdk-input
> lan0 rx queue 0
> buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1,
> totlen-nifb 0, trace handle 0x11
> ext-hdr-valid
> l4-cksum-computed l4-cksum-correct
> PKT MBUF: port 0, nb_segs 1, pkt_len 74
> buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr
> 0xe05874c0
> packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
> rss 0x0 fdir.hi 0x0 fdir.lo 0x0
> Packet Offload Flags
> PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet
> PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
> PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218
> Packet Types
> RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
> RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers
> IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
> fragment id 0xc4a2, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x7ddd
> 00:32:40:712830: ethernet-input
> frame: flags 0x3, hw-if-index 1, sw-if-index 1
> IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
> 00:32:40:712831: ip4-input
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
> fragment id 0xc4a2, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x7ddd
> 00:32:40:712831: ip4-lookup
> fib 0 dpo-idx 3 flow hash: 0x00000000
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
> fragment id 0xc4a2, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x7ddd
> 00:32:40:712832: ip4-glean
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
> fragment id 0xc4a2, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x7ddd
> 00:32:40:712833: ip4-drop
> ICMP: 172.30.0.2 -> 10.6.11.34
> tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
> fragment id 0xc4a2, flags DONT_FRAGMENT
> ICMP echo_request checksum 0x7ddd
> 00:32:40:712833: error-drop
> rx:lan0.218
> 00:32:40:712834: drop
> ip4-glean: ARP requests sent
>
>
>
> I believe for some reason the policy match is not happening. And I am not
> able to find examples of IPv4 over IPv6 tunnel, so I might be making a
> mistake in setting up the tunnel itself.
>
> Let me know if you need more info, and thank you so much for looking into
> this. Much appreciated.
>
>
>
> Thanks,
>
> Muthu
>
>
>
>
>
>
>
> On Mon, May 11, 2020 at 3:41 PM Filip Tehlar -X (ftehlar - PANTHEON TECH
> SRO at Cisco) <fteh...@cisco.com> wrote:
>
> Hi Muthu,
>
>
>
> you can enable packet tracing [1] and see what nodes are visited.
>
>
>
> Filip
>
>
>
> [1]
> https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script
>
> VPP/How To Use The Packet Generator and Packet Tracer - fd.io
> <https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script>
>
> Introduction. The VPP platform includes packet generation and packet
> tracing facilities. The following example shows steps that you might
> typically use to run a debug version of the vpp executable file, generate
> packets, and to analyze results.
>
> wiki.fd.io
>
>
> ------------------------------
>
> *From:* Muthu Raj <muthuraj.muth...@gmail.com>
> *Sent:* Thursday, May 7, 2020 7:04 PM
> *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) <
> fteh...@cisco.com>
> *Cc:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hello Filip,
>
>
>
> I just tried adding a loopback interface and setting the SPD on that
> interface.
>
> Here is how I tried it:
>
>
>
> create loopback interface
> set int state loop0 up
> set interface ip table loop0 0
> set int ip address loop0 11.11.11.11/32
> ipsec sa add 10 spi 1000 esp crypto-key <KEY> crypto-alg aes-cbc-128
> integ-key <KEY> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <KEY> crypto-alg aes-cbc-128
> integ-key <KEY> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
>
> ipsec spd add 1
> set interface ipsec spd loop0 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
>
> ipsec policy add spd 1 priority 10 outbound action protect sa 10
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ip route table 0 10.6.0.0/16 via loop0
>
>
>
> Corresponding configuration on the other side.
>
> Ping 10.6.0.1 results in these counters in show errors:
>
>
>
> 5 arp-input IP4 destination address
> is unset
> 5 ip4-glean ARP requests sent
>
> If there is any documentation that would help me gain clarity on what is
> going wrong, that would be really appreciated.
>
>
>
> Thanks again,
>
> Muthu
>
>
>
>
>
> On Thu, May 7, 2020 at 8:50 PM Muthu Raj <muthuraj.muth...@gmail.com>
> wrote:
>
> Thanks for replying Filip.
> I have indeed tried to set it up this way, and I think setting the spd on
> lan0.218 messes something up.
>
> I lose the ability to ping even other IPs on the same subnet as the
> address of the lan0.218 interface.
> eg., I can't ping 172.30.0.1 when lan0.218 is 172.30.0.2
>
> I see the ipsec4-output-feature IPSec policy (no match) counter
> going up by the exact number of ping packets, which means even local
> destinations are getting caught in this.
> I am not sure how to fix this.
>
> This is what show ipsec spd looks like:
> spd 1
> ip4-outbound:
> [0] priority 100 action bypass type ip4-outbound protocol IPSEC_ESP
> local addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> remote addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> packets 0 bytes 0
> [2] priority 10 action protect type ip4-outbound protocol any sa 10
> local addr range 10.6.0.0 - 10.6.255.255 port range 0 - 65535
> remote addr range 172.30.0.0 - 172.30.255.255 port range 0 - 65535
> packets 0 bytes 0
> ip6-outbound:
> ip4-inbound-protect:
> [3] priority 10 action protect type ip4-inbound-protect protocol any sa
> 20
> local addr range 10.6.0.0 - 10.6.255.255 port range 0 - 65535
> remote addr range 172.30.0.0 - 172.30.255.255 port range 0 - 65535
> packets 0 bytes 0
> ip6-inbound-protect:
> ip4-inbound-bypass:
> [1] priority 100 action bypass type ip4-inbound-bypass protocol
> IPSEC_ESP
> local addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> remote addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> packets 0 bytes 0
> ip6-inbound-bypass:
>
>
>
> Can you guide me on how to set the SPD the right way and if I should setup
> a separate loopback interface with a dummy IP from a different subnet
> (different from local-subnet) and send IPsec remote destinations there and
> set SPD on that dummy interface?
>
>
>
> Your help is much appreciated.
>
> Thanks,
>
> Muthu
>
>
>
> On Thu, May 7, 2020 at 7:11 PM Filip Tehlar -X (ftehlar - PANTHEON TECH
> SRO at Cisco) <fteh...@cisco.com> wrote:
>
> Hi Muthu,
>
>
>
> I don't see any reason why your approach shouldn't work.
>
>
>
> Do you have any specific problem with it?
>
>
>
> Filip
> ------------------------------
>
> *From:* Muthu Raj <muthuraj.muth...@gmail.com>
> *Sent:* Thursday, May 7, 2020 9:08 AM
> *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) <
> fteh...@cisco.com>
> *Cc:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
>
>
> Hello Filip,
>
> I was wondering if IPv6 IPsec can help us here while NAT traversal is
> being developed.
> Can we route IPv4 traffic over ipv6 ipsec tunnel?
> If we had two interfaces, one with the IPv4 IP which is part of local
> subnet and one with IPv6 subnet IP which will be used for transporting
> the ESP packets.
> The configuration I have in mind is like this:
> vpp# show int address
> lan0 (up):
> lan0.218 (up):
> L3 172.30.0.5/16
> local0 (dn):
> wan0 (up):
> wan0.67 (up):
> L3 <IPv6_HOME>
>
>
> On home side(172.30.0.0/16):
>
> ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DEST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DEST_IPv6>
> ipsec spd add 1
> set interface ipsec spd lan0.218 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 20 outbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ip route add 10.6.0.0/16 via lan0.218
>
> And similar configuration on the remote side.
>
> Let me know if I am going obviously wrong.
> This is what I could get from the docs available and there doesn't see
> to be examples for IPv6.
>
> I would like traffic flow from outside VPP source, like 172.30.0.2 on
> home side to flow to 172.30.0.5 (VPP) and flow over the tunnel for
> 10.6.0.0/16 destination.
> Thanks,
> Muthu
>
> On Tue, May 5, 2020 at 1:58 PM Filip Tehlar -X (ftehlar - PANTHEON
> TECH SRO at Cisco) <fteh...@cisco.com> wrote:
> >
> > Hi,
> >
> > NAT traversal in IKE isn't supported yet, but there is an ongoing work
> to add one:
> > https://gerrit.fd.io/r/c/vpp/+/26726
> >
> > Filip
> > ________________________________
> > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> on behalf of
> muthurajmuth...@gmail.com <muthurajmuth...@gmail.com>
> > Sent: Tuesday, May 5, 2020 8:49 AM
> > To: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> > Subject: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind
> NAT (AWS instance)
> >
> > Hello,
> >
> > I am trying out IPSec on VPP, and used the wiki[1] to create an IPSec
> > tunnel between an AWS instance(remote) and my home. The tunnel was
> > established successfully, and when pinging an IP on the remote side,
> > the icmp req flows over the tunnel, is seen by the remote box, and
> > responded back as well. I also see that the packets indeed end up
> > reaching my home VPP instance - however, they do not reach the last
> > hop. When I run show int, the ipip0 interface does not show the rx
> > counter at all, and when running `show errors` I do not see the
> > counter for the `ipsec4-tun-input` node either. Neither do I see the
> > `esp4-encrypt-tun' counter.
> >
> > My preliminary guess is that it has something to do with the fact that
> > on AWS we cannot see the public IP inside the instance and so that
> > cannot be assigned to the interface itself, so probably the ESP
> > packets are generated with source as the private IP address
> > corresponding to the public IP. With strongswan, we specify an
> > explicit sourceip parameter, like in the snippet below
> >
> > left=1.2.3.4
> > leftid=1.2.3.4
> > leftsubnet=172.16.0.0/16
> > right=4.5.6.7 #AWS public IP
> > rightsourceip=10.6.82.34 #AWS private IP for that public IP, seen
> > inside the instance.
> > rightsubnet=10.6.0.0/16
> >
> > So it is essentially like being NATed from the private IP to the public
> IP.
> >
> > I am attaching the ikev2 sa as seen from both sides.
> > How would I fix this issue?
> > Any help is appreciated very much.
> >
> > Thanks in advance.
> >
> >
> > This is from the home side. I've changed the IPs on home and remote
> > side. The private IP addresses have been left as it is.
> >
> > vpp# show ikev2 sa
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb
> >
> r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c
> > SK_d
> 1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde
> > SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c
> > r:04a71f139f2076058ceafb9be73eb359e43bc308
> > SK_e
> i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc
> >
> r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409
> > SK_p
> i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7
> >
> r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8
> > identifier (i) fqdn vpp.aws
> > identifier (r) fqdn vpp.home
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 147e7a05 spi(r) de36dcbc
> > SK_e
> i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229
> >
> r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532
> > SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3
> > r:9504182eb614d90aa8fe742122ec9d98c1b6e224
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420
> >
> > --------------------------------------------------------------------
> > Here is the AWS side
> >
> > vpp# show ikev2 sa
> > iip 1.2.3.4 ispi a72ae3cef809725c rip <aws private IP corresponding
> > to public IP> rspi b8b7b8ef09266a6d
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:d3e4299761fd93edd3df16456cb0ca9f717f67e57155fa7cb4cd0b9a1d371019
> >
> r:e9a33a33b901366438e262d225a418e9489839415562d3e3673107e0d81d830f
> > SK_d
> 7e4f795db87a02c5b4d5ea738945521473f5e449b783f3ac4b954be7716b7909
> > SK_a i:13639a11b6e96e65dd38d095a87fc1b5ceefdc6b
> > r:97c96809563dfe39c3d2762c1ff1bf0a8fbc3576
> > SK_e
> i:114661a058686bd4362d8515ce83a7d7de098af11b08084c407ad51843316135
> >
> r:d812542cfa988e6c302fc52d848fb2d7b7321d6c3e77ee04134338a21c0ccba8
> > SK_p
> i:a65ea61c70b3cb749dedc205b7715b4c278a4bc630c6508d89a55a00cd00a2cd
> >
> r:9e23352bac4d21f6f0d2ec8de82e556db3ddaba0ade0c4d664a020da3986d17b
> > identifier (i) fqdn vpp.home
> > identifier (r) fqdn vpp.aws
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 31c649f8 spi(r) 967b11c4
> > SK_e
> i:6a1b5898746bc922af1beba021768cd6417a0e8a4c555e5544781fee302cf633
> >
> r:2035a8be8fae47c284cef445381cef487bcd670bddc31558109c0303bc0f5399
> > SK_a i:da119e539529803a3d2a883c01a825211c782bd2
> > r:2330bc2dd9eb3741e3df649bcc3f7e5320fba512
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> > iip 1.2.3.4 ispi a72ae3cef809725c rip <aws private IP corresponding
> > to public IP> rspi b8b7b8ef09266a6d
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip <aws private IP corresponding
> > to public IP> rspi d6726c2768b2420
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb
> >
> r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c
> > SK_d
> 1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde
> > SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c
> > r:04a71f139f2076058ceafb9be73eb359e43bc308
> > SK_e
> i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc
> >
> r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409
> > SK_p
> i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7
> >
> r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8
> > identifier (i) fqdn vpp.home
> > identifier (r) fqdn vpp.aws
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 147e7a05 spi(r) de36dcbc
> > SK_e
> i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229
> >
> r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532
> > SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3
> > r:9504182eb614d90aa8fe742122ec9d98c1b6e224
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> >
> > [1]
> >
> https://wiki.fd.io/view/VPP/IPSec_and_IKEv2#IKEv2_negotiation_between_a_VPP_responder_and_a_VPP_initiator.2C_using_RSA_signature_authentication_method
> >
> > *I've used PSK in place of RSA signature.
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16363): https://lists.fd.io/g/vpp-dev/message/16363
Mute This Topic: https://lists.fd.io/mt/74050344/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
