Hi Filip,
I have traced the packet.
Here is how I setup the tunnel
<home subnet=172.30.0.0/16> || <SRC_IPv6> <=======> <DST_IPv6> || <Remote
Subnet=10.6.0.0/16>
ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
<DST_IPv6>
ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
integ-key <Key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
<DST_IPv6>
ipsec spd add 1
set interface ipsec spd lan0.218 1
ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
ipsec policy add spd 1 priority 10 outbound action protect sa 10
local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
10.6.255.255
ipsec policy add spd 1 priority 10 inbound action protect sa 20
local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
10.6.255.255
ip route add 10.6.0.0/16 via lan0.218
vpp# show int address
lan0 (up):
lan0.218 (up):
L3 172.30.0.5/16
local0 (up):
wan0 (up):
wan0.67 (up):
L3 <SRC_IPv6>
Corresponding config on remote side.
Trace:
00:32:40:825694: dpdk-input
lan0 rx queue 0
buffer 0x13e1d1: current data 0, length 74, buffer-pool 1, ref-count 1,
totlen-nifb 0, trace handle 0x11
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 74
buf_len 2176, data_len 74, ol_flags 0x181, data_off 128, phys_addr
0xe05874c0
packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_VLAN (0x0001) RX packet is a 802.1q VLAN packet
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid 218
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers
IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
ICMP: 172.30.0.2 -> 10.6.11.34
tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
fragment id 0xc4a2, flags DONT_FRAGMENT
ICMP echo_request checksum 0x7ddd
00:32:40:712830: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: 00:1b:21:88:af:89 -> b4:96:91:18:eb:bc 802.1q vlan 218
00:32:40:712831: ip4-input
ICMP: 172.30.0.2 -> 10.6.11.34
tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
fragment id 0xc4a2, flags DONT_FRAGMENT
ICMP echo_request checksum 0x7ddd
00:32:40:712831: ip4-lookup
fib 0 dpo-idx 3 flow hash: 0x00000000
ICMP: 172.30.0.2 -> 10.6.11.34
tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
fragment id 0xc4a2, flags DONT_FRAGMENT
ICMP echo_request checksum 0x7ddd
00:32:40:712832: ip4-glean
ICMP: 172.30.0.2 -> 10.6.11.34
tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
fragment id 0xc4a2, flags DONT_FRAGMENT
ICMP echo_request checksum 0x7ddd
00:32:40:712833: ip4-drop
ICMP: 172.30.0.2 -> 10.6.11.34
tos 0x00, ttl 64, length 84, checksum 0xb4be dscp CS0 ecn NON_ECN
fragment id 0xc4a2, flags DONT_FRAGMENT
ICMP echo_request checksum 0x7ddd
00:32:40:712833: error-drop
rx:lan0.218
00:32:40:712834: drop
ip4-glean: ARP requests sent
I believe for some reason the policy match is not happening. And I am not
able to find examples of IPv4 over IPv6 tunnel, so I might be making a
mistake in setting up the tunnel itself.
Let me know if you need more info, and thank you so much for looking into
this. Much appreciated.
Thanks,
Muthu
On Mon, May 11, 2020 at 3:41 PM Filip Tehlar -X (ftehlar - PANTHEON TECH
SRO at Cisco) <fteh...@cisco.com> wrote:
> Hi Muthu,
>
> you can enable packet tracing [1] and see what nodes are visited.
>
> Filip
>
> [1]
> https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script
> VPP/How To Use The Packet Generator and Packet Tracer - fd.io
> <https://wiki.fd.io/view/VPP/How_To_Use_The_Packet_Generator_and_Packet_Tracer#Step_3._Examine_the_setup_script>
> Introduction. The VPP platform includes packet generation and packet
> tracing facilities. The following example shows steps that you might
> typically use to run a debug version of the vpp executable file, generate
> packets, and to analyze results.
> wiki.fd.io
>
> ------------------------------
> *From:* Muthu Raj <muthuraj.muth...@gmail.com>
> *Sent:* Thursday, May 7, 2020 7:04 PM
> *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) <
> fteh...@cisco.com>
> *Cc:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
> Hello Filip,
>
> I just tried adding a loopback interface and setting the SPD on that
> interface.
> Here is how I tried it:
>
> create loopback interface
> set int state loop0 up
> set interface ip table loop0 0
> set int ip address loop0 11.11.11.11/32
> ipsec sa add 10 spi 1000 esp crypto-key <KEY> crypto-alg aes-cbc-128
> integ-key <KEY> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <KEY> crypto-alg aes-cbc-128
> integ-key <KEY> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DST_IPv6>
> ipsec spd add 1
> set interface ipsec spd loop0 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
> ipsec policy add spd 1 priority 10 outbound action protect sa 10
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ip route table 0 10.6.0.0/16 via loop0
>
> Corresponding configuration on the other side.
> Ping 10.6.0.1 results in these counters in show errors:
>
> 5 arp-input IP4 destination address
> is unset
> 5 ip4-glean ARP requests sent
> If there is any documentation that would help me gain clarity on what is
> going wrong, that would be really appreciated.
>
> Thanks again,
> Muthu
>
>
> On Thu, May 7, 2020 at 8:50 PM Muthu Raj <muthuraj.muth...@gmail.com>
> wrote:
>
> Thanks for replying Filip.
> I have indeed tried to set it up this way, and I think setting the spd on
> lan0.218 messes something up.
>
> I lose the ability to ping even other IPs on the same subnet as the
> address of the lan0.218 interface.
> eg., I can't ping 172.30.0.1 when lan0.218 is 172.30.0.2
>
> I see the ipsec4-output-feature IPSec policy (no match) counter
> going up by the exact number of ping packets, which means even local
> destinations are getting caught in this.
> I am not sure how to fix this.
>
> This is what show ipsec spd looks like:
> spd 1
> ip4-outbound:
> [0] priority 100 action bypass type ip4-outbound protocol IPSEC_ESP
> local addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> remote addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> packets 0 bytes 0
> [2] priority 10 action protect type ip4-outbound protocol any sa 10
> local addr range 10.6.0.0 - 10.6.255.255 port range 0 - 65535
> remote addr range 172.30.0.0 - 172.30.255.255 port range 0 - 65535
> packets 0 bytes 0
> ip6-outbound:
> ip4-inbound-protect:
> [3] priority 10 action protect type ip4-inbound-protect protocol any sa
> 20
> local addr range 10.6.0.0 - 10.6.255.255 port range 0 - 65535
> remote addr range 172.30.0.0 - 172.30.255.255 port range 0 - 65535
> packets 0 bytes 0
> ip6-inbound-protect:
> ip4-inbound-bypass:
> [1] priority 100 action bypass type ip4-inbound-bypass protocol
> IPSEC_ESP
> local addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> remote addr range 0.0.0.0 - 0.0.0.0 port range 0 - 65535
> packets 0 bytes 0
> ip6-inbound-bypass:
>
> Can you guide me on how to set the SPD the right way and if I should setup
> a separate loopback interface with a dummy IP from a different subnet
> (different from local-subnet) and send IPsec remote destinations there and
> set SPD on that dummy interface?
>
> Your help is much appreciated.
> Thanks,
> Muthu
>
>
> On Thu, May 7, 2020 at 7:11 PM Filip Tehlar -X (ftehlar - PANTHEON TECH
> SRO at Cisco) <fteh...@cisco.com> wrote:
>
> Hi Muthu,
>
> I don't see any reason why your approach shouldn't work.
>
> Do you have any specific problem with it?
>
> Filip
> ------------------------------
> *From:* Muthu Raj <muthuraj.muth...@gmail.com>
> *Sent:* Thursday, May 7, 2020 9:08 AM
> *To:* Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) <
> fteh...@cisco.com>
> *Cc:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> *Subject:* Re: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer
> behind NAT (AWS instance)
>
> Hello Filip,
>
> I was wondering if IPv6 IPsec can help us here while NAT traversal is
> being developed.
> Can we route IPv4 traffic over ipv6 ipsec tunnel?
> If we had two interfaces, one with the IPv4 IP which is part of local
> subnet and one with IPv6 subnet IP which will be used for transporting
> the ESP packets.
> The configuration I have in mind is like this:
> vpp# show int address
> lan0 (up):
> lan0.218 (up):
> L3 172.30.0.5/16
> local0 (dn):
> wan0 (up):
> wan0.67 (up):
> L3 <IPv6_HOME>
>
>
> On home side(172.30.0.0/16):
>
> ipsec sa add 10 spi 1000 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DEST_IPv6>
> ipsec sa add 20 spi 1001 esp crypto-key <key> crypto-alg aes-cbc-128
> integ-key <key> integ-alg sha1-96 tunnel-src <SRC_IPv6> tunnel-dst
> <DEST_IPv6>
> ipsec spd add 1
> set interface ipsec spd lan0.218 1
> ipsec policy add spd 1 priority 100 inbound action bypass protocol 50
> ipsec policy add spd 1 priority 100 outbound action bypass protocol 50
> ipsec policy add spd 1 priority 10 inbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ipsec policy add spd 1 priority 20 outbound action protect sa 20
> local-ip-range 172.30.0.0 - 172.30.255.255 remote-ip-range 10.6.0.0 -
> 10.6.255.255
> ip route add 10.6.0.0/16 via lan0.218
>
> And similar configuration on the remote side.
>
> Let me know if I am going obviously wrong.
> This is what I could get from the docs available and there doesn't see
> to be examples for IPv6.
>
> I would like traffic flow from outside VPP source, like 172.30.0.2 on
> home side to flow to 172.30.0.5 (VPP) and flow over the tunnel for
> 10.6.0.0/16 destination.
> Thanks,
> Muthu
>
> On Tue, May 5, 2020 at 1:58 PM Filip Tehlar -X (ftehlar - PANTHEON
> TECH SRO at Cisco) <fteh...@cisco.com> wrote:
> >
> > Hi,
> >
> > NAT traversal in IKE isn't supported yet, but there is an ongoing work
> to add one:
> > https://gerrit.fd.io/r/c/vpp/+/26726
> >
> > Filip
> > ________________________________
> > From: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> on behalf of
> muthurajmuth...@gmail.com <muthurajmuth...@gmail.com>
> > Sent: Tuesday, May 5, 2020 8:49 AM
> > To: vpp-dev@lists.fd.io <vpp-dev@lists.fd.io>
> > Subject: [SUSPECTED SPAM] [vpp-dev] Troubleshooting IPsec peer behind
> NAT (AWS instance)
> >
> > Hello,
> >
> > I am trying out IPSec on VPP, and used the wiki[1] to create an IPSec
> > tunnel between an AWS instance(remote) and my home. The tunnel was
> > established successfully, and when pinging an IP on the remote side,
> > the icmp req flows over the tunnel, is seen by the remote box, and
> > responded back as well. I also see that the packets indeed end up
> > reaching my home VPP instance - however, they do not reach the last
> > hop. When I run show int, the ipip0 interface does not show the rx
> > counter at all, and when running `show errors` I do not see the
> > counter for the `ipsec4-tun-input` node either. Neither do I see the
> > `esp4-encrypt-tun' counter.
> >
> > My preliminary guess is that it has something to do with the fact that
> > on AWS we cannot see the public IP inside the instance and so that
> > cannot be assigned to the interface itself, so probably the ESP
> > packets are generated with source as the private IP address
> > corresponding to the public IP. With strongswan, we specify an
> > explicit sourceip parameter, like in the snippet below
> >
> > left=1.2.3.4
> > leftid=1.2.3.4
> > leftsubnet=172.16.0.0/16
> > right=4.5.6.7 #AWS public IP
> > rightsourceip=10.6.82.34 #AWS private IP for that public IP, seen
> > inside the instance.
> > rightsubnet=10.6.0.0/16
> >
> > So it is essentially like being NATed from the private IP to the public
> IP.
> >
> > I am attaching the ikev2 sa as seen from both sides.
> > How would I fix this issue?
> > Any help is appreciated very much.
> >
> > Thanks in advance.
> >
> >
> > This is from the home side. I've changed the IPs on home and remote
> > side. The private IP addresses have been left as it is.
> >
> > vpp# show ikev2 sa
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb
> >
> r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c
> > SK_d
> 1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde
> > SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c
> > r:04a71f139f2076058ceafb9be73eb359e43bc308
> > SK_e
> i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc
> >
> r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409
> > SK_p
> i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7
> >
> r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8
> > identifier (i) fqdn vpp.aws
> > identifier (r) fqdn vpp.home
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 147e7a05 spi(r) de36dcbc
> > SK_e
> i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229
> >
> r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532
> > SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3
> > r:9504182eb614d90aa8fe742122ec9d98c1b6e224
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip 4.5.6.7 rspi d6726c2768b2420
> >
> > --------------------------------------------------------------------
> > Here is the AWS side
> >
> > vpp# show ikev2 sa
> > iip 1.2.3.4 ispi a72ae3cef809725c rip <aws private IP corresponding
> > to public IP> rspi b8b7b8ef09266a6d
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:d3e4299761fd93edd3df16456cb0ca9f717f67e57155fa7cb4cd0b9a1d371019
> >
> r:e9a33a33b901366438e262d225a418e9489839415562d3e3673107e0d81d830f
> > SK_d
> 7e4f795db87a02c5b4d5ea738945521473f5e449b783f3ac4b954be7716b7909
> > SK_a i:13639a11b6e96e65dd38d095a87fc1b5ceefdc6b
> > r:97c96809563dfe39c3d2762c1ff1bf0a8fbc3576
> > SK_e
> i:114661a058686bd4362d8515ce83a7d7de098af11b08084c407ad51843316135
> >
> r:d812542cfa988e6c302fc52d848fb2d7b7321d6c3e77ee04134338a21c0ccba8
> > SK_p
> i:a65ea61c70b3cb749dedc205b7715b4c278a4bc630c6508d89a55a00cd00a2cd
> >
> r:9e23352bac4d21f6f0d2ec8de82e556db3ddaba0ade0c4d664a020da3986d17b
> > identifier (i) fqdn vpp.home
> > identifier (r) fqdn vpp.aws
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 31c649f8 spi(r) 967b11c4
> > SK_e
> i:6a1b5898746bc922af1beba021768cd6417a0e8a4c555e5544781fee302cf633
> >
> r:2035a8be8fae47c284cef445381cef487bcd670bddc31558109c0303bc0f5399
> > SK_a i:da119e539529803a3d2a883c01a825211c782bd2
> > r:2330bc2dd9eb3741e3df649bcc3f7e5320fba512
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> > iip 1.2.3.4 ispi a72ae3cef809725c rip <aws private IP corresponding
> > to public IP> rspi b8b7b8ef09266a6d
> > iip 1.2.3.4 ispi d8607eea97ac12a9 rip <aws private IP corresponding
> > to public IP> rspi d6726c2768b2420
> > encr:aes-cbc-256 prf:hmac-sha2-256 integ:sha1-96 dh-group:modp-2048
> > nonce
> i:eb01e6ef107ba7018679bd239e25d4557f2465323caf0d3213b453ca59af3deb
> >
> r:1d9cb8f11cd69d4b2f73b182028d8aa8854a49bb3c99797f3994575c2994154c
> > SK_d
> 1eee29fff1ff234f1452006a79a7e27787e83331b29954300a70a9d6061f2fde
> > SK_a i:7f86a547c2d9cb2a4035e4926ca6e23c745c6c8c
> > r:04a71f139f2076058ceafb9be73eb359e43bc308
> > SK_e
> i:281c47cd100f69a3425031667150d3054124ff887d77a4a1f43fd7dece7486fc
> >
> r:3f72f8e973ee62962dc9dffd64d80af9e83993acbcd3690adf85044a23310409
> > SK_p
> i:79c096024c45499bd43b5d716c56e5152252c433b112195201dd5c4c23a1f1c7
> >
> r:fb7e3b35d57b2987bf61f04858a4afaeee10045c6001594f9f2e505b94d950d8
> > identifier (i) fqdn vpp.home
> > identifier (r) fqdn vpp.aws
> > child sa 0:
> > encr:aes-cbc-256 integ:sha1-96 esn:yes
> > spi(i) 147e7a05 spi(r) de36dcbc
> > SK_e
> i:31e22be618e3fe60faf935759e75fdc699f743486dd18f07de8b78747d10d229
> >
> r:30b10195fdb1cd5b7384a2db92d5a51fd9fab7f6fc7db775957e3dc862d72532
> > SK_a i:f98f3539966a66afec330c7cdf85fbe2794e01d3
> > r:9504182eb614d90aa8fe742122ec9d98c1b6e224
> > traffic selectors (i):
> > 0 type 7 protocol_id 0 addr 172.30.0.0 - 172.30.255.255 port 0 -
> 65535
> > traffic selectors (r):
> > 0 type 7 protocol_id 0 addr 10.6.0.0 - 10.6.255.255 port 0 - 65535
> >
> > [1]
> >
> https://wiki.fd.io/view/VPP/IPSec_and_IKEv2#IKEv2_negotiation_between_a_VPP_responder_and_a_VPP_initiator.2C_using_RSA_signature_authentication_method
> >
> > *I've used PSK in place of RSA signature.
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16310): https://lists.fd.io/g/vpp-dev/message/16310
Mute This Topic: https://lists.fd.io/mt/74050344/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
