Thanks for helping. I removed spd configs on both sides, but still no luck. I am pinging from r230 side.
It seems r230 is able to sending ping pkts over dpdk interface. However, on
r740 side, gre0 interface drops all of them. See the attached updated cfg
files and log files for more details.
IPSec: remote:10.10.10.11 spi:255129 (0x0003e499) seq 418
00:06:52:367944: esp4-decrypt-tun
esp: crypto aes-cbc-128 integrity sha1-96 pkt-seq 418 sa-seq 0 sa-seq-hi 0
00:06:52:367948: ip4-input-no-checksum
GRE: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 254, length 66, checksum 0x9464
fragment id 0x0000
GRE teb
00:06:52:367948: ip4-not-enabled
GRE: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 254, length 66, checksum 0x9464
fragment id 0x0000
GRE teb
00:06:52:367948: error-drop
rx:gre0
00:06:52:367949: drop
ip4-input: unknown ip protocol
On Fri, Oct 4, 2019 at 8:39 AM Neale Ranns (nranns) <nra...@cisco.com>
wrote:
>
>
> Hi Chuan,
>
>
>
> Please remove the SPD config. For tunnels all packets that ingress/egress
> the tunnel are decrypted/encrypted, so no policy is required. The presence
> of the SPD on the ingress eth0 link could be why it’s not working.
>
> Please provide packet traces when you are reporting packet loss problems,
> it helps us debug.
>
>
>
> For reference the setup for GRE TEB IPSec can be found in the python UT
> TestIpsecGreTebIfEsp.
>
>
>
> Regards,
>
> neale
>
>
>
>
>
> *From: *<vpp-dev@lists.fd.io> on behalf of "Chuan Han via Lists.Fd.Io"
> <chuanhan=google....@lists.fd.io>
> *Reply to: *"chuan...@google.com" <chuan...@google.com>
> *Date: *Friday 4 October 2019 at 02:15
> *To: *"John Lo (loj)" <l...@cisco.com>
> *Cc: *"vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> *Subject: *Re: [vpp-dev] How to configure l2 gre over ipsec in vpp 19.08
>
>
>
> Hi,
>
>
>
> Thanks for information.
>
>
>
> I am trying to configure l2 gre over ipsec transport mode.
>
>
>
> Here are my startup.cfg files. Can you help check if my configuration is
> correct or not?
>
>
>
> r230 and r740 are two servers which are directly connected.
>
>
>
> eth0 is the phy nic. host-veth1 is one endpoint of veth pair. the other
> end is connected to a network namespace with ip address 172.16.1.2.
>
>
>
> From the network namespace, I cannot ping the other end 172.16.1.1.
>
>
>
> On r230, I can see unknown ip protocol errors.
>
> vpp# sh errors
> Count Node Reason
> 5 null-node blackholed packets
> 5 ipsec4-output-feature IPSec policy (no match)
> 1 esp4-decrypt-tun ESP pkts received
> 1 ipsec4-tun-input good packets received
> 1 ipsec4-input-feature IPSEC pkts received
> 1 ip4-input unknown ip protocol
> 592 gre-encap GRE output packets
> encapsulated
> 592 ipsec4-output-feature IPSec policy bypass
> 592 esp4-encrypt-tun ESP pkts received
> 592 l2-output L2 output packets
> 592 l2-learn L2 learn packets
> 1 l2-learn L2 learn misses
> 592 l2-input L2 input packets
> 592 l2-flood L2 flood packets
> vpp# sh int
> Name Idx State MTU (L3/IP4/IP6/MPLS)
> Counter Count
> eth0 1 up 9000/0/0/0 rx
> packets 1
> rx
> bytes 166
> tx
> packets 592
> tx
> bytes 88816
> drops
> 5
> ip4
> 1
>
> rx-error 1
> gre0 3 up 9000/0/0/0 drops
> 1
> ip4
> 1
> host-veth1 2 up 9000/0/0/0 rx
> packets 592
> rx
> bytes 24892
> local0 0 down 0/0/0/0
> vpp# sh errors
> Count Node Reason
> 5 null-node blackholed packets
> 5 ipsec4-output-feature IPSec policy (no match)
> 1 esp4-decrypt-tun ESP pkts received
> 1 ipsec4-tun-input good packets received
> 1 ipsec4-input-feature IPSEC pkts received
> 1 ip4-input unknown ip protocol
> 592 gre-encap GRE output packets
> encapsulated
> 592 ipsec4-output-feature IPSec policy bypass
> 592 esp4-encrypt-tun ESP pkts received
> 592 l2-output L2 output packets
> 592 l2-learn L2 learn packets
> 1 l2-learn L2 learn misses
> 592 l2-input L2 input packets
> 592 l2-flood L2 flood packets
> vpp#
>
>
>
> On r740, I see the same errors:
>
>
>
> vpp# sh int
> Name Idx State MTU (L3/IP4/IP6/MPLS)
> Counter Count
> eth0 1 up 9000/0/0/0 rx
> packets 592
> rx
> bytes 88816
> tx
> packets 1
> tx
> bytes 166
> ip4
> 592
> gre0 3 up 9000/0/0/0 drops
> 592
> ip4
> 592
> host-veth1 2 up 9000/0/0/0 rx
> packets 1
> rx
> bytes 70
> local0 0 down 0/0/0/0
> vpp# sh errors
> Count Node Reason
> 592 esp4-decrypt-tun ESP pkts received
> 592 ipsec4-tun-input good packets received
> 592 ipsec4-input-feature IPSEC pkts received
> 592 ip4-input unknown ip protocol
> 1 gre-encap GRE output packets
> encapsulated
> 1 ipsec4-output-feature IPSec policy bypass
> 1 esp4-encrypt-tun ESP pkts received
> 1 l2-output L2 output packets
> 1 l2-learn L2 learn packets
> 1 l2-learn L2 learn misses
> 1 l2-input L2 input packets
> 1 l2-flood L2 flood packets
> vpp#
>
>
>
> On Wed, Oct 2, 2019 at 9:13 AM John Lo (loj) <l...@cisco.com> wrote:
>
> To create GRE tunnel in L2 mode, you can add “teb” keyword in the create
> CLI which makes the GRE tunnel work in transparent ethernet bridging mode:
>
>
>
> vpp# create gre ?
>
> create gre tunnel create gre tunnel src <addr>
> dst <addr> [instance <n>] [outer-fib-id <fib>] [*teb* | erspan
> <session-id>] [del]
>
>
>
> In theory, a GRE tunnel can be configured with IPSec, as described by
> Neale, irrespective of it being in teb mode or not. Neale, please correct
> me if it is not the case.
>
>
>
> Regards,
>
> John
>
>
>
> *From:* vpp-dev@lists.fd.io <vpp-dev@lists.fd.io> *On Behalf Of *Chuan
> Han via Lists.Fd.Io
> *Sent:* Wednesday, October 02, 2019 11:32 AM
> *To:* Neale Ranns (nranns) <nra...@cisco.com>
> *Cc:* vpp-dev@lists.fd.io
> *Subject:* Re: [vpp-dev] How to configure l2 gre over ipsec in vpp 19.08
>
>
>
> Gre is l3 in this case. Right? This limits the possible use cases.
>
>
>
> Is there any plan to support l2 gre over ipsec transport mode? It seems
> vpp 17 support s this feature. Not sure why it is dropped in 19.
>
>
>
> On Wed, Oct 2, 2019, 12:18 AM Neale Ranns (nranns) <nra...@cisco.com>
> wrote:
>
>
> Hi Chuan,
>
> IPSec and GRE is supported using the tunnel protection mechanism :
> https://wiki.fd.io/view/VPP/IPSec
>
> GRE over IPSec is only support when the SA is in tunnel mode. This means
> there is a double encap of the IP header ; once by the SA (in tunnel mode)
> and once by the tunnel itself. (Which has always been the case in VPP).
>
> Example config follows :
>
> DBGvpp# ipsec sa add 20 spi 200 crypto-key
> 6541686776336961656264656f6f6579 crypto-alg aes-cbc-128 tunnel-src
> 10.10.10.10 tunnel-dst 10.10.10.11
> DBGvpp# ipsec sa add 30 spi 300 crypto-key
> 6541686776336961656264656f6f6579 crypto-alg aes-cbc-128 tunnel-src
> 10.10.10.11 tunnel-dst 10.10.10.10
> DBGvpp# create gre tunnel src 10.10.10.10 dst 10.10.10.11
> gre0
> DBGvpp# ipsec tunnel protect gre0 sa-in 20 sa-out 30
> DBGvpp# sh ipsec protect
> gre0
> output-sa:
> [1] sa 30 (0x1e) spi 300 (0x0000012c) protocol:esp flags:[tunnel ]
> input-sa:
> [0] sa 20 (0x14) spi 200 (0x000000c8) protocol:esp flags:[tunnel
> Protect ]
>
> Regards,
> neale
>
>
> From: <vpp-dev@lists.fd.io> on behalf of "Chuan Han via Lists.Fd.Io"
> <chuanhan=google....@lists.fd.io>
> Reply to: "chuan...@google.com" <chuan...@google.com>
> Date: Wednesday 2 October 2019 at 02:08
> To: "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> Cc: "vpp-dev@lists.fd.io" <vpp-dev@lists.fd.io>
> Subject: [vpp-dev] How to configure l2 gre over ipsec in vpp 19.08
>
> Hi, vpp experts,
>
> I am trying to configure l2 gre over ipsec. I followed the steps here:
> https://docs.fd.io/vpp/16.12/ipsec_gre_doc.html
>
> I hit the following error:
> create ipsec: unknown input `gre tunnel src 10.10.10.10 dst...'
>
> My vpp version is v19.08.1-release
>
> It seems on this version the "create ipsec gre tunnel" command does not
> work. If so, is there any other way of configuring l2 gre over ipsec in
> 19.08?
>
> Please advise.
>
> Thanks.
> Chuan
>
>
r230.cfg
Description: Binary data
vpp# sh int addr
eth0 (up):
L3 10.10.10.10/24
gre0 (up):
L2 bridge bd-id 1 idx 1 shg 0
host-veth1 (up):
L2 bridge bd-id 1 idx 1 shg 0
local0 (dn):
vpp# sh int
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
eth0 1 up 9000/0/0/0 rx packets 900
rx bytes 135016
ip4 900
gre0 3 up 9000/0/0/0 drops 900
ip4 900
host-veth1 2 up 9000/0/0/0
local0 0 down 0/0/0/0
vpp# sh int
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
eth0 1 up 9000/0/0/0 rx packets 902
rx bytes 135316
ip4 902
gre0 3 up 9000/0/0/0 drops 902
ip4 902
host-veth1 2 up 9000/0/0/0
local0 0 down 0/0/0/0
vpp# sh error
Count Node Reason
904 esp4-decrypt-tun ESP pkts received
904 ipsec4-tun-input good packets received
904 ip4-input unknown ip protocol
vpp# sh error
Count Node Reason
906 esp4-decrypt-tun ESP pkts received
906 ipsec4-tun-input good packets received
906 ip4-input unknown ip protocol
vpp# trace add dpdk-input 10
vpp# sh trace
------------------- Start of thread 0 vpp_main -------------------
No packets in trace buffer
------------------- Start of thread 1 vpp_wk_0 -------------------
Packet 1
00:06:52:367932: dpdk-input
eth0 rx queue 0
buffer 0x9efae: current data 0, length 150, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000000
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 150
buf_len 2176, data_len 150, ol_flags 0x180, data_off 128, phys_addr 0xb81bec00
packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
Packet Types
RTE_PTYPE_L2_ETHER (0x0001) Ethernet packet
RTE_PTYPE_L3_IPV4 (0x0010) IPv4 packet without extension headers
IP4: b4:96:91:23:1e:d6 -> 24:6e:96:b4:b2:06
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
00:06:52:367938: ethernet-input
frame: flags 0x3, hw-if-index 1, sw-if-index 1
IP4: b4:96:91:23:1e:d6 -> 24:6e:96:b4:b2:06
00:06:52:367939: ip4-input-no-checksum
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
00:06:52:367942: ip4-lookup
fib 0 dpo-idx 5 flow hash: 0x00000000
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
00:06:52:367944: ip4-local
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
00:06:52:367944: ipsec4-tun-input
IPSec: remote:10.10.10.11 spi:255129 (0x0003e499) seq 418
00:06:52:367944: esp4-decrypt-tun
esp: crypto aes-cbc-128 integrity sha1-96 pkt-seq 418 sa-seq 0 sa-seq-hi 0
00:06:52:367948: ip4-input-no-checksum
GRE: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 254, length 66, checksum 0x9464
fragment id 0x0000
GRE teb
00:06:52:367948: ip4-not-enabled
GRE: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 254, length 66, checksum 0x9464
fragment id 0x0000
GRE teb
00:06:52:367948: error-drop
rx:gre0
00:06:52:367949: drop
ip4-input: unknown ip protocol
Packet 2
00:06:53:393296: dpdk-input
eth0 rx queue 0
buffer 0x9ef87: current data 0, length 150, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x1000001
ext-hdr-valid
l4-cksum-computed l4-cksum-correct
PKT MBUF: port 0, nb_segs 1, pkt_len 150
buf_len 2176, data_len 150, ol_flags 0x180, data_off 128, phys_addr 0xb81be240
packet_type 0x11 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
Packet Offload Flags
PKT_RX_IP_CKSUM_GOOD (0x0080) IP cksum of RX pkt. is valid
PKT_RX_L4_CKSUM_GOOD (0x0100) L4 cksum of RX pkt. is valid
vpp# sh int addr
eth0 (up):
L3 10.10.10.11/24
gre0 (up):
L2 bridge bd-id 1 idx 1 shg 0
host-veth1 (up):
L2 bridge bd-id 1 idx 1 shg 0
local0 (dn):
vpp# sh int
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
eth0 1 up 9000/0/0/0 tx packets 805
tx bytes 120766
gre0 3 up 9000/0/0/0
host-veth1 2 up 9000/0/0/0 rx packets 808
rx bytes 33964
drops 3
local0 0 down 0/0/0/0
vpp# sh err
Count Node Reason
810 gre-encap GRE output packets encapsulated
810 esp4-encrypt-tun ESP pkts received
810 l2-output L2 output packets
810 l2-learn L2 learn packets
2 l2-learn L2 learn misses
810 l2-input L2 input packets
810 l2-flood L2 flood packets
3 ethernet-input no error
vpp# sh int
Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
eth0 1 up 9000/0/0/0 tx packets 813
tx bytes 121966
gre0 3 up 9000/0/0/0
host-veth1 2 up 9000/0/0/0 rx packets 816
rx bytes 34300
drops 3
local0 0 down 0/0/0/0
vpp# sh err
Count Node Reason
820 gre-encap GRE output packets encapsulated
820 esp4-encrypt-tun ESP pkts received
820 l2-output L2 output packets
820 l2-learn L2 learn packets
2 l2-learn L2 learn misses
820 l2-input L2 input packets
820 l2-flood L2 flood packets
3 ethernet-input no error
vpp#
vpp# trace add dpdk-input 10
vpp# sh trace
------------------- Start of thread 0 vpp_main -------------------
No packets in trace buffer
------------------- Start of thread 1 vpp_wk_0 -------------------
No packets in trace buffer
------------------- Start of thread 2 vpp_wk_1 -------------------
No packets in trace buffer
vpp# trace add af-packet-input 10
vpp# sh trace
------------------- Start of thread 0 vpp_main -------------------
No packets in trace buffer
------------------- Start of thread 1 vpp_wk_0 -------------------
No packets in trace buffer
------------------- Start of thread 2 vpp_wk_1 -------------------
Packet 1
00:14:34:327762: af-packet-input
af_packet: hw_if_index 2 next-index 4
tpacket2_hdr:
status 0x20000001 len 42 snaplen 42 mac 66 net 80
sec 0x5d9790b4 nsec 0x1ae4dbdb vlan 0 vlan_tpid 0
00:14:34:327765: ethernet-input
ARP: 16:2e:9b:03:ea:d3 -> ff:ff:ff:ff:ff:ff
00:14:34:327766: l2-input
l2-input: sw_if_index 2 dst ff:ff:ff:ff:ff:ff src 16:2e:9b:03:ea:d3
00:14:34:327768: l2-learn
l2-learn: sw_if_index 2 dst ff:ff:ff:ff:ff:ff src 16:2e:9b:03:ea:d3 bd_index 1
00:14:34:327769: l2-flood
l2-flood: sw_if_index 2 dst ff:ff:ff:ff:ff:ff src 16:2e:9b:03:ea:d3 bd_index 1
00:14:34:327770: l2-output
l2-output: sw_if_index 3 dst ff:ff:ff:ff:ff:ff src 16:2e:9b:03:ea:d3 data 08 06 00 01 08 00 06 04 00 01 16 2e
00:14:34:327771: gre-encap
GRE: tunnel 0 len 42 src 10.10.10.11 dst 10.10.10.10
00:14:34:327771: adj-l2-midchain
adj-idx 3 : ethernet via 0.0.0.0 gre0: mtu:9000 4500000000000000fe2f94a60a0a0a0b0a0a0a0a00006558
stacked-on entry:11:
[@1]: ipv4 via 10.10.10.10 eth0: mtu:9000 246e96b4b206b49691231ed60800
00:14:34:327773: esp4-encrypt-tun
esp: sa-index 0 spi 255129 (0x0003e499) seq 855 sa-seq-hi 0 crypto aes-cbc-128 integrity sha1-96
00:14:34:327777: adj-midchain-tx-no-count
adj-midchain:[3]:ethernet via 0.0.0.0 gre0: mtu:9000 4500000000000000fe2f94a60a0a0a0b0a0a0a0a00006558
stacked-on entry:11:
[@1]: ipv4 via 10.10.10.10 eth0: mtu:9000 246e96b4b206b49691231ed60800
00:14:34:327778: ip4-rewrite
tx_sw_if_index 1 dpo-idx 1 : ipv4 via 10.10.10.10 eth0: mtu:9000 246e96b4b206b49691231ed60800 flow hash: 0x0000000e
00000000: 246e96b4b206b49691231ed608004500008800000000fd32951b0a0a0a0b0a0a
00000020: 0a0a0003e4990000035770d8b27234e68497540a61505ff88ce4bb3c
00:14:34:327778: eth0-output
eth0 l2_hdr_offset_valid l3_hdr_offset_valid
IP4: b4:96:91:23:1e:d6 -> 24:6e:96:b4:b2:06
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
00:14:34:327793: eth0-tx
eth0 tx queue 2
buffer 0x903f9: current data -82, length 150, buffer-pool 0, ref-count 1, totlen-nifb 0, trace handle 0x2000000
l2-hdr-offset 0 l3-hdr-offset 14
PKT MBUF: port 65535, nb_segs 1, pkt_len 150
buf_len 2176, data_len 150, ol_flags 0x0, data_off 46, phys_addr 0xe000fec0
packet_type 0x0 l2_len 0 l3_len 0 outer_l2_len 0 outer_l3_len 0
rss 0x0 fdir.hi 0x0 fdir.lo 0x0
IP4: b4:96:91:23:1e:d6 -> 24:6e:96:b4:b2:06
IPSEC_ESP: 10.10.10.11 -> 10.10.10.10
tos 0x00, ttl 253, length 136, checksum 0x951b
fragment id 0x0000
Packet 2
00:14:35:351106: af-packet-input
af_packet: hw_if_index 2 next-index 4
tpacket2_hdr:
status 0x20000001 len 42 snaplen 42 mac 66 net 80
sec 0x5d9790b5 nsec 0x1c52186e vlan 0 vlan_tpid 0
r740.cfg
Description: Binary data
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#14123): https://lists.fd.io/g/vpp-dev/message/14123 Mute This Topic: https://lists.fd.io/mt/34364734/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
