Re: [vpp-dev] VPP C++ API

[Klement Sekera via Lists.Fd.Io]

Hi,

can you please run in gdb (or open core) and pass us a stack trace?

Thanks,
Klement

On Sun, 2018-07-01 at 19:36 -0700, xpa...@gmail.com wrote:
> Hello,
> I'm trying to understand how works VOM wrapper for C++ API.
> VOM manager header: https://pastebin.com/BgG7td5s
> VOM manager cpp: https://pastebin.com/890jjUJm
> main.cpp: 
> 
> int main() {
>     TVppManager manager;
>     std::cout << "Starting\n";
>     manager.Start();
>  
>     sleep(60);
>  
>     manager.Stop();
> }
> 
> When I'm trying to run sudo ./main a I got error:
> Starting
> Calling connect
> Manager is connected
> Mon Jul  2 05:33:55 2018 [debug]hw.cpp:129 write() itf-events
> Mon Jul  2 05:33:55 2018 [debug]rpc_cmd.hpp:120 operator()() itf-
> events 0
> Started
> main: /place/home/xpahos/git/vpp/build-data/../src/vpp-
> api/vapi/vapi.c:696: vapi_msg_is_with_context: Assertion `id <=
> __vapi_metadata.count' failed.
> Aborted
> 
> I found that id looks like unsigned value overflow. How can I debug
> this? I don't see anything in trace dump.
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#9753): https://lists.fd.io/g/vpp-dev/message/9753
> Mute This Topic: https://lists.fd.io/mt/22999158/675704
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [ksek...@cisco.com]
> -=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9755): https://lists.fd.io/g/vpp-dev/message/9755
Mute This Topic: https://lists.fd.io/mt/22999158/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] mheap performance issue and fixup

[Kingwel Xie]

Hi Dave,

I notices you made some improvement to mheap in the latest code. I’m afraid 
that it would not work as you expected. Actually the same idea came to my mind 
and then I realized it doesn’t work well.

Here is your code:

  if (align > MHEAP_ELT_OVERHEAD_BYTES)
n_user_data_bytes = clib_max (n_user_data_bytes,
  align - 
MHEAP_ELT_OVERHEAD_BYTES);

Instead of allocating a very small object, you round it up to a bit bigger. 
Let’s take a typical case: you want 8 bytes, but aligned to 64B. Then you are 
actually allocating from free bin started with 64-8=56.

This is problematic if all meahp elements in free bin size 56B are not 64B 
aligned. It would become a performance issue when this 56B free bin happens to 
have a lot of elements, which means you have to go through all elements one by 
one.

It would not do better even if you turn to a bigger free bin, because you round 
the requested block size up to a bigger size.

As you can see in my patch, I add a modifier to the request block size when 
looking up an appropriate free bin:

In mheap_get_search_free_list:

  /* kingwel, lookup a free bin which is big enough to hold everything 
align+align_offset+lo_free_size+overhead */
  word modifier =
(align >
 MHEAP_USER_DATA_WORD_BYTES ? align + align_offset +
 sizeof (mheap_elt_t) : 0);
  bin = user_data_size_to_bin_index (n_user_bytes + modifier);

This is to ensure we can always locate an element without going through the 
whole list. Also I take the align_offset into consideration.

BTW, probably I need more time to work out a test case in test_mheap to prove 
my fix. It seems the latest code doesn’t generate test_mheap any more. I have 
to figure it out at first.

Regards,
Kingwel


From: Dave Barach (dbarach) 
Sent: Thursday, June 28, 2018 9:06 PM
To: Kingwel Xie ; Damjan Marion 
Cc: vpp-dev@lists.fd.io
Subject: RE: [vpp-dev] mheap performance issue and fixup

Allocating a large number of 16 byte objects @ 64 byte alignment will never 
work very well. If you pad the object such that the mheap header plus the 
object is exactly 64 bytes, the issue may go away.

With that hint, however, I’ll go build a test vector. It sounds like the mheap 
required size calculation might be a brick shy of a load.

D.

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Kingwel Xie
Sent: Thursday, June 28, 2018 2:25 AM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>; Damjan 
Marion mailto:dmar...@me.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] mheap performance issue and fixup

No problem. I’ll do that later.

Actually there has been a discussion about mheap performance, which describe 
the issue we are talking about. Please also check it again:

https://lists.fd.io/g/vpp-dev/topic/10642197#6399


From: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Sent: Thursday, June 28, 2018 3:38 AM
To: Damjan Marion mailto:dmar...@me.com>>; Kingwel Xie 
mailto:kingwel@ericsson.com>>
Cc: vpp-dev@lists.fd.io
Subject: RE: [vpp-dev] mheap performance issue and fixup

+1.

It would be super-helpful if you were to add test cases to 
.../src/vppinfra/test_mheap.c, and push a draft patch so we can reproduce / fix 
the problem(s).

Thanks... Dave

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Damjan Marion
Sent: Wednesday, June 27, 2018 3:27 PM
To: Kingwel Xie mailto:kingwel@ericsson.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] mheap performance issue and fixup


Dear Kingwei,

We finally managed to look at your mheap patches, sorry for delay.

Still we are not 100% convinced that there is a bug(s) in the mheap code.
Please note that that mheap code is stable, not changed frequently and used for 
years.

It will really help if you can provide test vectors for each issue you observed.
It will be much easier to understand the problem and confirm the fix if we are 
able to reproduce it in controlled environment.

thanks,

Damjan


From: mailto:vpp-dev@lists.fd.io>> on behalf of Kingwel 
Xie mailto:kingwel@ericsson.com>>
Date: Thursday, 19 April 2018 at 03:19
To: "Damjan Marion (damarion)" mailto:damar...@cisco.com>>
Cc: "vpp-dev@lists.fd.io" 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] mheap performance issue and fixup

Hi Damjan,

We will do it asap. Actually we are quite new to vPP and even don’t know how to 
make bug report and code contribution or so.

Regards,
Kingwel

From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Damjan Marion
Sent: Wednesday, April 18, 2018 11:30 PM
To: Kingwel Xie mailto:kingwel@ericsson.com>>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] mheap performance issue and fixup

Dear Kingwel,

Thank you for your em

Re: [vpp-dev] twamp

[Neale Ranns via Lists.Fd.Io]

Hi Avi,

None that I am aware of, but its inclusion would be welcome. 

/neale

-Original Message-
From:  on behalf of "Avi Cohen (A)" 
Date: Thursday, 28 June 2018 at 15:04
To: "vpp-dev@lists.fd.io" 
Subject: [vpp-dev] twamp

Hi,
Is there any plan to implement/integrate TWAMP into VPP ?
Regards
Avi



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9757): https://lists.fd.io/g/vpp-dev/message/9757
Mute This Topic: https://lists.fd.io/mt/22866386/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Is VPP IPSec implementation thread safe?

[Kingwel Xie]

Hi Vamsi, Damjan,

I’d like to contribute my two cents about IPSEC. We have been working on the 
improvement for quite some time.


  1.  Great that vPP supports IPSEC, but the code is mainly for PoC. It lacks 
of many features: buffer chain, AES-GCM/AES-CTR, UDP encap (seems already there 
in master track?) many hardcode, broken packet trace,  SEQ handling, etc.
  2.  Performance is not good, because of wrongly usage of openssl, buffer 
copying. We can see 100% up after fixing all these issues.
  3.  DPDK Ipsec has better performance but the quality of code is not good, 
many bugs.

If you are looking for a production IPSEC, vpp is a good start but you still 
have a lot things to do.

Regards,
Kingwel

From: vpp-dev@lists.fd.io  On Behalf Of Vamsi Krishna
Sent: Monday, July 02, 2018 12:05 PM
To: Damjan Marion 
Cc: Jim Thompson ; Dave Barach ; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Is VPP IPSec implementation thread safe?

Thanks Damjan.

Is the IPSec code using spd attached to an interface ("set interface ipsec spd 
 ") production quality?
How is the performance of this code in terms of throughput, are there any 
benchmarks that can be referred to?

Thanks
Vamsi

On Sat, Jun 30, 2018 at 1:35 AM, Damjan Marion 
mailto:dmar...@me.com>> wrote:

Dear Vamsi,

It was long long time ago when I wrote original ipsec code, so I don't remember 
details anymore.
For IKEv2, i would suggest using external implementation, as ikev2 stuff is 
just PoC quality code.

--
Damjan


On 29 Jun 2018, at 18:38, Vamsi Krishna 
mailto:vamsi...@gmail.com>> wrote:

Hi Damjan, Dave,

Can you please also answer the questions I had in the email just before Jim 
hijacked the thread.

Thanks
Vamsi

On Fri, Jun 29, 2018 at 3:06 PM, Damjan Marion 
mailto:dmar...@me.com>> wrote:

Hi Jim,

Atomic add sounds like a reasonable solution to me...

--
Damjan


On 28 Jun 2018, at 09:26, Jim Thompson 
mailto:j...@netgate.com>> wrote:

All,

I don't know if any of the previously-raised issues occur in real-life.  
Goodness knows we've run billions of IPsec packets in the test harnesses 
(harnessi?) here without seeing them.

There are a couple issues with IPsec and multicore that haven't been raised, 
however, so I'm gonna hijack the thread.

If multiple worker threads are configured in VPP, it seems like there’s the 
potential for problems with IPsec where the sequence number or replay window 
for an SA could get stomped on by two threads trying to update them at the 
same. We assume that this issue is well known since the following comment 
occurs at line 173 in src/vnet/ipsec/esp.h

/* TODO seq increment should be atomic to be accessed by multiple workers */

See: https://github.com/FDio/vpp/blob/master/src/vnet/ipsec/esp.h#L173

We've asked if anyone is working on this, and are willing to try and fix it, 
but would need some direction on what is the best way to accomplish same.

We could try to use locking, which would be straightforward but would add 
overhead.  Maybe that overhead could be offset some by requesting a block of 
sequence numbers upfront for all of the packets being processed instead of 
getting a sequence number and incrementing as each packet is processed.

There is also the clib_smp_atomic_add() call, which invokes 
__sync_fetch_and_add(addr,increment).  This is a GCC built_in that uses a 
memory barrier to avoid obtaining a lock.  We're not sure if there are 
drawbacks to using this.

See: http://gcc.gnu.org/onlinedocs/gcc-4.4.3/gcc/Atomic-Builtins.html

GRE uses clib_smp_atomic_add() for sequence number processing, see 
src/vnet/gre/gre.c#L409 and src/vnet/gre/gre.c#L421

Finally, there seem to be issues around AES-GCM nonce processing when operating 
multi-threaded.  If it is nonce processing, it can probably (also) be addressed 
via clib_smp_atomic_add(), but.. don't know yet.

We've raised these before, but haven't received much in the way of response.  
Again, we're willing to work on these, but would like a bit of 'guidance' from 
vpp-dev.

Thanks,

Jim (and the rest of Netgate)

On Thu, Jun 28, 2018 at 1:44 AM, Vamsi Krishna 
mailto:vamsi...@gmail.com>> wrote:
Hi Damjan, Dave,

Thanks for the quick reply.

It is really helpful. So the barrier ensures that the IPSec data structure 
access is thread safe.

Have a few more question on the IPSec implementation.
1. The inbound SA lookup (in ipsec-input) is actually going through the inbound 
policies for the given spd id linearly and matching a policy. The SA is picked 
based on the matching policy.
 This could have been an SAD hash table with key as (SPI, dst address, 
proto (ESP or AH) ), so that the SA can be looked up from the hash on receiving 
an ESP packet.
 Is there a particular reason it is implemented using a linear policy match?

2. There is also an IKEv2 responder implementation that adds/deletes IPSec 
tunnel interfaces. How does this work? Is there any documentation that can be 
referred to?

Thanks
Krishna

On Wed, Jun 27, 2018 at 6:23 PM, Dav

Re: [vpp-dev] VPP and DPDK PMD drivers

[Damjan Marion]

> On 2 Jul 2018, at 02:25, Kevin Wilson  wrote:
> 
>  
> Hello,
> My question is about VPP and DPDK PMD drivers: can any NIC which has support  
> in the DPDK official releases be used with VPP ?

Yes

-- 
Damjan


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9759): https://lists.fd.io/g/vpp-dev/message/9759
Mute This Topic: https://lists.fd.io/mt/22998130/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Is VPP IPSec implementation thread safe?

[Damjan Marion]

-- 
Damjan

> On 2 Jul 2018, at 11:14, Kingwel Xie  wrote:
> 
> Hi Vamsi, Damjan,
>  
> I’d like to contribute my two cents about IPSEC. We have been working on the 
> improvement for quite some time.
>  
> Great that vPP supports IPSEC, but the code is mainly for PoC. It lacks of 
> many features: buffer chain, AES-GCM/AES-CTR, UDP encap (seems already there 
> in master track?) many hardcode, broken packet trace,  SEQ handling, etc.
> Performance is not good, because of wrongly usage of openssl, buffer copying.

Buffer copying is needed, otherwise you have problem with cloned buffers. I.e. 
you still want original packet to be SPANed

> We can see 100% up after fixing all these issues.
> DPDK Ipsec has better performance but the quality of code is not good, many 
> bugs.
>  
> If you are looking for a production IPSEC, vpp is a good start but you still 
> have a lot things to do.

Contributions are welcome :)


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9760): https://lists.fd.io/g/vpp-dev/message/9760
Mute This Topic: https://lists.fd.io/mt/22720913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP ARM Build/Installation

[Brian Brooks]

Hi Jit,

On AArch64 machine:

$ git clone https://gerrit.fd.io/r/vpp
$ cd vpp
$ make build-release

Packages in nexus (not exactly sure which one):
https://nexus.fd.io/#view-repositories;fd.io.master.ubuntu-arm.xenial.main~browsestorage

Brian

From: vpp-dev@lists.fd.io  On Behalf Of Jit Mehta
Sent: Friday, June 29, 2018 11:28 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] VPP ARM Build/Installation

Hello,

Is there a way I can download and install aarch64 binaries?
If not, is there a wiki that lists how to build aarch64 targets?

Thanks,
Jit
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9761): https://lists.fd.io/g/vpp-dev/message/9761
Mute This Topic: https://lists.fd.io/mt/22895892/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP ARM Build/Installation

[Chris Luke]

Ubuntu aarch64 packages use the same recipe as amd64; 
https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages#Ubuntu.2FDebian

Chris

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Brian Brooks
Sent: Monday, July 02, 2018 12:56 PM
To: Jit Mehta ; vpp-dev@lists.fd.io
Subject: [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation

Hi Jit,

On AArch64 machine:

$ git clone https://gerrit.fd.io/r/vpp
$ cd vpp
$ make build-release

Packages in nexus (not exactly sure which one):
https://nexus.fd.io/#view-repositories;fd.io.master.ubuntu-arm.xenial.main~browsestorage

Brian

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Jit Mehta
Sent: Friday, June 29, 2018 11:28 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] VPP ARM Build/Installation

Hello,

Is there a way I can download and install aarch64 binaries?
If not, is there a wiki that lists how to build aarch64 targets?

Thanks,
Jit
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9762): https://lists.fd.io/g/vpp-dev/message/9762
Mute This Topic: https://lists.fd.io/mt/22895892/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP ARM Build/Installation

[Jit Mehta]

Thanks for your responses  Chris, Brian, Sirshak and Gabrielle.

I will try this out and post back if I run into any issues.

Thanks!
Jit

On Mon, Jul 2, 2018 at 9:59 AM, Luke, Chris  wrote:

> Ubuntu aarch64 packages use the same recipe as amd64;
> https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_
> packages#Ubuntu.2FDebian
>
>
>
> Chris
>
>
>
> *From:* vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] *On Behalf Of *Brian
> Brooks
> *Sent:* Monday, July 02, 2018 12:56 PM
> *To:* Jit Mehta ; vpp-dev@lists.fd.io
> *Subject:* [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation
>
>
>
> Hi Jit,
>
>
>
> On AArch64 machine:
>
>
>
> $ git clone https://gerrit.fd.io/r/vpp
>
> $ cd vpp
>
> $ make build-release
>
>
>
> Packages in nexus (not exactly sure which one):
>
> https://nexus.fd.io/#view-repositories;fd.io.master.
> ubuntu-arm.xenial.main~browsestorage
>
>
>
> Brian
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Jit Mehta
> *Sent:* Friday, June 29, 2018 11:28 AM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] VPP ARM Build/Installation
>
>
>
> Hello,
>
>
>
> Is there a way I can download and install aarch64 binaries?
>
> If not, is there a wiki that lists how to build aarch64 targets?
>
>
>
> Thanks,
>
> Jit
>
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sender immediately and do not disclose the
> contents to any other person, use it for any purpose, or store or copy the
> information in any medium. Thank you.
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9763): https://lists.fd.io/g/vpp-dev/message/9763
Mute This Topic: https://lists.fd.io/mt/22895892/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP ARM Build/Installation

[Jit Mehta]

All,

I see the below error when running 'apt-get update/install' during VPP
binaries installation based on these steps:
https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages#Ubuntu.2FDebian

Any pointers on how to fix this?

Thanks
Jit

root@ubuntu:/home/lab/Vpp/src# cat /etc/apt/sources.list.d/99fd.io.list

deb [trusted=yes]
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main/
./


root@ubuntu:/home/lab/Vpp/src# sudo apt-get update

Hit:1 http://ports.ubuntu.com/ubuntu-ports xenial InRelease

Hit:2 http://ports.ubuntu.com/ubuntu-ports xenial-updates InRelease

Ign:3
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ InRelease

Hit:4 http://ports.ubuntu.com/ubuntu-ports xenial-backports InRelease

Hit:5 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease

Ign:6
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Release

Ign:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Ign:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Ign:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Ign:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Ign:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Err:7
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Packages

  server certificate verification failed. CAfile:
/etc/ssl/certs/ca-certificates.crt CRLfile: none

Ign:8
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main
./ Translation-en

Reading package lists... Done

E: Failed to fetch
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main/./Packages
server certificate verification failed. CAfie

E: Some index files failed to download. They have been ignored, or old ones
used instead.


root@ubuntu:/home/lab/Vpp/src# sudo apt-get install vpp vpp-lib

Reading package lists... Done

Building dependency tree

Reading state information... Done

E: Unable to locate package vpp

E: Unable to locate package vpp-lib

root@ubuntu:/home/lab/Vpp/src#


On Mon, Jul 2, 2018 at 11:55 AM, Jit Mehta 
wrote:

> Thanks for your responses  Chris, Brian, Sirshak and Gabrielle.
>
> I will try this out and post back if I run into any issues.
>
> Thanks!
> Jit
>
> On Mon, Jul 2, 2018 at 9:59 AM, Luke, Chris 
> wrote:
>
>> Ubuntu aarch64 packages use the same recipe as amd64;
>> https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_pac
>> kages#Ubuntu.2FDebian
>>
>>
>>
>> Chris
>>
>>
>>
>> *From:* vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] *On Behalf Of *Brian
>> Brooks
>> *Sent:* Monday, July 02, 2018 12:56 PM
>> *To:* Jit Mehta ; vpp-dev@lists.fd.io
>> *Subject:* [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation
>>
>>
>>
>> Hi Jit,
>>
>>
>>
>> On AArch64 machine:
>>
>>
>>
>> $ git clone https://gerrit.fd.io/r/vpp
>>
>> $ cd vpp
>>
>> $ make build-release
>>
>>
>>
>> Packages in nexus (not exactly sure which one):
>>
>> https://nexus.fd.io/#view-repositories;fd.io.master.ubuntu-
>> arm.xenial.main~browsestorage
>>
>>
>>
>> Brian
>>
>>
>>
>> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Jit
>> Mehta
>> *Sent:* Friday, June 29, 2018 11:28 AM
>> *To:* vpp-dev@lists.fd.io
>> *Subject:* [vpp-dev] VPP ARM Build/Installation
>>
>>
>>
>> Hello,
>>
>>
>>
>> Is there a way I can download and install aarch64 binaries?
>>
>> If not, is there a wiki that lists how to build aarch64 targets?
>>
>>
>>
>> Thanks,
>>
>> Jit
>>
>> IMPORTANT NOTICE: The contents of this email and any attachments are
>> confidential and may also be privileged. If you are not the intended
>> recipient, please notify the sender immediately and do not disclose the
>> contents to any other person, use it for any purpose, or store or copy the
>> information in any medium. Thank you.
>>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#9763): https://lists.fd.io/g/vpp-dev/message/9763
> Mute This Topic: https://lists.fd.io/mt/22895892/900041
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [
> jitendra.harshad...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9764): https://lists.fd.io

Re: [vpp-dev] Is VPP IPSec implementation thread safe?

[Jim Thompson]

> On Jul 1, 2018, at 11:05 PM, Vamsi Krishna  wrote:
> 
> How is the performance of this code in terms of throughput, are there any 
> benchmarks that can be referred to? 

Four host setup (2 hosts for tunnel endpoints, 2 hosts outside tunnel as source 
& sink)
Source / sink Xeon E3-1275 v3 w/40G xl710
IPsec tunnel endpoints:  Intel i7-6950X (10C i7 @ 3.0GHz), xl710 40G NICs, 
Coleto Creek QAT card (https://store.netgate.com/ADI/QuickAssist8955.aspx 
)

Context:
“Kernel” is linux kernel
“User” is VPP

Testing performed in April 2017
 
Context Crypto processing   Crypto/AEAD algorithm   Integrity algorithm 
# of SAsTotal # streams Iperf3 TCP 1500 # of samples
Kernel  AES-NI  AES-CBC-128 SHA11   1   2.09 Gbps   16
Kernel  AES-NI  AES-CBC-128 SHA11   4   2.07 Gbps   16
Kernel  AES-NI  AES-CBC-128 SHA18   8   10.85 Gbps  6
Kernel  AES-NI  AES-GCM-128-16  1   1   5.06 Gbps   16
Kernel  AES-NI  AES-GCM-128-16  1   4   5.06 Gbps   16
Kernel  AES-NI  AES-GCM-128-16  8   8   25.25 Gbps  6
Kernel  QAT AES-CBC-128 SHA11   1   8.74 Gbps   16
Kernel  QAT AES-CBC-128 SHA11   4   8.74 Gbps   16
Kernel  QAT AES-CBC-128 SHA18   8   27.08 Gbps  6
UserVPP native (OpenSSL 1.0.1)  AES-CBC-128 SHA11   1   
2.03 Gbps   16
UserVPP native (OpenSSL 1.0.1)  AES-CBC-128 SHA11   4   
3.39 Gbps   16
UserVPP native (OpenSSL 1.0.1)  AES-CBC-128 SHA18   8   
9.45 Gbps   5
UserVPP AESNI MB cryptodev  AES-CBC-128 SHA11   1   7.42 
Gbps   6
UserVPP AESNI MB cryptodev  AES-CBC-128 SHA11   4   8.28 
Gbps   6
UserVPP AESNI GCM cryptodev AES-GCM-128-16  1   1   13.70 
Gbps  6
UserVPP AESNI GCM cryptodev AES-GCM-128-16  1   4   15.93 
Gbps  6
UserVPP QAT cryptodev   AES-CBC-128 SHA11   1   32.68 
Gbps  15
UserVPP QAT cryptodev   AES-CBC-128 SHA11   4   35.72 
Gbps  16
UserVPP QAT cryptodev   AES-CBC-128 SHA18   8   36.32 
Gbps  6
UserVPP QAT cryptodev   AES-GCM-128-16  1   1   32.73 
Gbps  6
UserVPP QAT cryptodev   AES-GCM-128-16  1   4   32.98 
Gbps  5








36.32Gbps is as close as you’re going to get to filling a 40Gbps NIC w/IPsec, 
due to framing overheads.
Tests using VPP w/ GCM not performed at 8 SAs / 8 streams due to issues I 
posted about last week.
We plan to repeat these with Skylake Xeon CPUs, a more modern VPP, 100Gbps NICs 
and a Lewisburg QAT device.


L3 forwarding (no IPsec, minimal routes, no ACLs) using the same setup:
Kernel, 1 stream, 64B: 804 kpps, 512B: 808 kpps, 1500B: 806 kpps
Kernel, 4 stream, 64B: 2.93 Mpps, 512B: 2.92 Mpps, 1500B: 2.91 Mpps
Kernel, 8 stream, 64B: 5.16 Mpps, 512B: 5.14 Mpps, 1500B: 3.28 Mpps

VPP, 1 stream, 64B: 14.05 Mpps, 512B: 8.84 Mpps, 1500B: 3.28 kpps
VPP, 4 stream, 64B: 32.23 Mpps, 512B: 9.39 Mpps, 1500B: 3.28 Mpps
VPP, 8 stream, 64B: 42.60 Mpps, 512B: 9.39 Mpps, 1500B: 3.28 Mpps



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9765): https://lists.fd.io/g/vpp-dev/message/9765
Mute This Topic: https://lists.fd.io/mt/22720913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Is VPP IPSec implementation thread safe?

[Kingwel Xie]

Hi Damjan,

Thanks for the heads-up. Never come to that. I’m still thinking it is 
acceptable if we are doing IPSec. Buffer copying is a significant overhead.

We are working on the code, will contribute when we think it is ready. There 
are so many corner cases of IPSec, hard to say we can cover all of them.

Regards,
Kingwel

From: vpp-dev@lists.fd.io  On Behalf Of Damjan Marion
Sent: Monday, July 02, 2018 7:43 PM
To: Kingwel Xie 
Cc: Vamsi Krishna ; Jim Thompson ; Dave 
Barach ; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Is VPP IPSec implementation thread safe?


--
Damjan


On 2 Jul 2018, at 11:14, Kingwel Xie 
mailto:kingwel@ericsson.com>> wrote:

Hi Vamsi, Damjan,

I’d like to contribute my two cents about IPSEC. We have been working on the 
improvement for quite some time.


  1.  Great that vPP supports IPSEC, but the code is mainly for PoC. It lacks 
of many features: buffer chain, AES-GCM/AES-CTR, UDP encap (seems already there 
in master track?) many hardcode, broken packet trace,  SEQ handling, etc.
  2.  Performance is not good, because of wrongly usage of openssl, buffer 
copying.

Buffer copying is needed, otherwise you have problem with cloned buffers. I.e. 
you still want original packet to be SPANed



  1.  We can see 100% up after fixing all these issues.
  2.  DPDK Ipsec has better performance but the quality of code is not good, 
many bugs.

If you are looking for a production IPSEC, vpp is a good start but you still 
have a lot things to do.

Contributions are welcome :)


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9766): https://lists.fd.io/g/vpp-dev/message/9766
Mute This Topic: https://lists.fd.io/mt/22720913/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation

[Chris Luke]

My best guess, not knowing your environment, is that you don’t have 
ca-certificates installed.

From: Jit Mehta 
Sent: Monday, July 2, 2018 16:50
To: Luke, Chris 
Cc: Brian Brooks ; vpp-dev@lists.fd.io
Subject: [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation

All,

I see the below error when running 'apt-get update/install' during VPP binaries 
installation based on these steps: 
https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages#Ubuntu.2FDebian

Any pointers on how to fix this?

Thanks
Jit


root@ubuntu:/home/lab/Vpp/src# cat /etc/apt/sources.list.d/99fd.io.list

deb [trusted=yes] 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main/ 
./



root@ubuntu:/home/lab/Vpp/src# sudo apt-get update

Hit:1 http://ports.ubuntu.com/ubuntu-ports xenial InRelease

Hit:2 http://ports.ubuntu.com/ubuntu-ports xenial-updates InRelease

Ign:3 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ InRelease

Hit:4 http://ports.ubuntu.com/ubuntu-ports xenial-backports InRelease

Hit:5 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease

Ign:6 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Release

Ign:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Ign:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Ign:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Ign:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Ign:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Err:7 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Packages

  server certificate verification failed. CAfile: 
/etc/ssl/certs/ca-certificates.crt CRLfile: none

Ign:8 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main 
./ Translation-en

Reading package lists... Done

E: Failed to fetch 
https://nexus.fd.io/content/repositories/fd.io.stable.1804.ubuntu.xenial.main/./Packages
  server certificate verification failed. CAfie

E: Some index files failed to download. They have been ignored, or old ones 
used instead.



root@ubuntu:/home/lab/Vpp/src# sudo apt-get install vpp vpp-lib

Reading package lists... Done

Building dependency tree

Reading state information... Done

E: Unable to locate package vpp

E: Unable to locate package vpp-lib

root@ubuntu:/home/lab/Vpp/src#


On Mon, Jul 2, 2018 at 11:55 AM, Jit Mehta 
mailto:jitendra.harshad...@gmail.com>> wrote:
Thanks for your responses  Chris, Brian, Sirshak and Gabrielle.

I will try this out and post back if I run into any issues.

Thanks!
Jit

On Mon, Jul 2, 2018 at 9:59 AM, Luke, Chris 
mailto:chris_l...@comcast.com>> wrote:
Ubuntu aarch64 packages use the same recipe as amd64; 
https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_packages#Ubuntu.2FDebian

Chris

From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Brian 
Brooks
Sent: Monday, July 02, 2018 12:56 PM
To: Jit Mehta 
mailto:jitendra.harshad...@gmail.com>>; 
vpp-dev@lists.fd.io
Subject: [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation

Hi Jit,

On AArch64 machine:

$ git clone https://gerrit.fd.io/r/vpp
$ cd vpp
$ make build-release

Packages in nexus (not exactly sure which one):
https://nexus.fd.io/#view-repositories;fd.io.master.ubuntu-arm.xenial.main~browsestorage

Brian

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Jit Mehta
Sent: Friday, June 29, 2018 11:28 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] VPP ARM Build/Installation

Hello,

Is there a way I can download and install aarch64 binaries?
If not, is there a wiki that lists how to build aarch64 targets?

Thanks,
Jit
IMPORTANT NOTICE: The contents of this email and any attachments are 
confidential and may also be privileged. If you are not the intended recipient, 
please notify the sender immediately and do not disclose the contents to any 
other person, use it for any purpose, or store or copy the information in any 
medium. Thank you.


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to

Re: [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation

[Jit Mehta]

You are right. :-)
Able to pull the binaries now. Thanks,!

Jit

On Mon, Jul 2, 2018 at 5:45 PM, Luke, Chris  wrote:

> My best guess, not knowing your environment, is that you don’t have
> ca-certificates installed.
>
>
>
> *From:* Jit Mehta 
> *Sent:* Monday, July 2, 2018 16:50
> *To:* Luke, Chris 
> *Cc:* Brian Brooks ; vpp-dev@lists.fd.io
>
> *Subject:* [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation
>
>
>
> All,
>
>
>
> I see the below error when running 'apt-get update/install' during VPP
> binaries installation based on these steps: https://wiki.fd.io/view/VPP/
> Installing_VPP_binaries_from_packages#Ubuntu.2FDebian
>
>
>
> Any pointers on how to fix this?
>
>
>
> Thanks
>
> Jit
>
>
>
> root@ubuntu:/home/lab/Vpp/src# cat /etc/apt/sources.list.d/99fd.io.list
>
> deb [trusted=yes] https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main/ ./
>
>
>
> root@ubuntu:/home/lab/Vpp/src# sudo apt-get update
>
> Hit:1 http://ports.ubuntu.com/ubuntu-ports xenial InRelease
>
> Hit:2 http://ports.ubuntu.com/ubuntu-ports xenial-updates InRelease
>
> Ign:3 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ InRelease
>
> Hit:4 http://ports.ubuntu.com/ubuntu-ports xenial-backports InRelease
>
> Hit:5 http://ports.ubuntu.com/ubuntu-ports xenial-security InRelease
>
> Ign:6 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Release
>
> Ign:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Ign:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Ign:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Ign:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Ign:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Err:7 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Packages
>
>   server certificate verification failed. CAfile: 
> /etc/ssl/certs/ca-certificates.crt
> CRLfile: none
>
> Ign:8 https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main ./ Translation-en
>
> Reading package lists... Done
>
> E: Failed to fetch https://nexus.fd.io/content/repositories/fd.io.stable.
> 1804.ubuntu.xenial.main/./Packages
> 
> server certificate verification failed. CAfie
>
> E: Some index files failed to download. They have been ignored, or old
> ones used instead.
>
>
>
> root@ubuntu:/home/lab/Vpp/src# sudo apt-get install vpp vpp-lib
>
> Reading package lists... Done
>
> Building dependency tree
>
> Reading state information... Done
>
> E: Unable to locate package vpp
>
> E: Unable to locate package vpp-lib
>
> root@ubuntu:/home/lab/Vpp/src#
>
>
>
>
>
> On Mon, Jul 2, 2018 at 11:55 AM, Jit Mehta 
> wrote:
>
> Thanks for your responses  Chris, Brian, Sirshak and Gabrielle.
>
>
>
> I will try this out and post back if I run into any issues.
>
>
>
> Thanks!
>
> Jit
>
>
>
> On Mon, Jul 2, 2018 at 9:59 AM, Luke, Chris 
> wrote:
>
> Ubuntu aarch64 packages use the same recipe as amd64;
> https://wiki.fd.io/view/VPP/Installing_VPP_binaries_from_
> packages#Ubuntu.2FDebian
>
>
>
> Chris
>
>
>
> *From:* vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] *On Behalf Of *Brian
> Brooks
> *Sent:* Monday, July 02, 2018 12:56 PM
> *To:* Jit Mehta ; vpp-dev@lists.fd.io
> *Subject:* [EXTERNAL] Re: [vpp-dev] VPP ARM Build/Installation
>
>
>
> Hi Jit,
>
>
>
> On AArch64 machine:
>
>
>
> $ git clone https://gerrit.fd.io/r/vpp
>
> $ cd vpp
>
> $ make build-release
>
>
>
> Packages in nexus (not exactly sure which one):
>
> https://nexus.fd.io/#view-repositories;fd.io.master.
> ubuntu-arm.xenial.main~browsestorage
>
>
>
> Brian
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Jit Mehta
> *Sent:* Friday, June 29, 2018 11:28 AM
> *To:* vpp-dev@lists.fd.io
> *Subject:* [vpp-dev] VPP ARM Build/Installation
>
>
>
> Hello,
>
>
>
> Is there a way I can download and install aarch64 binaries?
>
> If not, is there a wiki that lists how to build aarch64 targets?
>
>
>
> Thanks,
>
> Jit
>
> IMPORTANT NOTICE: The contents of this email and any attachments are
> confidential and may also be privileged. If you are not the intended
> recipient, please notify the sen

Re: [vpp-dev] VPP C++ API

[chetan bhasin]

Hi,

I also faced this type of issue,  there is a keep alive message,  which VPP
keep on sending to its client,  and if you don't have it's registration,
client app crashes.

Can you print the message id and you can refer various VPP CLI

show api messag
show api client
show api ring

etc

Regards,
Chetan Bhasin

On Mon, Jul 2, 2018, 13:10 Klement Sekera via Lists.Fd.Io  wrote:

> Hi,
>
> can you please run in gdb (or open core) and pass us a stack trace?
>
> Thanks,
> Klement
>
> On Sun, 2018-07-01 at 19:36 -0700, xpa...@gmail.com wrote:
> > Hello,
> > I'm trying to understand how works VOM wrapper for C++ API.
> > VOM manager header: https://pastebin.com/BgG7td5s
> > VOM manager cpp: https://pastebin.com/890jjUJm
> > main.cpp:
> >
> > int main() {
> > TVppManager manager;
> > std::cout << "Starting\n";
> > manager.Start();
> >
> > sleep(60);
> >
> > manager.Stop();
> > }
> >
> > When I'm trying to run sudo ./main a I got error:
> > Starting
> > Calling connect
> > Manager is connected
> > Mon Jul  2 05:33:55 2018 [debug]hw.cpp:129 write() itf-events
> > Mon Jul  2 05:33:55 2018 [debug]rpc_cmd.hpp:120 operator()() itf-
> > events 0
> > Started
> > main: /place/home/xpahos/git/vpp/build-data/../src/vpp-
> > api/vapi/vapi.c:696: vapi_msg_is_with_context: Assertion `id <=
> > __vapi_metadata.count' failed.
> > Aborted
> >
> > I found that id looks like unsigned value overflow. How can I debug
> > this? I don't see anything in trace dump.
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> >
> > View/Reply Online (#9753): https://lists.fd.io/g/vpp-dev/message/9753
> > Mute This Topic: https://lists.fd.io/mt/22999158/675704
> > Group Owner: vpp-dev+ow...@lists.fd.io
> > Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [ksek...@cisco.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#9755): https://lists.fd.io/g/vpp-dev/message/9755
> Mute This Topic: https://lists.fd.io/mt/22999158/856484
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [
> chetan.bhasin...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#9769): https://lists.fd.io/g/vpp-dev/message/9769
Mute This Topic: https://lists.fd.io/mt/22999158/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-