Re: [vpp-dev] [VCL] hoststack app crash with invalid memfd segment address

2019-11-21 Thread wanghanlin 






Hi Florin,
I have applied the patch, and found some problems in my case.  I have not right to post it in gerrit, so I post here.
1)evt->event_type should be set  with SESSION_CTRL_EVT_APP_DEL_SEGMENT rather than SESSION_CTRL_EVT_APP_ADD_SEGMENT. File: src/vnet/session/session_api.c, Line: 561, Function:mq_send_del_segment_cb2)session_send_fds may been called in the end of function mq_send_add_segment_cb, otherwise lock of app_mq can't been free here.File: src/vnet/session/session_api.c, Line: 519, Function:mq_send_add_segment_cb 

3) When vcl_segment_attach called in each worker thread, then ssvm_slave_init_memfd can been called in each worker thread and then ssvm_slave_init_memfd map address sequentially through map segment once in advance.  It's OK in only one thread, but maybe wrong in multiple worker threads. Suppose following scene: VPP allocate segment at address A1 and notify worker thread B1 to expect B1 also map segment at address A1,  and simultaneously VPP allocate segment at address A2 and notify worker thread B2 to expect B2 map segment at address A2. If B2 first process notify message, then ssvm_slave_init_memfd may map segment at address A1. Maybe VPP can add segment map address in notify message, and then worker thread just map segment at this address. Regards,Hanlin

 










wanghanlin







wanghan...@corp.netease.com








签名由
网易邮箱大师
定制

 

On 11/19/2019 09:50，wanghanlin wrote： 






Hi  Florin,VPP vsersion is v19.08.I'll apply this patch and check it. Thanks a lot!




Regards,Hanlin

 










wanghanlin







wanghan...@corp.netease.com








签名由
网易邮箱大师
定制

 

On 11/16/2019 00:50，Florin Coras wrote： 


Hi Hanlin,Just to make sure, are you running master or some older VPP?Regarding the issue you could be hitting lower, here’s [1] a patch that I have not yet pushed for merging because it leads to api changes for applications that directly use the session layer application interface instead of vcl. I haven’t tested it extensively, but the goal with it is to signal segment allocation/deallocation over the mq instead of the binary api.Finally, I’ve never tested LDP with Envoy, so not sure if that works properly. There’s ongoing work to integrate Envoy with VCL, so you may want to get in touch with the authors. Regards,Florin[1] https://gerrit.fd.io/r/c/vpp/+/21497On Nov 15, 2019, at 2:26 AM, wanghanlin  wrote:hi ALL,I accidentally got following crash stack when I used VCL with hoststack and memfd. But corresponding invalid rx_fifo address (0x2f42e2480) is valid in VPP process and also can be found in /proc/map. That is, shared memfd segment memory is not consistent between hoststack app and VPP.Generally, VPP allocate/dealloc the memfd segment and then notify hoststack app to attach/detach. But If just after VPP dealloc memfd segment and notify hoststack app, and then VPP allocate same memfd segment at once because of session connected, and then what happened now? Because hoststack app process dealloc message and connected message with diffrent threads, maybe rx_thread_fn just detach the memfd segment and not attach the same memfd segment, then unfortunately worker thread get the connected message. These are just my guess, maybe I misunderstand.(gdb) bt#0  0x7f7cde21ffbf in raise () from /lib/x86_64-linux-gnu/libpthread.so.0#1  0x01190a64 in Envoy::SignalAction::sigHandler (sig=11, info=, context=) at source/common/signal/signal_action.cc:73#2  #3  0x7f7cddc2e85e in vcl_session_connected_handler (wrk=0x7f7ccd4bad00, mp=0x224052f4a) at /home/wanghanlin/vpp-new/src/vcl/vppcom.c:471#4  0x7f7cddc37fec in vcl_epoll_wait_handle_mq_event (wrk=0x7f7ccd4bad00, e=0x224052f48, even

Re: [vpp-dev] Requirements for DPDK pmd/feature testing in CSIT vpp-device-test jobs

2019-11-21 Thread Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io 
> a +2 triggers a pre-commit gate where VOM and the dist builds and the 
> extended tests are run.
> If everything passes, the change is merged, if not, the +2 is removed.

+1 to that, but I highly doubt Gerrit supports such a workflow.

Additional features we would like to have:
+ Manual trigger to run the extended tests without also merging,
  (for debugging purposes after previous extended tests failed).
+ The gate job rebases on HEAD and only one run is allowed at a time
  (to avoid merge errors).
+ The gate job squashes multiple pending changes before testing
  (to keep up with the high frequency of +2 without introducing long queues).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Tuesday, November 19, 2019 12:10 PM
To: Paul Vinciguerra 
Cc: Dave Wallace ; vpp-dev 
Subject: Re: [vpp-dev] Requirements for DPDK pmd/feature testing in CSIT 
vpp-device-test jobs

Can we expand this discussion to discuss the VPP CI workflow?  I would like to 
see a decoupling of development and integration.
As I mentioned the other day,  It would be great if we could rebuild the 
containers whenever a commit updated the Makefile or the requirements.txt files.

I'd also like to throw out the idea of breaking up the verify job.  I think 
that if we were to remove VOM and the dist builds from verify and change the 
workflow so that a +2 triggers a pre-commit gate where VOM and the dist builds 
and the extended tests are run.  If everything passes, the change is merged, if 
not, the +2 is removed.  The existing csit job could be non-voting (so the csit 
folks could have a heads up) in the first phase, and voting in pre-commit-phase.

Paul

On Mon, Nov 18, 2019 at 12:18 PM Paul Vinciguerra via 
Lists.Fd.Io 
mailto:vinciconsulting@lists.fd.io>>
 wrote:
Hi Dave.

Where does the vpp-device-test live?

On Mon, Nov 18, 2019 at 11:13 AM Dave Wallace 
mailto:dwallac...@gmail.com>> wrote:
Folks,

Per the topic in last week's monthly VPP community meeting, the topic of DPDK 
pmd/feature testing in the CSIT devicetest job was discussed in the most recent 
CSIT community meeting (Wed 11/13).

In the beginning of the VPP project, DPDK pmd/feature testing was performed in 
the VIRL based CSIT test suites. DPDK was moved from the VPP core feature set 
into a plugin in VPP 17.04 and in later releases native device drivers were 
implemented.  Subsequently, DPDK testing was removed from the CSIT VIRL tests.  
Also the CSIT team put a plan put in place for all of the VIRL tests to be 
moved and the VIRL servers re-purposed.  In addition, the CSIT vpp-device-test 
job was created to provide test coverage of device level VPP features that 
cannot be tested in VPP's 'make test' framework. The plan for re-purposing the 
VIRL servers is complete and the vpp-device-test job is slated to become voting 
once it is stable enough for continuous-integration testing.

The CSIT team would like input from the VPP community on exactly what DPDK 
PMD's and/or features are required to be added to the CSIT vpp-device-test jobs.

Thanks,
-daw-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14617): https://lists.fd.io/g/vpp-dev/message/14617
Mute This Topic: https://lists.fd.io/mt/60208819/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14618): https://lists.fd.io/g/vpp-dev/message/14618
Mute This Topic: https://lists.fd.io/mt/60208819/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14655): https://lists.fd.io/g/vpp-dev/message/14655
Mute This Topic: https://lists.fd.io/mt/60208819/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Coverity run FAILED as of 2019-11-21 14:01:27 UTC

2019-11-21 Thread Noreply Jenkins 
Coverity run failed today.

Current number of outstanding issues are 2
Newly detected: 0
Eliminated: 0
More details can be found at  
https://scan.coverity.com/projects/fd-io-vpp/view_defects
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14656): https://lists.fd.io/g/vpp-dev/message/14656
Mute This Topic: https://lists.fd.io/mt/61078507/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [VCL] hoststack app crash with invalid memfd segment address

2019-11-21 Thread Jon Loeliger via Lists.Fd.Io 
On Thu, Nov 21, 2019 at 4:50 AM wanghanlin 
wrote:

> Hi Florin,
> I have applied the patch, and found some problems in my case.  I have not
> right to post it in gerrit, so I post here.
>

Why don't you register with Gerrit, like this:

https://wiki.fd.io/view/DEV/Setting_up_Gerrit

HTH,
jdl
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14657): https://lists.fd.io/g/vpp-dev/message/14657
Mute This Topic: https://lists.fd.io/mt/59126583/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Requirements for DPDK pmd/feature testing in CSIT vpp-device-test jobs

2019-11-21 Thread Jon Loeliger via Lists.Fd.Io 
On Thu, Nov 21, 2019 at 6:55 AM Vratko Polak -X (vrpolak - PANTHEON
TECHNOLOGIES at Cisco) via Lists.Fd.Io 
wrote:

> > a +2 triggers a pre-commit gate where VOM and the dist builds and the
> extended tests are run.
>
> > If everything passes, the change is merged, if not, the +2 is removed.
>
>
>
> +1 to that, but I highly doubt Gerrit supports such a workflow.
>
>
>
> Additional features we would like to have:
>
> ...
>
> + The gate job squashes multiple pending changes before testing
>
>   (to keep up with the high frequency of +2 without introducing long
> queues).
>

Please do not squash patches.


> Vratko.
>

Thanks,
jdl
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14658): https://lists.fd.io/g/vpp-dev/message/14658
Mute This Topic: https://lists.fd.io/mt/60208819/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Kubernetes and contiv vpp issue

2019-11-21 Thread Nathan Skrzypczak 
Hi Feroz,

If your plugin gets loaded in the vpp running directly on your machine but
not in contiv-vpp, it's likely that there is a version mismatch between the
two or something weird happening during contiv build process.
* You can check the loaded plugins directly in the debug cli (vppctl : show
plugins) to ensure check the plugin state at runtime.
* If both don't run the same version running  will
give two different commits.
* The problem might also come from vpp's config file
(/host/etc/vpp/contiv-vswitch.conf in contiv's case) but it's less probable
if it doesn't appear in the logs.

If this doesn't help finding the discrepancy, you can build a contiv-vpp
image from your working binaries with :
cd CONTIV_DIR && make contiv-agent contiv-init
cd VPP_DIR && make build-release ; make pkg-deb
cp $VPP_DIR/build-root/*.deb /scratch/
cd /scratch && wget
https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/dev/govpp.conf
cd /scratch && wget
https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/prod/vswitch/vpp.conf
cd /scratch && wget
https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/prod/vswitch/vppctl
You can then build a docker image with
https://gist.github.com/sknat/cc016e4905cb875f62c74e28e2cc5661

Hope this helps

Cheers
-Nathan



Le jeu. 21 nov. 2019 à 06:41, Mohamed feroz Abdul majeeth <
feroz...@gmail.com> a écrit :

> Hi Team,
>
> We add sample( sample.so) plugins with existing vpp plugins. I used 3.3.2
> tag contiv vpp and  copied sample plugins related code *patches *.diff
> under contiv vpp patches folder( vpp/ docker/ vpp- vswitch /vpp/ patches).
> Patches also applied correctly and able to create the contiv/vswitch docker
> image. Even I seen its generated sample.so in vpp_ plugins folder in
> *contiv-vpp/*vpp container which is having builded vpp code.
> I started kubernetes and apply contiv vpp yaml with
> contiv-vpp/vswitch:3.3.2 image. It's running and open the vppctl but not
> able to see sample plugins related cli commands so I seen logs(kubectl logs
> contiv-vswitch-lqxfp -n kube-system -c contiv-vswitch). In logs, there is
> no sample.so plugin loading information details.
>
> For verification, I followed same vpp build steps in bare metal which is
> follow in contiv vpp code, sample.so plugin is loaded when I launch the
> vpp and I'm able to see sample.so plugins cli commands details.
>
> In docker/container only, *sample**.so*  plugin is not
> loaded when launch the vpp.
>
> Please help me and guide me. How to solve this problem.
>
> Thanks,
> Feroz
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#14653): https://lists.fd.io/g/vpp-dev/message/14653
> Mute This Topic: https://lists.fd.io/mt/61032023/1706314
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [
> nathan.skrzypc...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14659): https://lists.fd.io/g/vpp-dev/message/14659
Mute This Topic: https://lists.fd.io/mt/61032023/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Change to Gerrit

2019-11-21 Thread Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io 
> implementing something like [0]

+1, but...

> skip the jobs and set verify label after the codestyle checks if no files 
> were changed

... how do you imagine this being implemented in ci-management?

Currently, the other jobs are triggered by checkstyle_success
Gerrit comment, but how would the new checkstyle check know
when to skip it?

I can imagine having two checkstyle jobs
(one skippable and commenting, one unskippable and not commenting),
but +1 from the latter job would not remove the previous -1 from the former.

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Wednesday, November 20, 2019 7:33 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Change to Gerrit

How would the group feel about implementing something like [0], so that changes 
to the commit message don't trigger rebuilds?

To enforce the commit message structure, we could skip the jobs and set verify 
label after the codestyle checks if no files were changed.
Maybe others don't care, but I don't like wasting cpu cycles/developer's time, 
and I weigh that before clarifying a commit message.

[0] 
https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_copyAllScoresIfNoCodeChange
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14660): https://lists.fd.io/g/vpp-dev/message/14660
Mute This Topic: https://lists.fd.io/mt/60969892/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] ldp write assert error

2019-11-21 Thread jiangxiaoming 
I used ab with ldp,  for nginx bench test.
Below is my start command:

> 
> sudo env \
> VCL_CONFIG=/tmp/vcl-test-3af8e.conf \
> VCL_DEBUG=1 \
> LDP_DEBUG=1 \
> LDP_SID_BIT=9 \
> gdb -x /tmp/gdb-3af8e
> 

I got the vppcom_session_write_inline assert failds. Obviously apache ab's 
write function send the wrong params.
But we in vppcom_session_write_inline only check the param: buf, not check the 
param: n.
If I add the param n check, apache ab tools will work well.
I think it's necessary to add the param:n check for vppcom_session_write_inline 
funcion.
Here is my patch: https://gerrit.fd.io/r/c/vpp/+/23584

> 
> vppcom_session_create:1142: vcl<15458:0>: created session 1
> fcntl:503: ldp<15458>: fd 513 vlsh 1, cmd 3
> fcntl:503: ldp<15458>: fd 513 vlsh 1, cmd 4
> connect:1255: ldp<15458>: fd 513: calling vls_connect(): vlsh 1 addr
> 0x55768f18 len 16
> vppcom_session_connect:1608: vcl<15458:0>: session handle 1: connecting to
> server IPv4 192.168.7.130 port 80 proto TCP
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 1
> connect:1255: ldp<15458>: fd 513: calling vls_connect(): vlsh 1 addr
> 0x55768f18 len 16
> vppcom_session_connect:1592: vcl<15458:0>: session handle 1 [0x10001]:
> session already connected to IPv4 192.168.7.130 port 80 proto TCP, state
> 0x1 (STATE_CONNECT)
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 2
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 1
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1968
> (vppcom_session_write_inline) assertion `n_write > 0' fails
> 
> Program received signal SIGABRT, Aborted.
> 0x75b98337 in __GI_raise (sig=sig@entry=6) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:55
> 55 return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
> Missing separate debuginfos, use: debuginfo-install
> keyutils-libs-1.5.8-3.el7.x86_64 libuuid-2.23.2-61.el7.x86_64
> pcre-8.32-17.el7.x86_64
> (gdb) bt
> #0  0x75b98337 in __GI_raise (sig=sig@entry=6) at
> ../nptl/sysdeps/unix/sysv/linux/raise.c:55
> #1  0x75b99a28 in __GI_abort () at abort.c:90
> #2  0x7506e8f3 in os_panic () at
> /home/dev/net-base/build/vpp/src/vppinfra/unix-misc.c:176
> #3  0x74fd6c93 in debugger () at
> /home/dev/net-base/build/vpp/src/vppinfra/error.c:84
> #4  0x74fd7062 in _clib_error (how_to_die=2, function_name=0x0,
> line_number=0, fmt=0x757483f0 "%s:%d (%s) assertion `%s' fails") at
> /home/dev/net-base/build/vpp/src/vppinfra/error.c:143
> #5  0x7571f70e in vppcom_session_write_inline (session_handle=1,
> buf=0x55763240 <_request>, n=0, is_flush=1 '\001') at
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1968
> #6  0x7571f81a in vppcom_session_write_msg (session_handle=1,
> buf=0x55763240 <_request>, n=0) at
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1986
> #7  0x757467f5 in vls_write_msg (vlsh=1, buf=0x55763240
> <_request>, nbytes=0) at
> /home/dev/net-base/build/vpp/src/vcl/vcl_locked.c:505
> #8  0x77bd0f3f in write (fd=513, buf=0x55763240 <_request>,
> nbytes=0) at /home/dev/net-base/build/vpp/src/vcl/ldp.c:424
> #9  0x7666e67b in apr_socket_send (sock=0x55787fc0,
> buf=0x55763240 <_request> "GET / HTTP/1.0\r\nConnection:
> Keep-Alive\r\nHost: 192.168.7.130\r\nUser-Agent:
> ApacheBench/2.3\r\nAccept: */*\r\n\r\n", len=len@entry=0x7fffdc20) at
> network_io/unix/sendrecv.c:41
> #10 0xb4f4 in write_request (c=c@entry=0x557870e0) at
> ab.c:707
> #11 0x7f9e in write_request (c=0x557870e0) at ab.c:1793
> #12 test () at ab.c:1871
> 
> 

> 
> 
>


gdb.log
Description: Binary data


gdb-3af8e
Description: Binary data


vcl-test-3af8e.conf
Description: Binary data
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14661): https://lists.fd.io/g/vpp-dev/message/14661
Mute This Topic: https://lists.fd.io/mt/61080258/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [VCL] hoststack app crash with invalid memfd segment address

2019-11-21 Thread Florin Coras 
Hi Hanlin, 

As Jon pointed out, you may want to register with gerrit. 

You comments with respect to points 1) and 2) are spot on. I’ve updated the 
patch to fix them. 

Regarding 3), if I understood your scenario correctly, it should not happen. 
The ssvm infra forces applications to map segments at fixed addresses. That is, 
for the scenario you’re describing lower, if B2 is processed first, 
ssvm_slave_init_memfd will map the segment at A2. Note how we first map the 
segment to read the shared header (sh) and then use sh->ssvm_va (which should 
be A2) to remap the segment at a fixed virtual address (va). 

Regards,
Florin

> On Nov 21, 2019, at 2:49 AM, wanghanlin  wrote:
> 
> Hi Florin,
> I have applied the patch, and found some problems in my case.  I have not 
> right to post it in gerrit, so I post here.
> 1)evt->event_type should be set  with SESSION_CTRL_EVT_APP_DEL_SEGMENT rather 
> than SESSION_CTRL_EVT_APP_ADD_SEGMENT. File: src/vnet/session/session_api.c, 
> Line: 561, Function:mq_send_del_segment_cb
> 2)session_send_fds may been called in the end of function 
> mq_send_add_segment_cb, otherwise lock of app_mq can't been free here.File: 
> src/vnet/session/session_api.c, Line: 519, Function:mq_send_add_segment_cb 
> 3) When vcl_segment_attach called in each worker thread, then 
> ssvm_slave_init_memfd can been called in each worker thread and then 
> ssvm_slave_init_memfd map address sequentially through map segment once in 
> advance.  It's OK in only one thread, but maybe wrong in multiple worker 
> threads. Suppose following scene: VPP allocate segment at address A1 and 
> notify worker thread B1 to expect B1 also map segment at address A1,  and 
> simultaneously VPP allocate segment at address A2 and notify worker thread B2 
> to expect B2 map segment at address A2. If B2 first process notify message, 
> then ssvm_slave_init_memfd may map segment at address A1. Maybe VPP can add 
> segment map address in notify message, and then worker thread just map 
> segment at this address. 
> 
> Regards,
> Hanlin
>   
> wanghanlin
> 
> wanghan...@corp.netease.com
>  
> 
> 签名由 网易邮箱大师  定制
> On 11/19/2019 09:50，wanghanlin 
>  wrote： 
> Hi  Florin,
> VPP vsersion is v19.08.
> I'll apply this patch and check it. Thanks a lot!
> 
> Regards,
> Hanlin
>   
> wanghanlin
> 
> wanghan...@corp.netease.com
>  
> 
> 签名由 网易邮箱大师  定制
> On 11/16/2019 00:50，Florin Coras 
>  wrote： 
> Hi Hanlin,
> 
> Just to make sure, are you running master or some older VPP?
> 
> Regarding the issue you could be hitting lower, here’s [1] a patch that I 
> have not yet pushed for merging because it leads to api changes for 
> applications that directly use the session layer application interface 
> instead of vcl. I haven’t tested it extensively, but the goal with it is to 
> signal segment allocation/deallocation over the mq instead of the binary api.
> 
> Finally, I’ve never tested LDP with Envoy, so not sure if that works 
> properly. There’s ongoing work to integrate Envoy with VCL, so you may want 
> to get in touch with the authors. 
> 
> Regards,
> Florin
> 
> [1] https://gerrit.fd.io/r/c/vpp/+/21497 
> 
> 
>> On Nov 15, 2019, at 2:26 AM, wanghanlin > > wrote:
>> 
>> hi ALL,
>> I accidentally got following crash stack when I used VCL with hoststack and 
>> memfd. But corresponding invalid rx_fifo address (0x2f42e2480) is valid in 
>> VPP process and also can be found in /proc/map. That is, shared memfd 
>> segment memory is not consistent between hoststack app and VPP.
>> Generally, VPP allocate/dealloc the memfd segment and then notify hoststack 
>> app to attach/detach. But If just after VPP dealloc memfd segment and notify 
>> hoststack app, and then VPP allocate same memfd segment at once because of 
>> session connected, and then what happened now? Because hoststack app process 
>> dealloc message and connected message with diffrent threads, maybe 
>> rx_thread_fn just detach the memfd segment and not attach the same memfd 
>> segment, then unfortunately worker thread get the connected mess

Re: [vpp-dev] ldp write assert error

2019-11-21 Thread Florin Coras 
Patch looks good! I’ll merge once the ci-infra issues are solved.

Since you’re doing perf testing, I’ll note again, although I’m sure you already 
know it, that ldp performance is somewhat lower than that of vcl under certain 
types of load, because of vls locking.

Thanks, 
Florin

> On Nov 21, 2019, at 7:39 AM, jiangxiaom...@outlook.com wrote:
> 
> I used ab with ldp,  for nginx bench test.
> Below is my start command:
> sudo env \
>  VCL_CONFIG=/tmp/vcl-test-3af8e.conf \
>  VCL_DEBUG=1 \
>  LDP_DEBUG=1 \
>  LDP_SID_BIT=9 \
>  gdb -x /tmp/gdb-3af8e
> I got the vppcom_session_write_inline assert failds. Obviously apache ab's 
> write function send the wrong params. 
> But we in vppcom_session_write_inline only check the param: buf, not check 
> the param: n.
> If I add the param n check, apache ab tools will work well.
> I think it's necessary to add the param:n check for 
> vppcom_session_write_inline  funcion.
> Here is my patch:  https://gerrit.fd.io/r/c/vpp/+/23584 
> 
> vppcom_session_create:1142: vcl<15458:0>: created session 1
> fcntl:503: ldp<15458>: fd 513 vlsh 1, cmd 3
> fcntl:503: ldp<15458>: fd 513 vlsh 1, cmd 4
> connect:1255: ldp<15458>: fd 513: calling vls_connect(): vlsh 1 addr 
> 0x55768f18 len 16
> vppcom_session_connect:1608: vcl<15458:0>: session handle 1: connecting to 
> server IPv4 192.168.7.130 port 80 proto TCP
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 1
> connect:1255: ldp<15458>: fd 513: calling vls_connect(): vlsh 1 addr 
> 0x55768f18 len 16
> vppcom_session_connect:1592: vcl<15458:0>: session handle 1 [0x10001]: 
> session already connected to IPv4 192.168.7.130 port 80 proto TCP, state 0x1 
> (STATE_CONNECT)
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 2
> epoll_ctl:2203: ldp<15458>: epfd 512 ep_vlsh 0, fd 513 vlsh 1, op 1
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1968 
> (vppcom_session_write_inline) assertion `n_write > 0' fails
>  
> Program received signal SIGABRT, Aborted.
> 0x75b98337 in __GI_raise (sig=sig@entry=6) at 
> ../nptl/sysdeps/unix/sysv/linux/raise.c:55
> 55   return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
> Missing separate debuginfos, use: debuginfo-install 
> keyutils-libs-1.5.8-3.el7.x86_64 libuuid-2.23.2-61.el7.x86_64 
> pcre-8.32-17.el7.x86_64
> (gdb) bt
> #0  0x75b98337 in __GI_raise (sig=sig@entry=6) at 
> ../nptl/sysdeps/unix/sysv/linux/raise.c:55
> #1  0x75b99a28 in __GI_abort () at abort.c:90
> #2  0x7506e8f3 in os_panic () at 
> /home/dev/net-base/build/vpp/src/vppinfra/unix-misc.c:176
> #3  0x74fd6c93 in debugger () at 
> /home/dev/net-base/build/vpp/src/vppinfra/error.c:84
> #4  0x74fd7062 in _clib_error (how_to_die=2, function_name=0x0, 
> line_number=0, fmt=0x757483f0 "%s:%d (%s) assertion `%s' fails") at 
> /home/dev/net-base/build/vpp/src/vppinfra/error.c:143
> #5  0x7571f70e in vppcom_session_write_inline (session_handle=1, 
> buf=0x55763240 <_request>, n=0, is_flush=1 '\001') at 
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1968
> #6  0x7571f81a in vppcom_session_write_msg (session_handle=1, 
> buf=0x55763240 <_request>, n=0) at 
> /home/dev/net-base/build/vpp/src/vcl/vppcom.c:1986
> #7  0x757467f5 in vls_write_msg (vlsh=1, buf=0x55763240 
> <_request>, nbytes=0) at /home/dev/net-base/build/vpp/src/vcl/vcl_locked.c:505
> #8  0x77bd0f3f in write (fd=513, buf=0x55763240 <_request>, 
> nbytes=0) at /home/dev/net-base/build/vpp/src/vcl/ldp.c:424
> #9  0x7666e67b in apr_socket_send (sock=0x55787fc0, 
> buf=0x55763240 <_request> "GET / HTTP/1.0\r\nConnection: 
> Keep-Alive\r\nHost: 192.168.7.130\r\nUser-Agent: ApacheBench/2.3\r\nAccept: 
> */*\r\n\r\n", len=len@entry=0x7fffdc20) at network_io/unix/sendrecv.c:41
> #10 0xb4f4 in write_request (c=c@entry=0x557870e0) at ab.c:707
> #11 0x7f9e in write_request (c=0x557870e0) at ab.c:1793
> #12 test () at ab.c:1871
>  
> 
>  
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> 
> View/Reply Online (#14661): https://lists.fd.io/g/vpp-dev/message/14661
> Mute This Topic: https://lists.fd.io/mt/61080258/675152
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [fcoras.li...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14663): https://lists.fd.io/g/vpp-dev/message/14663
Mute This Topic: https://lists.fd.io/mt/61080258/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Kubernetes and contiv vpp issue

2019-11-21 Thread Mohamed feroz Abdul majeeth 
Hi Nathan,


Thanks for your response, I will try what you said.
One more update:
My sample.so plugins are loaded in contiv - vpp/dev-vswitch( development
image).
I started kubernetes and apply contiv vpp yaml with
contiv-vpp/dev-vswitch:3.3.2 image( development image). It's running and
open the vppctl , able to see sample plugins related cli commands. I seen
logs(kubectl logs contiv-vswitch-lqxfp -n kube-system -c contiv-vswitch).
In logs, there is  sample.so  plugin loading information
details are available.

1.what is different between contiv-vpp/vswitch(production image) and
contiv-vpp/dev-vswitch( development image)

2. I applied sample.so plugins patches commonly.why it's not reflect in
contiv-vpp/vswitch and why it's reflect contiv-vpp/dev-vswitch.

Please help me to understand this things.

Thanks,
Feroz

On Thu, 21 Nov, 2019, 8:20 PM Nathan Skrzypczak, <
nathan.skrzypc...@gmail.com> wrote:

> Hi Feroz,
>
> If your plugin gets loaded in the vpp running directly on your machine but
> not in contiv-vpp, it's likely that there is a version mismatch between the
> two or something weird happening during contiv build process.
> * You can check the loaded plugins directly in the debug cli (vppctl :
> show plugins) to ensure check the plugin state at runtime.
> * If both don't run the same version running  will
> give two different commits.
> * The problem might also come from vpp's config file
> (/host/etc/vpp/contiv-vswitch.conf in contiv's case) but it's less probable
> if it doesn't appear in the logs.
>
> If this doesn't help finding the discrepancy, you can build a contiv-vpp
> image from your working binaries with :
> cd CONTIV_DIR && make contiv-agent contiv-init
> cd VPP_DIR && make build-release ; make pkg-deb
> cp $VPP_DIR/build-root/*.deb /scratch/
> cd /scratch && wget
> https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/dev/govpp.conf
> cd /scratch && wget
> https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/prod/vswitch/vpp.conf
> cd /scratch && wget
> https://github.com/contiv/vpp/blob/master/docker/vpp-vswitch/prod/vswitch/vppctl
> You can then build a docker image with
> https://gist.github.com/sknat/cc016e4905cb875f62c74e28e2cc5661
>
> Hope this helps
>
> Cheers
> -Nathan
>
>
>
> Le jeu. 21 nov. 2019 à 06:41, Mohamed feroz Abdul majeeth <
> feroz...@gmail.com> a écrit :
>
>> Hi Team,
>>
>> We add sample( sample.so) plugins with existing vpp plugins. I used
>> 3.3.2 tag contiv vpp and  copied sample plugins related code *patches *.diff
>> under contiv vpp patches folder( vpp/ docker/ vpp- vswitch /vpp/ patches).
>> Patches also applied correctly and able to create the contiv/vswitch docker
>> image. Even I seen its generated sample.so in vpp_ plugins folder in
>> *contiv-vpp/*vpp container which is having builded vpp code.
>> I started kubernetes and apply contiv vpp yaml with
>> contiv-vpp/vswitch:3.3.2 image. It's running and open the vppctl but not
>> able to see sample plugins related cli commands so I seen logs(kubectl logs
>> contiv-vswitch-lqxfp -n kube-system -c contiv-vswitch). In logs, there is
>> no sample.so plugin loading information details.
>>
>> For verification, I followed same vpp build steps in bare metal which is
>> follow in contiv vpp code, sample.so plugin is loaded when I launch the
>> vpp and I'm able to see sample.so plugins cli commands details.
>>
>> In docker/container only, *sample**.so*  plugin is not
>> loaded when launch the vpp.
>>
>> Please help me and guide me. How to solve this problem.
>>
>> Thanks,
>> Feroz
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>>
>> View/Reply Online (#14653): https://lists.fd.io/g/vpp-dev/message/14653
>> Mute This Topic: https://lists.fd.io/mt/61032023/1706314
>> Group Owner: vpp-dev+ow...@lists.fd.io
>> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [
>> nathan.skrzypc...@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
>
> View/Reply Online (#14659): https://lists.fd.io/g/vpp-dev/message/14659
> Mute This Topic: https://lists.fd.io/mt/61032023/1902043
> Group Owner: vpp-dev+ow...@lists.fd.io
> Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [feroz...@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14664): https://lists.fd.io/g/vpp-dev/message/14664
Mute This Topic: https://lists.fd.io/mt/61032023/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Support needed for ARM64 64k pagesize

2019-11-21 Thread Lei Zhang 
Hi,

Anyone knows if VPP can support ARM64 64K pagesize?
   Or anyone has experiences on below issue?

   Thank you!

Here is the kernel config.

# CONFIG_ARM64_4K_PAGES is not set

# CONFIG_ARM64_16K_PAGES is not set

CONFIG_ARM64_64K_PAGES=y

VPP reports a lot of errors when it starts up on ARM64 64k pagesize
enviroment.


load_one_plugin:189: Loaded plugin: stn_plugin.so (VPP Steals the NIC for
Container integration)

load_one_plugin:189: Loaded plugin: svs_plugin.so (Source VRF Select)

load_one_plugin:189: Loaded plugin: tlsopenssl_plugin.so (openssl based TLS
Engine)

load_one_plugin:117: Plugin disabled (default): unittest_plugin.so

load_one_plugin:189: Loaded plugin: vmxnet3_plugin.so (VMWare Vmxnet3
Device Plugin)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

register_node:476: process stack: Invalid argument (errno 22)

 The source code of the error is at mprotect(). It seems that VPP
memeory management may not support 64K pagesize.
#ifdef CLIB_UNIX
/*
 * Disallow writes to the bottom page of the stack, to
 * catch stack overflows.
 */
if (mprotect (p->stack, page_size, PROT_READ) < 0)
  clib_unix_warning ("process stack");
#endif
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14665): https://lists.fd.io/g/vpp-dev/message/14665
Mute This Topic: https://lists.fd.io/mt/61493265/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [VCL] hoststack app crash with invalid memfd segment address

2019-11-21 Thread wanghanlin 







Hi Florin,
Regarding 3), I think main problem maybe in function vl_socket_client_recv_fd_msg called by vcl_session_app_add_segment_handler.  Mutiple worker threads share the same scm->client_socket.fd, so B2 may receive the segment memfd belong to A1. Regards,Hanlin






 










wanghanlin







wanghan...@corp.netease.com








签名由
网易邮箱大师
定制

 

On 11/22/2019 01:44，Florin Coras wrote： 


Hi Hanlin, As Jon pointed out, you may want to register with gerrit. You comments with respect to points 1) and 2) are spot on. I’ve updated the patch to fix them. Regarding 3), if I understood your scenario correctly, it should not happen. The ssvm infra forces applications to map segments at fixed addresses. That is, for the scenario you’re describing lower, if B2 is processed first, ssvm_slave_init_memfd will map the segment at A2. Note how we first map the segment to read the shared header (sh) and then use sh->ssvm_va (which should be A2) to remap the segment at a fixed virtual address (va). Regards,FlorinOn Nov 21, 2019, at 2:49 AM, wanghanlin  wrote:Hi Florin,I have applied the patch, and found some problems in my case.  I have not right to post it in gerrit, so I post here.1)evt->event_type should be set  with SESSION_CTRL_EVT_APP_DEL_SEGMENT rather than SESSION_CTRL_EVT_APP_ADD_SEGMENT. File: src/vnet/session/session_api.c, Line: 561, Function:mq_send_del_segment_cb2)session_send_fds may been called in the end of function mq_send_add_segment_cb, otherwise lock of app_mq can't been free here.File: src/vnet/session/session_api.c, Line: 519, Function:mq_send_add_segment_cb 3) When vcl_segment_attach called in each worker thread, then ssvm_slave_init_memfd can been called in each worker thread and then ssvm_slave_init_memfd map address sequentially through map segment once in advance.  It's OK in only one thread, but maybe wrong in multiple worker threads. Suppose following scene: VPP allocate segment at address A1 and notify worker thread B1 to expect B1 also map segment at address A1,  and simultaneously VPP allocate segment at address A2 and notify worker thread B2 to expect B2 map segment at address A2. If B2 first process notify message, then ssvm_slave_init_memfd may map segment at address A1. Maybe VPP can add segment map address in notify message, and then worker thread just map segment at this address. Regards,Hanlinwanghanlinwanghan...@corp.netease.com签名由 网易邮箱大师 定制On 11/19/2019 09:50，wanghanlin wrote： Hi  Florin,VPP vsersion is v19.08.I'll apply this patch and check it. Thanks a lot!Regards,Hanlinwanghanlinwanghan...@corp.netease.com签名由 网易邮箱大师 定制On 11/16/2019 00:50，Florin Coras wrote： Hi Hanlin,Just to make sure, are you running master or some older VPP?Regarding the issue you could be hitting lower, here’s [1] a patch that I have not yet pushed for merging because it leads to api changes for applications that directly use the session layer application interface instead of vcl. I haven’t tested it extensively, but the goal with it is to signal segment allocation/deallocation over the mq instead of the binary api.Finally, I’ve never tested LDP with Envoy, so not sure if that works properly. There’s ongoing work to integrate Envoy with VCL, so you may want to get in touch with the authors. Regards,Florin[1] https://gerrit.fd.io/r/c/vpp/+/21497On Nov 15, 2019, at 2:26 AM, wanghanlin  wrote:hi ALL,I accidentally got following crash stack when I used VCL with hoststack and memfd. But corresponding invalid rx_fifo address (0x2f42e2480) is valid in VPP process and also can be found in /proc/map. That is, shared memfd segment memory is not consistent between hoststack app and VPP.Generally, VPP allocate/dealloc the memfd segment and then notify hoststack app to attach/detach. But If just after VPP dealloc memfd segment and notify hoststack app, and then VPP allocate same memfd segment at once because of session connected, and then what happened now? Because hoststack app process dealloc message and connected message with diffrent threads, maybe rx_thread_fn just detach the memfd segment and not attach the same memfd segment, then unfortunately worker thread get the connected message. These are just my guess, maybe I misunderstand.(gdb) bt#0  0x7f7cde21ffbf in raise () from /lib/x86_64-linux-