> -----邮件原件-----
> 发件人: Feifei Wang
> 发送时间: 2021年1月14日 14:24
> 收件人: Pavan Nikhilesh Bhagavatula <pbhagavat...@marvell.com>; Harry
> van Haaren <harry.van.haa...@intel.com>; Nikhil Rao
> <nikhil....@intel.com>; Pavan Nikhilesh
> <pbhagavat...@caviumnetworks.com>
> 抄送: dev@dpdk.org; jer...@marvell.com; nd <n...@arm.com>;
> sta...@dpdk.org; Ruifeng Wang <ruifeng.w...@arm.com>; Honnappa
> Nagarahalli <honnappa.nagaraha...@arm.com>; nd <n...@arm.com>
> 主题: 回复: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
>
> > -----邮件原件-----
> > 发件人: Pavan Nikhilesh Bhagavatula <pbhagavat...@marvell.com>
> > 发送时间: 2021年1月5日 18:09
> > 收件人: Feifei Wang <feifei.wa...@arm.com>; Harry van Haaren
> > <harry.van.haa...@intel.com>; Nikhil Rao <nikhil....@intel.com>; Pavan
> > Nikhilesh <pbhagavat...@caviumnetworks.com>
> > 抄送: dev@dpdk.org; jer...@marvell.com; nd <n...@arm.com>;
> > sta...@dpdk.org; Ruifeng Wang <ruifeng.w...@arm.com>; Honnappa
> > Nagarahalli <honnappa.nagaraha...@arm.com>
> > 主题: RE: [EXT] [PATCH v3] examples/eventdev: refactor ethdev port stop
> >
> Hi, Pavan
>
> > Hi Feifei,
> >
> > >Move eth stop code from "signal_handler" function to the end of "main"
> > >function. There are two reasons for this:
> > >
> > >First, this improves code maintenance and makes code look simple and
> > >clear. Based on this change, after receiving the interrupt signal,
> > >"fdata->done" is set as 1. Then the main thread will wait all worker
> > >lcores to jump out of the loop. Finally, the main thread will stop
> > >and then close eth dev port.
> > >
> > >Second, for older version, the main thread first stops eth dev port
> > >and then waits the end of worker lcore. This may cause errors because
> > >it may stop the eth dev port which worker lcores are using. This
> > >moving change can fix this by waiting all worker threads to exit and
> > >then stop the eth dev port.
> >
> > Apologies for the delayed reply,
> >
> > In case of event dev the workers don't interact with eth device
> > directly, Instead eth device "injects" packets into event device and
> > event device is responsible for scheduling them to the workers.
> >
> > If the producer is not stopped i.e. in this case eth device then the
> > worker threads might never exit and the main core would wait
> > indefinitely for workers to exit. This will be predominantly seen in
> > cases where there are only a few flows and large number of workers
> > causing a lot of intra thread dependency.
>
> For the case that the event device scheduling packets to the workers.
> Though the producer is not stopped (eth device), when the main core
> receive the interrupt signal, the "fdata->done"
> will be set as 1. Then all the workers load the value 1 of "fdata->done" and
> jump out of the loop to finish their thread.
>
And I also test the case you said above with this patch, when I send interrupt
signal(ctrl + c), IXIA
is running and eth device is also working, then the program can exit normally.
HW:
Traffic generator: IXIA
Nics: ixgbe 82599ES 10-Gigabit
Architecture: aarch64
CPU: Cortex-A72
With this patch:
______________________________________________________________________________________________
Command: sudo ./dpdk-eventdev_pipeline -l 0-2,8-11 --vdev=event_sw0 -- -r2 -t2
-e4 -w F00 -s4 -n0 -c32 -W1000 -D
.........
Config:
ports: 2
workers: 4
packets: 9223372036854775807
Queue-prio: 0
qid0 type: atomic
Cores available: 7
Cores used: 6
Eventdev 0: event_sw
Stages:
Stage 0, Type Atomic Priority = 128
Stage 1, Type Atomic Priority = 128
Stage 2, Type Atomic Priority = 128
Stage 3, Type Atomic Priority = 128
Port 0 modified RSS hash function based on hardware support,requested:0x3afbc
configured:0x38d34
Port 0 MAC: 90 e2 ba 56 ed 6c
Port 1 modified RSS hash function based on hardware support,requested:0x3afbc
configured:0x38d34
Port 1 MAC: 90 e2 ba 56 ed 6d
[dump_core_info()] lcore 1 executing NIC Rx
[dump_core_info()] lcore 1 executing NIC Tx
[dump_core_info()] lcore 2 executing scheduler
[dump_core_info()] lcore 8 executing worker, using eventdev port 0
[dump_core_info()] lcore 9 executing worker, using eventdev port 1
[dump_core_info()] lcore 10 executing worker, using eventdev port 2
[dump_core_info()] lcore 11 executing worker, using eventdev port 3
Command: ctrl + c
.........
worker 1 thread done. RX=0 TX=0
worker 2 thread done. RX=0 TX=0
worker 11 thread done. RX=1844023 TX=1844023
worker 9 thread done. RX=1844888 TX=1844888
worker 8 thread done. RX=1845490 TX=1845490
worker 10 thread done. RX=1844903 TX=1844903
Port Workload distribution:
worker 0 : 25.0 % (1845474 pkts)
worker 1 : 25.0 % (1844872 pkts)
worker 2 : 25.0 % (1844887 pkts)
worker 3 : 25.0 % (1844007 pkts)
Invalid port_id=0
Invalid port_id=1
______________________________________________________________________________________________
> Best Regards
> Feifei
> >
> > Regards,
> > Pavan.
> >
> > >
> > >In the meanwhile, remove wmb in signal_handler.
> > >
> > >This is because when the main lcore receive the stop signal, it
> > >stores
> > >1 into fdata->done. And then the worker lcores load "fdata->done" and
> > >jump out of the loop to stop running. Nothing should be stored after
> > >updating
> > >fdata->done, so the wmb is unnecessary.
> > >
> > >Fixes: 085edac2ca38 ("examples/eventdev_pipeline: support Tx
> > >adapter")
> > >Cc: pbhagavat...@marvell.com
> > >Cc: sta...@dpdk.org
> > >
> > >Suggested-by: Ruifeng Wang <ruifeng.w...@arm.com>
> > >Signed-off-by: Feifei Wang <feifei.wa...@arm.com>
> > >Reviewed-by: Ruifeng Wang <ruifeng.w...@arm.com>
> > >Reviewed-by: Honnappa Nagarahalli
> > ><honnappa.nagaraha...@arm.com>
> > >Acked-by: Harry van Haaren <harry.van.haa...@intel.com>
> > >---
> > >
> > >v2:
> > >1. Delete unused variable to fix build error 2. Reduce commit message
> > >length to fix coding style issues
> > >
> > >v3:
> > >1. Title improvement (Van Haaren)
> > >
> > > examples/eventdev_pipeline/main.c | 16 ++++------------
> > > 1 file changed, 4 insertions(+), 12 deletions(-)
> > >
> > >diff --git a/examples/eventdev_pipeline/main.c
> > >b/examples/eventdev_pipeline/main.c
> > >index 823f8b51c..fdbaf667b 100644
> > >--- a/examples/eventdev_pipeline/main.c
> > >+++ b/examples/eventdev_pipeline/main.c
> > >@@ -280,7 +280,6 @@ static void
> > > signal_handler(int signum)
> > > {
> > > static uint8_t once;
> > >- uint16_t portid;
> > >
> > > if (fdata->done)
> > > rte_exit(1, "Exiting on signal %d\n", signum); @@ -291,17
> > +290,6 @@
> > >signal_handler(int signum)
> > > rte_event_dev_dump(0, stdout);
> > > once = 1;
> > > fdata->done = 1;
> > >- rte_smp_wmb();
> > >-
> > >- RTE_ETH_FOREACH_DEV(portid) {
> > >- rte_event_eth_rx_adapter_stop(portid);
> > >- rte_event_eth_tx_adapter_stop(portid);
> > >- if (rte_eth_dev_stop(portid) < 0)
> > >- printf("Failed to stop port %u", portid);
> > >- }
> > >-
> > >- rte_eal_mp_wait_lcore();
> > >-
> > > }
> > > if (signum == SIGTSTP)
> > > rte_event_dev_dump(0, stdout);
> > >@@ -465,6 +453,10 @@ main(int argc, char **argv)
> > > }
> > >
> > > RTE_ETH_FOREACH_DEV(portid) {
> > >+ rte_event_eth_rx_adapter_stop(portid);
> > >+ rte_event_eth_tx_adapter_stop(portid);
> > >+ if (rte_eth_dev_stop(portid) < 0)
> > >+ printf("Failed to stop port %u", portid);
> > > rte_eth_dev_close(portid);
> > > }
> > >
> > >--
> > >2.25.1
