> > Hi Akhil, Konstantin, > > Do you have any further comments?
Will try to have a proper look today/tomorrow. Sorry for delay. Konstantin > > Thanks, > Anoob > > > -----Original Message----- > > From: dev <dev-boun...@dpdk.org> On Behalf Of Anoob Joseph > > Sent: Monday, January 20, 2020 7:15 PM > > To: Akhil Goyal <akhil.go...@nxp.com>; Radu Nicolau > > <radu.nico...@intel.com>; Thomas Monjalon <tho...@monjalon.net> > > Cc: Anoob Joseph <ano...@marvell.com>; Jerin Jacob Kollanukkaran > > <jer...@marvell.com>; Narayana Prasad Raju Athreya > > <pathr...@marvell.com>; Ankur Dwivedi <adwiv...@marvell.com>; > > Archana Muniganti <march...@marvell.com>; Tejasree Kondoj > > <ktejas...@marvell.com>; Vamsi Krishna Attunuru > > <vattun...@marvell.com>; Lukas Bartosik <lbarto...@marvell.com>; > > Konstantin Ananyev <konstantin.anan...@intel.com>; dev@dpdk.org > > Subject: [dpdk-dev] [PATCH v2 00/12] add eventmode to ipsec-secgw > > > > This series introduces event-mode additions to ipsec-secgw. This effort is > > parallel to the similar changes in l2fwd (l2fwd-event app) & l3fwd. > > > > With this series, ipsec-secgw would be able to run in eventmode. The worker > > thread (executing loop) would be receiving events and would be submitting > > it back to the eventdev after the processing. This way, multicore scaling > > and > > h/w assisted scheduling is achieved by making use of the eventdev > > capabilities. > > > > Since the underlying event device would be having varying capabilities, the > > worker thread could be drafted differently to maximize performance. > > This series introduces usage of multiple worker threads, among which the > > one to be used will be determined by the operating conditions and the > > underlying device capabilities. > > > > For example, if an event device - eth device pair has Tx internal port, then > > application can do tx_adapter_enqueue() instead of regular > > event_enqueue(). So a thread making an assumption that the device pair has > > internal port will not be the right solution for another pair. The > > infrastructure > > added with these patches aims to help application to have multiple worker > > threads, there by extracting maximum performance from every device > > without affecting existing paths/use cases. > > > > The eventmode configuration is predefined. All packets reaching one eth > > port will hit one event queue. All event queues will be mapped to all event > > ports. So all cores will be able to receive traffic from all ports. > > When schedule_type is set as RTE_SCHED_TYPE_ORDERED/ATOMIC, event > > device will ensure the ordering. Ordering would be lost when tried in > > PARALLEL. > > > > Following command line options are introduced, > > > > --transfer-mode: to choose between poll mode & event mode > > --schedule-type: to specify the scheduling type > > (RTE_SCHED_TYPE_ORDERED/ > > RTE_SCHED_TYPE_ATOMIC/ > > RTE_SCHED_TYPE_PARALLEL) > > > > Additionally the event mode introduces two modes of processing packets: > > > > Driver-mode: This mode will have bare minimum changes in the application > > to support ipsec. There woudn't be any lookup etc done in > > the application. And for inline-protocol use case, the > > thread would resemble l2fwd as the ipsec processing would be > > done entirely in the h/w. This mode can be used to benchmark > > the raw performance of the h/w. All the application side > > steps (like lookup) can be redone based on the requirement > > of the end user. Hence the need for a mode which would > > report the raw performance. > > > > App-mode: This mode will have all the features currently implemented with > > ipsec-secgw (non librte_ipsec mode). All the lookups etc > > would follow the existing methods and would report numbers > > that can be compared against regular ipsec-secgw benchmark > > numbers. > > > > The driver mode is selected with existing --single-sa option (used also by > > poll > > mode). When --single-sa option is used in conjution with event mode then > > index passed to --single-sa is ignored. > > > > Example commands to execute ipsec-secgw in various modes on OCTEON > > TX2 platform, > > > > #Inbound and outbound app mode > > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=128 -w > > 0002:03:00.0,ipsec_in_max_spi=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log- > > level=8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f > > aes-gcm.cfg -- > > transfer-mode event --schedule-type parallel > > > > #Inbound and outbound driver mode > > ipsec-secgw -w 0002:02:00.0,ipsec_in_max_spi=128 -w > > 0002:03:00.0,ipsec_in_max_spi=128 -w 0002:0e:00.0 -w 0002:10:00.1 --log- > > level=8 -c 0x1 -- -P -p 0x3 -u 0x1 --config "(1,0,0),(0,0,0)" -f > > aes-gcm.cfg -- > > transfer-mode event --schedule-type parallel --single-sa 0 > > > > This series adds non burst tx internal port workers only. It provides > > infrastructure for non internal port workers, however does not define any. > > Also, only inline ipsec protocol mode is supported by the worker threads > > added. > > > > Following are planned features, > > 1. Add burst mode workers. > > 2. Add non internal port workers. > > 3. Verify support for Rx core (the support is added but lack of h/w to > > verify). > > 4. Add lookaside protocol support. > > > > Following are features that Marvell won't be attempting. > > 1. Inline crypto support. > > 2. Lookaside crypto support. > > > > For the features that Marvell won't be attempting, new workers can be > > introduced by the respective stake holders. > > > > This series is tested on Marvell OCTEON TX2. > > > > Changes in v2: > > * Remove --process-dir option. Instead use existing unprotected port mask > > option (-u) to decide wheter port handles inbound or outbound traffic. > > * Remove --process-mode option. Instead use existing --single-sa option > > to select between app and driver modes. > > * Add handling of PKT_RX_SEC_OFFLOAD_FAIL result in app worker thread. > > * Fix passing of req_rx_offload flags to create_default_ipsec_flow(). > > * Move destruction of flows to a location where eth ports are stopped > > and closed. > > * Print error and exit when event mode --schedule-type option is used > > in poll mode. > > * Reduce number of goto statements replacing them with loop constructs. > > * Remove sec_session_fixed table and replace it with locally build > > table in driver worker thread. Table is indexed by port identifier > > and holds first inline session pointer found for a given port. > > * Print error and exit when sessions other than inline are configured > > in event mode. > > * When number of event queues is less than number of eth ports then > > map all eth ports to one event queue. > > * Cleanup and minor improvements in code as suggested by Konstantin > > > > Deferred to v3: > > * The final patch updates the hardcoded number of buffers in a pool. > > Also, there was a discussion on the update of number of qp. Both the > > above can be handled properly, if we can remove the logic which limits > > one core to only use one crypto qp. If we can allow one qp per > > lcore_param, every eth queue can have it's own crypto qp and that would > > solve the requirements with OCTEON TX2 inline ipsec support as well. > > > > Patch with the mentioned change, > > https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__patches.dpdk.org_patch_64408_&d=DwIDAg&c=nKjWec2b6R0mOyPaz > > 7xtfQ&r=BPcGOOudUMrTDQ9YbgKcOkO5ChYiUPPlPNIEvTOhjNE&m=rg71UQ > > 1CwRYPFy30QuJQZd1Lam_kwYg15N2h5GN2iD4&s=yHzfRBRuunl4JWV97vufk > > 7aycUc472ahPVnQ9Tt6SeY&e= > > > > * Update ipsec-secgw documentation to describe the new options as well as > > event mode support. > > > > This series depends on the PMD changes submitted in the following set, > > https://urldefense.proofpoint.com/v2/url?u=http- > > 3A__patches.dpdk.org_project_dpdk_list_-3Fseries- > > 3D8203&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=BPcGOOudUMrTDQ9Y > > bgKcOkO5ChYiUPPlPNIEvTOhjNE&m=rg71UQ1CwRYPFy30QuJQZd1Lam_kwY > > g15N2h5GN2iD4&s=g2wtO9tOQTYHa9os1ECz5uwgpz9JmjTlGbEl- > > Cp6WAw&e= > > > > Ankur Dwivedi (1): > > examples/ipsec-secgw: add default rte_flow for inline Rx > > > > Anoob Joseph (5): > > examples/ipsec-secgw: add framework for eventmode helper > > examples/ipsec-secgw: add eventdev port-lcore link > > examples/ipsec-secgw: add Rx adapter support > > examples/ipsec-secgw: add Tx adapter support > > examples/ipsec-secgw: add routines to display config > > > > Lukasz Bartosik (6): > > examples/ipsec-secgw: add routines to launch workers > > examples/ipsec-secgw: add support for internal ports > > examples/ipsec-secgw: add eventmode to ipsec-secgw > > examples/ipsec-secgw: add driver mode worker > > examples/ipsec-secgw: add app mode worker > > examples/ipsec-secgw: add cmd line option for bufs > > > > examples/ipsec-secgw/Makefile | 2 + > > examples/ipsec-secgw/event_helper.c | 1714 > > +++++++++++++++++++++++++++++++++++ > > examples/ipsec-secgw/event_helper.h | 312 +++++++ examples/ipsec- > > secgw/ipsec-secgw.c | 502 ++++++++-- > > examples/ipsec-secgw/ipsec-secgw.h | 86 ++ > > examples/ipsec-secgw/ipsec.c | 7 + > > examples/ipsec-secgw/ipsec.h | 36 +- > > examples/ipsec-secgw/ipsec_worker.c | 656 ++++++++++++++ > > examples/ipsec-secgw/ipsec_worker.h | 39 + > > examples/ipsec-secgw/meson.build | 4 +- > > examples/ipsec-secgw/sa.c | 11 - > > 11 files changed, 3275 insertions(+), 94 deletions(-) create mode 100644 > > examples/ipsec-secgw/event_helper.c > > create mode 100644 examples/ipsec-secgw/event_helper.h > > create mode 100644 examples/ipsec-secgw/ipsec-secgw.h > > create mode 100644 examples/ipsec-secgw/ipsec_worker.c > > create mode 100644 examples/ipsec-secgw/ipsec_worker.h > > > > -- > > 2.7.4
