Introduce event dma adapter interface to transfer packets between dma device and event device.
Signed-off-by: Amit Prakash Shukla <amitpraka...@marvell.com> Acked-by: Jerin Jacob <jer...@marvell.com> --- MAINTAINERS | 6 + doc/api/doxy-api-index.md | 1 + doc/guides/eventdevs/features/default.ini | 8 + doc/guides/prog_guide/event_dma_adapter.rst | 264 ++++ doc/guides/prog_guide/eventdev.rst | 8 +- .../img/event_dma_adapter_op_forward.svg | 1086 +++++++++++++++++ .../img/event_dma_adapter_op_new.svg | 1079 ++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/rel_notes/release_23_11.rst | 6 + lib/eventdev/eventdev_pmd.h | 171 ++- lib/eventdev/eventdev_private.c | 10 + lib/eventdev/meson.build | 1 + lib/eventdev/rte_event_dma_adapter.h | 581 +++++++++ lib/eventdev/rte_eventdev.h | 44 + lib/eventdev/rte_eventdev_core.h | 8 +- lib/eventdev/version.map | 16 + lib/meson.build | 2 +- 17 files changed, 3286 insertions(+), 6 deletions(-) create mode 100644 doc/guides/prog_guide/event_dma_adapter.rst create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg create mode 100644 doc/guides/prog_guide/img/event_dma_adapter_op_new.svg create mode 100644 lib/eventdev/rte_event_dma_adapter.h diff --git a/MAINTAINERS b/MAINTAINERS index a926155f26..4ebbbe8bb3 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -540,6 +540,12 @@ F: lib/eventdev/*crypto_adapter* F: app/test/test_event_crypto_adapter.c F: doc/guides/prog_guide/event_crypto_adapter.rst +Eventdev DMA Adapter API +M: Amit Prakash Shukla <amitpraka...@marvell.com> +T: git://dpdk.org/next/dpdk-next-eventdev +F: lib/eventdev/*dma_adapter* +F: doc/guides/prog_guide/event_dma_adapter.rst + Raw device API M: Sachin Saxena <sachin.sax...@oss.nxp.com> M: Hemant Agrawal <hemant.agra...@nxp.com> diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index fdeda13932..b7df7be4d9 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -29,6 +29,7 @@ The public API headers are grouped by topics: [event_eth_tx_adapter](@ref rte_event_eth_tx_adapter.h), [event_timer_adapter](@ref rte_event_timer_adapter.h), [event_crypto_adapter](@ref rte_event_crypto_adapter.h), + [event_dma_adapter](@ref rte_event_dma_adapter.h), [rawdev](@ref rte_rawdev.h), [metrics](@ref rte_metrics.h), [bitrate](@ref rte_bitrate.h), diff --git a/doc/guides/eventdevs/features/default.ini b/doc/guides/eventdevs/features/default.ini index 00360f60c6..73a52d915b 100644 --- a/doc/guides/eventdevs/features/default.ini +++ b/doc/guides/eventdevs/features/default.ini @@ -44,6 +44,14 @@ internal_port_op_fwd = internal_port_qp_ev_bind = session_private_data = +; +; Features of a default DMA adapter. +; +[DMA adapter Features] +internal_port_op_new = +internal_port_op_fwd = +internal_port_vchan_ev_bind = + ; ; Features of a default Timer adapter. ; diff --git a/doc/guides/prog_guide/event_dma_adapter.rst b/doc/guides/prog_guide/event_dma_adapter.rst new file mode 100644 index 0000000000..701e50d042 --- /dev/null +++ b/doc/guides/prog_guide/event_dma_adapter.rst @@ -0,0 +1,264 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright (c) 2023 Marvell. + +Event DMA Adapter Library +========================= + +DPDK :doc:`Eventdev library <eventdev>` provides event driven programming model with features +to schedule events. :doc:`DMA Device library <dmadev>` provides an interface to DMA poll mode +drivers that support DMA operations. Event DMA Adapter is intended to bridge between the event +device and the DMA device. + +Packet flow from DMA device to the event device can be accomplished using software and hardware +based transfer mechanisms. The adapter queries an eventdev PMD to determine which mechanism to +be used. The adapter uses an EAL service core function for software based packet transfer and +uses the eventdev PMD functions to configure hardware based packet transfer between DMA device +and the event device. DMA adapter uses a new event type called ``RTE_EVENT_TYPE_DMADEV`` to +indicate the source of event. + +Application can choose to submit an DMA operation directly to an DMA device or send it to an DMA +adapter via eventdev based on ``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD`` capability. The +first mode is known as the event new (``RTE_EVENT_DMA_ADAPTER_OP_NEW``) mode and the second as the +event forward (``RTE_EVENT_DMA_ADAPTER_OP_FORWARD``) mode. Choice of mode can be specified while +creating the adapter. In the former mode, it is the application's responsibility to enable +ingress packet ordering. In the latter mode, it is the adapter's responsibility to enable +ingress packet ordering. + + +Adapter Modes +------------- + +RTE_EVENT_DMA_ADAPTER_OP_NEW mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode, application submits DMA operations directly to an DMA +device. The adapter then dequeues DMA completions from the DMA device and enqueues them as events +to the event device. This mode does not ensure ingress ordering as the application directly +enqueues to the dmadev without going through DMA/atomic stage. In this mode, events dequeued +from the adapter are treated as new events. The application has to specify event information +(response information) which is needed to enqueue an event after the DMA operation is completed. + +.. _figure_event_dma_adapter_op_new: + +.. figure:: img/event_dma_adapter_op_new.* + + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` mode + + +RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), the application should use +``rte_event_dma_adapter_enqueue()`` API to enqueue DMA operations as events to DMA adapter. If +not, application retrieves DMA adapter's event port using ``rte_event_dma_adapter_event_port_get()`` +API, links its event queue to this port and starts enqueuing DMA operations as events to eventdev +using ``rte_event_enqueue_burst()``. The adapter then dequeues the events and submits the DMA +operations to the dmadev. After the DMA operation is complete, the adapter enqueues events to the +event device. + +Applications can use this mode when ingress packet ordering is needed. In this mode, events +dequeued from the adapter will be treated as forwarded events. Application has to specify event +information (response information) needed to enqueue the event after the DMA operation has +completed. + +.. _figure_event_dma_adapter_op_forward: + +.. figure:: img/event_dma_adapter_op_forward.* + + Working model of ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode + + +API Overview +------------ + +This section has a brief introduction to the event DMA adapter APIs. The application is expected +to create an adapter which is associated with a single eventdev, then add dmadev and vchan to the +adapter instance. + + +Create an adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +An adapter instance is created using ``rte_event_dma_adapter_create()``. This function is called +with event device to be associated with the adapter and port configuration for the adapter to +setup an event port (if the adapter needs to use a service function). + +Adapter can be started in ``RTE_EVENT_DMA_ADAPTER_OP_NEW`` or ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` +mode. + +.. code-block:: c + + enum rte_event_dma_adapter_mode mode; + struct rte_event_dev_info dev_info; + struct rte_event_port_conf conf; + uint8_t evdev_id; + uint8_t dma_id; + int ret; + + ret = rte_event_dev_info_get(dma_id, &dev_info); + + conf.new_event_threshold = dev_info.max_num_events; + conf.dequeue_depth = dev_info.max_event_port_dequeue_depth; + conf.enqueue_depth = dev_info.max_event_port_enqueue_depth; + mode = RTE_EVENT_DMA_ADAPTER_OP_FORWARD; + ret = rte_event_dma_adapter_create(dma_id, evdev_id, &conf, mode); + + +``rte_event_dma_adapter_create_ext()`` function can be used by the application to have a finer +control on eventdev port allocation and setup. The ``rte_event_dma_adapter_create_ext()`` +function is passed a callback function. The callback function is invoked if the adapter creates +a service function and uses an event port for it. The callback is expected to fill the +``struct rte_event_dma_adapter_conf`` structure passed to it. + +In the ``RTE_EVENT_DMA_ADAPTER_OP_FORWARD`` mode, if the event PMD and DMA PMD supports internal +event port (``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD``), events with DMA operations should +be enqueued to the DMA adapter using ``rte_event_dma_adapter_enqueue()`` API. If not, the event port +created by the adapter can be retrieved using ``rte_event_dma_adapter_event_port_get()`` API. An +application can use this event port to link with an event queue, on which it enqueues events +towards the DMA adapter using ``rte_event_enqueue_burst()``. + +.. code-block:: c + + uint8_t dma_adpt_id, evdev_id, dma_dev_id, dma_ev_port_id, app_qid; + struct rte_event ev; + uint32_t cap; + int ret; + + // Fill in event info and update event_ptr with rte_dma_op + memset(&ev, 0, sizeof(ev)); + . + . + ev.event_ptr = op; + + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD) { + ret = rte_event_dma_adapter_enqueue(evdev_id, app_ev_port_id, ev, nb_events); + } else { + ret = rte_event_dma_adapter_event_port_get(dma_adpt_id, &dma_ev_port_id); + ret = rte_event_queue_setup(evdev_id, app_qid, NULL); + ret = rte_event_port_link(evdev_id, dma_ev_port_id, &app_qid, NULL, 1); + ev.queue_id = app_qid; + ret = rte_event_enqueue_burst(evdev_id, app_ev_port_id, ev, nb_events); + } + + +Event device configuration for service based adapter +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When ``rte_event_dma_adapter_create()`` is used for creating adapter instance, +``rte_event_dev_config::nb_event_ports`` is automatically incremented, and event device is +reconfigured with additional event port during service initialization. This event device +reconfigure logic also increments the ``rte_event_dev_config::nb_single_link_event_port_queues`` +parameter if the adapter event port config is of type ``RTE_EVENT_PORT_CFG_SINGLE_LINK``. + +Applications using this mode of adapter creation need not configure the event device with +``rte_event_dev_config::nb_event_ports`` and +``rte_event_dev_config::nb_single_link_event_port_queues`` parameters required for DMA adapter when +the adapter is created using the above-mentioned API. + + +Querying adapter capabilities +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_dma_adapter_caps_get()`` function allows the application to query the adapter +capabilities for an eventdev and dmadev combination. This API provides whether dmadev and eventdev +are connected using internal HW port or not. + +.. code-block:: c + + rte_event_dma_adapter_caps_get(dev_id, dma_dev_id, &cap); + + +Adding vchan to the adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +dmadev device id and vchan are configured using dmadev APIs. For more information +see :doc:`here <dmadev>`. + +.. code-block:: c + + struct rte_dma_vchan_conf vchan_conf; + struct rte_dma_conf dev_conf; + uint8_t dev_id = 0; + uint16_t vchan = 0; + + rte_dma_configure(dev_id, &dev_conf); + rte_dma_vchan_setup(dev_id, vchan, &vchan_conf); + +These dmadev id and vchan are added to the instance using the +``rte_event_dma_adapter_vchan_add()`` API. The same is removed using +``rte_event_dma_adapter_vchan_del()`` API. If hardware supports +``RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND`` capability, event information must be passed to the add API. + +.. code-block:: c + + uint32_t cap; + int ret; + + ret = rte_event_dma_adapter_caps_get(evdev_id, dma_dev_id, &cap); + if (cap & RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND) { + struct rte_event event; + + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, &conf); + } else + rte_event_dma_adapter_vchan_add(id, dma_dev_id, vchan, NULL); + + +Configuring service function +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If the adapter uses a service function, the application is required to assign a service core to +the service function as show below. + +.. code-block:: c + + uint32_t service_id; + + if (rte_event_dma_adapter_service_id_get(dma_id, &service_id) == 0) + rte_service_map_lcore_set(service_id, CORE_ID); + + +Set event response information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the RTE_EVENT_DMA_ADAPTER_OP_FORWARD / RTE_EVENT_DMA_ADAPTER_OP_NEW mode, the application +specifies the dmadev ID and vchan ID in ``struct rte_event_dma_adapter_op`` and the event +information (response information) needed to enqueue an event after the DMA operation has +completed. The response information is specified in ``struct rte_event`` and appended to the +``struct rte_event_dma_adapter_op``. + + +Start the adapter instance +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The application calls ``rte_event_dma_adapter_start()`` to start the adapter. This function calls +the start callbacks of the eventdev PMDs for hardware based eventdev-dmadev connections and +``rte_service_run_state_set()`` to enable the service function if one exists. + +.. code-block:: c + + rte_event_dma_adapter_start(id); + +.. Note:: + + The eventdev to which the event_dma_adapter is connected should be started before calling + rte_event_dma_adapter_start(). + + +Get adapter statistics +~~~~~~~~~~~~~~~~~~~~~~ + +The ``rte_event_dma_adapter_stats_get()`` function reports counters defined in struct +``rte_event_dma_adapter_stats``. The received packet and enqueued event counts are a sum of the +counts from the eventdev PMD callbacks if the callback is supported, and the counts maintained by +the service function, if one exists. + +Set/Get adapter runtime configuration parameters +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The runtime configuration parameters of adapter can be set/get using +``rte_event_dma_adapter_runtime_params_set()`` and +``rte_event_dma_adapter_runtime_params_get()`` respectively. +The parameters that can be set/get are defined in +``struct rte_event_dma_adapter_runtime_params``. diff --git a/doc/guides/prog_guide/eventdev.rst b/doc/guides/prog_guide/eventdev.rst index 2c83176846..ff55115d0d 100644 --- a/doc/guides/prog_guide/eventdev.rst +++ b/doc/guides/prog_guide/eventdev.rst @@ -333,7 +333,8 @@ eventdev. .. Note:: EventDev needs to be started before starting the event producers such - as event_eth_rx_adapter, event_timer_adapter and event_crypto_adapter. + as event_eth_rx_adapter, event_timer_adapter, event_crypto_adapter and + event_dma_adapter. Ingress of New Events ~~~~~~~~~~~~~~~~~~~~~ @@ -445,8 +446,9 @@ using ``rte_event_dev_stop_flush_callback_register()`` function. .. Note:: The event producers such as ``event_eth_rx_adapter``, - ``event_timer_adapter`` and ``event_crypto_adapter`` - need to be stopped before stopping the event device. + ``event_timer_adapter``, ``event_crypto_adapter`` and + ``event_dma_adapter`` need to be stopped before stopping + the event device. Summary ------- diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg new file mode 100644 index 0000000000..b7fe1fecf2 --- /dev/null +++ b/doc/guides/prog_guide/img/event_dma_adapter_op_forward.svg @@ -0,0 +1,1086 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright (c) 2023 Marvell. --> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + width="720px" + height="486px" + id="svg13237" + version="1.1" + inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)" + sodipodi:docname="event_dma_adapter_op_forward.svg" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns="http://www.w3.org/2000/svg" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/"> + <defs + id="defs13239"> + <marker + inkscape:stockid="Arrow1Sstart" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Sstart" + style="overflow:visible"> + <path + id="path8416" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.2) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Send" + style="overflow:visible;"> + <path + id="path8419" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="DiamondL" + orient="auto" + refY="0.0" + refX="0.0" + id="DiamondL" + style="overflow:visible"> + <path + id="path8483" + d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="DotL" + orient="auto" + refY="0.0" + refX="0.0" + id="DotL" + style="overflow:visible"> + <path + id="path8465" + d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8) translate(7.4, 1)" /> + </marker> + <marker + inkscape:stockid="SquareL" + orient="auto" + refY="0.0" + refX="0.0" + id="SquareL" + style="overflow:visible"> + <path + id="path8474" + d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0.0" + refX="0.0" + id="TriangleOutL" + style="overflow:visible"> + <path + id="path8546" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path8404" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8) translate(12.5,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Mend" + style="overflow:visible;"> + <path + id="path8413" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.4) rotate(180) translate(10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow2Lend" + style="overflow:visible;"> + <path + id="path8425" + style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + transform="scale(1.1) rotate(180) translate(1,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Lend" + style="overflow:visible;"> + <path + id="path8407" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.8) rotate(180) translate(12.5,0)" /> + </marker> + <filter + id="filter_2" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15" /> + </filter> + <filter + id="filter_2-3" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1" /> + </filter> + <filter + id="filter_2-0" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-7" /> + </filter> + <filter + id="filter_2-0-8" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-7-7" /> + </filter> + <filter + id="filter_2-3-9" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-6" /> + </filter> + <filter + id="filter_2-3-6" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-63" /> + </filter> + <filter + id="filter_2-3-91" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-3" /> + </filter> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-5" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart-7" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8404-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.8,0,0,0.8,10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-51" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-3" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-62" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-9" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-2" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart-7-9" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8404-0-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.8,0,0,0.8,10,0)" /> + </marker> + <filter + id="filter_2-3-6-1" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-63-8" /> + </filter> + <filter + id="filter_2-3-92" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-2" /> + </filter> + <filter + id="filter_2-3-94" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-7" /> + </filter> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart-7-6" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8404-0-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.8,0,0,0.8,10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-55" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-4" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1" + inkscape:cx="233.5" + inkscape:cy="288" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1920" + inkscape:window-height="1017" + inkscape:window-x="-8" + inkscape:window-y="-8" + inkscape:window-maximized="1" + inkscape:snap-nodes="false" + inkscape:showpageshadow="2" + inkscape:pagecheckerboard="0" + inkscape:deskcolor="#d1d1d1"> + <inkscape:grid + type="xygrid" + id="grid13454" /> + </sodipodi:namedview> + <metadata + id="metadata13242"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + id="layer1" + inkscape:label="Layer 1" + inkscape:groupmode="layer"> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-4" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,88.874699,-812.39909)"> + <title + id="title22-7-5">Square</title> + <desc + id="desc24-7-8">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-5" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-7" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" /> + </g> + <g + id="g13515-33"> + <g + id="g13534-8"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-95" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none" + d="M 312.28671,240.74335 H 227.99897" + id="path17209" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)" + d="m 221.6484,77.57125 h 94.28101" + id="path17209-8" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,314.24227,-811.89589)"> + <title + id="title22-7">Square</title> + <desc + id="desc24-7">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" /> + </g> + <g + id="g13515"> + <g + id="g13534"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.724714;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none" + d="M 89.025329,74.39932 H 24.750043" + id="path17209-3" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-4.325033,28.642983)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-3" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-1.93108,192.80833)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-1" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.751412;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none" + d="M 18.763392,120.7432 H 87.758545" + id="path17209-3-0" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" + transform="matrix(0.73232502,0,0,0.75477602,-218.16394,72.68276)" /> + <path + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-2" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" + transform="matrix(0.73232502,0,0,0.75477602,-217.40136,26.716271)" /> + <g + id="g29167-4" + transform="matrix(0.73232502,0,0,0.75477602,-217.31662,28.007562)"> + <text + id="text29163-9" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-9" + sodipodi:role="line">1</tspan></text> + </g> + <g + id="g29167-9" + transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,28.689051)"> + <text + id="text29163-3" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-3" + sodipodi:role="line">2</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none" + d="M 181,214.66098 V 145.33902" + id="path17211-7-1-6" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <g + id="g29167" + transform="matrix(0.73232502,0,0,0.75477602,-218.07919,73.10621)"> + <text + id="text29163" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165" + sodipodi:role="line">8</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.678033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none" + d="m 131,145.8531 v 69.32197" + id="path17211-7-1" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-140.37076,129.97088)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-8" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <g + id="g29167-2" + transform="matrix(0.73232502,0,0,0.75477602,-140.28602,131.01695)"> + <text + id="text29163-92" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-8" + sodipodi:role="line">7</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.718986px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)" + d="m 317.1405,116 h -94.281" + id="path17209-8-0" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-3.4914,66.68745)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-6" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <g + id="g29167-46" + transform="matrix(0.73232502,0,0,0.75477602,-4.40666,67.48829)"> + <text + id="text29163-1" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-5" + sodipodi:role="line">3</tspan></text> + </g> + <path + transform="matrix(0.73232502,0,0,0.75477602,-90.692582,130.31695)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-8-6" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <g + id="g29167-6" + transform="matrix(0.73232502,0,0,0.75477602,-90.84634,131.60918)"> + <text + id="text29163-17" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-2" + sodipodi:role="line">4</tspan></text> + </g> + <g + id="g29167-2-0" + transform="matrix(0.73232502,0,0,0.75477602,-2.424397,194.0216)"> + <text + id="text29163-92-6" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-8-2" + sodipodi:role="line">5</tspan></text> + </g> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-8" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,93.82055,-648.98949)"> + <title + id="title22-7-97">Square</title> + <desc + id="desc24-7-3">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-6" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-12" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-92)" /> + </g> + <g + id="g13515-9"> + <g + id="g13534-3"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-1" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-84" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,314.82055,-648.98949)"> + <title + id="title22-7-50">Square</title> + <desc + id="desc24-7-36">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-1" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-0" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-94)" /> + </g> + <g + id="g13515-6"> + <g + id="g13534-32"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-0" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.712265;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:none;marker-end:url(#Arrow1Lend)" + d="M 313.14387,285 H 228.85613" + id="path17209-7" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-2.692582,236.31695)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-1-6" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <g + id="g29167-2-0-5" + transform="matrix(0.73232502,0,0,0.75477602,-2.424397,237.0216)"> + <text + id="text29163-92-6-6" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-8-2-9" + sodipodi:role="line">6</tspan></text> + </g> + <g + id="g29167-4-3" + transform="matrix(0.73232502,0,0,0.75477602,-154.60784,51.117791)"> + <text + id="text29163-9-6" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-9-7" + sodipodi:role="line">Eventdev</tspan></text> + </g> + <g + id="g29167-4-3-5" + transform="matrix(0.73232502,0,0,0.75477602,-144.65044,201.97821)"> + <text + id="text29163-9-6-3" + y="70" + x="412.93716" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="70" + x="412.93716" + id="tspan29165-9-7-5" + sodipodi:role="line">DMA</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="100.26363" + x="412.93716" + sodipodi:role="line" + id="tspan3201">Adapter</tspan></text> + </g> + <g + id="g29167-4-3-5-6" + transform="matrix(0.73232502,0,0,0.75477602,79.53518,46.62529)"> + <text + id="text29163-9-6-3-2" + y="48.801659" + x="412.93716" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="48.801659" + x="412.93716" + sodipodi:role="line" + id="tspan3155">Application</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="98.801659" + x="412.93716" + sodipodi:role="line" + id="tspan3201-1">in ordered</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="148.80167" + x="412.93716" + sodipodi:role="line" + id="tspan3161">stage</tspan></text> + </g> + <g + id="g29167-4-3-5-2" + transform="matrix(0.73232502,0,0,0.75477602,77.535182,213.62529)"> + <text + id="text29163-9-6-3-7" + y="70" + x="412.93716" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="70" + x="412.93716" + sodipodi:role="line" + id="tspan3201-9">DMA Device</tspan></text> + </g> + <g + id="g29167-4-3-5-3" + transform="matrix(0.73232502,0,0,0.75477602,188.53518,-3.37471)"> + <text + id="text29163-9-6-3-6" + y="70" + x="375.65271" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="375.65271" + sodipodi:role="line" + id="tspan3201-6">1. Events from the previous stage.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="93.538376" + x="375.65271" + sodipodi:role="line" + id="tspan3260"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="117.07675" + x="375.65271" + sodipodi:role="line" + id="tspan3262">2. Application in ordered stage</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="140.61513" + x="375.65271" + sodipodi:role="line" + id="tspan3288"> dequeues events from eventdev.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="164.1535" + x="375.65271" + sodipodi:role="line" + id="tspan3264"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="187.69188" + x="375.65271" + sodipodi:role="line" + id="tspan3266">3. Application enqueues DMA</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="211.23026" + x="375.65271" + sodipodi:role="line" + id="tspan3290"> operations as events to eventdev.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="234.76863" + x="375.65271" + sodipodi:role="line" + id="tspan3268"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="258.30701" + x="375.65271" + sodipodi:role="line" + id="tspan3270">4. DMA adapter dequeues event</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="281.84537" + x="375.65271" + sodipodi:role="line" + id="tspan3292"> from eventdev.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="305.38376" + x="375.65271" + sodipodi:role="line" + id="tspan3272"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="328.92212" + x="375.65271" + sodipodi:role="line" + id="tspan3274">5. DMA adapter submits DMA</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="352.46051" + x="375.65271" + sodipodi:role="line" + id="tspan3294"> operations to DMA Device (Atomic</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="375.99887" + x="375.65271" + sodipodi:role="line" + id="tspan3296"> stage)</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="399.53726" + x="375.65271" + sodipodi:role="line" + id="tspan3276"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="423.07562" + x="375.65271" + sodipodi:role="line" + id="tspan3278">6. DMA adapter dequeues DMA</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="446.61401" + x="375.65271" + sodipodi:role="line" + id="tspan3298"> completions from DMA Device</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="470.15237" + x="375.65271" + sodipodi:role="line" + id="tspan3280"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="493.69073" + x="375.65271" + sodipodi:role="line" + id="tspan3282">7. DMA adapter enqueues events</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="517.22913" + x="375.65271" + sodipodi:role="line" + id="tspan3300"> to the eventdev</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="540.76752" + x="375.65271" + sodipodi:role="line" + id="tspan3284"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18.8307px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="564.30585" + x="375.65271" + sodipodi:role="line" + id="tspan3286">8. Events to the next stage</tspan></text> + </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg new file mode 100644 index 0000000000..e9e8bb2b98 --- /dev/null +++ b/doc/guides/prog_guide/img/event_dma_adapter_op_new.svg @@ -0,0 +1,1079 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright (c) 2023 Marvell. --> +<!-- Created with Inkscape (http://www.inkscape.org/) --> + +<svg + width="720px" + height="486px" + id="svg13237" + version="1.1" + inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)" + sodipodi:docname="event_dma_adapter_op_new.svg" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns="http://www.w3.org/2000/svg" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/"> + <defs + id="defs13239"> + <marker + inkscape:stockid="Arrow1Sstart" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Sstart" + style="overflow:visible"> + <path + id="path8416" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.2) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Send" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Send" + style="overflow:visible;"> + <path + id="path8419" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.2) rotate(180) translate(6,0)" /> + </marker> + <marker + inkscape:stockid="DiamondL" + orient="auto" + refY="0.0" + refX="0.0" + id="DiamondL" + style="overflow:visible"> + <path + id="path8483" + d="M 0,-7.0710768 L -7.0710894,0 L 0,7.0710589 L 7.0710462,0 L 0,-7.0710768 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="DotL" + orient="auto" + refY="0.0" + refX="0.0" + id="DotL" + style="overflow:visible"> + <path + id="path8465" + d="M -2.5,-1.0 C -2.5,1.7600000 -4.7400000,4.0 -7.5,4.0 C -10.260000,4.0 -12.5,1.7600000 -12.5,-1.0 C -12.5,-3.7600000 -10.260000,-6.0 -7.5,-6.0 C -4.7400000,-6.0 -2.5,-3.7600000 -2.5,-1.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8) translate(7.4, 1)" /> + </marker> + <marker + inkscape:stockid="SquareL" + orient="auto" + refY="0.0" + refX="0.0" + id="SquareL" + style="overflow:visible"> + <path + id="path8474" + d="M -5.0,-5.0 L -5.0,5.0 L 5.0,5.0 L 5.0,-5.0 L -5.0,-5.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="TriangleOutL" + orient="auto" + refY="0.0" + refX="0.0" + id="TriangleOutL" + style="overflow:visible"> + <path + id="path8546" + d="M 5.77,0.0 L -2.88,5.0 L -2.88,-5.0 L 5.77,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Lstart" + style="overflow:visible"> + <path + id="path8404" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt" + transform="scale(0.8) translate(12.5,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Mend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Mend" + style="overflow:visible;"> + <path + id="path8413" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.4) rotate(180) translate(10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow2Lend" + style="overflow:visible;"> + <path + id="path8425" + style="fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + transform="scale(1.1) rotate(180) translate(1,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow1Lend" + style="overflow:visible;"> + <path + id="path8407" + d="M 0.0,0.0 L 5.0,-5.0 L -12.5,0.0 L 5.0,5.0 L 0.0,0.0 z " + style="fill-rule:evenodd;stroke:#000000;stroke-width:1.0pt;" + transform="scale(0.8) rotate(180) translate(12.5,0)" /> + </marker> + <filter + id="filter_2" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15" /> + </filter> + <filter + id="filter_2-3" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1" /> + </filter> + <filter + id="filter_2-0" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-7" /> + </filter> + <filter + id="filter_2-0-8" + color-interpolation-filters="sRGB"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-7-7" /> + </filter> + <filter + id="filter_2-3-9" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-6" /> + </filter> + <filter + id="filter_2-3-6" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-63" /> + </filter> + <filter + id="filter_2-3-91" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-3" /> + </filter> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-5" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-3" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-6" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lstart" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lstart-7" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8404-0" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(0.8,0,0,0.8,10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-51" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-1" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-3" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-6" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-62" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-9" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <marker + inkscape:stockid="Arrow1Lend" + orient="auto" + refY="0" + refX="0" + id="Arrow1Lend-2" + style="overflow:visible"> + <path + inkscape:connector-curvature="0" + id="path8407-7" + d="M 0,0 5,-5 -12.5,0 5,5 0,0 z" + style="fill-rule:evenodd;stroke:#000000;stroke-width:1pt" + transform="matrix(-0.8,0,0,-0.8,-10,0)" /> + </marker> + <filter + id="filter_2-3-91-3" + color-interpolation-filters="sRGB" + x="-0.086178862" + y="-0.086178862" + width="1.1723577" + height="1.1723577"> + <feGaussianBlur + stdDeviation="2" + id="feGaussianBlur15-1-3-6" /> + </filter> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1" + inkscape:cx="385.5" + inkscape:cy="234" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="false" + inkscape:window-width="1920" + inkscape:window-height="1017" + inkscape:window-x="-8" + inkscape:window-y="-8" + inkscape:window-maximized="1" + inkscape:snap-nodes="false" + inkscape:showpageshadow="2" + inkscape:pagecheckerboard="0" + inkscape:deskcolor="#d1d1d1"> + <inkscape:grid + type="xygrid" + id="grid13454" /> + </sodipodi:namedview> + <metadata + id="metadata13242"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title /> + </cc:Work> + </rdf:RDF> + </metadata> + <g + id="layer1" + inkscape:label="Layer 1" + inkscape:groupmode="layer"> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-0" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,323.2187,-540.25927)"> + <title + id="title22-7-6">Square</title> + <desc + id="desc24-7-4">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-0" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-9" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-9)" /> + </g> + <g + id="g13515-4"> + <g + id="g13534-5"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-4" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-4" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.165886,88.874699,-447.8809)"> + <title + id="title22-7-5">Square</title> + <desc + id="desc24-7-8">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-5" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-7" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91)" /> + </g> + <g + id="g13515-33"> + <g + id="g13534-8"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-95" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-9" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,88.874699,-538.24651)"> + <title + id="title22-7-9">Square</title> + <desc + id="desc24-7-5">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-2" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-1" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-6)" /> + </g> + <g + id="g13515-3"> + <g + id="g13534-0"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-9" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)" + d="M 220.66064,98.57125 H 321.88592" + id="path17209-8" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.5671361,322.24227,-811.89589)"> + <title + id="title22-7">Square</title> + <desc + id="desc24-7">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3)" /> + </g> + <g + id="g13515"> + <g + id="g13534"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <g + id="g13518" + transform="matrix(0.73232502,0,0,0.75477602,25.29268,348.89752)"> + <g + id="g13526"> + <flowRoot + xml:space="preserve" + id="flowRoot13464-9" + style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + transform="translate(-12.00521,-129.65179)"><flowRegion + id="flowRegion13466-1" + style="font-family:sans-serif"><rect + id="rect13468-2" + width="195.99997" + height="112.00001" + x="273.33334" + y="175.33333" + style="text-align:center;text-anchor:middle" /></flowRegion><flowPara + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:20px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + id="flowPara13511"> </flowPara></flowRoot> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.751455;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart);marker-end:none" + d="m 176.26096,124.64833 v 69.24854" + id="path17209-3" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-4.325033,50.642983)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-3" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.743466px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-mid:none;marker-end:url(#Arrow1Lend)" + d="M 322.61264,375 H 221.38736" + id="path17209-8-0" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,0.0689171,324.80833)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-1" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.629082px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)" + d="M 155,324.19955 V 264.82863" + id="path17211-7-1" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,-116.37076,245.97088)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-8" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.75059;stroke-linecap:butt;stroke-linejoin:miter;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;marker-start:url(#Arrow1Lstart-7);marker-end:none" + d="m 126.26097,124.99178 v 69.24941" + id="path17209-3-0" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" + transform="matrix(0.73232502,0,0,0.75477602,-146.16394,110.68276)" /> + <path + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-2" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" + transform="matrix(0.73232502,0,0,0.75477602,-95.40136,110.71627)" /> + <g + id="g29167-4" + transform="matrix(0.73232502,0,0,0.75477602,-95.31662,112.00756)"> + <text + id="text29163-9" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-9" + sodipodi:role="line">1</tspan></text> + </g> + <g + id="g29167-9" + transform="matrix(0.73232502,0,0,0.75477602,-4.9726112,50.689051)"> + <text + id="text29163-3" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-3" + sodipodi:role="line">2</tspan></text> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:1.04033px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow1Lend)" + d="M 388.20118,147.93341 V 321.89308" + id="path17211-7" + inkscape:connector-type="orthogonal" + inkscape:connector-curvature="0" /> + <path + transform="matrix(0.73232502,0,0,0.75477602,116.5086,136.68745)" + sodipodi:type="arc" + style="fill:#539de6;fill-opacity:1;stroke:#0000ea;stroke-width:1;stroke-linecap:square;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none" + id="path29161-6" + sodipodi:cx="371" + sodipodi:cy="64.5" + sodipodi:rx="17" + sodipodi:ry="15.5" + d="m 388,64.5 a 17,15.5 0 1 1 -34,0 17,15.5 0 1 1 34,0 z" /> + <g + id="g29167-46" + transform="matrix(0.73232502,0,0,0.75477602,116.59334,137.48829)"> + <text + id="text29163-1" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-5" + sodipodi:role="line">3</tspan></text> + </g> + <g + id="g29167-6" + transform="matrix(0.73232502,0,0,0.75477602,0.1536639,325.60918)"> + <text + id="text29163-17" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-2" + sodipodi:role="line">4</tspan></text> + </g> + <g + id="g29167" + transform="matrix(0.73232502,0,0,0.75477602,-146.07919,111.10621)"> + <text + id="text29163" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165" + sodipodi:role="line">6</tspan></text> + </g> + <g + id="g29167-4-3" + transform="matrix(0.73232502,0,0,0.75477602,-117.60784,180.11779)"> + <text + id="text29163-9-6" + y="70" + x="321.30356" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="321.30356" + id="tspan29165-9-7" + sodipodi:role="line">Eventdev</tspan></text> + </g> + <g + id="g29167-4-3-5" + transform="matrix(0.73232502,0,0,0.75477602,55.34956,26.97821)"> + <text + id="text29163-9-6-3" + y="70" + x="454.74152" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="70" + x="454.74152" + id="tspan29165-9-7-5" + sodipodi:role="line">A<tspan + style="line-height:100%;font-family:sans-serif" + id="tspan3374">tomic Stage</tspan></tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="94.210899" + x="454.74152" + sodipodi:role="line" + id="tspan3320">+</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="115.7317" + x="454.74152" + sodipodi:role="line" + id="tspan3322">Enqueue to</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:100%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="137.2525" + x="454.74152" + sodipodi:role="line" + id="tspan3324">DMA Device</tspan></text> + </g> + <g + id="g29167-2" + transform="matrix(0.73232502,0,0,0.75477602,-116.28602,248.01695)"> + <text + id="text29163-92" + y="70" + x="365" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:18px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="365" + id="tspan29165-8" + sodipodi:role="line">5</tspan></text> + </g> + <flowRoot + xml:space="preserve" + id="flowRoot3376" + style="font-style:normal;font-weight:normal;line-height:0.01%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1"><flowRegion + id="flowRegion3378" + style="font-family:sans-serif"><rect + id="rect3380" + width="100" + height="37" + x="109" + y="259" /></flowRegion><flowPara + id="flowPara3382" + style="font-size:18px;line-height:1.25;font-family:sans-serif"> </flowPara></flowRoot> + <g + id="g29167-4-3-1" + transform="matrix(0.73232502,0,0,0.75477602,109.34956,323.97821)"> + <text + id="text29163-9-6-8" + y="70" + x="321.30356" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="321.30356" + id="tspan29165-9-7-7" + sodipodi:role="line">DMA Device</tspan></text> + </g> + <g + id="g29167-4-3-1-9" + transform="matrix(0.73232502,0,0,0.75477602,-114.48565,314.20704)"> + <text + id="text29163-9-6-8-2" + y="70" + x="368.01718" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="70" + x="368.01718" + id="tspan29165-9-7-7-0" + sodipodi:role="line">DMA</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:center;writing-mode:lr-tb;text-anchor:middle" + y="100.26363" + x="368.01718" + sodipodi:role="line" + id="tspan3488">Adapter</tspan></text> + </g> + <g + id="g29167-4-3-1-9-2" + transform="matrix(0.73232502,0,0,0.75477602,250.96804,192.62529)"> + <text + id="text29163-9-6-8-2-3" + y="-188.35481" + x="318.61978" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;text-align:center;letter-spacing:0px;word-spacing:0px;text-anchor:middle;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-188.35481" + x="318.61978" + sodipodi:role="line" + id="tspan3543">1. Application dequeues</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-161.45381" + x="318.61978" + sodipodi:role="line" + id="tspan3196"> events from the previous</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-134.55281" + x="318.61978" + sodipodi:role="line" + id="tspan3232"> stage</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-107.65182" + x="318.61978" + sodipodi:role="line" + id="tspan3519"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-80.750816" + x="318.61978" + sodipodi:role="line" + id="tspan3551">2. Application prepares the</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-53.849815" + x="318.61978" + sodipodi:role="line" + id="tspan3203"> DMA operations.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-26.948814" + x="318.61978" + sodipodi:role="line" + id="tspan3523"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="-0.0478158" + x="318.61978" + sodipodi:role="line" + id="tspan3541">3. DMA operations are</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="26.853184" + x="318.61978" + sodipodi:role="line" + id="tspan3207"> submitted to dmadev</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="53.754185" + x="318.61978" + sodipodi:role="line" + id="tspan3209"> by application.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="80.655182" + x="318.61978" + sodipodi:role="line" + id="tspan3527"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="107.55618" + x="318.61978" + sodipodi:role="line" + id="tspan3547">4. DMA adapter dequeues</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="134.45718" + x="318.61978" + sodipodi:role="line" + id="tspan3216"> DMA completions from</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="161.35818" + x="318.61978" + sodipodi:role="line" + id="tspan3218"> DMA device.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="188.25919" + x="318.61978" + sodipodi:role="line" + id="tspan3531"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="215.16019" + x="318.61978" + sodipodi:role="line" + id="tspan3549">5. DMA adapter enqueues</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="242.06119" + x="318.61978" + sodipodi:role="line" + id="tspan3222"> events to the eventdev.</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="268.96219" + x="318.61978" + sodipodi:role="line" + id="tspan3535"> </tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="295.86319" + x="318.61978" + sodipodi:role="line" + id="tspan3537">6. Application dequeues from</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="322.76419" + x="318.61978" + sodipodi:role="line" + id="tspan3224"> eventdev and prepare for</tspan><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:21.5208px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="349.66519" + x="318.61978" + sodipodi:role="line" + id="tspan3226"> further processing</tspan></text> + </g> + <g + style="font-size:12px;fill:none;stroke-linecap:square;stroke-miterlimit:3;overflow:visible" + id="shape1-1-2-4-7" + v:mID="1" + v:groupContext="shape" + transform="matrix(2.1604167,0,0,1.165886,90.820551,-587.97129)"> + <title + id="title22-7-5-5">Square</title> + <desc + id="desc24-7-8-3">Atomic Queue #1</desc> + <v:userDefs> + <v:ud + v:nameU="visVersion" + v:val="VT0(15):26" /> + </v:userDefs> + <v:textBlock + v:margins="rect(4,4,4,4)" /> + <v:textRect + cx="30.75" + cy="581.25" + width="61.5" + height="61.5" /> + <g + id="shadow1-2-9-5-5" + v:groupContext="shadow" + v:shadowOffsetX="0.345598" + v:shadowOffsetY="-1.97279" + v:shadowType="1" + transform="translate(0.345598,1.97279)" + class="st1" + style="visibility:visible"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st2" + id="rect27-8-7-6" + style="fill:#5b9bd5;fill-opacity:0.22000002;stroke:#5b9bd5;stroke-opacity:0.22000002;filter:url(#filter_2-3-91-3)" /> + </g> + <g + id="g13515-33-2"> + <g + id="g13534-8-9"> + <rect + x="0" + y="550.5" + width="61.5" + height="61.5" + class="st3" + id="rect29-1-95-1" + style="fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25" /> + </g> + </g> + </g> + <g + id="g29167-4-3-2" + transform="matrix(0.73232502,0,0,0.75477602,-125.66199,44.027402)"> + <text + id="text29163-9-6-7" + y="70" + x="321.30356" + style="font-style:normal;font-weight:normal;line-height:0%;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none" + xml:space="preserve"><tspan + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:24.2109px;line-height:125%;font-family:sans-serif;-inkscape-font-specification:Sans;text-align:start;writing-mode:lr-tb;text-anchor:start" + y="70" + x="321.30356" + id="tspan29165-9-7-0" + sodipodi:role="line">Application</tspan></text> + </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 52a6d9e7aa..beaa4b8869 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -60,6 +60,7 @@ Programmer's Guide event_ethernet_tx_adapter event_timer_adapter event_crypto_adapter + event_dma_adapter qos_framework power_man packet_classif_access_ctrl diff --git a/doc/guides/rel_notes/release_23_11.rst b/doc/guides/rel_notes/release_23_11.rst index b34ddc0860..b66c364e21 100644 --- a/doc/guides/rel_notes/release_23_11.rst +++ b/doc/guides/rel_notes/release_23_11.rst @@ -84,6 +84,12 @@ New Features for creating Rx adapter instance for the applications desire to control both the event port allocation and event buffer size. +* **Added event DMA adapter library.** + + * Added the Event DMA Adapter Library. This library extends the event-based + model by introducing APIs that allow applications to enqueue/dequeue DMA + operations to/from dmadev as events scheduled by an event device. + * **Updated Marvell cnxk eventdev driver.** * Added support for ``remaining_ticks_get`` timer adapter PMD callback diff --git a/lib/eventdev/eventdev_pmd.h b/lib/eventdev/eventdev_pmd.h index f62f42e140..f7227c0bfd 100644 --- a/lib/eventdev/eventdev_pmd.h +++ b/lib/eventdev/eventdev_pmd.h @@ -178,8 +178,12 @@ struct rte_eventdev { event_tx_adapter_enqueue_t txa_enqueue; /**< Pointer to PMD eth Tx adapter enqueue function. */ event_crypto_adapter_enqueue_t ca_enqueue; + /**< Pointer to PMD crypto adapter enqueue function. */ - uint64_t reserved_64s[4]; /**< Reserved for future fields */ + event_dma_adapter_enqueue_t dma_enqueue; + /**< Pointer to PMD DMA adapter enqueue function. */ + + uint64_t reserved_64s[3]; /**< Reserved for future fields */ void *reserved_ptrs[3]; /**< Reserved for future fields */ } __rte_cache_aligned; @@ -1320,6 +1324,156 @@ typedef int (*eventdev_eth_tx_adapter_queue_stop) #define eventdev_stop_flush_t rte_eventdev_stop_flush_t +/** + * Retrieve the event device's DMA adapter capabilities for the + * specified DMA device + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param[out] caps + * A pointer to memory filled with event adapter capabilities. + * It is expected to be pre-allocated & initialized by caller. + * + * @return + * - 0: Success, driver provides event adapter capabilities for the + * dmadev. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_caps_get_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, uint32_t *caps); + +/** + * Add DMA vchan queue to event device. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param vchan_id + * dmadev vchan queue identifier. + * + * @param event + * Event information required for binding dmadev vchan to event queue. + * This structure will have a valid value for only those HW PMDs supporting + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND capability. + * + * @return + * - 0: Success, dmadev vchan added successfully. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_vchan_add_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + uint16_t vchan_id, + const struct rte_event *event); + +/** + * Delete DMA vhcan to event device. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(, dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param vchan_id + * dmadev vchan identifier. + * + * @return + * - 0: Success, dmadev vchan deleted successfully. + * - <0: Error code returned by the driver function. + * + */ +typedef int (*eventdev_dma_adapter_vchan_del_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + uint16_t vchan_id); + +/** + * Start DMA adapter. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id + * have been added to the event device. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * - 0: Success, DMA adapter started successfully. + * - <0: Error code returned by the driver function. + */ +typedef int (*eventdev_dma_adapter_start_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + +/** + * Stop DMA adapter. This callback is invoked if + * the caps returned from rte_event_dma_adapter_caps_get(.., dmadev_id) + * has RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_* set and vchan for dmadev_id + * have been added to the event device. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * - 0: Success, DMA adapter stopped successfully. + * - <0: Error code returned by the driver function. + */ +typedef int (*eventdev_dma_adapter_stop_t)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + +struct rte_event_dma_adapter_stats; + +/** + * Retrieve DMA adapter statistics. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @param[out] stats + * Pointer to stats structure + * + * @return + * Return 0 on success. + */ +typedef int (*eventdev_dma_adapter_stats_get)(const struct rte_eventdev *dev, + const int16_t dma_dev_id, + struct rte_event_dma_adapter_stats *stats); + +/** + * Reset DMA adapter statistics. + * + * @param dev + * Event device pointer + * + * @param dma_dev_id + * DMA device identifier + * + * @return + * Return 0 on success. + */ +typedef int (*eventdev_dma_adapter_stats_reset)(const struct rte_eventdev *dev, + const int16_t dma_dev_id); + + /** Event device operations function pointer table */ struct eventdev_ops { eventdev_info_get_t dev_infos_get; /**< Get device info. */ @@ -1440,6 +1594,21 @@ struct eventdev_ops { eventdev_eth_tx_adapter_queue_stop eth_tx_adapter_queue_stop; /**< Stop Tx queue assigned to Tx adapter instance */ + eventdev_dma_adapter_caps_get_t dma_adapter_caps_get; + /**< Get DMA adapter capabilities */ + eventdev_dma_adapter_vchan_add_t dma_adapter_vchan_add; + /**< Add vchan queue to DMA adapter */ + eventdev_dma_adapter_vchan_del_t dma_adapter_vchan_del; + /**< Delete vchan queue from DMA adapter */ + eventdev_dma_adapter_start_t dma_adapter_start; + /**< Start DMA adapter */ + eventdev_dma_adapter_stop_t dma_adapter_stop; + /**< Stop DMA adapter */ + eventdev_dma_adapter_stats_get dma_adapter_stats_get; + /**< Get DMA stats */ + eventdev_dma_adapter_stats_reset dma_adapter_stats_reset; + /**< Reset DMA stats */ + eventdev_selftest dev_selftest; /**< Start eventdev Selftest */ diff --git a/lib/eventdev/eventdev_private.c b/lib/eventdev/eventdev_private.c index 1d3d9d357e..18ed8bf3c8 100644 --- a/lib/eventdev/eventdev_private.c +++ b/lib/eventdev/eventdev_private.c @@ -81,6 +81,14 @@ dummy_event_crypto_adapter_enqueue(__rte_unused void *port, return 0; } +static uint16_t +dummy_event_dma_adapter_enqueue(__rte_unused void *port, __rte_unused struct rte_event ev[], + __rte_unused uint16_t nb_events) +{ + RTE_EDEV_LOG_ERR("event DMA adapter enqueue requested for unconfigured event device"); + return 0; +} + void event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) { @@ -97,6 +105,7 @@ event_dev_fp_ops_reset(struct rte_event_fp_ops *fp_op) .txa_enqueue_same_dest = dummy_event_tx_adapter_enqueue_same_dest, .ca_enqueue = dummy_event_crypto_adapter_enqueue, + .dma_enqueue = dummy_event_dma_adapter_enqueue, .data = dummy_data, }; @@ -117,5 +126,6 @@ event_dev_fp_ops_set(struct rte_event_fp_ops *fp_op, fp_op->txa_enqueue = dev->txa_enqueue; fp_op->txa_enqueue_same_dest = dev->txa_enqueue_same_dest; fp_op->ca_enqueue = dev->ca_enqueue; + fp_op->dma_enqueue = dev->dma_enqueue; fp_op->data = dev->data->ports; } diff --git a/lib/eventdev/meson.build b/lib/eventdev/meson.build index 6edf98dfa5..21347f7c4c 100644 --- a/lib/eventdev/meson.build +++ b/lib/eventdev/meson.build @@ -25,6 +25,7 @@ sources = files( ) headers = files( 'rte_event_crypto_adapter.h', + 'rte_event_dma_adapter.h', 'rte_event_eth_rx_adapter.h', 'rte_event_eth_tx_adapter.h', 'rte_event_ring.h', diff --git a/lib/eventdev/rte_event_dma_adapter.h b/lib/eventdev/rte_event_dma_adapter.h new file mode 100644 index 0000000000..e924ab673d --- /dev/null +++ b/lib/eventdev/rte_event_dma_adapter.h @@ -0,0 +1,581 @@ +/* SPDX-License-Identifier: BSD-3-Clause + * Copyright (c) 2023 Marvell. + */ + +#ifndef RTE_EVENT_DMA_ADAPTER +#define RTE_EVENT_DMA_ADAPTER + +/** + * @file rte_event_dma_adapter.h + * + * @warning + * @b EXPERIMENTAL: + * All functions in this file may be changed or removed without prior notice. + * + * DMA Event Adapter API. + * + * Eventdev library provides adapters to bridge between various components for providing new + * event source. The event DMA adapter is one of those adapters which is intended to bridge + * between event devices and DMA devices. + * + * The DMA adapter adds support to enqueue / dequeue DMA operations to / from event device. The + * packet flow between DMA device and the event device can be accomplished using both SW and HW + * based transfer mechanisms. The adapter uses an EAL service core function for SW based packet + * transfer and uses the eventdev PMD functions to configure HW based packet transfer between the + * DMA device and the event device. + * + * The application can choose to submit a DMA operation directly to an DMA device or send it to the + * DMA adapter via eventdev based on RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. The + * first mode is known as the event new (RTE_EVENT_DMA_ADAPTER_OP_NEW) mode and the second as the + * event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode. The choice of mode can be specified while + * creating the adapter. In the former mode, it is an application responsibility to enable ingress + * packet ordering. In the latter mode, it is the adapter responsibility to enable the ingress + * packet ordering. + * + * + * Working model of RTE_EVENT_DMA_ADAPTER_OP_NEW mode: + * + * +--------------+ +--------------+ + * | | | DMA stage | + * | Application |---[2]-->| + enqueue to | + * | | | dmadev | + * +--------------+ +--------------+ + * ^ ^ | + * | | [3] + * [6] [1] | + * | | | + * +--------------+ | + * | | | + * | Event device | | + * | | | + * +--------------+ | + * ^ | + * | | + * [5] | + * | v + * +--------------+ +--------------+ + * | | | | + * | DMA adapter |<--[4]---| dmadev | + * | | | | + * +--------------+ +--------------+ + * + * + * [1] Application dequeues events from the previous stage. + * [2] Application prepares the DMA operations. + * [3] DMA operations are submitted to dmadev by application. + * [4] DMA adapter dequeues DMA completions from dmadev. + * [5] DMA adapter enqueues events to the eventdev. + * [6] Application dequeues from eventdev for further processing. + * + * In the RTE_EVENT_DMA_ADAPTER_OP_NEW mode, application submits DMA operations directly to DMA + * device. The DMA adapter then dequeues DMA completions from DMA device and enqueue events to the + * event device. This mode does not ensure ingress ordering, if the application directly enqueues + * to dmadev without going through DMA / atomic stage i.e. removing item [1] and [2]. + * + * Events dequeued from the adapter will be treated as new events. In this mode, application needs + * to specify event information (response information) which is needed to enqueue an event after the + * DMA operation is completed. + * + * + * Working model of RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode: + * + * +--------------+ +--------------+ + * --[1]-->| |---[2]-->| Application | + * | Event device | | in | + * <--[8]--| |<--[3]---| Ordered stage| + * +--------------+ +--------------+ + * ^ | + * | [4] + * [7] | + * | v + * +----------------+ +--------------+ + * | |--[5]->| | + * | DMA adapter | | dmadev | + * | |<-[6]--| | + * +----------------+ +--------------+ + * + * + * [1] Events from the previous stage. + * [2] Application in ordered stage dequeues events from eventdev. + * [3] Application enqueues DMA operations as events to eventdev. + * [4] DMA adapter dequeues event from eventdev. + * [5] DMA adapter submits DMA operations to dmadev (Atomic stage). + * [6] DMA adapter dequeues DMA completions from dmadev + * [7] DMA adapter enqueues events to the eventdev + * [8] Events to the next stage + * + * In the event forward (RTE_EVENT_DMA_ADAPTER_OP_FORWARD) mode, if the HW supports the capability + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD, application can directly submit the DMA + * operations to the dmadev. If not, application retrieves the event port of the DMA adapter + * through the API, rte_event_DMA_adapter_event_port_get(). Then, links its event queue to this + * port and starts enqueuing DMA operations as events to the eventdev. The adapter then dequeues + * the events and submits the DMA operations to the dmadev. After the DMA completions, the adapter + * enqueues events to the event device. + * + * Application can use this mode, when ingress packet ordering is needed. Events dequeued from the + * adapter will be treated as forwarded events. In this mode, the application needs to specify the + * dmadev ID and queue pair ID (request information) needed to enqueue an DMA operation in addition + * to the event information (response information) needed to enqueue an event after the DMA + * operation has completed. + * + * The event DMA adapter provides common APIs to configure the packet flow from the DMA device to + * event devices for both SW and HW based transfers. The DMA event adapter's functions are: + * + * - rte_event_dma_adapter_create_ext() + * - rte_event_dma_adapter_create() + * - rte_event_dma_adapter_free() + * - rte_event_dma_adapter_vchan_add() + * - rte_event_dma_adapter_vchan_del() + * - rte_event_dma_adapter_start() + * - rte_event_dma_adapter_stop() + * - rte_event_dma_adapter_stats_get() + * - rte_event_dma_adapter_stats_reset() + * + * The application creates an instance using rte_event_dma_adapter_create() or + * rte_event_dma_adapter_create_ext(). + * + * dmadev queue pair addition / deletion is done using the rte_event_dma_adapter_vchan_add() / + * rte_event_dma_adapter_vchan_del() APIs. If HW supports the capability + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND, event information must be passed to the + * add API. + * + */ + +#include <stdint.h> + +#include <rte_common.h> +#include <rte_dmadev_pmd.h> +#include <rte_eventdev.h> + +#ifdef __cplusplus +extern "C" { +#endif + +/** + * A structure used to hold event based DMA operation entry. All the information + * required for a DMA transfer shall be populated in "struct rte_event_dma_adapter_op" + * instance. + */ +struct rte_event_dma_adapter_op { + struct rte_dma_sge *src_seg; + /**< Source segments. */ + struct rte_dma_sge *dst_seg; + /**< Destination segments. */ + uint16_t nb_src; + /**< Number of source segments. */ + uint16_t nb_dst; + /**< Number of destination segments. */ + uint64_t flags; + /**< Flags related to the operation. + * @see RTE_DMA_OP_FLAG_* + */ + int16_t dma_dev_id; + /**< DMA device ID to be used */ + uint16_t vchan; + /**< DMA vchan ID to be used */ + struct rte_mempool *op_mp; + /**< Mempool from which op is allocated. */ +}; + +/** + * DMA event adapter mode + */ +enum rte_event_dma_adapter_mode { + RTE_EVENT_DMA_ADAPTER_OP_NEW, + /**< Start the DMA adapter in event new mode. + * @see RTE_EVENT_OP_NEW. + * + * Application submits DMA operations to the dmadev. Adapter only dequeues the DMA + * completions from dmadev and enqueue events to the eventdev. + */ + + RTE_EVENT_DMA_ADAPTER_OP_FORWARD, + /**< Start the DMA adapter in event forward mode. + * @see RTE_EVENT_OP_FORWARD. + * + * Application submits DMA requests as events to the DMA adapter or DMA device based on + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability. DMA completions are enqueued + * back to the eventdev by DMA adapter. + */ +}; + +/** + * Adapter configuration structure that the adapter configuration callback function is expected to + * fill out. + * + * @see rte_event_dma_adapter_conf_cb + */ +struct rte_event_dma_adapter_conf { + uint8_t event_port_id; + /** < Event port identifier, the adapter enqueues events to this port and dequeues DMA + * request events in RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. + */ + + uint32_t max_nb; + /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't + * treated as a requirement; batching may cause the adapter to process more than max_nb DMA + * ops. + */ +}; + +/** + * Adapter runtime configuration parameters + */ +struct rte_event_dma_adapter_runtime_params { + uint32_t max_nb; + /**< The adapter can return early if it has processed at least max_nb DMA ops. This isn't + * treated as a requirement; batching may cause the adapter to process more than max_nb DMA + * ops. + * + * Callback function passed to rte_event_dma_adapter_create_ext() configures the adapter + * with default value of max_nb. + * rte_event_dma_adapter_runtime_params_set() allows to re-configure max_nb during runtime + * (after adding at least one queue pair) + * + * This is valid for the devices without RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD or + * RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW capability. + */ + + uint32_t rsvd[15]; + /**< Reserved fields for future expansion */ +}; + +/** + * Function type used for adapter configuration callback. The callback is used to fill in members of + * the struct rte_event_dma_adapter_conf, this callback is invoked when creating a SW service for + * packet transfer from dmadev vchan to the event device. The SW service is created within the + * function, rte_event_dma_adapter_vchan_add(), if SW based packet transfers from dmadev vchan + * to the event device are required. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param conf + * Structure that needs to be populated by this callback. + * @param arg + * Argument to the callback. This is the same as the conf_arg passed to the + * rte_event_dma_adapter_create_ext(). + */ +typedef int (*rte_event_dma_adapter_conf_cb)(uint8_t id, uint8_t evdev_id, + struct rte_event_dma_adapter_conf *conf, void *arg); + +/** + * A structure used to retrieve statistics for an event DMA adapter instance. + */ +struct rte_event_dma_adapter_stats { + uint64_t event_poll_count; + /**< Event port poll count */ + + uint64_t event_deq_count; + /**< Event dequeue count */ + + uint64_t dma_enq_count; + /**< dmadev enqueue count */ + + uint64_t dma_enq_fail_count; + /**< dmadev enqueue failed count */ + + uint64_t dma_deq_count; + /**< dmadev dequeue count */ + + uint64_t event_enq_count; + /**< Event enqueue count */ + + uint64_t event_enq_retry_count; + /**< Event enqueue retry count */ + + uint64_t event_enq_fail_count; + /**< Event enqueue fail count */ +}; + +/** + * Create a new event DMA adapter with the specified identifier. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param conf_cb + * Callback function that fills in members of a struct rte_event_dma_adapter_conf struct passed + * into it. + * @param mode + * Flag to indicate the mode of the adapter. + * @see rte_event_dma_adapter_mode + * @param conf_arg + * Argument that is passed to the conf_cb function. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_create_ext(uint8_t id, uint8_t evdev_id, + rte_event_dma_adapter_conf_cb conf_cb, + enum rte_event_dma_adapter_mode mode, void *conf_arg); + +/** + * Create a new event DMA adapter with the specified identifier. This function uses an internal + * configuration function that creates an event port. This default function reconfigures the event + * device with an additional event port and set up the event port using the port_config parameter + * passed into this function. In case the application needs more control in configuration of the + * service, it should use the rte_event_dma_adapter_create_ext() version. + * + * @param id + * Adapter identifier. + * @param evdev_id + * Event device identifier. + * @param port_config + * Argument of type *rte_event_port_conf* that is passed to the conf_cb function. + * @param mode + * Flag to indicate the mode of the adapter. + * @see rte_event_dma_adapter_mode + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_create(uint8_t id, uint8_t evdev_id, + struct rte_event_port_conf *port_config, + enum rte_event_dma_adapter_mode mode); + +/** + * Free an event DMA adapter + * + * @param id + * Adapter identifier. + * @return + * - 0: Success + * - <0: Error code on failure, If the adapter still has queue pairs added to it, the function + * returns -EBUSY. + */ +__rte_experimental +int rte_event_dma_adapter_free(uint8_t id); + +/** + * Retrieve the event port of an adapter. + * + * @param id + * Adapter identifier. + * + * @param [out] event_port_id + * Application links its event queue to this adapter port which is used in + * RTE_EVENT_DMA_ADAPTER_OP_FORWARD mode. + * + * @return + * - 0: Success + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_event_port_get(uint8_t id, uint8_t *event_port_id); + +/** + * Add a vchan to an event DMA adapter. + * + * @param id + * Adapter identifier. + * @param dmadev_id + * dmadev identifier. + * @param vchan + * DMA device vchan identifier. If vchan is set -1, adapter adds all the + * preconfigured vchan to the instance. + * @param event + * If HW supports dmadev vchan to event queue binding, application is expected to fill in + * event information, else it will be NULL. + * @see RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND + * + * @return + * - 0: Success, vchan added correctly. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_vchan_add(uint8_t id, int16_t dmadev_id, uint16_t vchan, + const struct rte_event *event); + +/** + * Delete a vchan from an event DMA adapter. + * + * @param id + * Adapter identifier. + * @param dmadev_id + * DMA device identifier. + * @param vchan + * DMA device vchan identifier. + * + * @return + * - 0: Success, vchan deleted successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_vchan_del(uint8_t id, int16_t dmadev_id, uint16_t vchan); + +/** + * Retrieve the service ID of an adapter. If the adapter doesn't use a rte_service function, this + * function returns -ESRCH. + * + * @param id + * Adapter identifier. + * @param [out] service_id + * A pointer to a uint32_t, to be filled in with the service id. + * + * @return + * - 0: Success + * - <0: Error code on failure, if the adapter doesn't use a rte_service function, this function + * returns -ESRCH. + */ +__rte_experimental +int rte_event_dma_adapter_service_id_get(uint8_t id, uint32_t *service_id); + +/** + * Start event DMA adapter + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, adapter started successfully. + * - <0: Error code on failure. + * + * @note The eventdev and dmadev to which the event_dma_adapter is connected should be started + * before calling rte_event_dma_adapter_start(). + */ +__rte_experimental +int rte_event_dma_adapter_start(uint8_t id); + +/** + * Stop event DMA adapter + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, adapter stopped successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stop(uint8_t id); + +/** + * Initialize the adapter runtime configuration parameters + * + * @param params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_init(struct rte_event_dma_adapter_runtime_params *params); + +/** + * Set the adapter runtime configuration parameters + * + * @param id + * Adapter identifier + * + * @param params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params with configuration + * parameter values. The reserved fields of this structure must be initialized to zero and the valid + * fields need to be set appropriately. This struct can be initialized using + * rte_event_dma_adapter_runtime_params_init() API to default values or application may reset this + * struct and update required fields. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_set(uint8_t id, + struct rte_event_dma_adapter_runtime_params *params); + +/** + * Get the adapter runtime configuration parameters + * + * @param id + * Adapter identifier + * + * @param[out] params + * A pointer to structure of type struct rte_event_dma_adapter_runtime_params containing valid + * adapter parameters when return value is 0. + * + * @return + * - 0: Success + * - <0: Error code on failure + */ +__rte_experimental +int rte_event_dma_adapter_runtime_params_get(uint8_t id, + struct rte_event_dma_adapter_runtime_params *params); + +/** + * Retrieve statistics for an adapter + * + * @param id + * Adapter identifier. + * @param [out] stats + * A pointer to structure used to retrieve statistics for an adapter. + * + * @return + * - 0: Success, retrieved successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stats_get(uint8_t id, struct rte_event_dma_adapter_stats *stats); + +/** + * Reset statistics for an adapter. + * + * @param id + * Adapter identifier. + * + * @return + * - 0: Success, statistics reset successfully. + * - <0: Error code on failure. + */ +__rte_experimental +int rte_event_dma_adapter_stats_reset(uint8_t id); + +/** + * Enqueue a burst of DMA operations as event objects supplied in *rte_event* structure on an event + * DMA adapter designated by its event *evdev_id* through the event port specified by *port_id*. + * This function is supported if the eventdev PMD has the + * #RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD capability flag set. + * + * The *nb_events* parameter is the number of event objects to enqueue that are supplied in the + * *ev* array of *rte_event* structure. + * + * The rte_event_dma_adapter_enqueue() function returns the number of event objects it actually + * enqueued. A return value equal to *nb_events* means that all event objects have been enqueued. + * + * @param evdev_id + * The identifier of the device. + * @param port_id + * The identifier of the event port. + * @param ev + * Points to an array of *nb_events* objects of type *rte_event* structure which contain the + * event object enqueue operations to be processed. + * @param nb_events + * The number of event objects to enqueue, typically number of + * rte_event_port_attr_get(...RTE_EVENT_PORT_ATTR_ENQ_DEPTH...) available for this port. + * + * @return + * The number of event objects actually enqueued on the event device. The return value can be + * less than the value of the *nb_events* parameter when the event devices queue is full or if + * invalid parameters are specified in a *rte_event*. If the return value is less than *nb_events*, + * the remaining events at the end of ev[] are not consumed and the caller has to take care of them, + * and rte_errno is set accordingly. Possible errno values include: + * - EINVAL: The port ID is invalid, device ID is invalid, an event's queue ID is invalid, or an + * event's sched type doesn't match the capabilities of the destination queue. + * - ENOSPC: The event port was backpressured and unable to enqueue one or more events. This + * error code is only applicable to closed systems. + */ +__rte_experimental +uint16_t rte_event_dma_adapter_enqueue(uint8_t evdev_id, uint8_t port_id, struct rte_event ev[], + uint16_t nb_events); + +#ifdef __cplusplus +} +#endif + +#endif /* RTE_EVENT_DMA_ADAPTER */ diff --git a/lib/eventdev/rte_eventdev.h b/lib/eventdev/rte_eventdev.h index 2ba8a7b090..41743f91b1 100644 --- a/lib/eventdev/rte_eventdev.h +++ b/lib/eventdev/rte_eventdev.h @@ -1197,6 +1197,8 @@ struct rte_event_vector { */ #define RTE_EVENT_TYPE_ETH_RX_ADAPTER 0x4 /**< The event generated from event eth Rx adapter */ +#define RTE_EVENT_TYPE_DMADEV 0x5 +/**< The event generated from dma subsystem */ #define RTE_EVENT_TYPE_VECTOR 0x8 /**< Indicates that event is a vector. * All vector event types should be a logical OR of EVENT_TYPE_VECTOR. @@ -1462,6 +1464,48 @@ int rte_event_crypto_adapter_caps_get(uint8_t dev_id, uint8_t cdev_id, uint32_t *caps); +/* DMA adapter capability bitmap flag */ +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_NEW 0x1 +/**< Flag indicates HW is capable of generating events in + * RTE_EVENT_OP_NEW enqueue operation. DMADEV will send + * packets to the event device as new events using an + * internal event port. + */ + +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_OP_FWD 0x2 +/**< Flag indicates HW is capable of generating events in + * RTE_EVENT_OP_FORWARD enqueue operation. DMADEV will send + * packets to the event device as forwarded event using an + * internal event port. + */ + +#define RTE_EVENT_DMA_ADAPTER_CAP_INTERNAL_PORT_VCHAN_EV_BIND 0x4 +/**< Flag indicates HW is capable of mapping DMA vchan to event queue. */ + +/** + * Retrieve the event device's DMA adapter capabilities for the + * specified dmadev device + * + * @param dev_id + * The identifier of the device. + * + * @param dmadev_id + * The identifier of the dmadev device. + * + * @param[out] caps + * A pointer to memory filled with event adapter capabilities. + * It is expected to be pre-allocated & initialized by caller. + * + * @return + * - 0: Success, driver provides event adapter capabilities for the + * dmadev device. + * - <0: Error code returned by the driver function. + * + */ +__rte_experimental +int +rte_event_dma_adapter_caps_get(uint8_t dev_id, uint8_t dmadev_id, uint32_t *caps); + /* Ethdev Tx adapter capability bitmap flags */ #define RTE_EVENT_ETH_TX_ADAPTER_CAP_INTERNAL_PORT 0x1 /**< This flag is sent when the PMD supports a packet transmit callback diff --git a/lib/eventdev/rte_eventdev_core.h b/lib/eventdev/rte_eventdev_core.h index c27a52ccc0..83e8736c71 100644 --- a/lib/eventdev/rte_eventdev_core.h +++ b/lib/eventdev/rte_eventdev_core.h @@ -42,6 +42,10 @@ typedef uint16_t (*event_crypto_adapter_enqueue_t)(void *port, uint16_t nb_events); /**< @internal Enqueue burst of events on crypto adapter */ +typedef uint16_t (*event_dma_adapter_enqueue_t)(void *port, struct rte_event ev[], + uint16_t nb_events); +/**< @internal Enqueue burst of events on DMA adapter */ + struct rte_event_fp_ops { void **data; /**< points to array of internal port data pointers */ @@ -65,7 +69,9 @@ struct rte_event_fp_ops { /**< PMD Tx adapter enqueue same destination function. */ event_crypto_adapter_enqueue_t ca_enqueue; /**< PMD Crypto adapter enqueue function. */ - uintptr_t reserved[5]; + event_dma_adapter_enqueue_t dma_enqueue; + /**< PMD DMA adapter enqueue function. */ + uintptr_t reserved[4]; } __rte_cache_aligned; extern struct rte_event_fp_ops rte_event_fp_ops[RTE_EVENT_MAX_DEVS]; diff --git a/lib/eventdev/version.map b/lib/eventdev/version.map index 7ce09a87bb..b81eb2919c 100644 --- a/lib/eventdev/version.map +++ b/lib/eventdev/version.map @@ -133,6 +133,22 @@ EXPERIMENTAL { rte_event_timer_remaining_ticks_get; # added in 23.11 + rte_event_dma_adapter_caps_get; + rte_event_dma_adapter_create; + rte_event_dma_adapter_create_ext; + rte_event_dma_adapter_enqueue; + rte_event_dma_adapter_event_port_get; + rte_event_dma_adapter_free; + rte_event_dma_adapter_runtime_params_get; + rte_event_dma_adapter_runtime_params_init; + rte_event_dma_adapter_runtime_params_set; + rte_event_dma_adapter_service_id_get; + rte_event_dma_adapter_start; + rte_event_dma_adapter_stats_get; + rte_event_dma_adapter_stats_reset; + rte_event_dma_adapter_stop; + rte_event_dma_adapter_vchan_add; + rte_event_dma_adapter_vchan_del; rte_event_eth_rx_adapter_create_ext_with_params; }; diff --git a/lib/meson.build b/lib/meson.build index 53155be8e9..8330d90820 100644 --- a/lib/meson.build +++ b/lib/meson.build @@ -33,6 +33,7 @@ libraries = [ 'compressdev', 'cryptodev', 'distributor', + 'dmadev', # eventdev depends on this 'efd', 'eventdev', 'gpudev', @@ -48,7 +49,6 @@ libraries = [ 'rawdev', 'regexdev', 'mldev', - 'dmadev', 'rib', 'reorder', 'sched', -- 2.25.1