HI,
We are using OVS with the DPDK datapath and leveraging the OVS hardware offload
feature with Marvell CN10K accelerators to achieve high performance.
However, we are encountering a system crash whenever a non-PMD thread (carrying
an invalid rte_lcore_id) attempts to transmit a packet, in scenarios like
ARP resolution and managing bridge traffic, non-PMD threads (e.g., the main
thread) flush some packets out using dp_netdev_pmd_flush_output_packets().
CN10K PMD TX routine requires each worker thread to write packets in its own
dedicated region and hit the corresponding doorbell to ensure synchronization.
To avoid conditional checks in the TX fast path, base address of these regions
is calculated based on lcore_id.
When a thread with an invalid lcore_id attempts to transmit, it calculates an
incorrect base address and triggers the doorbell, resulting in a crash when the
hardware
accesses this invalid address.
To mitigate this crash we came up with a workaround where a non-pmd thread
registers for a valid lcore_id (using rte_thread_register()) before
transmission
and deregister back (rte_thread_unregister()) on completion, as shown in the
following patch:
diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
index 99ff9b369..6edc5eb29 100644
--- a/lib/dpif-netdev.c
+++ b/lib/dpif-netdev.c
@@ -5599,6 +5599,7 @@ dp_netdev_pmd_flush_output_packets(struct
dp_netdev_pmd_thread *pmd,
{
struct tx_port *p;
int output_cnt = 0;
+ bool dpdk_attached;
if (!pmd->n_output_batches) {
return 0;
@@ -5607,7 +5608,11 @@ dp_netdev_pmd_flush_output_packets(struct
dp_netdev_pmd_thread *pmd,
HMAP_FOR_EACH (p, node, &pmd->send_port_cache) {
if (!dp_packet_batch_is_empty(&p->output_pkts)
&& (force || pmd->ctx.now >= p->flush_time)) {
+ if (pmd->core_id == NON_PMD_CORE_ID)
+ dpdk_attached = dpdk_attach_thread(0);
output_cnt +=
dp_netdev_pmd_flush_output_on_port(pmd, p);
+ if (dpdk_attached && pmd->core_id == NON_PMD_CORE_ID)
+ dpdk_detach_thread();
}
}
return output_cnt;
Is the above change acceptable in OVS? Could you please suggest a solution for
our problem? We strictly want to avoid any condition checks in our TX fast path
routine.
Thanks
Harman
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
