The ThunderX CN88XX NIC Virtual Function driver uses mailbox interface to communicate to physical function driver. Each of VF has it's own pair of mailbox registers to read from and write to. The mailbox registers has no protection from possible races, so it has to be implemented at software side.
After long term testing by loop of 'ip link set <ifname> up/down' command it was found that there are two possible scenarios when race condition appears: 1. VF receives link change message from PF and VF send RX mode configuration message to PF in the same time from separate thread. 2. PF receives RX mode configuration from VF and in the same time, in separate thread PF detects link status change and sends appropriate message to particular VF. Both cases leads to mailbox data to be rewritten, NIC VF messaging control data to be updated incorrectly and communication sequence gets broken. This patch series is to address race condition with VF & PF communication. Vadim Lomovtsev (8): net: thunderx: correct typo in macro name net: thunderx: replace global nicvf_rx_mode_wq work queue for all VFs to private for each of them. net: thunderx: make CFG_DONE message to run through generic send-ack sequence net: thunderx: add nicvf_send_msg_to_pf result check for set_rx_mode_task net: thunderx: rework xcast message structure to make it fit into 64 bit net: thunderx: add mutex to protect mailbox from concurrent calls for same VF net: thunderx: implement helpers to read mailbox IRQ status net: thunderx: check status of mailbox IRQ before sending a message drivers/net/ethernet/cavium/thunder/nic.h | 12 +-- .../net/ethernet/cavium/thunder/nic_main.c | 58 +++++++------ .../net/ethernet/cavium/thunder/nicvf_main.c | 82 +++++++++++++------ .../ethernet/cavium/thunder/nicvf_queues.c | 14 ++++ .../ethernet/cavium/thunder/nicvf_queues.h | 1 + .../net/ethernet/cavium/thunder/thunder_bgx.c | 2 +- .../net/ethernet/cavium/thunder/thunder_bgx.h | 2 +- 7 files changed, 112 insertions(+), 59 deletions(-) -- 2.17.2