On Wed 24 Jun 2020 at 13:30, Pablo Neira Ayuso <pa...@netfilter.org> wrote: > On Wed, Jun 24, 2020 at 01:22:29PM +0300, Vlad Buslov wrote: >> Hi Pablo, >> >> I've encountered a new issue with indirect offloads infrastructure. The >> issue is that on driver offload its indirect callbacks are not removed >> from blocks and any following offloads operations on block that has such >> callback in its offloads cb list causes call to unmapped address. >> >> Steps to reproduce: >> >> echo 1 >/sys/class/net/ens1f0/device/sriov_numvfs >> echo 0000:81:00.2 > /sys/bus/pci/drivers/mlx5_core/unbind >> devlink dev eswitch set pci/0000:81:00.0 mode switchdev >> >> ip link add vxlan1 type vxlan dstport 4789 external >> ip addr add 192.168.1.1 dev ens1f0 >> link set up dev ens1f0 >> ip link set up dev ens1f0 >> tc qdisc add dev vxlan1 ingress >> tc filter add dev vxlan1 protocol ip ingress flower enc_src_ip 192.168.1.2 >> enc_dst_ip 192.168.1.1 enc_key_id 42 enc_dst_port 4789 action tunnel_key >> unset action mirred egress redirect dev ens1f0_0 >> tc -s filter show dev vxlan1 ingress >> >> rmmod mlx5_ib >> rmmod mlx5_core >> tc -s filter show dev vxlan1 ingress > > On module removal, the representors are gone and the ->cleanup > callback should be exercised, this callback removes the flow_block and > removes the rules in the driver. > > Can you check if the ->cleanup callback is exercised?
I added some traces. On module unload mlx5e_cleanup_rep_tx() and flow_indr_dev_unregister() are called, but not tc_block_indr_cleanup(). Maybe this is the problem that wenxu fixed in one of his patches? I'll try to reproduce on net branch. > >> Resulting dmesg: >> >> [ 153.747853] BUG: unable to handle page fault for address: ffffffffc114cee0 >> [ 153.747975] #PF: supervisor instruction fetch in kernel mode >> [ 153.748071] #PF: error_code(0x0010) - not-present page >> [ 153.748189] PGD 5b6c12067 P4D 5b6c12067 PUD 5b6c14067 PMD 35b76b067 PTE 0 >> [ 153.748328] Oops: 0010 [#1] SMP KASAN PTI >> [ 153.748403] CPU: 1 PID: 1909 Comm: tc Not tainted 5.8.0-rc1+ #1170 >> [ 153.748507] Hardware name: Supermicro SYS-2028TP-DECR/X10DRT-P, BIOS 2.0b >> 03/30/2017 >> [ 153.748638] RIP: 0010:0xffffffffc114cee0 >> [ 153.748709] Code: Bad RIP value. >> [ 153.748767] RSP: 0018:ffff88834895ef00 EFLAGS: 00010246 >> [ 153.748858] RAX: 0000000000000000 RBX: ffff888330a30078 RCX: >> ffffffffb2da70ba >> [ 153.748975] RDX: ffff888333635d80 RSI: ffff88834895efa0 RDI: >> 0000000000000002 >> [ 153.752948] RBP: 0000000000000002 R08: 0000000000000001 R09: >> ffffed106614600c >> [ 153.759173] R10: ffff888330a3005f R11: ffffed106614600b R12: >> ffff88834895efa0 >> [ 153.765419] R13: 0000000000000000 R14: ffffffffc114cee0 R15: >> ffff8883470efe00 >> [ 153.771689] FS: 00007f6f6ac12480(0000) GS:ffff888362e40000(0000) >> knlGS:0000000000000000 >> [ 153.777983] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 >> [ 153.784187] CR2: ffffffffc114ceb6 CR3: 000000035eb9e005 CR4: >> 00000000001606e0 >> [ 153.790567] Call Trace: >> [ 153.796844] ? tc_setup_cb_call+0xd8/0x170 >> [ 153.803164] ? fl_hw_update_stats+0x117/0x280 [cls_flower] >> [ 153.809516] ? 0xffffffffc1328000 >> [ 153.815766] ? _find_next_bit.constprop.0+0x3e/0xf0 >> [ 153.822079] ? __nla_reserve+0x4c/0x60 > [...] >> >> I can come up with something to fix mlx5 but it looks like all other >> drivers that support indirect devices are also susceptible to similar >> issue. > > How does the fix you have in mind look like? To call flow_indr_dev_unregister() from right path. But you are right, it is already called, so we just need to determine why it doesn't perform the proper cleanup. > > Thanks.
