Hi, I am currently playing with SYNPROXY target to optimize SYN filtering performance and by occasion found that TCP SYN packets containing port 0 can result in a soft lockup when conntrack is enabled just by itself, given high packet ratio (I`ve reached 450kpps so far with 60b packets on a /32<->/32 flood with enabled flow control at the media level and middle-level E3 Xeon on receiver side). Same flood with port > 0 going just well, producing same ceil numbers but without visible lockups in kernel log. I`ve tested the issue on a broad range of 3.x kernels and all of them are seemingly affected. Fast and dirty grep revealed special conditions for port 0 only for protocol-specific helpers, but there are none of them.
Please find both same captures and traceback below.
[ 152.001957] ixgbe 0000:01:00.0 eth8: NIC Link is Up 10 Gbps, Flow Control: RX/TX [ 157.326410] sched: RT throttling activated [ 180.038105] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [rcuos/0:9] [ 180.038128] Modules linked in: xt_CT iptable_raw ipt_SYNPROXY nf_synproxy_core nf_conntrack_ipv4 nf_defrag_ipv4 xt_tcpudp xt_conntrack nf_conntrack iptable_filter ip_tables x_tables tun openvswitch(O) libcrc32c nfsd auth_rpcgss nfs_acl nfs lockd grace fscache sunrpc bridge stp llc w83627ehf hwmon_vid jc42 loop fuse dm_crypt joydev hid_generic usbhid x86_pkg_temp_thermal intel_powerclamp ast coretemp igb ttm drm_kms_helper kvm_intel(O) drm iTCO_wdt iTCO_vendor_support sg pcspkr kvm(O) syscopyarea sysfillrect sysimgblt video thermal tpm_tis i2c_algo_bit ipmi_si ipmi_msghandler tpm i2c_i801 8250_fintek fan battery shpchp button ie31200_edac edac_core xhci_pci lpc_ich mfd_core xhci_hcd processor crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper [ 180.038157] ablk_helper cryptd mpt2sas raid_class ehci_pci ixgbe(O) ehci_hcd vxlan ip6_udp_tunnel udp_tunnel usbcore ptp usb_common pps_core dca dm_mirror dm_region_hash dm_log dm_mod [ 180.038165] CPU: 0 PID: 9 Comm: rcuos/0 Tainted: G O 3.18.10-default #5 [ 180.038166] Hardware name: Supermicro X10SL7-F/X10SL7-F, BIOS 2.00 04/24/2014 [ 180.038167] task: ffff880409624c20 ti: ffff880409630000 task.ti: ffff880409630000 [ 180.038168] RIP: 0010:[<ffffffff8155b972>] [<ffffffff8155b972>] dev_gro_receive+0x182/0x370 [ 180.038172] RSP: 0018:ffff88041fc03d48 EFLAGS: 00000296 [ 180.038173] RAX: ffffffff81cf6920 RBX: 0000000180200020 RCX: 0000000067632533 [ 180.038174] RDX: ffff880405580000 RSI: ffff8800d8392100 RDI: ffff8804054dc440 [ 180.038175] RBP: ffff88041fc03d98 R08: 000000000000002e R09: 0000000000000000 [ 180.038175] R10: ffff8800d8392100 R11: ffffea001019de00 R12: ffff88041fc03cb8 [ 180.038176] R13: ffffffff8165b83d R14: ffff88041fc03d98 R15: ffff8800d8392100 [ 180.038177] FS: 0000000000000000(0000) GS:ffff88041fc00000(0000) knlGS:0000000000000000 [ 180.038178] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 180.038179] CR2: 00007fed089d21b0 CR3: 0000000001c15000 CR4: 00000000001407f0 [ 180.038180] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 180.038180] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 180.038181] Stack: [ 180.038181] 0000000000000280 000000000000000e 000000000000000b ffffffff81cf8860 [ 180.038183] ffff88041fc03d98 ffff8804054dc640 ffff8800d8392100 ffff8804054dc440 [ 180.038184] 000000000000000c ffff880407023fb0 ffff88041fc03dc8 ffffffff8155c0d0 [ 180.038185] Call Trace: [ 180.038186] <IRQ> [ 180.038189] [<ffffffff8155c0d0>] napi_gro_receive+0x30/0x100 [ 180.038196] [<ffffffffa012cc39>] ixgbe_clean_rx_irq+0x8d9/0x1030 [ixgbe] [ 180.038200] [<ffffffffa012e588>] ixgbe_poll+0x478/0x690 [ixgbe] [ 180.038203] [<ffffffff8150da90>] ? show_no_turbo+0x90/0x90 [ 180.038204] [<ffffffff8155bd99>] net_rx_action+0x149/0x250 [ 180.038208] [<ffffffff8106629f>] __do_softirq+0xdf/0x260 [ 180.038210] [<ffffffff8165c4fc>] do_softirq_own_stack+0x1c/0x30 [ 180.038211] <EOI> [ 180.038213] [<ffffffff810664d5>] do_softirq+0x65/0x70 [ 180.038214] [<ffffffff81066574>] __local_bh_enable_ip+0x94/0xa0 [ 180.038216] [<ffffffff810bfa15>] rcu_nocb_kthread+0x155/0x580 [ 180.038219] [<ffffffff8109fcc0>] ? finish_wait+0x80/0x80 [ 180.038220] [<ffffffff810bf8c0>] ? rcu_eqs_exit_common.isra.60+0xe0/0xe0 [ 180.038222] [<ffffffff8107fad9>] kthread+0xc9/0xe0 [ 180.038224] [<ffffffff8107fa10>] ? kthread_create_on_node+0x1a0/0x1a0 [ 180.038226] [<ffffffff8165a798>] ret_from_fork+0x58/0x90 [ 180.038227] [<ffffffff8107fa10>] ? kthread_create_on_node+0x1a0/0x1a0 [ 180.038228] Code: 00 48 8b 05 91 8a 79 00 48 89 45 c8 4c 8b 75 c8 49 81 fe e0 43 cf 81 49 8d 46 e0 0f 84 b6 01 00 00 66 44 39 28 74 0a 48 8b 40 20 <eb> db 0f 1f 40 00 48 83 78 10 00 74 ef 48 8b 93 d8 00 00 00 48
0-80.pcap
Description: application/vnd.tcpdump.pcap
80-80.pcap
Description: application/vnd.tcpdump.pcap
