Hi Alfredo,
I am very grateful to your kind reply. I deployed the latest version in
a testing environment yesterday; I will tell you more details if this
problem is reproduced.
Forget to say, in order to change the parameters, I complied the
kernel module of PF_RING and libpcap library from source code. Is my action
the cause of this problem?
Here is log when PF_RING load kernel module:
pf_ring: module verification failed: signature and/or required key missing
- tainting kernel.
Besides, I wonder whether this problem is an underlying BUG in the
kernel module of PF_RING or there is something wrong with my application
calling PF_RING’s kernel module. If there is something wrong with my
application method, I need to check my logic of application program. Now I
am very confused about this problem.
Looking forward to your reply. Thank you.
2017-10-30 15:53 GMT+08:00 Alfredo Cardigliano <[email protected]>:
> Hi Bowen Li
> any chance you can move to latest stable and check if you are still able
> to reproduce it?
>
> Thank you
> Alfredo
>
> On 30 Oct 2017, at 08:15, Bowen Li <[email protected]> wrote:
>
> Hi all,
> Recently，I run bro cluster with PF_RING 6.4.1 as the package
> capturing framework. However, I found a strange scene: sometimes when the
> cluster restarts, the kernel module of PF_RING will block. The problem
> seems to generate when operates /proc to apply read-write lock, causing
> kernel CPU soft lockup and then the whole server is stuck. I want to know
> whether the deadlock is caused by the resource competition when the cluster
> progress applies read-write lock at the same time, or other reasons.
> Here is the log in /var/log/message:
>
> Oct 22 01:26:10 TEST kernel: BUG: soft lockup - CPU#16 stuck for 22s!
> [bro:17375]
> Oct 22 01:26:10 TEST kernel: Modules linked in: binfmt_misc xt_CHECKSUM
> iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat
> nf_nat_ipv4 nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack
> nf_conntrack ipt_REJECT tun bridge stp llc ebtable_filter ebtables
> ip6table_filter ip6_tables iptable_filter intel_powerclamp coretemp
> intel_rapl kvm_intel iTCO_wdt kvm mei_me iTCO_vendor_support mxm_wmi mei
> lpc_ich ipmi_ssif sb_edac crc32_pclmul ghash_clmulni_intel aesni_intel lrw
> gf128mul glue_helper ablk_helper edac_core ipmi_si cryptd ioTESTma
> ipmi_msghandler shpchp wmi sg dca i2c_i801 pcspkr mfd_core nfsd auth_rpcgss
> nfs_acl lockd grace pf_ring(OE) sunrpc ip_tables ext4 mbcache jbd2 sd_mod
> crc_t10dif crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt
> i2c_algo_bit drm_kms_helper ttm crct10dif_pclmul crct10dif_common
> Oct 22 01:26:10 TEST kernel: crc32c_intel drm isci e1000e libsas ahci
> scsi_transport_sas libahci ptp i2c_core libata pps_core ntb
> Oct 22 01:26:10 TEST kernel: CPU: 16 PID: 17375 Comm: bro Tainted: G
> OE ------------ 3.10.0-327.13.1.el7.x86_64 #1
> Oct 22 01:26:10 TEST kernel: Hardware name: Supermicro
> X9DRL-3F/iF/X9DRL-3F/iF, BIOS 3.2 09/22/2015
> Oct 22 01:26:10 TEST kernel: task: ffff880fdad73980 ti: ffff8800287b8000
> task.ti: ffff8800287b8000
> Oct 22 01:26:10 TEST kernel: RIP: 0010:[<ffffffff81301909>]
> [<ffffffff81301909>] __write_lock_failed+0x9/0x20
> Oct 22 01:26:10 TEST kernel: RSP: 0018:ffff8800287bbe88 EFLAGS: 00000297
> Oct 22 01:26:10 TEST kernel: RAX: 0000000000000000 RBX: 0000000092e79345
> RCX: 0000000000000000
> Oct 22 01:26:10 TEST kernel: RDX: 0000000000000000 RSI: ffff88104aabb000
> RDI: ffffffffa03c6324
> Oct 22 01:26:10 TEST kernel: RBP: ffff8800287bbe88 R08: 0000000000017600
> R09: 0000000000000080
> Oct 22 01:26:10 TEST kernel: R10: 0000000000000000 R11: 000000000000001b
> R12: 00007f21f432e000
> Oct 22 01:26:10 TEST kernel: R13: 0000000000000032 R14: ffff8800000000a8
> R15: 0000000000000000
> Oct 22 01:26:10 TEST kernel: FS: 00007f21f4e56880(0000)
> GS:ffff88085fd00000(0000) knlGS:0000000000000000
> Oct 22 01:26:10 TEST kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
> 0000000080050033
> Oct 22 01:26:10 TEST kernel: CR2: 00007f21f432eee0 CR3: 0000000648323000
> CR4: 00000000000407e0
> Oct 22 01:26:10 TEST kernel: DR0: 0000000000000000 DR1: 0000000000000000
> DR2: 0000000000000000
> Oct 22 01:26:10 TEST kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
> DR7: 0000000000000400
> Oct 22 01:26:10 TEST kernel: Stack:
> Oct 22 01:26:10 TEST kernel: ffff8800287bbe98 ffffffff8163ce37
> ffff8800287bbeb8 ffffffffa03afe1b
> Oct 22 01:26:10 TEST kernel: ffff88104aabb000 ffff88101a631800
> ffff8800287bbee8 ffffffffa03b6903
> Oct 22 01:26:10 TEST kernel: 0000000000000003 0000000000000300
> 0000000000000000 ffffffff81a25e00
> Oct 22 01:26:10 TEST kernel: Call Trace:
> Oct 22 01:26:10 TEST kernel: [<ffffffff8163ce37>] _raw_write_lock+0x17/0x20
> Oct 22 01:26:10 TEST kernel: [<ffffffffa03afe1b>] ring_proc_add+0x1b/0xb0
> [pf_ring]
> Oct 22 01:26:10 TEST kernel: [<ffffffffa03b6903>] ring_create+0x2e3/0x4a0
> [pf_ring]
> Oct 22 01:26:10 TEST kernel: [<ffffffff815103e0>] __sock_create+0x110/0x260
> Oct 22 01:26:10 TEST kernel: [<ffffffff81511ab1>] SyS_socket+0x61/0xf0
> Oct 22 01:26:10 TEST kernel: [<ffffffff8163d948>] ? page_fault+0x28/0x30
> Oct 22 01:26:10 TEST kernel: [<ffffffff81645ec9>]
> system_call_fastpath+0x16/0x1b
> Oct 22 01:26:10 TEST kernel: Code: 66 90 48 89 01 31 c0 66 66 90 c3 b8 f2
> ff ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5
> f0 ff 07 f3 90 <83> 3f 01 75 f9 f0 ff 0f 75 f1 5d c3 66 66 2e 0f 1f 84 00
> 00 00
>
> Info in /proc:
>
> PF_RING Version : 6.4.1 ((detached:eded7ce625e60620639
> 050575b1fba0aa0412374)
> Total rings : 16
>
> Standard (non ZC) Options
> Ring slots : 65534
> Slot version : 16
> Capture TX : No [RX only]
> IP Defragment : No
> Socket Mode : Standard
> Total plugins : 0
> Cluster Fragment Queue : 0
> Cluster Fragment Discard : 0
>
> cpu and memory info:
>
> Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz * 2
>
> total used free shared buff/cache
> available
> Mem: 62G 33G 25G 784M 4.0G
> 27G
> Swap: 31G 0B 31G
>
> Any insight would be helpful.
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
>
>
> _______________________________________________
> Ntop-misc mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop-misc
>
_______________________________________________
Ntop-misc mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop-misc
