> On 30 Oct 2017, at 13:12, Bowen Li <[email protected]> wrote: > > Hi Alfredo, > I am very grateful to your kind reply. I deployed the latest version in a > testing environment yesterday; I will tell you more details if this problem > is reproduced. > Forget to say, in order to change the parameters, I complied the kernel > module of PF_RING and libpcap library from source code. Is my action the > cause of this problem?
This is discouraged to avoid out-of-sync issues, however it should work as long as the kernel and library versions are in sync. > Here is log when PF_RING load kernel module: > > pf_ring: module verification failed: signature and/or required key missing - > tainting kernel. This is not causing the issue, it’s just a warning. Alfredo > Besides, I wonder whether this problem is an underlying BUG in the kernel > module of PF_RING or there is something wrong with my application calling > PF_RING’s kernel module. If there is something wrong with my application > method, I need to check my logic of application program. Now I am very > confused about this problem. > Looking forward to your reply. Thank you. > > 2017-10-30 15:53 GMT+08:00 Alfredo Cardigliano <[email protected] > <mailto:[email protected]>>: > Hi Bowen Li > any chance you can move to latest stable and check if you are still able to > reproduce it? > > Thank you > Alfredo > >> On 30 Oct 2017, at 08:15, Bowen Li <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi all, >> Recently，I run bro cluster with PF_RING 6.4.1 as the package capturing >> framework. However, I found a strange scene: sometimes when the cluster >> restarts, the kernel module of PF_RING will block. The problem seems to >> generate when operates /proc to apply read-write lock, causing kernel CPU >> soft lockup and then the whole server is stuck. I want to know whether the >> deadlock is caused by the resource competition when the cluster progress >> applies read-write lock at the same time, or other reasons. >> Here is the log in /var/log/message: >> >> Oct 22 01:26:10 TEST kernel: BUG: soft lockup - CPU#16 stuck for 22s! >> [bro:17375] >> Oct 22 01:26:10 TEST kernel: Modules linked in: binfmt_misc xt_CHECKSUM >> iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 >> nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 xt_conntrack nf_conntrack ipt_REJECT >> tun bridge stp llc ebtable_filter ebtables ip6table_filter ip6_tables >> iptable_filter intel_powerclamp coretemp intel_rapl kvm_intel iTCO_wdt kvm >> mei_me iTCO_vendor_support mxm_wmi mei lpc_ich ipmi_ssif sb_edac >> crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper >> ablk_helper edac_core ipmi_si cryptd ioTESTma ipmi_msghandler shpchp wmi sg >> dca i2c_i801 pcspkr mfd_core nfsd auth_rpcgss nfs_acl lockd grace >> pf_ring(OE) sunrpc ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif >> crct10dif_generic mgag200 syscopyarea sysfillrect sysimgblt i2c_algo_bit >> drm_kms_helper ttm crct10dif_pclmul crct10dif_common >> Oct 22 01:26:10 TEST kernel: crc32c_intel drm isci e1000e libsas ahci >> scsi_transport_sas libahci ptp i2c_core libata pps_core ntb >> Oct 22 01:26:10 TEST kernel: CPU: 16 PID: 17375 Comm: bro Tainted: G >> OE ------------ 3.10.0-327.13.1.el7.x86_64 #1 >> Oct 22 01:26:10 TEST kernel: Hardware name: Supermicro >> X9DRL-3F/iF/X9DRL-3F/iF, BIOS 3.2 09/22/2015 >> Oct 22 01:26:10 TEST kernel: task: ffff880fdad73980 ti: ffff8800287b8000 >> task.ti: ffff8800287b8000 >> Oct 22 01:26:10 TEST kernel: RIP: 0010:[<ffffffff81301909>] >> [<ffffffff81301909>] __write_lock_failed+0x9/0x20 >> Oct 22 01:26:10 TEST kernel: RSP: 0018:ffff8800287bbe88 EFLAGS: 00000297 >> Oct 22 01:26:10 TEST kernel: RAX: 0000000000000000 RBX: 0000000092e79345 >> RCX: 0000000000000000 >> Oct 22 01:26:10 TEST kernel: RDX: 0000000000000000 RSI: ffff88104aabb000 >> RDI: ffffffffa03c6324 >> Oct 22 01:26:10 TEST kernel: RBP: ffff8800287bbe88 R08: 0000000000017600 >> R09: 0000000000000080 >> Oct 22 01:26:10 TEST kernel: R10: 0000000000000000 R11: 000000000000001b >> R12: 00007f21f432e000 >> Oct 22 01:26:10 TEST kernel: R13: 0000000000000032 R14: ffff8800000000a8 >> R15: 0000000000000000 >> Oct 22 01:26:10 TEST kernel: FS: 00007f21f4e56880(0000) >> GS:ffff88085fd00000(0000) knlGS:0000000000000000 >> Oct 22 01:26:10 TEST kernel: CS: 0010 DS: 0000 ES: 0000 CR0: >> 0000000080050033 >> Oct 22 01:26:10 TEST kernel: CR2: 00007f21f432eee0 CR3: 0000000648323000 >> CR4: 00000000000407e0 >> Oct 22 01:26:10 TEST kernel: DR0: 0000000000000000 DR1: 0000000000000000 >> DR2: 0000000000000000 >> Oct 22 01:26:10 TEST kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 >> DR7: 0000000000000400 >> Oct 22 01:26:10 TEST kernel: Stack: >> Oct 22 01:26:10 TEST kernel: ffff8800287bbe98 ffffffff8163ce37 >> ffff8800287bbeb8 ffffffffa03afe1b >> Oct 22 01:26:10 TEST kernel: ffff88104aabb000 ffff88101a631800 >> ffff8800287bbee8 ffffffffa03b6903 >> Oct 22 01:26:10 TEST kernel: 0000000000000003 0000000000000300 >> 0000000000000000 ffffffff81a25e00 >> Oct 22 01:26:10 TEST kernel: Call Trace: >> Oct 22 01:26:10 TEST kernel: [<ffffffff8163ce37>] _raw_write_lock+0x17/0x20 >> Oct 22 01:26:10 TEST kernel: [<ffffffffa03afe1b>] ring_proc_add+0x1b/0xb0 >> [pf_ring] >> Oct 22 01:26:10 TEST kernel: [<ffffffffa03b6903>] ring_create+0x2e3/0x4a0 >> [pf_ring] >> Oct 22 01:26:10 TEST kernel: [<ffffffff815103e0>] __sock_create+0x110/0x260 >> Oct 22 01:26:10 TEST kernel: [<ffffffff81511ab1>] SyS_socket+0x61/0xf0 >> Oct 22 01:26:10 TEST kernel: [<ffffffff8163d948>] ? page_fault+0x28/0x30 >> Oct 22 01:26:10 TEST kernel: [<ffffffff81645ec9>] >> system_call_fastpath+0x16/0x1b >> Oct 22 01:26:10 TEST kernel: Code: 66 90 48 89 01 31 c0 66 66 90 c3 b8 f2 ff >> ff ff 66 66 90 c3 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 48 89 e5 f0 >> ff 07 f3 90 <83> 3f 01 75 f9 f0 ff 0f 75 f1 5d c3 66 66 2e 0f 1f 84 00 00 00 >> >> Info in /proc: >> >> PF_RING Version : 6.4.1 >> ((detached:eded7ce625e60620639050575b1fba0aa0412374) >> Total rings : 16 >> >> Standard (non ZC) Options >> Ring slots : 65534 >> Slot version : 16 >> Capture TX : No [RX only] >> IP Defragment : No >> Socket Mode : Standard >> Total plugins : 0 >> Cluster Fragment Queue : 0 >> Cluster Fragment Discard : 0 >> >> cpu and memory info: >> >> Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz * 2 >> >> total used free shared buff/cache >> available >> Mem: 62G 33G 25G 784M 4.0G >> 27G >> Swap: 31G 0B 31G >> >> Any insight would be helpful. >> _______________________________________________ >> Ntop-misc mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop-misc >> <http://listgateway.unipi.it/mailman/listinfo/ntop-misc> > > _______________________________________________ > Ntop-misc mailing list > [email protected] <mailto:[email protected]> > http://listgateway.unipi.it/mailman/listinfo/ntop-misc > <http://listgateway.unipi.it/mailman/listinfo/ntop-misc> > > _______________________________________________ > Ntop-misc mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop-misc
signature.asc
Description: Message signed with OpenPGP
_______________________________________________ Ntop-misc mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop-misc
