On 2011/04/13 05:28, Gary Gatten wrote: > What's "-n 0" ? I don't have 4.0.3, but I just have "-n". I'm curious to see > what you find. When I have ip lookups / name res enabled it's not good, > consumes "all" cpu and eventually dies. I'm several revs behind though so it > will be interesting to see how this impacts you.
[-n <mode> | --numeric-ip-addresses <mode>]
Numeric IP addresses DNS resolution mode:
0 - No DNS resolution at all
1 - DNS resolution for local hosts only
2 - DNS resolution for remote hosts only
Ntop still running so far.
Conor.
>
>
>
> ----- Original Message -----
> From: Schrodinger [mailto:[email protected]]
> Sent: Wednesday, April 13, 2011 04:44 AM
> To: [email protected] <[email protected]>
> Subject: Re: [Ntop] known double free bug in 4.0.3?
>
> On 2011/04/12 09:51, Gary Gatten wrote:
> > Oh yeah, there's known issues with the tcp session tracking. At one point
> > it was disabled in the code, is it back again?
> >
> > So -z to disable sessions and -n to disable name/ip resolution. Start by
> > disabling one, or both I guess. My bet is your problems go away.
>
> I ran Ntop yesterday with -z and came in this morning to find it has
> crashed again.
>
> I am running with -n 0 and -z now to see how it goes.
>
> Conor.
>
> >
> > When I was troubleshooting I had about a dozen different binaries with
> > different features, such as disable IPv6, and each one of those a half
> > dozen different startup args.
> >
> > ----- Original Message -----
> > From: Schrodinger [mailto:[email protected]]
> > Sent: Tuesday, April 12, 2011 09:16 AM
> > To: [email protected] <[email protected]>
> > Subject: Re: [Ntop] known double free bug in 4.0.3?
> >
> > On 2011/04/12 08:44, Gary Gatten wrote:
> > > Certainly not an expert, but I see text like "dequeueAddress" and think
> > > name/ip lookups. Disable this (-z?) and see how things go. I've had
> > > issues with the resolution process that disabling fixed. Maybe some of
> > > those same issues aren't fully patched and are impacting you?
> >
> > I would want the TCP session tracking though, this could be useful for
> > accounting and security. How would this affect Netflow accounting?
> >
> > Conor.
> >
> > >
> > > ----- Original Message -----
> > > From: Schrodinger [mailto:[email protected]]
> > > Sent: Tuesday, April 12, 2011 08:23 AM
> > > To: [email protected] <[email protected]>
> > > Subject: Re: [Ntop] known double free bug in 4.0.3?
> > >
> > > On 2011/04/12 07:28, Gary Gatten wrote:
> > > > If it dies so often, perhaps try running it with a bunch of stuff
> > > > disabled; name lookups, decoders, rrd (as much as possible), etc. and
> > > > see if that makes a difference.
> > >
> > > Gary,
> > >
> > > Error messages below are from Ntop running normally, not in gdb and also
> > > running in gdb + gdb backtrace.
> > >
> > > "While running the main Ntop process in gdb"
> > >
> > > Conor.
> > >
> > > >
> > > > Have you tried running in gdb? I found many of my issues there, except
> > > > once when it would die every hour running "normal", but running in gdb
> > > > it would NOT crash! Typical :). The '-k (or -K?)' MAY have had
> > > > something to do with that.
> > > >
> > > > ----- Original Message -----
> > > > From: Schrodinger [mailto:[email protected]]
> > > > Sent: Tuesday, April 12, 2011 04:01 AM
> > > > To: [email protected] <[email protected]>
> > > > Subject: Re: [Ntop] known double free bug in 4.0.3?
> > > >
> > > > Hi,
> > > >
> > > > I have this exact same problem, "double free or corruption". I have
> > > > compiled Ntop with debugging and electric fence and the problem still
> > > > persists. Usually Ntop can't last more than a few hours at a time :( I
> > > > am using Ntop to collect Netflow from different sources via thge native
> > > > Netflow plugin.
> > > >
> > > > I had compiled Ntop with additional debugging to try and locate the
> > > > problem.
> > > >
> > > > [user@host ntop-4.0.3]# export CFLAGS+=-g
> > > > [user@host ntop-4.0.3]# echo $CFLAGS
> > > > -g
> > > > [user@host ntop-4.0.3]# ./autogen.sh --prefix=/usr/local/ntop-4.0.3
> > > > --with-memorydebug=2
> > > >
> > > > ===================================================================
> > > >
> > > > This is your ntop 4.0.3 configuration:
> > > >
> > > > Host System Type : x86_64-redhat-linux-gnu
> > > > Preprocessor (cppflags) : -DLINUX -DMEMORY_DEBUG=3
> > > > -I/usr/local/include -I/opt/local/include
> > > > Compiler (cflags) : gcc -g -DMEMORY_DEBUG_UNLIMITED -DMUTEX_DEBUG
> > > > -I/usr/local/include -I/opt/local/include -Wshadow -Wpointer-arith
> > > > -Wmissing-prototypes -Wmissing-declarations -Wnested-externs -fPIC
> > > > -DPIC
> > > > Defines : -DHAVE_CONFIG_H
> > > > Loader (ldflags) : -L/usr/local/lib -L/opt/local/lib
> > > > Include path :
> > > > System Libs : -lpthread -lcrypt -lc -lssl -lcrypto -lrrd_th
> > > > -lgdbm -lz -lGeoIP
> > > > Locale : /usr/lib/locale
> > > > Memory Debug : 3
> > > >
> > > > External packages:
> > > >
> > > > LBL pcap .h : standard system headers
> > > > LBL pcap library : standard system libraries
> > > > GNU gdbm .h : standard system headers
> > > > GNU gdbm library : standard system libraries
> > > > zlib .h : standard system headers
> > > > zlib library : standard system libraries
> > > > openSSL .h : standard system headers
> > > > openSSL library : standard system libraries
> > > >
> > > > Install directories:
> > > >
> > > > Default prefix: /usr/local
> > > > Install into: /usr/local/ntop-4.0.3 (default or via --prefix
> > > > request)
> > > >
> > > > Data files are in /usr/local/ntop-4.0.3/share/ntop
> > > > Config files are in /usr/local/ntop-4.0.3/etc/ntop
> > > > Run directory is /usr/local/ntop-4.0.3/var/ntop
> > > > Plugin files are in /usr/local/ntop-4.0.3/lib/ntop/plugins
> > > > Database files are in /usr/local/ntop-4.0.3/var/ntop
> > > >
> > > > -------------------------------------------------------------------
> > > >
> > > > Here are some logs then that I was able to glean from the system log
> > > > and/or gdb.
> > > >
> > > > 127.0.0.1 - - [09/Nov/2010:09:11:48 +0000] "GET /MochiKit/Sortable.js
> > > > HTTP/1.1" 200 83 - - 1
> > > > *** glibc detected *** ./bin/ntop: double free or corruption (fasttop):
> > > > 0x00002a
> > > > aab41ede00 ***
> > > > ======= Backtrace: =========
> > > > /lib64/libc.so.6[0x2abf39c0430f]
> > > > /lib64/libc.so.6(cfree+0x4b)[0x2abf39c0476b]
> > > > /usr/local/ntop-4.0.3/lib/libntop-4.0.3.so(ntop_safefree+0x16)[0x2abf392de346]
> > > > /usr/local/ntop-4.0.3/lib/libntop-4.0.3.so(dequeueAddress+0x3cd)[0x2abf392d36dd]
> > > > /lib64/libpthread.so.0[0x2abf3974473d]
> > > > /lib64/libc.so.6(clone+0x6d)[0x2abf39c65f6d]
> > > >
> > > > While running the main Ntop process in gdb:
> > > >
> > > > Detaching after fork from child process 22963.
> > > > Detaching after fork from child process 22964.
> > > > Detaching after fork from child process 22965.
> > > > Detaching after fork from child process 22966.
> > > > Detaching after fork from child process 22968.
> > > > Detaching after fork from child process 22969.
> > > > Detaching after fork from child process 22970.
> > > > [...]
> > > > Detaching after fork from child process 22971.
> > > > *** glibc detected *** /usr/local/ntop-4.0.3/bin/ntop: double free or
> > > > corruption (fasttop): 0x00002aaab43181d0 ***
> > > > ======= Backtrace: =========
> > > > /lib64/libc.so.6[0x2aaaab8b930f]
> > > > /lib64/libc.so.6(cfree+0x4b)[0x2aaaab8b976b]
> > > > /usr/local/ntop-4.0.3/lib/libntop-4.0.3.so(dequeueAddress+0x422)[0x2aaaaaf71497]
> > > > /lib64/libpthread.so.0[0x2aaaab3f973d]
> > > > /lib64/libc.so.6(clone+0x6d)[0x2aaaab91af6d]
> > > > ======= Memory map: ========
> > > > 00400000-00410000 r-xp 00000000 68:02 7962633
> > > > /usr/local/ntop-4.0.3/bin/ntop
> > > > 00610000-00611000 rw-p 00010000 68:02 7962633
> > > > /usr/local/ntop-4.0.3/bin/ntop
> > > > 00611000-01892000 rw-p 00611000 00:00 0
> > > > [heap]
> > > > 40000000-40001000 ---p 40000000 00:00 0
> > > > 40001000-40a01000 rw-p 40001000 00:00 0
> > > > 40a01000-40a02000 ---p 40a01000 00:00 0
> > > > 40a02000-41402000 rw-p 40a02000 00:00 0
> > > > 41402000-41403000 ---p 41402000 00:00 0
> > > > 41403000-41e03000 rw-p 41403000 00:00 0
> > > > 41e03000-41e04000 ---p 41e03000 00:00 0
> > > > 41e04000-42804000 rw-p 41e04000 00:00 0
> > > > 42804000-42805000 ---p 42804000 00:00 0
> > > > 42805000-43205000 rw-p 42805000 00:00 0
> > > > 43205000-43206000 ---p 43205000 00:00 0
> > > > 43206000-43c06000 rw-p 43206000 00:00 0
> > > > 43c06000-43c07000 ---p 43c06000 00:00 0
> > > > [...]
> > > > 2aaaaff4d000-2aaab014c000 ---p 00020000 68:01 163900
> > > > /lib64/libexpat.so.0.5.0
> > > > 2aaab014c000-2aaab014f000 rw-p 0001f000 68:01 163900
> > > > /lib64/libexpat.so.0.5.0
> > > > 2aaab014f000-2aaab0150000 rw-p 2aaab014f000 00:00 0
> > > > 2aaab0150000-2aaab018b000 r-xp 00000000 68:01 163978
> > > > /lib64/libsepol.so.1
> > > > 2aaab018b000-2aaab038b000 ---p 0003b000 68:01 163978
> > > > /lib64/libsepol.so.1
> > > > 2aaab038b000-2aaab038c000 rw-p 0003b000 68:01 163978
> > > > /lib64/lib
> > > > Program received signal SIGABRT, Aborted.
> > > > [Switching to Thread 0x41e02940 (LWP 22660)]
> > > > 0x00002aaaab877265 in raise () from /lib64/libc.so.6
> > > > (gdb)
> > > > (gdb)
> > > >
> > > > (gdb) bt
> > > > #0 0x00002aaaab877265 in raise () from /lib64/libc.so.6
> > > > #1 0x00002aaaab878d10 in abort () from /lib64/libc.so.6
> > > > #2 0x00002aaaab8b184b in __libc_message () from /lib64/libc.so.6
> > > > #3 0x00002aaaab8b930f in _int_free () from /lib64/libc.so.6
> > > > #4 0x00002aaaab8b976b in free () from /lib64/libc.so.6
> > > > #5 0x00002aaaaaf71497 in dequeueAddress (_i=0x0) at address.c:260
> > > > #6 0x00002aaaab3f973d in start_thread () from /lib64/libpthread.so.0
> > > > #7 0x00002aaaab91af6d in clone () from /lib64/libc.so.6
> > > > (gdb)
> > > >
> > > > I am happy to provide any more information you might need or perform
> > > > any
> > > > additional debugging tasks that would help track this issue down.
> > > >
> > > > Thanks,
> > > > Conor.
> > > > --
> > > > +---------------------------------------------------------------+
> > > > It was a new day yesterday, but it's an old day now.
> > > > MSN: [email protected]
> > > > ICQ: 112562229
> > > > GPG: http://www.konundrum.org/schro.asc
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > <font size="1">
> > > > <div style='border:none;border-bottom:double windowtext
> > > > 2.25pt;padding:0in 0in 1.0pt 0in'>
> > > > </div>
> > > > "This email is intended to be reviewed by only the intended recipient
> > > > and may contain information that is privileged and/or confidential.
> > > > If you are not the intended recipient, you are hereby notified that
> > > > any review, use, dissemination, disclosure or copying of this email
> > > > and its attachments, if any, is strictly prohibited. If you have
> > > > received this email in error, please immediately notify the sender by
> > > > return email and delete this email from your system."
> > > > </font>
> > > >
> > > > _______________________________________________
> > > > Ntop mailing list
> > > > [email protected]
> > > > http://listgateway.unipi.it/mailman/listinfo/ntop
> > >
> > > --
> > > +---------------------------------------------------------------+
> > > It was a new day yesterday, but it's an old day now.
> > > MSN: [email protected]
> > > ICQ: 112562229
> > > GPG: http://www.konundrum.org/schro.asc
> > >
> > >
> > >
> > >
> > >
> > > <font size="1">
> > > <div style='border:none;border-bottom:double windowtext
> > > 2.25pt;padding:0in 0in 1.0pt 0in'>
> > > </div>
> > > "This email is intended to be reviewed by only the intended recipient
> > > and may contain information that is privileged and/or confidential.
> > > If you are not the intended recipient, you are hereby notified that
> > > any review, use, dissemination, disclosure or copying of this email
> > > and its attachments, if any, is strictly prohibited. If you have
> > > received this email in error, please immediately notify the sender by
> > > return email and delete this email from your system."
> > > </font>
> > >
> > > _______________________________________________
> > > Ntop mailing list
> > > [email protected]
> > > http://listgateway.unipi.it/mailman/listinfo/ntop
> >
> > --
> > +---------------------------------------------------------------+
> > It was a new day yesterday, but it's an old day now.
> > MSN: [email protected]
> > ICQ: 112562229
> > GPG: http://www.konundrum.org/schro.asc
> >
> >
> >
> >
> >
> > <font size="1">
> > <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in
> > 0in 1.0pt 0in'>
> > </div>
> > "This email is intended to be reviewed by only the intended recipient
> > and may contain information that is privileged and/or confidential.
> > If you are not the intended recipient, you are hereby notified that
> > any review, use, dissemination, disclosure or copying of this email
> > and its attachments, if any, is strictly prohibited. If you have
> > received this email in error, please immediately notify the sender by
> > return email and delete this email from your system."
> > </font>
> >
> > _______________________________________________
> > Ntop mailing list
> > [email protected]
> > http://listgateway.unipi.it/mailman/listinfo/ntop
>
> --
> +---------------------------------------------------------------+
> It was a new day yesterday, but it's an old day now.
> MSN: [email protected]
> ICQ: 112562229
> GPG: http://www.konundrum.org/schro.asc
>
>
>
>
>
> <font size="1">
> <div style='border:none;border-bottom:double windowtext 2.25pt;padding:0in
> 0in 1.0pt 0in'>
> </div>
> "This email is intended to be reviewed by only the intended recipient
> and may contain information that is privileged and/or confidential.
> If you are not the intended recipient, you are hereby notified that
> any review, use, dissemination, disclosure or copying of this email
> and its attachments, if any, is strictly prohibited. If you have
> received this email in error, please immediately notify the sender by
> return email and delete this email from your system."
> </font>
>
> _______________________________________________
> Ntop mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop
--
+---------------------------------------------------------------+
It was a new day yesterday, but it's an old day now.
MSN: [email protected]
ICQ: 112562229
GPG: http://www.konundrum.org/schro.asc
pgpvG4mjy3JNO.pgp
Description: PGP signature
_______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop
