Hi Vito,
Well, from the crash information it looks related to a malloc() call, ie.
pmacct is requesting memory to the operating system for something. I see
possible one of the following two scenarios: a) system went out of memory
or b) the daemon has been hitting some OS limit to the amount of memory a
process can allocate.
Is the pktgen tool generating heavy traffic within a single stream? Or is
it randomly generating new traffic streams? The latter can lead pmacct to
request memory to the OS given the fine-grained aggregation method you are
using ('aggregate' directive). Let me know on this.
Cheers,
Paolo
PS: on a sidenote, just reading the config you posted: the combination of
sql_refresh_time, sql_history and sql_dont_try_update is not valid.
You should either suppress sql_dont_try_update or set sql_refresh_time
to 300 secs or bring sql_history to 1 minute.
On Fri, Jun 24, 2011 at 06:46:13PM +0200, [email protected] wrote:
> Hi all,
>
> i'm running pmacctd under my L2 bridged machine between two other pc as
> follow:
>
>
> inner <------> bridge <-----> outer
>
>
> Testing the mysql plugin under heavy traffic, generated with pktgen from
> the inner (81 Mb/s) and outer machines (130 Mb/s),
> the plugin crashes reporting the following error.
>
> Have you any hints or suggestion?
>
> best regards
> vito
>
>
>
>
>
> #########################################
> # cat /etc/pmacct/pmacctd.conf
> ! pmacctd configuration
> !
> !
> !
> debug: true
> daemonize: false
> pidfile: /var/run/pmacctd.pid
> syslog: daemon
> !
> ! interested in in and outbound traffic
> aggregate: src_mac,dst_mac,src_host,dst_host,src_port,dst_port,proto,flows
> ! on this network
> pcap_filter: tcp or udp
> ! on this interface
> interface: br0
> !
> ! storage methods
> plugins: mysql
> sql_host: localhost
> sql_user: pmacct
> sql_passwd: 1nation
> sql_db: pmacct
> sql_table: acct_v4
> sql_table_version: 4
> ! refresh the db every minute
> sql_refresh_time: 60
> ! reduce the size of the insert/update clause
> !sql_optimize_clauses: true
> ! accumulate values in each row for up to 2 minute
> sql_history: 5m
> ! create new rows on the minute, hour, day boundaries
> sql_history_roundoff: m
> ! in case of emergency, log to this file
> sql_recovery_logfile: /var/lib/pmacct/recovery_log
> sql_dont_try_update: true
> #########################################
>
>
>
>
> # pmacctd -f /etc/pmacct/pmacctd.conf -S local2 -d
> DEBUG ( /etc/pmacct/pmacctd.conf ): plugin name/type: 'default'/'core'.
> DEBUG ( /etc/pmacct/pmacctd.conf ): plugin name/type: 'default'/'mysql'.
> DEBUG ( /etc/pmacct/pmacctd.conf ): debug:true
> DEBUG ( /etc/pmacct/pmacctd.conf ): daemonize:false
> DEBUG ( /etc/pmacct/pmacctd.conf ): pidfile:/var/run/pmacctd.pid
> DEBUG ( /etc/pmacct/pmacctd.conf ): syslog:daemon
> DEBUG ( /etc/pmacct/pmacctd.conf ):
> aggregate:src_mac,dst_mac,src_host,dst_host,src_port,dst_port,proto,flows
> DEBUG ( /etc/pmacct/pmacctd.conf ): pcap_filter:tcp or udp
> DEBUG ( /etc/pmacct/pmacctd.conf ): interface:br0
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_host:localhost
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_user:pmacct
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_passwd:1nation
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_db:pmacct
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_table:acct_v4
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_table_version:4
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_refresh_time:60
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_history:5m
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_history_roundoff:m
> DEBUG ( /etc/pmacct/pmacctd.conf ):
> sql_recovery_logfile:/var/lib/pmacct/recovery_log
> DEBUG ( /etc/pmacct/pmacctd.conf ): sql_dont_try_update:true
> DEBUG ( /etc/pmacct/pmacctd.conf ): syslog:local2
> DEBUG ( /etc/pmacct/pmacctd.conf ): debug:true
> *** glibc detected *** pmacctd: MySQL Plugin [default]: break adjusted
> to free malloc space: 0x0000000000fb3e60 ***
> ======= Backtrace: =========
> /lib/libc.so.6(+0x71ad6)[0x7fc2971bbad6]
> /lib/libc.so.6(+0x751cd)[0x7fc2971bf1cd]
> /lib/libc.so.6(__libc_malloc+0x70)[0x7fc2971c0930]
> pmacctd: MySQL Plugin [default](sql_cache_insert+0x458)[0x450ed0]
> pmacctd: MySQL Plugin [default](mysql_plugin+0xbd9)[0x44d03a]
> pmacctd: MySQL Plugin [default](load_plugins+0x742)[0x41a2c2]
> pmacctd: MySQL Plugin [default](main+0x19fa)[0x4163c8]
> /lib/libc.so.6(__libc_start_main+0xfd)[0x7fc297168c4d]
> pmacctd: MySQL Plugin [default][0x414759]
> ======= Memory map: ========
> 00400000-004bb000 r-xp 00000000 08:01 2444015
> /usr/local/sbin/pmacctd
> 006ba000-006bf000 rw-p 000ba000 08:01 2444015
> /usr/local/sbin/pmacctd
> 006bf000-00714000 rw-p 00000000 00:00 0
> 00a08000-00fd5000 rw-p 00000000 00:00 0
> [heap]
> 7fc290000000-7fc290021000 rw-p 00000000 00:00 0
> 7fc290021000-7fc294000000 ---p 00000000 00:00 0
> 7fc296163000-7fc296179000 r-xp 00000000 08:01 19775491
> /lib/libgcc_s.so.1
> 7fc296179000-7fc296378000 ---p 00016000 08:01 19775491
> /lib/libgcc_s.so.1
> 7fc296378000-7fc296379000 rw-p 00015000 08:01 19775491
> /lib/libgcc_s.so.1
> 7fc296379000-7fc296a3a000 rw-p 00000000 00:00 0
> 7fc296a3a000-7fc296cfb000 rw-s 00000000 00:04 34877
> /dev/zero (deleted)
> 7fc296cfb000-7fc296d10000 r-xp 00000000 08:01 19775509
> /lib/libnsl-2.11.2.so
> 7fc296d10000-7fc296f0f000 ---p 00015000 08:01 19775509
> /lib/libnsl-2.11.2.so
> 7fc296f0f000-7fc296f10000 r--p 00014000 08:01 19775509
> /lib/libnsl-2.11.2.so
> 7fc296f10000-7fc296f11000 rw-p 00015000 08:01 19775509
> /lib/libnsl-2.11.2.so
> 7fc296f11000-7fc296f13000 rw-p 00000000 00:00 0
> 7fc296f13000-7fc296f1b000 r-xp 00000000 08:01 19775510
> /lib/libcrypt-2.11.2.so
> 7fc296f1b000-7fc29711a000 ---p 00008000 08:01 19775510
> /lib/libcrypt-2.11.2.so
> 7fc29711a000-7fc29711b000 r--p 00007000 08:01 19775510
> /lib/libcrypt-2.11.2.so
> 7fc29711b000-7fc29711c000 rw-p 00008000 08:01 19775510
> /lib/libcrypt-2.11.2.so
> 7fc29711c000-7fc29714a000 rw-p 00000000 00:00 0
> 7fc29714a000-7fc2972a2000 r-xp 00000000 08:01 19775499
> /lib/libc-2.11.2.so
> 7fc2972a2000-7fc2974a1000 ---p 00158000 08:01 19775499
> /lib/libc-2.11.2.so
> 7fc2974a1000-7fc2974a5000 r--p 00157000 08:01 19775499
> /lib/libc-2.11.2.so
> 7fc2974a5000-7fc2974a6000 rw-p 0015b000 08:01 19775499
> /lib/libc-2.11.2.so
> 7fc2974a6000-7fc2974ab000 rw-p 00000000 00:00 0
> 7fc2974ab000-7fc2974c2000 r-xp 00000000 08:01 19775500
> /lib/libpthread-2.11.2.so
> 7fc2974c2000-7fc2976c1000 ---p 00017000 08:01 19775500
> /lib/libpthread-2.11.2.so
> 7fc2976c1000-7fc2976c2000 r--p 00016000 08:01 19775500
> /lib/libpthread-2.11.2.so
> 7fc2976c2000-7fc2976c3000 rw-p 00017000 08:01 19775500
> /lib/libpthread-2.11.2.so
> 7fc2976c3000-7fc2976c7000 rw-p 00000000 00:00 0
> 7fc2976c7000-7fc2976de000 r-xp 00000000 08:01 2426919
> /usr/lib/libz.so.1.2.3.4
> 7fc2976de000-7fc2978dd000 ---p 00017000 08:01 2426919
> /usr/lib/libz.so.1.2.3.4
> 7fc2978dd000-7fc2978de000 rw-p 00016000 08:01 2426919
> /usr/lib/libz.so.1.2.3.4
> 7fc2978de000-7fc29795e000 r-xp 00000000 08:01 19775508
> /lib/libm-2.11.2.so
> 7fc29795e000-7fc297b5e000 ---p 00080000 08:01 19775508
> /lib/libm-2.11.2.so
> 7fc297b5e000-7fc297b5f000 r--p 00080000 08:01 19775508
> /lib/libm-2.11.2.so
> 7fc297b5f000-7fc297b60000 rw-p 00081000 08:01 19775508
> /lib/libm-2.11.2.so
> 7fc297b60000-7fc297b62000 r-xp 00000000 08:01 19775498
> /lib/libdl-2.11.2.so
> 7fc297b62000-7fc297d62000 ---p 00002000 08:01 19775498
> /lib/libdl-2.11.2.so
> 7fc297d62000-7fc297d63000 r--p 00002000 08:01 19775498
> /lib/libdl-2.11.2.so
> 7fc297d63000-7fc297d64000 rw-p 00003000 08:01 19775498
> /lib/libdl-2.11.2.so
> 7fc297d64000-7fc297f33000 r-xp 00000000 08:01 2429601
> /usr/lib/libmysqlclient.so.16.0.0
> 7fc297f33000-7fc298133000 ---p 001cf000 08:01 2429601
> /usr/lib/libmysqlclient.so.16.0.0
> 7fc298133000-7fc29813a000 r--p 001cf000 08:01 2429601
> /usr/lib/libmysqlclient.so.16.0.0
> 7fc29813a000-7fc298184000 rw-p 001d6000 08:01 2429601
> /usr/lib/libmysqlclient.so.16.0.0
> 7fc298184000-7fc298185000 rw-p 00000000 00:00 0
> 7fc298185000-7fc2981b9000 r-xp 00000000 08:01 2429072
> /usr/lib/libpcap.so.1.1.1
> 7fc2981b9000-7fc2983b8000 ---p 00034000 08:01 2429072
> /usr/lib/libpcap.so.1.1.1
> 7fc2983b8000-7fc2983ba000 rw-p 00033000 08:01 2429072
> /usr/lib/libpcap.so.1.1.1
> 7fc2983ba000-7fc2983bb000 rw-p 00000000 00:00 0
> 7fc2983bb000-7fc2983d9000 r-xp 00000000 08:01 19775511
> /lib/ld-2.11.2.so
> 7fc298534000-7fc2985d2000 rw-p 00000000 00:00 0
> 7fc2985d4000-7fc2985d5000 rw-s 00000000 00:04 34878
> /dev/zero (deleted)
> 7fc2985d5000-7fc2985d8000 rw-p 00000000 00:00 0
> 7fc2985d8000-7fc2985d9000 r--p 0001d000 08:01 19775511
> /lib/ld-2.11.2.so
> 7fc2985d9000-7fc2985da000 rw-p 0001e000 08:01 19775511
> /lib/ld-2.11.2.so
> 7fc2985da000-7fc2985db000 rw-p 00000000 00:00 0
> 7fff47f4a000-7fff47f5f000 rw-p 00000000 00:00 0
> [stack]
> 7fff47f86000-7fff47f87000 r-xp 00000000 00:00 0
> [vdso]
> ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0
> [vsyscall]
>
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
