Hi Paolo,
I'm seeing a significant memory leak in the print plugin in 1.5.2.
# service nfacctd restart
Shutting down sflow collection services (nfacctd): [ OK ]
Starting sflow collection services (nfacctd): INFO ( default/core ):
Daemonizing. Hmm, bye bye screen.
[ OK ]
# pgrep nfacctd
874
876
# while [ 1 ]; do sleep 60; date; ps uwp 876; done
Thu Jan 7 15:53:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.1 0.6 115276 104432 ? S 15:52 0:01 nfacctd:
Print Plugin [default]
Thu Jan 7 15:54:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.0 0.6 120884 109980 ? S 15:52 0:01 nfacctd:
Print Plugin [default]
Thu Jan 7 15:55:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.0 0.7 128408 117608 ? S 15:52 0:02 nfacctd:
Print Plugin [default]
Thu Jan 7 15:56:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.1 0.8 147680 136856 ? S 15:52 0:03 nfacctd:
Print Plugin [default]
Thu Jan 7 15:57:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.1 0.9 160748 149944 ? S 15:52 0:03 nfacctd:
Print Plugin [default]
Thu Jan 7 15:58:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.1 1.0 180548 169668 ? S 15:52 0:04 nfacctd:
Print Plugin [default]
Thu Jan 7 15:59:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.1 1.1 203384 192604 ? S 15:52 0:05 nfacctd:
Print Plugin [default]
Thu Jan 7 16:00:56 CST 2016
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 876 1.3 1.5 272024 261120 ? S 15:52 0:07 nfacctd:
Print Plugin [default]
^C
# strings - /usr/sbin/nfacctd | fgrep 1.5
NetFlow Accounting Daemon, nfacctd 1.5.2
/opt/pmacct/pmacct-1.5.2/src
/opt/pmacct/pmacct-1.5.2/src/nfprobe_plugin
/opt/pmacct/pmacct-1.5.2/src/sfprobe_plugin
/opt/pmacct/pmacct-1.5.2/src/bgp
/opt/pmacct/pmacct-1.5.2/src/tee_plugin
/opt/pmacct/pmacct-1.5.2/src/isis
/opt/pmacct/pmacct-1.5.2/src/bmp
Any thoughts? Suggestions?
Thanks,
Ed
On Mon, Sep 7, 2015 at 1:01 PM, Paolo Lucente <[email protected]> wrote:
> VERSION.
> 1.5.2
>
>
> DESCRIPTION.
> pmacct is a small set of passive network monitoring tools to account,
> classify,
> aggregate, replicate and export IPv4 and IPv6 traffic; a pluggable
> architecture
> allows to store collected data into memory tables, RDBMS (MySQL,
> PostgreSQL,
> SQLite), noSQL databases (MongoDB, BerkeleyDB) AMQP message exchanges
> (RabbitMQ) and flat-files and export through NetFlow or sFlow protocols to
> remote collectors. pmacct offers customizable historical data breakdown,
> BGP
> and IGP correlation, BMP stats and events, GeoIP lookups, sampling and
> renormalization, filtering, tagging and triggers. Libpcap, Linux
> Netlink/ULOG,
> sFlow v2/v4/v5, NetFlow v5/v8/v9 and IPFIX are supported. Replication of
> incoming NetFlow and sFlow datagrams is also available. Statistics can be
> easily
> exported to tools like RRDtool, Net-SNMP, MRTG, GNUPlot and Cacti.
>
>
> HOMEPAGE.
> http://www.pmacct.net/
>
>
> DOWNLOAD.
> http://www.pmacct.net/pmacct-1.5.2.tar.gz
>
>
> CHANGELOG.
> + Introduced support for a RabbitMQ broker to be used for queueing and
> data exchange between Core Process and plugins. This is in alternative to
> the home-grown circular queue implementation. plugin_pipe_amqp directive,
> along with all other plugin_pipe_amqp_* directives, can be set globally
> or apply on a per plugin basis (ie. it is a valid scenario, if multiple
> plugins are instantiated, that some make use of home-grown queueing,
> while others use RabbitMQ based queueing).
> + Introducting support for Maximind GeoIP v2 (libmaxminddb) library: if
> pmacct is compiled with --enable-geoipv2, this defines full pathname to
> a Maxmind GeoIP database v2 (libmaxminddb) Only the binary database
> format is supported (ie. it is not possible to load distinct CSVs for
> IPv4 and IPv6 addresses).
> + Introduced infrastructure for sFlow counters and support specifically for
> generic, ethernet and vlan counters. Counters are exported in JSON format
> to files, specified via sfacctd_counter_file. The supplied filename can
> contain as variable the sFlow agent IP address.
> + Introduced a new thread_stack config directive to allow to modify the
> thread stack size. Natanael Copa reported that some libc implementations,
> ie. musl libc, may set a stack size that is too small by default.
> + Introduced networks_file_no_lpm feature: it applies when aggregation
> method includes src_net and/or dst_net and nfacctd_net (or equivalents)
> and/or nfacctd_as_new (or equivalents) are set to longest (or fallback):
> an IP prefix defined as part of the supplied networks_file wins always,
> even if it's not longest.
> + tee plugin: added support for (non-)transparent IPv6 replication [further
> QA required]
> + plugin_common.c, sql_common.c: added log message to estimate base cache
> memory usage.
> + print, AMQP, MongoDB plugins; sfacctd, BGP, BMP daemons: introducing
> timestamps_since_epoch to write timestamps in 'since Epoch' format.
> + nfacctd: flow bytes counter can now be sourced via element ID #352
> (layer2OctetDeltaCount) in addition to element ID's already supported.
> Thanks to Jonathan Thorpe for his support.
> + Introducing proc_priority: redefines the process scheduling priority,
> equivalent to using the 'nice' tool. Each daemon process, ie. core,
> plugins, etc., can define a different priority.
> ! fix, BMP daemon: improved preliminar checks in bmp_log_msg() and added
> missing SIGHUP signal handling to reload bmp_daemon_msglog_file files.
> ! fix, bgp_logdump.c: under certain configuration conditions call to both
> write_and_free_json() and write_and_free_json_amqp() was leading to SEGV.
> Thanks to Yuriy Lachin for reporting the issue.
> ! fix, BGP daemon: improved BGP dump output: more accurate timestamping of
> dump_init, dump_close events. dump_close now mentions amount of entries
> and tables dumped. Thanks to Yuriy Lachin for brainstorming around this.
> ! fix, cfg.c: raised amount of allowed config lines from 256 to 8K.
> ! fix, print/AMQP/MongoDB plugins: SEGV observed when IPFIX vlen variables
> were stored in the pending_queries_queue structure (ie. as a result of a
> time mismatch among the IPFIX exporter and the collector box).
> ! fix, vlen primitives: when 'raw' semantics was selected, print_hex() was
> returning wrong hex string length (one char short). As a consequence
> occasionally some extra dirty chars were seen at the end of the converted
> string.
> ! fix, vlen primitives: memory leak verified in print/AMQP/MongoDB plugins.
> ! fix, print, MongoDB & AMQP plugins: dirty values printed as part of the
> 'proto' under certain conditions. Thanks to Rene Stoutjesdijk for his
> support resolving the issue.
> ! fix, amqp_common.c: amqp_exchange_declare() call changed so to address
> the change of rabbitmq-c API for support of auto_delete & internal for
> exchange.declare. Backward compatibility with rabbitmq-c <= 0.5.2 is
> also taken care of. Thanks to Brent Van Dussen for reporting the issue.
> ! fix, compiling on recent FreeBSD: solved some errors catched by the -Wall
> compiler flag. Thanks to Stephen Fulton for reporting the issue. Most of
> the patch is courtesy by Mike Bowie.
> ! fix, print/AMQP/MongoDB plugins: enforcing cleanup of malloc()ed structs
> part of entries added to the pending queue, ie. because seen as future
> entries due to a mismatch of the collector clock with the one of NetFlow/
> IPFIX exporter(s). This may have lead to data inconsistencies.
> ! fix, amqp_common.c: Content type was only specified for messages
> published
> when the amqp_persistent_msg configuration option is specified. This info
> should always be applied to describe the payload of the message. Patch is
> courtesy by Will Dowling.
> ! fix, amqp_plugin.c: generate an error on compile if --enable-rabbitmq is
> specified without --enable-jansson. It's clear in the documentation that
> both are required for AMQP support, but if built without jansson it will
> silently not publish messages to AMQP. Patch is courtesy by Will Dowling.
> ! fix, amqp_common.c: modified the content type to "application/json" in
> line with RFC4627. Patch is courtesy by Will Dowling.
> ! fix, setsockopt(): u_int64_t pipe_size vars changed to int, in line with
> typical OS buffer limits (Linux, Solaris). Introduced check supplied pipe
> size values are not bigger than INT_MAX. Many thanks to Markus Weber for
> reporting the issue.
> ! fix, nl.c: removed pretag_free_label() from pcap_cb() and ensuring init
> of pptrs. Under certain conditions SEGVs could be noticed.
> ! fix, flow stitching: when print/AMQP/MongoDB plugins were making use of
> the pending queries queue, ie. to compensate for time offsets/flows in
> the future, the stitching feature could potentially lead to SEGV due to
> unsettled pointers.
> ! fix, pgsql plugin: SEGV were noticed when insert/update queries to the
> PostgreSQL database were returning different than PGRES_COMMAND_OK, hence
> triggering the reprocess mechanism. Thanks very much to Alan Turower for
> his support.
> ! fix, improved logging of elements received/sent at buffering point
> between
> core process and plugins. Also added explicit start/end purge log message
> for cases in which there is no data to purge.
> ! fix, signals.c: ignore_falling_child() now logs if a child process exited
> with abnormal conditions; this is useful to track writer processes
> (created
> by plugins) are terminated by a signal, ie. SEGV. This is already the
> case
> for plugins themselves, with the Core Process reporting a simlar log
> message in case of abnormal exit. Thanks very much to Rene Stoutjesdijk
> for his support.
> ! fix, preprocess-data.h: added supported functions minf, minb, minbpp and
> minppf to non SQL plugins. Thanks to Jared Deyo for reporting the issue.
> ! fix, nfprobe_plugin.c: IP protocol was not set up correctly for IPv6
> traffic in NetFlow v9/IPFIX. Thanks to Gabriel Vermeulen his support
> solving the issue.
>
>
> NOTES.
> See UPGRADE file.
>
>
> Cheers,
> Paolo
>
> _______________________________________________
> pmacct-discussion mailing list
> http://www.pmacct.net/#mailinglists
>
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
