Hi Paolo,
I just did a fresh CVS checkout and tried to build it. Alas, it failed.
It complains about not finding ../bmp/bmp.h as referenced in
bgp/bgp_logdump.c. And my checkout doesn't even have that directory (bmp).
Regards,
Ruben
On 26-8-2015 01:02, Paolo Lucente wrote:
Hi Ruben,
Yes, there is some config. You can read chapter "Internal buffering and
queueing" of the QUICKSTART document:
https://github.com/paololucente/pmacct/blob/master/pmacct/QUICKSTART
Essentially plugin_pipe_size becomes something not to care anymore and
plugin_pipe_amqp goes to 'true'. Then any extra directive is to config
aspects to connect to the RabbitMQ exchange (user, password, etc.).
Cheers,
Paolo
On Sun, Aug 23, 2015 at 11:44:31PM +0200, Ruben Laban wrote:
Hi Paolo,
This sure sounds interesting. I'll see if I can get around to
deploying latest CVS to my test setup asap. Does the change of the
queuing mechanism introduce a change in configuration as well? Like
new/changed options, different sizing recommendations, etc.
Regards,
Ruben Laban
On 22-8-2015 19:41, Paolo Lucente wrote:
Hi Ruben,
Your email is very timely and i understand such fluctuations between low
and high traffic periods can happen in a libpcap deployment. A new feature
that has been introduced as part of 1.5.2 (which is currently in the CVS
and about to be released) is passing buffers inside pmacct - so from the
core process to the plugins - using a RabbitMQ broker, as an alternative
to the homegrown queueing infrastructure. This should help you working
with small buffers (so that data does not get "delayed" in case of low
traffic times) while keeping the queue size dynamic.
Let me know if this looks of interest to you and, if yes, whether maybe
you have room to give it a try before 1.5.2 release, say throughout next
week.
Cheers,
Paolo
On Fri, Aug 21, 2015 at 08:04:21AM +0200, Ruben Laban wrote:
Hi,
Lately I've been experimenting a lot with plugin_buffer_size,
plugin_pipe_size and print_cache_size. My current conclusion is that
it is in no way possible to tune pmacct in such a way that it will
deal with low and (very) high traffic levels accurately:
* Tune for low traffic: loss of data at high traffic
* Tune for high traffic: data gets "delayed" at low traffic
Am I missing something here is this something I'll have to live with
(or otherwise work around).
What I was hoping for was that by using large buffers (to
accommodate high traffic levels) I'd get to deal properly with low
and high traffic levels. I kinda expected buffers to flushed
whenever data is being flushed to disk (using print plugin right
now). This does not seem to be the case (completely).
The fact that tuning for high traffic levels requires (a lot) more
memory is easier to live with, memory's cheap after all.
Some background info:
Ubuntu 14.01
Pmacct 1.5.1
libpcap, without PF_RING or anything
Current config:
!
debug: false
!
daemonize: true
!
core_proc_name: bond0
!
aggregate[bond0-all]: src_mac, dst_mac, src_host, dst_host,
src_port, dst_port
aggregate[bond0-aggr]: none
!
plugins: print[bond0-all], print[bond0-aggr]
!
!plugin_buffer_size: 1024
!plugin_buffer_size: 4096
!plugin_pipe_size: 10240000
plugin_pipe_size: 40960000
!
files_umask: 027
!files_uid: 0
files_gid: 112
!
interface: bond0.1234
!
promisc: false
!
syslog: daemon
!
pidfile: /var/run/pmacctd-bond0.pid
!
print_output_file[bond0-all]: /tmp/pmacct-bond0-all-%Y%m%d-%H%M.csv
print_output_file[bond0-aggr]: /tmp/pmacct-bond0-aggr-%Y%m%d-%H%M.csv
!
print_refresh_time: 60
!
print_time_roundoff: m
!
print_history: 1m
!
print_cache_entries: 323333
!print_cache_entries: 100003
!
print_output_file_append: true
!
print_markers: true
!
!print_output: json
print_output: csv
Regards,
Ruben Laban
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
_______________________________________________
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
