Hi Ruben, Can you give it another try? I don't manage to reproduce this issue with the CVS - all appears to work good for me. As a workaround, should you still have problems, you can download the latest daily package from the webapage: http://www.pmacct.net/pmacct-daily.tar.gz . As yet another alternative you can check out latest code on GitHub. Please let me know.
Cheers, Paolo On Wed, Aug 26, 2015 at 09:41:29AM +0200, Ruben Laban wrote: > Hi Paolo, > > I just did a fresh CVS checkout and tried to build it. Alas, it > failed. It complains about not finding ../bmp/bmp.h as referenced in > bgp/bgp_logdump.c. And my checkout doesn't even have that directory > (bmp). > > Regards, > Ruben > > On 26-8-2015 01:02, Paolo Lucente wrote: > >Hi Ruben, > > > >Yes, there is some config. You can read chapter "Internal buffering and > >queueing" of the QUICKSTART document: > > > >https://github.com/paololucente/pmacct/blob/master/pmacct/QUICKSTART > > > >Essentially plugin_pipe_size becomes something not to care anymore and > >plugin_pipe_amqp goes to 'true'. Then any extra directive is to config > >aspects to connect to the RabbitMQ exchange (user, password, etc.). > > > >Cheers, > >Paolo > > > >On Sun, Aug 23, 2015 at 11:44:31PM +0200, Ruben Laban wrote: > >>Hi Paolo, > >> > >>This sure sounds interesting. I'll see if I can get around to > >>deploying latest CVS to my test setup asap. Does the change of the > >>queuing mechanism introduce a change in configuration as well? Like > >>new/changed options, different sizing recommendations, etc. > >> > >>Regards, > >>Ruben Laban > >> > >>On 22-8-2015 19:41, Paolo Lucente wrote: > >>>Hi Ruben, > >>> > >>>Your email is very timely and i understand such fluctuations between low > >>>and high traffic periods can happen in a libpcap deployment. A new feature > >>>that has been introduced as part of 1.5.2 (which is currently in the CVS > >>>and about to be released) is passing buffers inside pmacct - so from the > >>>core process to the plugins - using a RabbitMQ broker, as an alternative > >>>to the homegrown queueing infrastructure. This should help you working > >>>with small buffers (so that data does not get "delayed" in case of low > >>>traffic times) while keeping the queue size dynamic. > >>> > >>>Let me know if this looks of interest to you and, if yes, whether maybe > >>>you have room to give it a try before 1.5.2 release, say throughout next > >>>week. > >>> > >>>Cheers, > >>>Paolo > >>> > >>>On Fri, Aug 21, 2015 at 08:04:21AM +0200, Ruben Laban wrote: > >>>>Hi, > >>>> > >>>>Lately I've been experimenting a lot with plugin_buffer_size, > >>>>plugin_pipe_size and print_cache_size. My current conclusion is that > >>>>it is in no way possible to tune pmacct in such a way that it will > >>>>deal with low and (very) high traffic levels accurately: > >>>> > >>>>* Tune for low traffic: loss of data at high traffic > >>>>* Tune for high traffic: data gets "delayed" at low traffic > >>>> > >>>>Am I missing something here is this something I'll have to live with > >>>>(or otherwise work around). > >>>> > >>>>What I was hoping for was that by using large buffers (to > >>>>accommodate high traffic levels) I'd get to deal properly with low > >>>>and high traffic levels. I kinda expected buffers to flushed > >>>>whenever data is being flushed to disk (using print plugin right > >>>>now). This does not seem to be the case (completely). > >>>>The fact that tuning for high traffic levels requires (a lot) more > >>>>memory is easier to live with, memory's cheap after all. > >>>> > >>>>Some background info: > >>>>Ubuntu 14.01 > >>>>Pmacct 1.5.1 > >>>>libpcap, without PF_RING or anything > >>>> > >>>>Current config: > >>>> > >>>>! > >>>>debug: false > >>>>! > >>>>daemonize: true > >>>>! > >>>>core_proc_name: bond0 > >>>>! > >>>>aggregate[bond0-all]: src_mac, dst_mac, src_host, dst_host, > >>>>src_port, dst_port > >>>>aggregate[bond0-aggr]: none > >>>>! > >>>>plugins: print[bond0-all], print[bond0-aggr] > >>>>! > >>>>!plugin_buffer_size: 1024 > >>>>!plugin_buffer_size: 4096 > >>>>!plugin_pipe_size: 10240000 > >>>>plugin_pipe_size: 40960000 > >>>>! > >>>>files_umask: 027 > >>>>!files_uid: 0 > >>>>files_gid: 112 > >>>>! > >>>>interface: bond0.1234 > >>>>! > >>>>promisc: false > >>>>! > >>>>syslog: daemon > >>>>! > >>>>pidfile: /var/run/pmacctd-bond0.pid > >>>>! > >>>>print_output_file[bond0-all]: /tmp/pmacct-bond0-all-%Y%m%d-%H%M.csv > >>>>print_output_file[bond0-aggr]: /tmp/pmacct-bond0-aggr-%Y%m%d-%H%M.csv > >>>>! > >>>>print_refresh_time: 60 > >>>>! > >>>>print_time_roundoff: m > >>>>! > >>>>print_history: 1m > >>>>! > >>>>print_cache_entries: 323333 > >>>>!print_cache_entries: 100003 > >>>>! > >>>>print_output_file_append: true > >>>>! > >>>>print_markers: true > >>>>! > >>>>!print_output: json > >>>>print_output: csv > >>>> > >>>>Regards, > >>>>Ruben Laban > >>>> > >>>>_______________________________________________ > >>>>pmacct-discussion mailing list > >>>>http://www.pmacct.net/#mailinglists > >>> > >>>_______________________________________________ > >>>pmacct-discussion mailing list > >>>http://www.pmacct.net/#mailinglists > >>> > >> > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
