On 2012-06-16 1656, Paolo Lucente wrote: > Hi Tiernan, Hi Paolo,
> It is not clear to me whether you tried to set higher values for both > plugin_buffer_size and plugin_pipe_size - say to 100MB and 1MB or 10MB > respectively. A value of 16384 for plugin_buffer_size is definitely > > too small. I assume you meant buffer and pipe sizes the other way around? The documentation seems to indicate that plugin_pipe_size should be bigger than plugin_buffer_size. Being that I've got a few plugins configured setting the pipe size to 100MB causes pmacct to chew around 600MB of ram which seems a little high (it's only a counter after all? There's only a few hosts that it's being aggregated on). Fortunately the following also worked fine for stopping the missing data messages in a more manageable memory footprint: plugin_pipe_size: 10485760 plugin_buffer_size: 1048576 This seems to have fixed the bogus data entries too, although during high load the CPU usage skyrockets. With 100MB pipe size and 10MB buffer size the CPU usage was about the same. Is there any way to get it down? A bit of an architectural question but why is such a large buffer needed between the core and plugin processes? Where is the aggregation being done, plugin side or core side? It seems like the core process is matching the filter then passing each packet header individually to the plugins incurring copying and requiring such large buffers - I could be wrong and I've not studied the internals of pmacct - but could resource usage be reduced by doing the aggregation and packet/byte counting in the core process then handing those figures off to the plugin every second or so help in high traffic instances? Possibly save moving around data in memory as much. > Cheers, > Paolo _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
