Ahh perfect, sounds like nfacctd_time_new is exactly what I need. Thank you Paolo.
Ed > On Oct 22, 2015, at 10:58 AM, Paolo Lucente <[email protected]> wrote: > > Hi Edward, > > Mario is right. > > Plus you can set nfacctd_time_new to true to make nfacctd use the time > of arrival at the collector (rather than individual flow start times) > for time binning. This approach will be less precise than using flow > start times; a few considerations at this propo: 1) if flow timers at > the router side are set low, it will only be slightly less precise; then > again if this is acceptable or not depends on the use-case; 2) it does > guarantee only the last/current file is going to be updated. > > Cheers, > Paolo > >> On Thu, Oct 22, 2015 at 07:51:23AM +0000, Jentsch, Mario wrote: >> Hey Edward, >> >> each file contains the data of one timebin. Flows spread over a timeframe >> longer than one timebin cause pmacct to create/update multiple files. >> Depending on how long your Netflow exporter keeps the flow records before it >> flushes them to the collector, the created/updated files will be more or >> less far in the past. >> >> Regards, >> Mario >> >> From: pmacct-discussion [mailto:[email protected]] On >> Behalf Of Edward Henigin >> Sent: Wednesday, October 21, 2015 10:34 PM >> To: pmacct-discuss. >> Subject: Re: [pmacct-discussion] multiple nfacctd files being written >> >> And sometimes the filenames look correct but again they all come out at the >> same time: >> >> (root) packet1:/opt/pmacct/data# ls -lt | head >> total 6670424 >> -rw------- 1 root root 3005237 Oct 21 15:28 nfacct-20151021-1525.csv >> -rw------- 1 root root 1461133 Oct 21 15:28 nfacct-20151021-1527.csv >> -rw------- 1 root root 2292406 Oct 21 15:28 nfacct-20151021-1526.csv >> -rw------- 1 root root 3505033 Oct 21 15:27 nfacct-20151021-1524.csv >> -rw------- 1 root root 2178942 Oct 21 15:26 nfacct-20151021-1523.csv >> -rw------- 1 root root 2551924 Oct 21 15:24 nfacct-20151021-1522.csv >> -rw------- 1 root root 3633218 Oct 21 15:23 nfacct-20151021-1521.csv >> -rw------- 1 root root 2407956 Oct 21 15:22 nfacct-20151021-1520.csv >> -rw------- 1 root root 2669403 Oct 21 15:21 nfacct-20151021-1519.csv >> (root) packet1:/opt/pmacct/data# perl -le 'foreach (qw{1519 1520 1521 1522 >> 1523 1524 1525 1526 1527}){print("$_: ". >> localtime((stat("nfacct-20151021-$_.csv"))[9]))}' >> 1519: Wed Oct 21 15:21:46 2015 >> 1520: Wed Oct 21 15:22:50 2015 >> 1521: Wed Oct 21 15:23:53 2015 >> 1522: Wed Oct 21 15:24:58 2015 >> 1523: Wed Oct 21 15:26:02 2015 >> 1524: Wed Oct 21 15:27:06 2015 >> 1525: Wed Oct 21 15:28:10 2015 >> 1526: Wed Oct 21 15:28:10 2015 >> 1527: Wed Oct 21 15:28:10 2015 >> >> Seems related? >> >> >> On Wed, Oct 21, 2015 at 3:28 PM, Edward Henigin >> <[email protected]<mailto:[email protected]>> wrote: >> Hi Paolo, >> >> Running pmacct 1.5.2, simply using the print plugin, I'm getting multiple >> files coming out at the same time with filenames suggesting they should be >> coming out at different times, and sometimes very strange filenames like the >> data is very old. >> >> ls output: >> >> (root) packet1:/opt/pmacct/data# ls -lt | head >> total 6649800 >> -rw------- 1 root root 1785873 Oct 21 15:20 nfacct-20151021-1450.csv >> -rw------- 1 root root 2740509 Oct 21 15:20 nfacct-20151021-1518.csv >> -rw------- 1 root root 2597403 Oct 21 15:20 nfacct-20151021-1519.csv >> -rw------- 1 root root 2778987 Oct 21 15:19 nfacct-20151021-1517.csv >> -rw------- 1 root root 3017902 Oct 21 15:18 nfacct-20151021-1516.csv >> -rw------- 1 root root 2860626 Oct 21 15:17 nfacct-20151021-1515.csv >> -rw------- 1 root root 3013418 Oct 21 15:16 nfacct-20151021-1514.csv >> -rw------- 1 root root 3433555 Oct 21 15:15 nfacct-20151021-1513.csv >> -rw------- 1 root root 2752513 Oct 21 15:14 nfacct-20151021-1512.csv >> >> and timestamps to the second: >> >> (root) packet1:/opt/pmacct/data# perl -le 'foreach (qw{1512 1513 1514 1515 >> 1516 1517 1518 1450}){print("$_: ". >> localtime((stat("nfacct-20151021-$_.csv"))[9]))}' >> 1512: Wed Oct 21 15:14:18 2015 >> 1513: Wed Oct 21 15:15:22 2015 >> 1514: Wed Oct 21 15:16:26 2015 >> 1515: Wed Oct 21 15:17:30 2015 >> 1516: Wed Oct 21 15:18:34 2015 >> 1517: Wed Oct 21 15:19:38 2015 >> 1518: Wed Oct 21 15:20:42 2015 >> 1450: Wed Oct 21 15:20:42 2015 >> >> Where is filename "...-1450" coming from, and why is it coming out at the >> same time as -1518? >> >> Configuration: >> >> ! nfacctd configuration file >> aggregate: peer_src_ip,in_iface,dst_host,dst_mask >> plugins: print >> plugin_buffer_size: 10240 >> imt_buckets: 157 >> imt_mem_pools_number: 256 >> imt_mem_pools_size: 32768 >> syslog: daemon >> daemonize: true >> ! >> print_refresh_time: 64 >> print_history: 1m >> print_output: csv >> print_output_file: /opt/pmacct/data/nfacct-%Y%m%d-%H%M.csv >> print_output_file_append: true >> ! >> nfacctd_port: 2055 >> >> Syslog: >> >> Oct 21 15:15:21 packet1 nfacctd[558]: INFO ( default/print ): *** Purging >> cache - START (PID: 558) *** >> Oct 21 15:15:22 packet1 nfacctd[558]: INFO ( default/print ): *** Purging >> cache - END (PID: 558, QN: 80643/80847, ET: 1) *** >> Oct 21 15:16:25 packet1 nfacctd[593]: INFO ( default/print ): *** Purging >> cache - START (PID: 593) *** >> Oct 21 15:16:26 packet1 nfacctd[593]: INFO ( default/print ): *** Purging >> cache - END (PID: 593, QN: 74092/74503, ET: 1) *** >> Oct 21 15:17:29 packet1 nfacctd[651]: INFO ( default/print ): *** Purging >> cache - START (PID: 651) *** >> Oct 21 15:17:30 packet1 nfacctd[651]: INFO ( default/print ): *** Purging >> cache - END (PID: 651, QN: 74211/74621, ET: 1) *** >> Oct 21 15:18:33 packet1 nfacctd[690]: INFO ( default/print ): *** Purging >> cache - START (PID: 690) *** >> Oct 21 15:18:34 packet1 nfacctd[690]: INFO ( default/print ): *** Purging >> cache - END (PID: 690, QN: 71470/72233, ET: 1) *** >> Oct 21 15:19:37 packet1 nfacctd[739]: INFO ( default/print ): *** Purging >> cache - START (PID: 739) *** >> Oct 21 15:19:38 packet1 nfacctd[739]: INFO ( default/print ): *** Purging >> cache - END (PID: 739, QN: 69195/73637, ET: 1) *** >> Oct 21 15:20:41 packet1 nfacctd[779]: INFO ( default/print ): *** Purging >> cache - START (PID: 779) *** >> Oct 21 15:20:42 packet1 nfacctd[779]: INFO ( default/print ): *** Purging >> cache - END (PID: 779, QN: 67848/83867, ET: 1) *** >> >> >> Ed > >> _______________________________________________ >> pmacct-discussion mailing list >> http://www.pmacct.net/#mailinglists > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
