On 08/10/2015 10:33, Николай Кинаш wrote: > I have a very large sequence of files from nfcapd.. I want to get fast > TOP 30 flows ordered by bytes via nfdump. Right now, i can get them in > 30 minutes. I wonder about parallelize of nfdump. Have developers > tried to create the parallel nfdump?(maybe there is some architecture > restriction, in which parallel version can't be done) > Are developers plan to create the parallel nfdump? Is it possible? Sounds like what you want is to summarise nfdump files with map-reduce.
It would be an interesting project for someone to implement a Hadoop RecordReader for nfdump files; and/or for nfcapd to stream records into Kafka and then process with Storm or Samza, which would give you continuous real-time reporting. All of those tools are unfortunately in Java :-( ------------------------------------------------------------------------------ _______________________________________________ Nfdump-discuss mailing list Nfdump-discuss@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfdump-discuss
