Whether or not flume can handle 20k eps will depend on several factors. The main ones being: 1. What is the avg size of event 2. What source will you be using
With that said I have seen a single flume agent handle well over 20k eps using the multiport syslog source. Here is a link to a presentation given by Arvind Prabhakar on planning a flume deployment. http://goo.gl/FsfmmC -Jeff On Wed, Sep 24, 2014 at 10:53 PM, Blade Liu <hafzc...@gmail.com> wrote: > Hi, > > I'm going to deploy Flume in production systems, but a little worried > about its performance in real-world environment. Could anyone tell me about > Flume's actual performance in production environment? say, if Flume can > deal with 20,000 events per second from a single source(and what about > 100-200 sources with one final HDFS sink). > > In addition, to reach good performance of tens of thousands of events per > second, how many servers(agents) should be used? More agents(and more > tiers), better performance? > > Thanks very much for your suggestions. > > > Cheers, > Blade >
