Re: Collecting thousands of sources

2014-09-05 Thread Javi Roman
Hello, The scenario which is describing Juanfra is related with the question I made a few days ago [1]. You can not install Flume agents in the SNMP managed devices, and you can not modify any software in the SNMP managed devide for use Flume client SDK (if I understand correctly your idea Ashish

Re: Collecting thousands of sources

2014-09-05 Thread Ashish
Jovi, I have NMS background, so understand your concern. Answers inline On Fri, Sep 5, 2014 at 12:44 PM, Javi Roman wrote: > Hello, > > The scenario which is describing Juanfra is related with the question > I made a few days ago [1]. > I am not sure about this, Juanfra is a better person to

Re: Collecting thousands of sources

2014-09-05 Thread JuanFra Rodriguez Cardoso
Thanks, both of you! @Ashish, Javi's thoughts are right. My use case is focused on sources for consuming SNMP traps. I came here from the already open discussion [1] with hopes anyone was facing with this problem. Your solution based on Async SNMP walker would help us to scale that thousands of a

Re: Collecting thousands of sources

2014-09-05 Thread Ashish
On Fri, Sep 5, 2014 at 4:01 PM, JuanFra Rodriguez Cardoso < juanfra.rodriguez.card...@gmail.com> wrote: > Thanks, both of you! > > @Ashish, Javi's thoughts are right. My use case is focused on sources for > consuming SNMP traps. I came here from the already open discussion [1] with > hopes anyone

Re: Flume-kafka problem

2014-09-05 Thread Santiago Mola
Hi, 2014-08-29 11:27 GMT+02:00 镕 : > > In my project i used flume and kafkaSink, but with a strange phenomeno, the flume didnot work after an hour, that mean the files in flume spooldir cannot be process normally, > it seems like something wrong but with no errors,the flume-ng server log has not

Re: Collecting thousands of sources

2014-09-05 Thread JuanFra Rodriguez Cardoso
Got it Ashish! Thanks for your support and suggestions Cheers, JuanFra El 05/09/2014 13:28, "Ashish" escribió: > > > > On Fri, Sep 5, 2014 at 4:01 PM, JuanFra Rodriguez Cardoso < > juanfra.rodriguez.card...@gmail.com> wrote: > >> Thanks, both of you! >> >> @Ashish, Javi's thoughts are right. My

Re: Newbie - Sink question

2014-09-05 Thread Kevin Warner
Thanks Andrew, Ashish and Sharinder for your response. I have a large number of JSON files which are 2K size each on Tomcat servers. We are using rsync to get the files from the Tomcat servers to the EC2 compute instances. Lets say we have 4 Tomcat servers, do we need 4 machines (EC2) with Flume o

Re: Newbie - Sink question

2014-09-05 Thread Ashish
On Sat, Sep 6, 2014 at 4:42 AM, Kevin Warner wrote: > Thanks Andrew, Ashish and Sharinder for your response. > > I have a large number of JSON files which are 2K size each on Tomcat > servers. We are using rsync to get the files from the Tomcat servers to the > EC2 compute instances. Lets say we