multi-tier avro agents configuration problem

2014-08-28 Thread Blade Liu
Hi folks, I ran into a configuration problem of setting up multi-tier avro agents. The flow is as follows, and data is generated on tier1(slave3) by using "flume-ng avro-client --conf conf -H localhost -p 41000 -F /etc/hosts" tier1: slave3, avro source->avro sink tier2: slave2, avro source->logge

Re: Flume 1.4 HDFS Sink Cannot Reconnect

2014-08-28 Thread Andrew O'Neill
Hello, Did anyone have a chance to look at this issue? Thanks, [cid:15AE4044-2A22-4E71-959A-C79917667E52] Andrew O'Neill | Paytronix 74 Bridge Street, Suite 400 Newton, MA 02458 p. 617.649.3300 x256 [cid:4AEC1A82-7442-4A90-A364-8908C948D63F]

Re: multi-tier avro agents configuration problem

2014-08-28 Thread Jeff Lord
I think you want this to bind to slave2 or even better the appropriate ip tier2.sources.source2.bind= slave3 If that doesn't work please send the log snippet. On Thursday, August 28, 2014, Blade Liu wrote: > Hi folks, > > I ran into a configuration problem of setting up multi-tier avro age

FileChannel Replays consistently take a long time

2014-08-28 Thread Gautham Varada
I’ve the same issue but in my case the data on the file channel is > 50g and flume is just spinning the CPU and ports haven’t opened in the last 8 hrs.

Re: FileChannel Replays consistently take a long time

2014-08-28 Thread Gary Malouf
Check out https://issues.apache.org/jira/browse/FLUME-2450 On Thu, Aug 28, 2014 at 10:01 PM, Gautham Varada wrote: > I’ve the same issue but in my case the data on the file channel is > 50g > and flume is just spinning the CPU and ports haven’t opened in the last 8 > hrs.

Re: FileChannel Replays consistently take a long time

2014-08-28 Thread Gautham Varada
Thanks Gary. Have you tried this patch ? Sent from my iPhone > On 29-Aug-2014, at 7:47 am, Gary Malouf wrote: > > Check out https://issues.apache.org/jira/browse/FLUME-2450 > > >> On Thu, Aug 28, 2014 at 10:01 PM, Gautham Varada wrote: >> I’ve the same issue but in my case the data on the fi

Re: FileChannel Replays consistently take a long time

2014-08-28 Thread Gary Malouf
I've tried out an earlier version of it that helped a lot. Hari was making changes to not require a user to change their memory settings. We try to stay on stable release versions of Flume in prod, so we'll look to upgrade then. On Thu, Aug 28, 2014 at 10:47 PM, Gautham Varada wrote: > Thanks