Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Bernardo de Seabra
; > We will work towards upgrading to CDH 4.2.1 soon, and hopefully, this > issue is resolved. > > ~Rahul. > > -- > *From:* Hari Shreedharan > *To:* "user@flume.apache.org" > *Sent:* Monday, May 13, 2013 7:58 PM > > *Sub

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Rahul Ravindran
ubject: Re: IOException with HDFS-Sink:flushOrSync The patch also made it to Hadoop 2.0.3. On Monday, May 13, 2013, Hari Shreedharan wrote: Looks like CDH4.2.1 does have that patch:  http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.1.CHANGES.txt (but it was not in CDH4.1.2) > > > >

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Hari Shreedharan
The patch also made it to Hadoop 2.0.3. On Monday, May 13, 2013, Hari Shreedharan wrote: > Looks like CDH4.2.1 does have that patch: > http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.1.CHANGES.txt (but > it was not in CDH4.1.2) > > > Hari > > -- > Hari Shreedharan > > On Monday, May 1

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Hari Shreedharan
Looks like CDH4.2.1 does have that patch: http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.1.CHANGES.txt (but it was not in CDH4.1.2) Hari -- Hari Shreedharan On Monday, May 13, 2013 at 7:23 PM, Rahul Ravindran wrote: > We are using cdh 4.1.2 - Hadoop version 2.0.0. Looks like c

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Rahul Ravindran
We are using cdh 4.1.2 - Hadoop version 2.0.0. Looks like cdh 4.2.1 also uses the same Hadoop version. Any suggestions on any mitigations? Sent from my phone.Excuse the terseness. On May 13, 2013, at 7:12 PM, Hari Shreedharan wrote: > What version of Hadoop are you using? Looks like you are ge

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Hari Shreedharan
What version of Hadoop are you using? Looks like you are getting hit by https://issues.apache.org/jira/browse/HADOOP-6762. Hari -- Hari Shreedharan On Monday, May 13, 2013 at 6:50 PM, Matt Wise wrote: > So we've just had this happen twice to two different flume machines... we're > using

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Matt Wise
So we've just had this happen twice to two different flume machines... we're using the HDFS sink as well, but ours is writing to an S3N:// URL. Both times our sink stopped working and the filechannel clogged up immediately causing serious problems. A restart of Flume worked -- but the filechanne

Re: IOException with HDFS-Sink:flushOrSync

2013-05-13 Thread Rahul Ravindran
Pinging again since this has been happening a lot more frequently recently From: Rahul Ravindran To: User-flume Sent: Tuesday, May 7, 2013 8:42 AM Subject: IOException with HDFS-Sink:flushOrSync Hi,    We have noticed this a few times now where we appear t