Glad to hear that was the issue. dave
On Tue, Oct 29, 2013 at 12:53 AM, wesley dias <[email protected]>wrote: > Thanks Roshan. But issue is now resolved. I compiled flume 1.4 from source > using protobuf 2.5 and this solved the below mentioned issue. My guess is > that the distribution available for flume 1.4 from the release site must > have been compiled using protobuf 2.4 > > Regards, > > Wesley > > ------------------------------ > Date: Mon, 28 Oct 2013 13:41:52 -0700 > > Subject: Re: Exception in thread > "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink > From: [email protected] > To: [email protected] > > HDP already includes hadoop 2.2.0 + Flume 1.4. You'd be better of just > using instead of building and testing that combo out on your own. > > http://hortonworks.com/products/hdp-2/?b=2#overview > > > On Mon, Oct 28, 2013 at 1:19 PM, wesley dias <[email protected]>wrote: > > Hi David, > > Thanks for the quick reply.Does this mean that I would need to compile the > flume from source using protobuf 2.5 instead of directly using the flume > distribution available for download from the site? Is flume 1.4 based of > protobuf 2.4? > I am not sure of the same hence I am asking these questions. > > Regards, > > Wesley > ------------------------------ > Date: Mon, 28 Oct 2013 16:11:16 -0400 > Subject: Re: Exception in thread > "SinkRunner-PollingRunner-DefaultSinkProcessor" on writing to HDFS sink > From: [email protected] > To: [email protected] > > > Looks like you have a protocol buffer version problem. Do you have a > dependency on 2.5.0 version of protocol buffers in your pom? > > see > > http://code.google.com/p/protobuf/issues/detail?id=493 > > > > On Mon, Oct 28, 2013 at 4:02 PM, wesley dias <[email protected]>wrote: > > Hi, > > I am using the latest flume build 1.4 along with hadoop 2.2.0 to collect > twitter stream and write to HDFS sink. However, on writing to HDFS I get > the following error: > > 2013-10-28 09:41:23,481 (SinkRunner-PollingRunner-**DefaultSinkProcessor) > [ERROR - > org.apache.flume.sink.hdfs.**HDFSEventSink.process(**HDFSEventSink.java:422)] > process failed > java.lang.VerifyError: class org.apache.hadoop.hdfs.**protocol.proto.** > ClientNamenodeProtocolProtos$**SetOwnerRequestProto overrides final > method getUnknownFields.()Lcom/**google/protobuf/**UnknownFieldSet; > at java.lang.ClassLoader.**defineClass1(Native Method) > at java.lang.ClassLoader.**defineClass(ClassLoader.java:**800) > at java.security.**SecureClassLoader.defineClass(** > SecureClassLoader.java:142) > at java.net.URLClassLoader.**defineClass(URLClassLoader.**java:449) > at java.net.URLClassLoader.**access$100(URLClassLoader.**java:71) > at java.net.URLClassLoader$1.run(**URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(**URLClassLoader.java:355) > at java.security.**AccessController.doPrivileged(**Native Method) > at java.net.URLClassLoader.**findClass(URLClassLoader.java:**354) > at java.lang.ClassLoader.**loadClass(ClassLoader.java:**425) > at sun.misc.Launcher$**AppClassLoader.loadClass(**Launcher.java:308) > at java.lang.ClassLoader.**loadClass(ClassLoader.java:**358) > at java.lang.Class.**getDeclaredMethods0(Native Method) > at java.lang.Class.**privateGetDeclaredMethods(**Class.java:2531) > at java.lang.Class.**privateGetPublicMethods(Class.**java:2651) > at java.lang.Class.**privateGetPublicMethods(Class.**java:2661) > at java.lang.Class.getMethods(**Class.java:1467) > > I am currently new to flume and any help shall be really appreciated. > > Regards, > > Wesley > > > > > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity > to which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
