Spark now shades its own protobuf dependency so protobuf 2.4.1 should't be getting pulled in unless you are directly using akka yourself. Are you?
Does your project have other dependencies that might be indirectly pulling in protobuf 2.4.1? It would be helpful if you could list all of your dependencies including the exact Spark version and other libraries. - Patrick On Sun, Mar 30, 2014 at 10:03 PM, Vipul Pandey <vipan...@gmail.com> wrote: > I'm using ScalaBuff (which depends on protobuf2.5) and facing the same > issue. any word on this one? > On Mar 27, 2014, at 6:41 PM, Kanwaldeep <kanwal...@gmail.com> wrote: > > > We are using Protocol Buffer 2.5 to send messages to Spark Streaming 0.9 > with > > Kafka stream setup. I have protocol Buffer 2.5 part of the uber jar > deployed > > on each of the spark worker nodes. > > The message is compiled using 2.5 but then on runtime it is being > > de-serialized by 2.4.1 as I'm getting the following exception > > > > java.lang.VerifyError (java.lang.VerifyError: class > > com.snc.sinet.messages.XServerMessage$XServer overrides final method > > getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;) > > java.lang.ClassLoader.defineClass1(Native Method) > > java.lang.ClassLoader.defineClassCond(ClassLoader.java:631) > > java.lang.ClassLoader.defineClass(ClassLoader.java:615) > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141) > > > > Suggestions on how I could still use ProtoBuf 2.5. Based on the article - > > https://spark-project.atlassian.net/browse/SPARK-995 we should be able > to > > use different version of protobuf in the application. > > > > > > > > > > > > -- > > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Using-ProtoBuf-2-5-for-messages-with-Spark-Streaming-tp3396.html > > Sent from the Apache Spark User List mailing list archive at Nabble.com. > >
