Re: problem with kafka createDirectStream ..

2016-12-09 Thread Cody Koeninger
I'd say unzip your actual assembly jar and verify whether the kafka consumer classes are 0.10.1 or 0.10.0. We've seen reports of odd behavior with 0.10.1 classes. Possibly unrelated, but good to eliminate. On Fri, Dec 9, 2016 at 10:38 AM, Debasish Ghosh wrote: > oops .. it's 0.10.0 .. sorry for

Re: problem with kafka createDirectStream ..

2016-12-09 Thread Debasish Ghosh
oops .. it's 0.10.0 .. sorry for the confusion .. On Fri, Dec 9, 2016 at 10:07 PM, Debasish Ghosh wrote: > My assembly contains the 0.10.1 classes .. Here are the dependencies > related to kafka & spark that my assembly has .. > > libraryDependencies ++= Seq( > "org.apache.kafka" % "kaf

Re: problem with kafka createDirectStream ..

2016-12-09 Thread Debasish Ghosh
My assembly contains the 0.10.1 classes .. Here are the dependencies related to kafka & spark that my assembly has .. libraryDependencies ++= Seq( "org.apache.kafka" % "kafka-streams" % "0.10.0.0", "org.apache.spark" %% "spark-streaming-kafka-0-10" % spark,

Re: problem with kafka createDirectStream ..

2016-12-09 Thread Cody Koeninger
When you say 0.10.1 do you mean broker version only, or does your assembly contain classes from the 0.10.1 kafka consumer? On Fri, Dec 9, 2016 at 10:19 AM, debasishg wrote: > Hello - > > I am facing some issues with the following snippet of code that reads from > Kafka and creates DStream. I am u