Re: Kafka version dependency in Spark 1.2

2014-11-10 Thread Helena
Version 0.8.2-beta is published. I'd consider waiting on this, it has quite a few nice changes coming. https://archive.apache.org/dist/kafka/0.8.2-beta/RELEASE_NOTES.html I started the 0.8.1.1 upgrade in a branch a few weeks ago but abandoned it because I wasn't sure if there was interest beyond m

Re: Kafka version dependency in Spark 1.2

2014-11-10 Thread Sean McNamara
> Can the 0.8.1.1 client still talk to 0.8.0 versions of Kafka Yes it can. "0.8.1 is fully compatible with 0.8." It is buried on this page: http://kafka.apache.org/documentation.html In addition to the pom version bump SPARK-2492 would bring the kafka streaming receiver (which was originally

Re: Kafka version dependency in Spark 1.2

2014-11-10 Thread Matei Zaharia
Just curious, what are the pros and cons of this? Can the 0.8.1.1 client still talk to 0.8.0 versions of Kafka, or do you need it to match your Kafka version exactly? Matei > On Nov 10, 2014, at 9:48 AM, Bhaskar Dutta wrote: > > Hi, > > Is there any plan to bump the Kafka version dependency