Thanks cody, so is it means if old kafka consumer 0.8.1.1 works with kafka cluster version 0.8.2 then spark streaming 1.3 should also work?
I have tested standalone consumer kafka consumer 0.8.0 with kafka cluster 0.8.2 and that works. On Thu, Jul 9, 2015 at 9:58 PM, Cody Koeninger <c...@koeninger.org> wrote: > It's the consumer version. Should work with 0.8.2 clusters. > > On Thu, Jul 9, 2015 at 11:10 AM, Shushant Arora <shushantaror...@gmail.com > > wrote: > >> Does spark streaming 1.3 requires kafka version 0.8.1.1 and is not >> compatible with kafka 0.8.2 ? >> >> As per maven dependency of spark streaming 1.3 with kafka >> >> >> <dependencies><dependency><groupId>org.apache.spark</groupId><artifactId>spark-streaming_2.10</artifactId><version> >> 1.3.0 >> </version><scope>provided</scope><exclusions><exclusion><artifactId>spark-core_2.10</artifactId><groupId>org.apache.spark</groupId></exclusion></exclusions></dependency> >> >> <dependency><groupId>org.apache.kafka</groupId><artifactId>kafka_2.10</artifactId><version> >> 0.8.1.1 >> </version><scope>compile</scope><exclusions><exclusion><artifactId>jmxri</artifactId><groupId>com.sun.jmx</groupId></exclusion><exclusion><artifactId>jmxtools</artifactId><groupId>com.sun.jdmk</groupId></exclusion><exclusion><artifactId>jopt-simple</artifactId><groupId>net.sf.jopt-simple</groupId></exclusion><exclusion><artifactId>slf4j-simple</artifactId><groupId>org.slf4j</groupId></exclusion><exclusion><artifactId>zookeeper</artifactId><groupId>org.apache.zookeeper</groupId></exclusion></exclusions></dependency> >> >> >> Is 0.8.1.1 is consumer version or it means spark streaming 1.3 is >> compatible with kafka cluster version 0.8.1.1 only and not with 0.8.2 ? >> >> >
