I can't speak for the Spark Community, but checking their code, DirectKafkaStream and KafkaRDD use the SimpleConsumer API:
https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/DirectKafkaInputDStream.scala https://github.com/apache/spark/blob/master/external/kafka/src/main/scala/org/apache/spark/streaming/kafka/KafkaRDD.scala On Fri, Aug 28, 2015 at 11:32 AM, Cassa L <lcas...@gmail.com> wrote: > Hi I am using below Spark jars with Direct Stream API. > spark-streaming-kafka_2.10 > > When I look at its pom.xml, Kafka libraries that its pulling in is > <groupId>org.apache.kafka</groupId> > <artifactId>kafka_${scala.binary.version}</artifactId> > <version>0.8.2.1</version> > > > I believe this DirectStream API uses SimpleConsumer API. Can someone from > Spark community confirm too? > > Thanks, > LCassa. > > On Fri, Aug 28, 2015 at 11:12 AM, Sriharsha Chintalapani <ka...@harsha.io> > wrote: > > > SSL is supported for new producer and consumer api and old api (simple > > consumer and high-level consumer) is not supported. > > I think spark uses simple consumer? if so its not supported. > > > > Thanks, > > Harsha > > > > > > On August 28, 2015 at 11:00:30 AM, Cassa L (lcas...@gmail.com) wrote: > > > > Hi, > > I was going through SSL setup of Kafka. > > > https://cwiki.apache.org/confluence/display/KAFKA/Deploying+SSL+for+Kafka > > However, I am also using Spark-Kafka streaming to read data from Kafka. > Is > > there a way to activate SSL for spark streaming API or not possible at > > all? > > > > Thanks, > > LCassa > > > > >
