Re: Spark structured streaming + kafka

2018-07-09 Thread Pulkit Manchanda
You were right. It worked. Thanks Pulkit On Mon, Jul 9, 2018 at 5:29 PM, Liam Clarke wrote: > In your SBT script you've specified that your Kafka data source jar is > provided. > > Please read the documentation of dependency scopes. > https://maven.apache.org/guides/introduction/introduction-to

Re: Spark structured streaming + kafka

2018-07-09 Thread Liam Clarke
In your SBT script you've specified that your Kafka data source jar is provided. Please read the documentation of dependency scopes. https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope Kind regards, Liam Clarke On Tue, 10 Jul. 2018, 9:24 am Pu

Spark structured streaming + kafka

2018-07-09 Thread Pulkit Manchanda
Hi, I am trying to do structured streaming with kafka as source. I am unable to get pass of this code. val df = spark .readStream .format("org.apache.spark.sql.kafka010.KafkaSourceProvider") .option("kafka.bootstrap.servers", "localhost:8082") .option("subscribe", "jsontest") .load() T