Re: [E] Re: Pyspark Kafka Structured Stream not working.

2020-05-07 Thread Jungtaek Lim
..@gmail.com] > *Sent:* Wednesday, May 06, 2020 8:24 PM > *To:* Vijayant Kumar > *Cc:* user@spark.apache.org > *Subject:* [E] Re: Pyspark Kafka Structured Stream not working. > > > > *[EXTERNAL EMAIL]* DO NOT CLICK links or attachments unless you recognize > the sender and kno

RE: [E] Re: Pyspark Kafka Structured Stream not working.

2020-05-06 Thread Vijayant Kumar
: [E] Re: Pyspark Kafka Structured Stream not working. [EXTERNAL EMAIL] DO NOT CLICK links or attachments unless you recognize the sender and know the content is safe. Hi, 1. You seem to use DStream (Spark Streaming), not Structured Streaming. 2. I'm not familiar with pyspark, but looks lik

Re: Pyspark Kafka Structured Stream not working.

2020-05-06 Thread Jungtaek Lim
Hi, 1. You seem to use DStream (Spark Streaming), not Structured Streaming. 2. I'm not familiar with pyspark, but looks like the error message is very clear - Kafka doesn't allow such name for "client.id". The error message guides the naming rule, so you may need to be adopted with such convention