Re: Spark Kafka adapter questions

2018-08-20 Thread Ted Yu
ver-0 > > 18/08/20 22:29:34 INFO AbstractCoordinator: Discovered coordinator FQDN>:9093 (id: 2147483647 rack: null) for group > spark-kafka-source-1aa50598-99d1-4c53-a73c-fa6637a219b2--1338794993-driver-0. > > > > > > Also, I’m not sure if it’s relevant but I am running on Datab

RE: Spark Kafka adapter questions

2018-08-20 Thread Basil Hariri
github master on 8/8/18 and Kafka v2.0. Thanks so much for your help, let me know if there’s anything else I can provide Sincerely, Basil From: Ted Yu Sent: Friday, August 17, 2018 4:20 PM To: basil.har...@microsoft.com.invalid Cc: dev Subject: Re: Spark Kafka adapter questions If you have

Re: Spark Kafka adapter questions

2018-08-17 Thread Ted Yu
If you have picked up all the changes for SPARK-18057, the Kafka “broker” supporting v1.0+ should be compatible with Spark's Kafka adapter. Can you post more details about the “failed to send SSL close message” errors ? (The default Kafka version is 2.0.0 in Spark Kafka adapter after SPARK-18057