Re: Connectors using new Kafka consumer API

2016-11-10 Thread Mark Grover
Ok, I understand your point, thanks. Let me see what I can be done there. I may come back if it doesn't work out there:-) On Wed, Nov 9, 2016 at 9:25 AM, Cody Koeninger wrote: > Ok... in general it seems to me like effort would be better spent > trying to help upstream, as opposed to us making a

Re: Connectors using new Kafka consumer API

2016-11-09 Thread Cody Koeninger
Ok... in general it seems to me like effort would be better spent trying to help upstream, as opposed to us making a 5th slightly different interface to kafka (currently have 0.8 receiver, 0.8 dstream, 0.10 dstream, 0.10 structured stream) On Tue, Nov 8, 2016 at 10:05 PM, Mark Grover wrote: > I t

Re: Connectors using new Kafka consumer API

2016-11-08 Thread Mark Grover
I think they are open to others helping, in fact, more than one person has worked on the JIRA so far. And, it's been crawling really slowly and that's preventing adoption of Spark's new connector in secure Kafka environments. On Tue, Nov 8, 2016 at 7:59 PM, Cody Koeninger wrote: > Have you asked

Re: Connectors using new Kafka consumer API

2016-11-08 Thread Cody Koeninger
Have you asked the assignee on the Kafka jira whether they'd be willing to accept help on it? On Tue, Nov 8, 2016 at 5:26 PM, Mark Grover wrote: > Hi all, > We currently have a new direct stream connector, thanks to work by Cody and > others on SPARK-12177. > > However, that can't be used in secu