Re: Spark Streaming - Direct Approach

2016-07-15 Thread Cody Koeninger
We've been running direct stream jobs in production for over a year, with uptimes in the range of months. I'm pretty slammed with work right now, but when I get time to submit a PR for the 0.10 docs i'll remove the experimental note from 0.8 On Mon, Jul 11, 2016 at 4:35 PM, Tathagata Das wrote:

Re: Spark Streaming - Direct Approach

2016-07-11 Thread Tathagata Das
Aah, the docs have not been updated. They are totally in production in many place. Others should chime in as well. On Mon, Jul 11, 2016 at 1:43 PM, Mail.com wrote: > Hi All, > > Can someone please confirm if streaming direct approach for reading Kafka > is still experimental or can it be used fo

Re: Spark Streaming - Direct Approach

2016-07-11 Thread Andy Davidson
Hi Pradeep I can not comment about experimental or production, how ever I recently started a POC using direct approach. Its been running off and on for about 2 weeks. In general it seems to work really well. One thing that is not clear to me is how the cursor is manage. E.G. I have my topic set t