RE: [spark-streaming] kafka source and flow control

2014-08-12 Thread Gwenhael Pasquiers
issue with file (hdfs) inputs ? how can I be sure the input won’t “overflow” the process chain ? From: Tobias Pfeiffer [mailto:t...@preferred.jp] Sent: mardi 12 août 2014 02:58 To: Gwenhael Pasquiers Cc: u...@spark.incubator.apache.org Subject: Re: [spark-streaming] kafka source and flow control

RE: [spark-streaming] kafka source and flow control

2014-08-11 Thread Gwenhael Pasquiers
hdfs’s maximum). I think the kind of self-regulating system you describe would be too difficult to implement and probably unreliable (even more with the fact that we have multiple slaves). From: Tobias Pfeiffer [mailto:t...@preferred.jp] Sent: lundi 11 août 2014 11:44 To: Gwenhael Pasquiers

RE: [spark-streaming] kafka source and flow control

2014-08-11 Thread Gwenhael Pasquiers
[mailto:t...@preferred.jp] Sent: lundi 11 août 2014 11:44 To: Gwenhael Pasquiers Subject: Re: [spark-streaming] kafka source and flow control Hi, On Mon, Aug 11, 2014 at 6:19 PM, gpasquiers mailto:gwenhael.pasqui...@ericsson.com>> wrote: I’m using spark-streaming in a cloudera environm