回复:Re: SparkStreaming batch processing time question

2015-03-31 Thread luohui20001
hummm, got it. Thank you Akhil. Thanks&Best regards! 罗辉 San.Luo - 原始邮件 - 发件人:Akhil Das 收件人:罗辉 抄送人:user 主题:Re: SparkStreaming batch processing time question 日期:2015年04月01日 14点31分 It will add scheduling delay for the new batch. The new b

Re: SparkStreaming batch processing time question

2015-03-31 Thread Akhil Das
It will add scheduling delay for the new batch. The new batch data will be processed after finish up the previous batch, when the time is too high, sometimes it will throw fetch failures as the batch data could get removed from memory. Thanks Best Regards On Wed, Apr 1, 2015 at 11:35 AM, wrote:

SparkStreaming batch processing time question

2015-03-31 Thread luohui20001
hi guys: I got a question when reading http://spark.apache.org/docs/latest/streaming-programming-guide.html#setting-the-right-batch-interval. What will happen to the streaming data if the batch processing time is bigger than the batch interval? Will the next batch data be dal