Re: [Streaming] Triggering an action in absence of data

2014-09-01 Thread Tobias Pfeiffer
Hi, On Mon, Sep 1, 2014 at 9:25 PM, Aniket Bhatnagar wrote: > > No state/foreach methods get called when no data has arrived. > Have you double-checked this? I am pretty sure that, for example, foreachRDD gets called (with an empty RDD) even when there was no data received. Tobias

[Streaming] Triggering an action in absence of data

2014-09-01 Thread Aniket Bhatnagar
Hi all I am struggling to implement a use case wherein I need to trigger an action in case no data has been received for X amount of time. I haven't been able to figure out an easy way to do this. No state/foreach methods get called when no data has arrived. I thought of generating a 'tick' DStrea