But how will I specify my state there?
On Tue, Feb 24, 2015 at 12:50 AM Arush Kharbanda <ar...@sigmoidanalytics.com>
wrote:
> You can use a reduceByKeyAndWindow with your specific time window. You can
> specify the inverse function in reduceByKeyAndWindow.
>
> On Tue, Feb 24, 2015 at 1:36 PM, Ashish Sharma <ashishonl...@gmail.com>
> wrote:
>
>> So say I want to calculate top K users visiting a page in the past 2
>> hours updated every 5 mins.
>>
>> so here I want to maintain something like this
>>
>> Page_01 => {user_01:32, user_02:3, user_03:7...}
>> ...
>>
>> Basically a count of number of times a user visited a page. Here my key
>> is page name/id and state is the hashmap.
>>
>> Now in updateStateByKey I get the previous state and new events coming
>> *in* the window. Is there a way to also get the events going *out* of the
>> window? This was I can incrementally update the state over a rolling window.
>>
>> What is the efficient way to do it in spark streaming?
>>
>> Thanks
>> Ashish
>>
>
>
>
> --
>
> [image: Sigmoid Analytics] <http://htmlsig.com/www.sigmoidanalytics.com>
>
> *Arush Kharbanda* || Technical Teamlead
>
> ar...@sigmoidanalytics.com || www.sigmoidanalytics.com
>
