Re: Maintaining overall cumulative data in Spark Streaming

How to we reset the aggregated statistics to null? Regards, Sandeep Giri, +1 347 781 4573 (US) +91-953-899-8962 (IN) www.KnowBigData.com. Phone: +1-253-397-1945 (Office) [image: linkedin icon] [image: other site icon]

RE: Maintaining overall cumulative data in Spark Streaming

Yes, update state by key worked. Though there are some more complications. On Oct 30, 2015 8:27 AM, "skaarthik oss" wrote: > Did you consider UpdateStateByKey operation? > > > > *From:* Sandeep Giri [mailto:sand...@knowbigdata.com] > *Sent:* Thursday, October 29, 2015 3:09 PM > *To:* user ; dev

RE: Maintaining overall cumulative data in Spark Streaming

Did you consider UpdateStateByKey operation? From: Sandeep Giri [mailto:sand...@knowbigdata.com] Sent: Thursday, October 29, 2015 3:09 PM To: user ; dev Subject: Maintaining overall cumulative data in Spark Streaming Dear All, If a continuous stream of text is coming in and you have t

Re: Maintaining overall cumulative data in Spark Streaming

-dev +user Hi Sandeep, Perhaps (flat)mapping values and using an accumulator? > El 29/10/2015, a las 23:08, Sandeep Giri escribió: > > Dear All, > > If a continuous stream of text is coming in and you have to keep publishing > the overall word count so far since 0:00 today, what would you d