Re: Incremental Analysis with Spark

2015-11-25 Thread chester
For the 2nd use case, can you save the result for first 29 days, then just get the last day result and add yourself ? This can be done outside of spark. Does that work for you Sent from my iPad > On Nov 25, 2015, at 9:46 PM, Sachith Withana wrote: > > Hi folks! > > I'm wondering if Sparks

Incremental Analysis with Spark

2015-11-25 Thread Sachith Withana
Hi folks! I'm wondering if Sparks supports or hopes to support incremental data analysis. There are few use cases that prompted me to wonder. ex: If we need to summarize last 30 days worth of data everyday, 1. Does Spark support time range based query execution ? select * from foo where timesta