I do not recommend large windows. You can have small windows, store the data and then do the reports for one hour or one day on stored data.
> On 09 May 2016, at 05:19, "kramer2...@126.com" <kramer2...@126.com> wrote: > > We have some stream data need to be calculated and considering use spark > stream to do it. > > We need to generate three kinds of reports. The reports are based on > > 1. The last 5 minutes data > 2. The last 1 hour data > 3. The last 24 hour data > > The frequency of reports is 5 minutes. > > After reading the docs, the most obvious way to solve this seems to set up a > spark stream with 5 minutes interval and two window which are 1 hour and 1 > day. > > > But I am worrying that if the window is too big for one day and one hour. I > do not have much experience on spark stream, so what is the window length in > your environment? > > Any official docs talking about this? > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-big-the-spark-stream-window-could-be-tp26899.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
