You could set them up in the same streaming context. Have the batch interval as 10 second, do the 10 second operations on the input stream, then apply a window of 5 minutes on the input stream (assuming same input stream in both cases) and then do the 5 minute operation on the windowed stream.
TD On Tue, Jul 15, 2014 at 5:22 PM, gorenuru <goren...@gmail.com> wrote: > Because I want to have different streams with different durations. > > Fornexample, one triggers snapshot analysis each 5 minutes and another > each 10 seconds > > On Tue, Jul 15, 2014 at 3:59 pm, Tathagata Das [via Apache Spark User > List] <[hidden email] <http://user/SendEmail.jtp?type=node&node=9852&i=0>> > wrote: > > Why do you need to create multiple streaming contexts at all? > > TD > > > > On Tue, Jul 15, 2014 at 3:43 PM, gorenuru <<a href="about:blank" > rel="nofollow">[hidden email]> wrote: > >> Oh, sad to hear that :( >> From my point of view, creating separate spark context for each stream is >> to >> expensive. >> Also, it's annoying because we have to be responsible for proper akka and >> UI >> port determination for each context. >> >> Do you know about any plans to support it? >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-streams-at-the-same-time-tp9819p9842.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> > > > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-streams-at-the-same-time-tp9819p9843.html > To unsubscribe from Multiple streams at the same time, click here. > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > > ------------------------------ > View this message in context: Re: Multiple streams at the same time > <http://apache-spark-user-list.1001560.n3.nabble.com/Multiple-streams-at-the-same-time-tp9819p9852.html> > > Sent from the Apache Spark User List mailing list archive > <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >
