Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-27 Thread Márton Balassi
Today we had a discussion with Robert on this issue. I would like to eventually have the streaming grouped and the windowing buffers/state maybe along with the crucial state of the user in the managed memory. If we had this separating the two modes could became less important as streaming would als

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-18 Thread Paris Carbone
+1 I agree it’s a proper way to go. On 18 Feb 2015, at 10:41, Max Michels mailto:m...@apache.org>> wrote: +1 On Tue, Feb 17, 2015 at 2:40 PM, Aljoscha Krettek mailto:aljos...@apache.org>> wrote: +1 On Tue, Feb 17, 2015 at 1:34 PM, Till Rohrmann mailto:trohrm...@apache.org>> wrote: +1 On Tu

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-18 Thread Max Michels
+1 On Tue, Feb 17, 2015 at 2:40 PM, Aljoscha Krettek wrote: > +1 > > On Tue, Feb 17, 2015 at 1:34 PM, Till Rohrmann wrote: >> +1 >> >> On Tue, Feb 17, 2015 at 1:34 PM, Kostas Tzoumas wrote: >> >>> +1 >>> >>> On Tue, Feb 17, 2015 at 12:14 PM, Márton Balassi >>> wrote: >>> >>> > When it comes to

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Aljoscha Krettek
+1 On Tue, Feb 17, 2015 at 1:34 PM, Till Rohrmann wrote: > +1 > > On Tue, Feb 17, 2015 at 1:34 PM, Kostas Tzoumas wrote: > >> +1 >> >> On Tue, Feb 17, 2015 at 12:14 PM, Márton Balassi >> wrote: >> >> > When it comes to the current use cases I'm for this separation. >> > @Ufuk: As Gyula has alre

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Till Rohrmann
+1 On Tue, Feb 17, 2015 at 1:34 PM, Kostas Tzoumas wrote: > +1 > > On Tue, Feb 17, 2015 at 12:14 PM, Márton Balassi > wrote: > > > When it comes to the current use cases I'm for this separation. > > @Ufuk: As Gyula has already pointed out with the current design of > > integration it should not

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Kostas Tzoumas
+1 On Tue, Feb 17, 2015 at 12:14 PM, Márton Balassi wrote: > When it comes to the current use cases I'm for this separation. > @Ufuk: As Gyula has already pointed out with the current design of > integration it should not be a problem. Even if we submitted programs to > the wrong clusters it wou

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Márton Balassi
When it comes to the current use cases I'm for this separation. @Ufuk: As Gyula has already pointed out with the current design of integration it should not be a problem. Even if we submitted programs to the wrong clusters it would only cause performance issues. Eventually it would be nice to have

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Gyula Fóra
So the current setup is to share results between the two apis by files. So I dont see any reason why this couldnt work with the 2 cluster setup. It makes deployment a little trickier but still feasible. On Tue, Feb 17, 2015 at 11:55 AM, Ufuk Celebi wrote: > I think this separation reflects the w

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Ufuk Celebi
I think this separation reflects the way that Flink is used currently anyways. I would be in favor of it as well. - What about the ongoing efforts (I think by Gyula) to combine both the batch and stream processing APIs? I assume that this would only effect the performance and wouldn't pose a funda

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Gyula Fóra
+1 Let's do this soon to avoid performance issues for streaming. On Tue, Feb 17, 2015 at 11:39 AM, Fabian Hueske wrote: > sounds like a good idea to me. > +1 > > 2015-02-17 11:28 GMT+01:00 Stephan Ewen : > > > Hi everyone! > > > > What do you think about making the streaming execution mode of th

Re: [DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Fabian Hueske
sounds like a good idea to me. +1 2015-02-17 11:28 GMT+01:00 Stephan Ewen : > Hi everyone! > > What do you think about making the streaming execution mode of the system > explicit? That means that people start a Flink cluster explicitly in Batch > mode or in Streaming mode. > > The rational behin

[DISCUSS] Dedicated streaming mode and start scripts

2015-02-17 Thread Stephan Ewen
Hi everyone! What do you think about making the streaming execution mode of the system explicit? That means that people start a Flink cluster explicitly in Batch mode or in Streaming mode. The rational behind this idea is that I am not sure how batch and streaming clusters are really shared in a