Oh sorry, my fault. When I wrote it, I had iterations in mind. What I actually wanted to say, how "resuming from intermediate results" will work with (non-nested) "non-Flink" iterations? And with iterations I mean something like this:
while(...): - change params - submit to cluster where the executed Flink-program is more or less the same at each iterations. But with changing input sets, which are reused between different loop iterations. I might got something wrong, because in our group we mentioned caching a lá Spark for Flink and someone came up that "pinning" will do that. Is that somewhat right? Thanks and Cheers, Max On Mon, Jul 20, 2015 at 1:06 PM, Maximilian Michels <m...@apache.org> wrote: > "So it is up to debate how the support for resuming from intermediate > results will look like." -> What's the current state of that debate? > > Since there is no support for nested iterations that I know of, the debate > how intermediate results are integrated has not started yet. > > >> "Intermediate results are not produced within the iterations cycles." -> >> Ok, if there are none, what does it have to do with that debate? :-) >> > > I was referring to the existing support for intermediate results within > iterations. If we were to implement nested iterations, this could > (possibly) change. This is all very theoretical because there are no plans > to support nested iterations. > > Hope this clarifies. Otherwise, please restate your question because I > might have misunderstood. > > Cheers, > Max > > > On Mon, Jul 20, 2015 at 12:11 PM, Maximilian Alber < > alber.maximil...@gmail.com> wrote: > >> Thanks for the answer! But I need some clarification: >> >> "So it is up to debate how the support for resuming from intermediate >> results will look like." -> What's the current state of that debate? >> "Intermediate results are not produced within the iterations cycles." -> >> Ok, if there are none, what does it have to do with that debate? :-) >> >> Cheers, >> Max >> >> On Mon, Jul 20, 2015 at 10:50 AM, Maximilian Michels <m...@apache.org> >> wrote: >> >>> Hi Max, >>> >>> You are right, there is no support for nested iterations yet. As far as >>> I know, there are no concrete plans to add support for it. So it is up to >>> debate how the support for resuming from intermediate results will look >>> like. Intermediate results are not produced within the iterations cycles. >>> Same would be true for nested iterations. So the behavior for resuming from >>> intermediate results should be alike for nested iterations. >>> >>> Cheers, >>> Max >>> >>> On Fri, Jul 17, 2015 at 4:26 PM, Maximilian Alber < >>> alber.maximil...@gmail.com> wrote: >>> >>>> Hi Flinksters, >>>> >>>> as far as I know, there is still no support for nested iterations >>>> planned. Am I right? >>>> >>>> So my question is how such use cases should be handled in the future. >>>> More specific: when pinning/caching will be available, you suggest to >>>> use that feature and program in "Spark" style? Or is there some other, more >>>> flexible, mechanism planned for loops? >>>> >>>> Cheers, >>>> Max >>>> >>> >>> >> >
