nd you don't write
> >> intermediate results to a sink, but just use the DataSet from the
> >> previous step. The disadvantage of this is that you might end up with
> >> a too large Flink job, which might also hurt performance.
> >>
> >> Best,
> >>
t; previous step. The disadvantage of this is that you might end up with
>> a too large Flink job, which might also hurt performance.
>>
>> Best,
>> Gábor
>>
>>
>>
>>
>>
>>
>> 2016-09-01 18:09 GMT+02:00 Supun Kamburugamuve :
>>
of this is that you might end up with
> a too large Flink job, which might also hurt performance.
>
> Best,
> Gábor
>
>
>
>
>
>
> 2016-09-01 18:09 GMT+02:00 Supun Kamburugamuve :
> > Hi,
> >
> > Does Flink support nested iterations? We are tr
e disadvantage of this is that you might end up with
a too large Flink job, which might also hurt performance.
Best,
Gábor
2016-09-01 18:09 GMT+02:00 Supun Kamburugamuve :
> Hi,
>
> Does Flink support nested iterations? We are trying to develop a complex
> machine learning al
Hi,
Does Flink support nested iterations? We are trying to develop a complex
machine learning algorithm which has 3 iterations nested.
Best,
Supun..
If I would like to test it, I just need to merge those two
>>>> pull requests into my current branch?
>>>>
>>>> Cheers,
>>>> Max
>>>>
>>>> On Mon, Jul 20, 2015 at 4:02 PM, Maximilian Michels
>>>> wrote:
>>>>
Alber <
>> alber.maximil...@gmail.com> wrote:
>>
>>> Thanks!
>>>
>>> Ok, cool. If I would like to test it, I just need to merge those two
>>> pull requests into my current branch?
>>>
>>> Cheers,
>>> Max
>>>
>>>
>
>> Cheers,
>> Max
>>
>> On Mon, Jul 20, 2015 at 4:02 PM, Maximilian Michels
>> wrote:
>>
>>> Now that makes more sense :) I thought by "nested iterations" you meant
>>> iterations in Flink that can be nested, i.e. starting an ite
pull
> requests into my current branch?
>
> Cheers,
> Max
>
> On Mon, Jul 20, 2015 at 4:02 PM, Maximilian Michels
> wrote:
>
>> Now that makes more sense :) I thought by "nested iterations" you meant
>> iterations in Flink that can be nested, i.e. st
Thanks!
Ok, cool. If I would like to test it, I just need to merge those two pull
requests into my current branch?
Cheers,
Max
On Mon, Jul 20, 2015 at 4:02 PM, Maximilian Michels wrote:
> Now that makes more sense :) I thought by "nested iterations" you meant
> iterations in F
Now that makes more sense :) I thought by "nested iterations" you meant
iterations in Flink that can be nested, i.e. starting an iteration inside
an iteration.
The caching/pinning of intermediate results is still a work in progress in
Flink. It is actually in a state where it could be
pinning" will do that. Is that
somewhat right?
Thanks and Cheers,
Max
On Mon, Jul 20, 2015 at 1:06 PM, Maximilian Michels wrote:
> "So it is up to debate how the support for resuming from intermediate
> results will look like." -> What's the current state of that debat
"So it is up to debate how the support for resuming from intermediate
results will look like." -> What's the current state of that debate?
Since there is no support for nested iterations that I know of, the debate
how intermediate results are integrated has not started yet.
gt;
Ok, if there are none, what does it have to do with that debate? :-)
Cheers,
Max
On Mon, Jul 20, 2015 at 10:50 AM, Maximilian Michels wrote:
> Hi Max,
>
> You are right, there is no support for nested iterations yet. As far as I
> know, there are no concrete plans to add support for
Hi Max,
You are right, there is no support for nested iterations yet. As far as I
know, there are no concrete plans to add support for it. So it is up to
debate how the support for resuming from intermediate results will look
like. Intermediate results are not produced within the iterations
Hi Flinksters,
as far as I know, there is still no support for nested iterations planned.
Am I right?
So my question is how such use cases should be handled in the future.
More specific: when pinning/caching will be available, you suggest to use
that feature and program in "Spark" st
I think running the program multiple times is a reasonable way to start
working on this.
I would try and see whether this can be re-written to a non-nested
iterations case. Nestes iterations algorithms may have much more overhead
to start with.
Stephan
On Tue, Apr 14, 2015 at 3:53 PM, Benoît
the heap, if you >need access to it.
Cheers,
Till
On Tue, Apr 14, 2015 at 3:11 PM, Stephan Ewen wrote:
Hi Benoît!
You are right, the nested iterations are currently not supported.
The test you found actually checks that the Optimizer gives a good
error message when encountering nes
Hi Benoît!
You are right, the nested iterations are currently not supported.
The test you found actually checks that the Optimizer gives a good error
message when encountering nested iterations.
Can you write your program as one iterations (the inner) and start the
program multiple times to
l
On Tue, Apr 14, 2015 at 3:11 PM, Stephan Ewen wrote:
> Hi Benoît!
>
> You are right, the nested iterations are currently not supported.
>
> The test you found actually checks that the Optimizer gives a good error
> message when encountering nested iterations.
>
> Can
Hello,
I'm implementing an algorithm which requires nested iterations, and, from
what I understood, this feature was not yet available in Flink [1], and my
experiments with 2 nested bulk iterations seem to confirm that. However I
came across a Flink unit test [2] using nested itera
21 matches
Mail list logo
