You can take a look at the ALS implementation. There I did something
similar.
On Wed, Aug 12, 2015 at 10:27 AM, Sachin Goel
wrote:
> Since the random splits need to be done on any data set a user provides, I
> think making a persistent source would be the best solution then.
>
>
> -- Sachin Goel
Since the random splits need to be done on any data set a user provides, I
think making a persistent source would be the best solution then.
-- Sachin Goel
Computer Science, IIT Delhi
m. +91-9871457685
On Wed, Aug 12, 2015 at 1:37 PM, Till Rohrmann
wrote:
> One branch does not occupy a single
One branch does not occupy a single slot. A slot is usually shared by
operators from multiple branches. Only subtasks of the same operator cannot
be placed into the same slot. Thus, it's not an argument against it.
Most if not all input formats assign the input splits on a first comes
first serve
Hi Till
Thanks for the reply.
If you think about it however, having several diverging computational paths
from an intermediate point will probably require re-computation anyway, in
case the number of these paths is even higher than the slots available.
Could that be an argument against a possible i
At the moment, Flink does not support the calculation of intermediate
results from which you can continue your computation. When you execute jobs
which share parts of its job graph, then they are recomputed. When your job
contains operators with non-deterministic output, then there is no
guarantee
