I'll test with java 8 streams.
Thanks,
Nuno
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Java-parallel-streams-vs-IterativeStream-tp13655p13737.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at
Nabble.com
Ah yes, I forgot Java8 streams. That could probably be your best option. Yes!
> On 14. Jun 2017, at 16:25, nragon wrote:
>
> I've mentioned java 8 stream beacuse avoids leaving map, thus decreasing
> network io, if not chained, and takes advantage of multiple cpus. Guess will
> have to test it.
I've mentioned java 8 stream beacuse avoids leaving map, thus decreasing
network io, if not chained, and takes advantage of multiple cpus. Guess will
have to test it.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Java-parallel-streams-vs-It
I see, I’m afraid that is not easily possible except by doing a custom stateful
function that waits for all elements to arrive and combines them again.
Another thing you could look at is the async I/O operator:
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/asyncio.html
That would work but after FlatMap, T> I would have to downstream
all elements into one.
--
View this message in context:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Java-parallel-streams-vs-IterativeStream-tp13655p13685.html
Sent from the Apache Flink User Mailing List
I think for that you would unpack to List of values, for example with a
FlatMap, T>. This would emit each element of the list as a separate
element. Then, downstream operations can operate on each element individually
and you will exploit parallelism in the cluster.
Best,
Aljoscha
> On 13. Jun
Iterate until all elements were changed perhaps. But just wanted to know if
there areimplementations out there using java 8 streams, in cases where you
want to parallelize a map function even if it is function scoped.
So, in my case, if the computation for each list element is to heavy, how
can one
Hi,
How would you use IterativeStream? In Flink IterativeStream is a pipeline-level
concept whereas your problem seems to be scoped to one user function.
Best,
Aljoscha
> On 12. Jun 2017, at 19:17, nragon wrote:
>
> In my map functions i have an object containing a list which must be changed,
