@Stephan, why would there be a problem with the reliability of updates?
Also, what would be the best way to achieve this functionality? Preferably
using the current API.
Cheers!
Sachin
-- Sachin Goel
Computer Science, IIT Delhi
m. +91-9871457685
On Tue, Jul 21, 2015 at 4:17 PM, Stephan Ewen wr
I see the use case for that, but I don't think that this should be realized
through Flink's network stack. The network stack is designed for continuous
streams, backpressure, and configurable throughput/latency. I think it is
not the best solution for asynchronous messaging.
Also, for the ML field
@Max, broadcast variables have to be declared before the program is
executed. I want to be able to do something whereby I can send data inside
a map operation to all nodes. This would perhaps have an effect similar to
the recent discussion on making Accumulators available before Job
completion. [I
You are probably looking for a parameter server tool.
How about setting up one of these memory grids to use that? Apache Ignite,
or Apache Geode, or one of those.
On Fri, Jul 17, 2015 at 6:19 PM, Sachin Goel
wrote:
> Hi all
> Is it possible to send and receive key,value pairs at runtime? I woul
Hi Sachin,
Do you know about Broadcast Variables? They allow you to transfer a DataSet
to all nodes.
https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#broadcast-variables
Let us know if that fits your needs.
Cheers,
Max
On Fri, Jul 17, 2015 at 6:19 PM, Sachin G
Hi all
Is it possible to send and receive key,value pairs at runtime? I would like
to broadcast values at runtime so they are available on every node. This
somehow seems essential for an implementation of Asynchronous batch
iterations.
What I would like is to have two functions in the RuntimeConte