Thanks a lot, David and Fabian.
I will give this a try.
Cheers,Chirag
On Monday, 1 October, 2018, 3:48:42 PM IST, David Anderson
wrote:
Hi Chirag,
The community is also looking at an approach that involves using Bravo[1][2] to
bootstrap state by loading the initial version of the state i
Hi Chirag,
The community is also looking at an approach that involves using
Bravo[1][2] to bootstrap state by loading the initial version of the state
into a savepoint.
[1] https://github.com/king/bravo
[2]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/Proposal-Utilities-for-read
Hi Chirag,
Flink 1.5.0 added support for BroadcastState which should address your
requirement of replicating the data. [1]
The replicated data is stored in the configured state backend which can
also be RocksDB.
Regarding the reload, I would recommend Lasse's approach of having a custom
source t
Thanks Lasse, that is rightly put. That's the only solution I can think of too.
Only thing which I can't get my head around is using the coMap and coFlatMap
functions with such a stream. Since they dont support side outputs, is there a
way my lookup map/flatmap function simply consume a stream?
Hi Lasse,
One approach I’ve used in a similar situation is to have a “UnionedSource”
wrapper that first emits the (bounded) data that will be loaded in-memory, and
then starts running the source that emits the continuous stream of data.
This outputs an Either, which I then split, and broadcast
Hi.
We have created our own database source that pools the data with a configured
interval. We then use a co processed function. It takes to input one from our
database and one from our data input. I require that you keyby with the
attributes you use lookup in your map function.
To delay your
Hi,
I saw Apache Flink User Mailing List archive. - static/dynamic lookups in flink
streaming being discussed, and then I saw this FLIP
https://cwiki.apache.org/confluence/display/FLINK/FLIP-17+Side+Inputs+for+DataStream+API.
I know we havent made much progress on this topic. I still wanted to