Re: sharing data between 2 pipelines

2022-05-10 Thread yidan zhao
why not put the two pipelines together. David Anderson 于2022年5月11日周三 00:13写道: > > This sounds like it might be a use case for something like a > KeyedCoProcessFunction (or possibly a KeyedBroadcastProcessFunction, > depending on the details). These operators can receive inputs from two > diffe

Re: sharing data between 2 pipelines

2022-05-10 Thread David Anderson
This sounds like it might be a use case for something like a KeyedCoProcessFunction (or possibly a KeyedBroadcastProcessFunction, depending on the details). These operators can receive inputs from two different sources, and share state between them. The rides and fares exercise [1] from the flink-

sharing data between 2 pipelines

2022-05-10 Thread Sigalit Eliazov
Hi all i have 2 pipelines: A. receives information from kafka and "holds" that info B. a pipeline which is triggered by a scheduler and every x minutes should send the info i received in pipeline A to another kafka topic As i understood i cannot use the flink state for this since these are differe