Thanks for your response. The use case is 2 pipelines: A. receives information from kafka and "holds" that info B. a pipeline which is triggered by a scheduler and every x minutes should send the info i received in pipeline A to another kafka topic
Currently these pipelines are implemented by BEAM and I use DB in order to save the output of pipeline A, so when B is triggered I select the relevant info from DB. I would prefer to drop the DB and use internal state if possible. I also tried to merge those pipelines and use the spec state (@stateId) but it did not work. the state was not available for the operator which reads from the scheduler . Thanks Sigalit On Sun, May 15, 2022 at 6:03 PM Reuven Lax <re...@google.com> wrote: > Beam supports side inputs, which might help you. Can you describe your use > case? > > On Sun, May 15, 2022 at 7:34 AM Sigalit Eliazov <e.siga...@gmail.com> > wrote: > >> Hello >> does beam have support for something similar to >> KeyedBroadcastProcessFunction which exists in flink? >> I am looking for an option to have broadcast state in beam so it can be >> shared between different operators >> >> Thanks >> Sigalit >> >
