Hi, Thanks for pointing me in the right direction. I think I can union the two streams as they have the same type and in the process function have a shared state since now both are the same stream.
I will figure this around connecting streams and using keyed state stores. Sachin On Wed, Aug 7, 2024 at 4:23 PM Schwalbe Matthias < matthias.schwa...@viseca.ch> wrote: > Hi Sachin, > > > > Just as an idea, while you cannot easily share state across operators, you > can do so within the same operator: > > - For two such input streams you could connect() the two streams into > a ConnectedStreams and then process() by means of a KeyedCoProcessFunction > - For more than two input streams, implement some > MultipleInputStreamOperator … > - In both cases you can yield multiple independent output streams (if > need be), by means of multiple side outputs (see here e.g. > > org.apache.flink.streaming.api.functions.co.KeyedCoProcessFunction.Context#output) > > > > I do that all the time 😊 > > > > WDYT? > > > > Sincere Flink greetings > > > > Thias > > > > > > > > *From:* Sachin Mittal <sjmit...@gmail.com> > *Sent:* Wednesday, August 7, 2024 12:37 PM > *To:* user@flink.apache.org > *Subject:* Can we share states across tasks/operators > > > > Hi, > > I have a stream which starts from a source and is keyed by a field f. > > With the stream process function, I can emit the processed record > downstream and also update state based on the records it received for > the same key. > > > > Now I have another stream which starts from another source and is of the > same type as the first stream and it is also keyed by the same field f. > > > > In its process function I want to access the last state updated by the > first stream's process function for the same key, do some processing > (update the state) and also send the record downstream. > > > > Is there any way I can achieve this in Flink by connecting to the same > state store ? > > > > Is there any concept of global state in Flink if I cannot achieve this by > using keyed states associated with an operator's process function ? > > > > Any other way you can think of achieving the same ? > > > > Thanks > > Sachin > > > Diese Nachricht ist ausschliesslich für den Adressaten bestimmt und > beinhaltet unter Umständen vertrauliche Mitteilungen. Da die > Vertraulichkeit von e-Mail-Nachrichten nicht gewährleistet werden kann, > übernehmen wir keine Haftung für die Gewährung der Vertraulichkeit und > Unversehrtheit dieser Mitteilung. Bei irrtümlicher Zustellung bitten wir > Sie um Benachrichtigung per e-Mail und um Löschung dieser Nachricht sowie > eventueller Anhänge. Jegliche unberechtigte Verwendung oder Verbreitung > dieser Informationen ist streng verboten. > > This message is intended only for the named recipient and may contain > confidential or privileged information. As the confidentiality of email > communication cannot be guaranteed, we do not accept any responsibility for > the confidentiality and the intactness of this message. If you have > received it in error, please advise the sender by return e-mail and delete > this message and any attachments. Any unauthorised use or dissemination of > this information is strictly prohibited. >
