RE: Sharing data among flink operator pipeline

2025-08-25 Thread Kamal Mittal via user
Ok thanks. From: Kyle Lahnakoski Sent: 25 August 2025 18:51 To: Kamal Mittal ; user@flink.apache.org Subject: Re: Sharing data among flink operator pipeline You don't often get email from kyle.lahnako...@xe.com. Learn why this is important<https://aka.ms/LearnAboutSenderIdentificatio

Re: Sharing data among flink operator pipeline

2025-08-25 Thread Kyle Lahnakoski via user
@flink.apache.org Subject: RE: Sharing data among flink operator pipeline No, I am not talking about Kafka Message Key. Rather key/value pair which comes separately as below. Headers org.apache.kafka.clients.consumer.ConsumerRecord.headers() From: Nikola Milutinovic Sent: 25 August 2025 14:56 To: user

RE: Sharing data among flink operator pipeline

2025-08-25 Thread Kamal Mittal via user
mailto:user@flink.apache.org>> Date: Thursday, August 21, 2025 at 11:35 AM To: user@flink.apache.org<mailto:user@flink.apache.org> mailto:user@flink.apache.org>> Subject: Sharing data among flink operator pipeline Hello, I have a scenario like for kafka source where kafka heade

Re: Sharing data among flink operator pipeline

2025-08-25 Thread shenzhongwei
unsubscribe 原始邮件 发件人:Nikola Milutinovic

Re: Sharing data among flink operator pipeline

2025-08-25 Thread Nikola Milutinovic
: Thursday, August 21, 2025 at 11:35 AM To: user@flink.apache.org Subject: Sharing data among flink operator pipeline Hello, I have a scenario like for kafka source where kafka headers also come along with kafka record/event. Kafka headers fetched need to share/pass to next/parallel operators in

Re: Sharing data among flink operator pipeline

2025-08-21 Thread shenzhongwei
unsubscribe 原始邮件 发件人:Pedro Mázala

Re: Sharing data among flink operator pipeline

2025-08-21 Thread Pedro Mázala
Hello there, Kamal! Whenever I need to keep passing data downstream, I reflect it on my POJO. Usually, I call those fields metadata and access them downstream. The state will also segment data by operator, in other words, an operator cannot access stored data from other operators (unless you use b

Sharing data among flink operator pipeline

2025-08-21 Thread Kamal Mittal via user
Hello, I have a scenario like for kafka source where kafka headers also come along with kafka record/event. Kafka headers fetched need to share/pass to next/parallel operators in pipeline. So, is there any way to share data across operator pipeline? Explored keyed state which has limitation th