Re: Waiting for a signal on one stream to start processing on another

2023-03-08 Thread Yuval Itzchakov
Thank you Mason and Ken for your replies The source itself isn't actually hybrid. The requirement is to materialize a stream first since the stream itself may contain updates to existing keys, changing their meanings, i.e. we can have key A go from medium importance to high importance and vice ver

Seeking Advice on Minimizing Downtime During Application Logic Upgrades in Flink Cluster

2023-03-08 Thread Ruibin Xing
Hi Community, I need some advice on upgrading the application logic in our Apache Flink cluster without causing any downtime. We're using Flink for our most important data pipeline, and we need to keep upgrading the application logic to meet our business needs. However, we're worried about the dow

Re: Waiting for a signal on one stream to start processing on another

2023-03-08 Thread Mason Chen
Hi Yuval, It seems you are trying to perform bootstrapping on a Flink job by doing the bounded read first. A good pattern to follow is to use HybridSource [1] and the docs have some examples with File and Kafka sources. The point of switching can be coordinated by the source so that you can dynami

Waiting for a signal on one stream to start processing on another

2023-03-08 Thread Yuval Itzchakov
Hi, I have a use-case where I have two streams, call them A and B. I need to consume stream A up to a certain point, and only then start processing on stream B. What could be a way to go about this?

Re: Create generic DeserializationSchema (Scala)

2023-03-08 Thread Ana Gómez González
Thank you Alexey! It worked perfectly. I was missing the ClassTag correct use. *Ana Gómez González* El lun, 6 mar 2023 a las 23:34, Alexey Novakov () escribió: > Hi Ana, > > I think you will need to deal with ClassTag to k