Unsubscribe

Re: Combining multiple stages into a multi-stage processing pipeline

session > windows instead of introducing side outputs. > > Best, > Yunfeng > > On Sun, Apr 7, 2024 at 12:41 AM Mark Petronic > wrote: > > > > I am looking for some design advice for a new Flink application and I am > relatively new to Flink - I have one, fairly straightfor

Combining multiple stages into a multi-stage processing pipeline

I am looking for some design advice for a new Flink application and I am relatively new to Flink - I have one, fairly straightforward Flink application in production so far. For this new application, I want to create a three-stage processing pipeline. Functionally, I am seeing this as ONE long dat

Guidance on general design approach for Flink

I am working on a new application to perform real time anomaly detection using Flink. I am relatively new to Flink but have already one application in production that is fairly basic and of low throughput. This next one will be more complex and much higher throughput. My query is about handling la

Best practice way to conditionally discard a window and not serialize the results

I am reading stats from Kinesis, deserializing them into a stat POJO and then doing something like this using an aggregated window with no defined processWindow function: timestampedStats .keyBy(v -> v.groupKey()) .window(TumblingEventTimeWindows.of(Time.seconds(appCfg.get

Checkpoint/Savepoint restore of S3 file reads using continuous read mode

I am trying to understand the Flink design pattern for consuming files from S3 continuously as they appear. I have written the below minimal program to do that and it works as expected wrt detecting newly-uploaded S3 files within the configured 5 second monitoring poll period. Then it just prints t

Unsubscribe

Web UI not working with createLocalEnvironmentWithWebUI()

I am learning Flink for a new project. I am trying to understand the development/debug environment to help me step through my code to better learn Flink. I am using the Intellij community edition for my IDE and Flink 1.17.0. I am using this simple Flink app to demonstrate my issue. //