Re: Using state processor for a custom windowed aggregate function

2024-08-02 Thread Alexis Sarda-Espinosa
Hi Matthias, Thank you for looking into this. That change makes the example work, but my real job still has issues. There is a key difference that might be causing the problem, but that's not so easy to replicate in the example I made. Essentially, I'm trying to modify the partition key of an ope

Re: Flink 1.20.0 missing linux/amd64 images

2024-08-02 Thread weijie guo
Hi, Thanks for noticing us. Actually `amd64` architectures already in the metadata file in flink-docker image repo, but it's not available in official docker-hub repo. This is uploaded by docker official, I will reach out them asap. Best regards, Weijie Bjarke Tornager 于2024年8月2日周五 20:42写道:

RE: Using state processor for a custom windowed aggregate function

2024-08-02 Thread Schwalbe Matthias
Hi Alexis, I've worked it out: The input of your com.test.Application.StateReader#readWindow(..., Iterable elements, ...) is of the projection type com.test.Application.AggregateFunctionForMigration: AggregateFunction<..., OUT = GenericService>. I.e. you need to implement com.test.Application

Re: S3 schema for jar location?

2024-08-02 Thread Ahmed Hamdy
Hi Maxim, You need to add the s3 filesystem in the Flink plugins directory in the operator to be able to work with S3, this is similar to any other Filesystem and similar to how Flink itself works. Flink offers 2 S3 filesystem implementations - flink-s3-fs-hadoop[1] for extension s3a://*** - flink-

Flink 1.20.0 missing linux/amd64 images

2024-08-02 Thread Bjarke Tornager
Hi, It looks like no Flink 1.20.0 linux/amd64 images are published on the official Flink Dockerhub repo, however the linux/arm64 images are available. Will you publish the images for linux/amd64 as well? Best regards, Bjarke Tornager

Re: Checkpoint failures due to other subtasks sharing the ChannelState file (Caused the Job to Stall)

2024-08-02 Thread Dhruv Patel
We have also enabled unaligned checkpoints. Could it be because of that? We were experience slowness and intermittent packet loss when this issue occurred. On Wed, Jul 31, 2024 at 7:43 PM Dhruv Patel wrote: > Hi Everyone, > > We are observing an interesting issue with continuous checkpoint >