Re: Savepoints and Checkpoints missing files

2025-06-10 Thread Aleksandr Pilipenko
Hi Nikola, The state files that are being inlined into _metadata file have the same keys as if they were outside, the difference would be in the type. Not inlined file with path A will have exactly the same path as a reference if it is inlined into _metadata. So as long as there are no errors duri

Re: Classloading issue with Parquet Avro format after update to 1.20

2025-05-26 Thread Aleksandr Pilipenko
nt ClassLoader. > No guarantees, but maybe it will solve your problem. > > On Mon, 2025-05-12 at 11:26 +0100, Aleksandr Pilipenko wrote: > > Hi all, > > After updating one of our Flink jobs from 1.18 to 1.20 we started to see a > classloading issue when using file source wi

Classloading issue with Parquet Avro format after update to 1.20

2025-05-12 Thread Aleksandr Pilipenko
Hi all, After updating one of our Flink jobs from 1.18 to 1.20 we started to see a classloading issue when using file source with Parquet Avro format, which looks like a regression: java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration at org.apache.flink.formats.parquet.avro.A

Re: Location of source for flink-connector-aws-kinesis-streams ?

2025-01-31 Thread Aleksandr Pilipenko
Hi, Connector sources are located in Flink AWS connectors repo: https://github.com/apache/flink-connector-aws/tree/main/flink-connector-aws/flink-connector-aws-kinesis-streams Regards, Aleksandr On Fri, 31 Jan 2025 at 17:21, Anderson, Michael via user < user@flink.apache.org> wrote: > Hi, > > I

Re: Taskslots usage

2024-07-12 Thread Aleksandr Pilipenko
Hello Alexandre, Flink does not use TaskSlot per each task by default, but rather task slot will hold a slice of the entire pipeline (up to 1 subtasks of each operator, depending on the operator parallelism) [1]. So if your job parallelism is 1 - only a single task slot will be occupied. If you w

Re: issues with Flink kinesis connector

2024-05-23 Thread Aleksandr Pilipenko
Hi Nick, You need to use another method to add sink to your job - sinkTo. KinesisStreamsSink implements newer Sink interface, while addSink expect old SinkFunction. You can see this by looking at method signatures[1] and in usage examples in documentation[2] [1] https://github.com/apache/flink/bl