Re: Classloading issue with Parquet Avro format after update to 1.20

2025-05-26 Thread Pavel Dmitriev
Hi Aleksandr, you could try the next configuration option: classloader.parent-first-patterns.additional: "org.apache.hadoop" to force Flink to load Hadoop classes on the parent ClassLoader. No guarantees, but maybe it will solve your problem. On Mon, 2025-05-12 at 11:26 +0100, Aleksandr Pilipen

Timeout of open() method in RichFunction

2025-02-17 Thread Pavel Dmitriev
Hi everyone, I have a question about `open(Configuration parameters)` method in `RichFunction`. Is there any timeout for this method, defined in Flink Configuration (https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/ ) or elsewhere? Context: I need to download in

Re: Using data classes in pyflink

2025-01-13 Thread Pavel Dmitriev
Hi Oleksii, it also can be related to the problem, that you have two different classloaders, because Flink creates a separate classloader for every running pipeline. It is especially related when you run the pipeline in Session Mode. Not sure whether it helps or not, but you can try the next con

Elastic Scaling in Apache Flink and minimum parallelism

2024-12-02 Thread Pavel Dmitriev
Hi, I have a question about Elsatic Scaling in Apache Flink. Is there any possibility to set min-parallelism for a pipeline when the pipeline keeps silent (no payload) for a long time to minimize the latency when the payload appears again? For now when there is no data coming into the pipeline (