How to setup Regions for Fault Tolerance in Flink when using Side Outputs

2020-11-24 Thread Eifler, Patrick
Hi all, We are trying to setup regions to enable Flink to only stop failing tasks based on region instead of failing the entire stream. We are using one main stream that is reading from a kafka topic and a bunch of side outputs for processing each event from that topic differently. For the proce

Re: How to setup Regions for Fault Tolerance in Flink when using Side Outputs

2020-11-25 Thread Eifler, Patrick
many E: patrick.eif...@sony.com From: Till Rohrmann Date: Tuesday, 24. November 2020 at 17:20 To: "Eifler, Patrick" Cc: "user@flink.apache.org" Subject: Re: How to setup Regions for Fault Tolerance in Flink when using Side Outputs Hi Patrick, Flink supports regional

RocksDB: Memory is slowly increasing over time

2023-10-17 Thread Eifler, Patrick
Hello, We are running Flink jobs on K8s and using RocksDB as state backend. It is connected to S3 for checkpointing. We have multiple states in the job (mapstate and value states). We are seeing a slow but stable increase over time on the memory consumption. We only see this in our jobs connect

Flink 1.17 with Scala 2.13 or Scala 3

2024-01-09 Thread Eifler, Patrick
Hi, The Flink image still has Scala 2.12 dependencies. I tried to run a flink job written in Scala 2.13 avoiding the usage of all flink-scala api’s but getting an incompatibility issue (regarding scala.collections) that I would normally expect in a project were Scala 2.13 and Scala 2.12 are run

Re: Flink 1.17 with Scala 2.13 or Scala 3

2024-01-10 Thread Eifler, Patrick
f it is possible to completely exclude Scala from the Flink docker image. Best, Patrick -- Patrick Eifler Staff Software Engineer FTG Data Engineering Sony Interactive Entertainment Kemperplatz 1 10785 Berlin From: Martijn Visser Date: Tuesday, 9. January 2024 at 16:41 To: Eifler, Patrick Cc: