Flink job unable to restore from savepoint

Hello, We have been facing this weird issue of not being able to restore from savepoint, when we have a significant load on flink jobs. " *org.apache.flink.util.FlinkRuntimeException: Error while deserializing the user key.* *at org.apache.flink.contrib.streaming.state.RocksDBMapState$RocksD

[discuss] [jdbc] Support Ignore deleting is required？

When using the jdbc sink connector, there are a lot of scenarios where we ignore delete (-D) record when writing out data. Our jdbc sink connector scenario needs to add a configuration sink.ignore-delete attribute. The default value is false, and data deletion is not ignored. The user can set t

Discussion thread : Proposal to add Conditions in Flink CRD's Status field

 Hello, Starting discussion thread here to discuss a proposal to add Conditions field in the CR status of Flink Deployment and FlinkSessionJob. Here is the google doc with details. Please provide your thoughts/inputs. https://docs.google.com/document/d/12wlJCL_Vq2KZnABzK7OR7gAd1jZMmo0MxgXQXq

Re: need flink support framework for dependency injection

Hi Ganesh, I disagree. I don’t think Flink needs a dependency injection framework. I have implemented many complex jobs without one. Can you please articulate why you think it needs a dependency injection framework, along with some use cases that will show its benefit? I would rather see more

need flink support framework for dependency injection

Re: Temporal join on rolling aggregate

Hey Sebastien et al, have you tried rewriting the rolling aggregate as a window-over query? A window-over aggregation creates an append-only stream which should preserve the timestamp/watermark of the source. You can then add a deduplication

Re: Understanding RocksDBStateBackend in Flink on Yarn on AWS EMR

Usually, you should use the HDFS nameservice instead of the NameNode hostname:port to avoid NN failover. And you could find the supported nameservice in the hdfs-site.xml in the key *dfs.nameservices*. Best, Yang On Fri, Mar 22, 2024 at 8:33 PM Sachin Mittal wrote: > So, when we create an EMR