[DISCUSS] Add config to enable job stop with savepoint on exceeding tolerable checkpoint Failures

2023-09-04 Thread Dongwoo Kim
Hi all, I have a proposal that aims to enhance the flink application's resilience in cases of unexpected failures in checkpoint storages like S3 or HDFS, *[Background]* When using self managed S3-compatible object storage, we faced checkpoint async failures lasting for an extended period more than

Re: [DISCUSS] Add config to enable job stop with savepoint on exceeding tolerable checkpoint Failures

2023-09-06 Thread Dongwoo Kim
execution.checkpointing.tolerable-failed-checkpoints` times, then > > stopWithSavepoint is likely to fail as well. > > If stopWithSavepoint succeeds or fails, will the job just stop? I am > > more curious about how this option works with the restart strategy? > > > > Bes

[Discuss] CRD for flink sql gateway in the flink k8s operator

2023-09-14 Thread Dongwoo Kim
Hi all, I've been working on setting up a flink SQL gateway in a k8s environment and it got me thinking — what if we had a CRD for this? So I have quick questions below. 1. Is there ongoing work to create a CRD for the Flink SQL Gateway? 2. If not, would the community be open to considering a CRD

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

2023-09-14 Thread Dongwoo Kim
t; > I don't completely understand what would be a content of such CRD, > > could > > > > you give a minimal example how the Flink SQL Gateway CR yaml would > look > > > > like? > > > > > > > > Adding a CRD would mean you need to add some

Re: [Discuss] CRD for flink sql gateway in the flink k8s operator

2023-09-19 Thread Dongwoo Kim
t; > > > > > > On Fri, Sep 15, 2023 at 10:18 AM Yangze Guo < > > karma...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > Thanks for bringing this up, Dongwoo. Flink SQL Gatew

PR review request for sql-gateway feature

2024-07-18 Thread Dongwoo Kim
Hi devs, A month ago, I created a jira issue and pr for a custom query operation validator in Flink SQL Gateway to block unsafe queries. Since there has been no feedback yet, I would appreciate it if someone could review it or provide some feedback. I'm curious if this feature can be added and if

Re: PR review request for sql-gateway feature

2024-07-18 Thread Dongwoo Kim
/cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals > > On Thu, Jul 18, 2024 at 2:50 PM Dongwoo Kim > wrote: > > > Hi devs, > > > > A month ago, I created a jira issue and pr for a custom query operation > > validator in Flink SQL Gateway to

Re: PR review request for sql-gateway feature

2024-07-22 Thread Dongwoo Kim
behavior as a plan advisor, as it lacks the necessary tools for > obtaining the upsert key or statistics. > > Best, > Shengkai > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-280%3A+Introduce+EXPLAIN+PLAN_ADVICE+to+provide+SQL+advice > > Dongwoo Kim