Re: PR review request for sql-gateway feature

2024-07-18 Thread Shengkai Fang
Hi Dongwoo, Thanks for your proposal! I think the idea about OperationValidator is similar to the plan advice[1], which can give advice if the statement has problems. I wonder if it would be more beneficial to develop the concept within the existing infrastructure, rather than introducing a new c

[RESULT][VOTE] FLIP-460: Display source/sink I/O metrics on Flink Web UI

2024-07-18 Thread Zhanghao Chen
Hi everyone, I'm delighted to announce that FLIP-460 [1] has been accepted. There were 13 votes in favor: - Yong Fang (binding) - Robert Metzger (binding) - Leonard Xu (binding) - Becket Qin (binding) - Ahmed Hamdy (non-binding) - Aleksandr Pilipenko (non-binding) - Xintong Song (binding) - Rui F

[jira] [Created] (FLINK-35867) 99% state lost after task auto restart due to "rocksdb block checksum mismatch" exception

2024-07-18 Thread heiya07 (Jira)
heiya07 created FLINK-35867: --- Summary: 99% state lost after task auto restart due to "rocksdb block checksum mismatch" exception Key: FLINK-35867 URL: https://issues.apache.org/jira/browse/FLINK-35867 Proje

[jira] [Created] (FLINK-35866) unsupported type Short for numeric minus

2024-07-18 Thread sucb (Jira)
sucb created FLINK-35866: Summary: unsupported type Short for numeric minus Key: FLINK-35866 URL: https://issues.apache.org/jira/browse/FLINK-35866 Project: Flink Issue Type: Bug Components

Re: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-07-18 Thread Martijn Visser
Hi David, The FLIP is updated. Cheers, Martijn On Tue, Jul 16, 2024 at 6:33 PM David Radley wrote: > Hello all, > > > > I have prototyped the new design and confirmed it works. I have documented > the design in google doc > https://docs.google.com/document/d/1J1E-cE-X2H3-kw4rNjLn71OGPQk_Yl1iG

Re: Request to make ContinuousFileReaderOperator public

2024-07-18 Thread Martijn Visser
Hi Darin, Your comment "I think our use case is rather unique and I'm not sure who else would benefit" makes me think then why we should actually support it in Flink, given that you're mentioning the use case is rather unique? Especially since the SourceFunction has many flaws, and has been long o

Re: PR review request for sql-gateway feature

2024-07-18 Thread Dongwoo Kim
Hi Martijn, Thanks for the guidance on making a FLIP. Before proceeding, I’d like to gather some initial thoughts from the community to see if we agree on the usefulness of this feature. Could anyone share their opinions on the custom query operation validator for Flink SQL Gateway? Thanks in adv

RE: Potential Kafka Connector FLIP: Large Message Handling

2024-07-18 Thread David Radley
Hi Kevin, That sounds good, unfortunately the image did not come through in the email for me,kind regards, David From: Kevin Lam Date: Wednesday, 10 July 2024 at 19:20 To: dev@flink.apache.org Subject: [EXTERNAL] Re: Potential Kafka Connector FLIP: Large Message Handling Hey David, Yes my p

Re: PR review request for sql-gateway feature

2024-07-18 Thread Martijn Visser
Hi Dongwoo, I think this should first have been a FLIP [1], given that it proposes to make something pluggable with custom interfaces. Best regards, Martijn [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals On Thu, Jul 18, 2024 at 2:50 PM Dongwoo Kim wrote: >

PR review request for sql-gateway feature

2024-07-18 Thread Dongwoo Kim
Hi devs, A month ago, I created a jira issue and pr for a custom query operation validator in Flink SQL Gateway to block unsafe queries. Since there has been no feedback yet, I would appreciate it if someone could review it or provide some feedback. I'm curious if this feature can be added and if

[jira] [Created] (FLINK-35865) When Sync SqlServer table with smallint type as pk

2024-07-18 Thread Zhongqiang Gong (Jira)
Zhongqiang Gong created FLINK-35865: --- Summary: When Sync SqlServer table with smallint type as pk Key: FLINK-35865 URL: https://issues.apache.org/jira/browse/FLINK-35865 Project: Flink Issu

[jira] [Created] (FLINK-35864) Add CONV function

2024-07-18 Thread Dylan He (Jira)
Dylan He created FLINK-35864: Summary: Add CONV function Key: FLINK-35864 URL: https://issues.apache.org/jira/browse/FLINK-35864 Project: Flink Issue Type: Sub-task Components: Table SQ

[DISCUSS] FLIP-470: Support Adaptive Broadcast Join

2024-07-18 Thread Xia Sun
Hi devs, Junrui Lee, Lei Yang, and I would like to initiate a discussion about FLIP-470: Support Adaptive Broadcast Join[1]. In general, Broadcast Hash Join is currently the most efficient join strategy available in Flink. However, its prerequisite is that the input data on one side must be suffi

Re: [VOTE] Release 1.20.0, release candidate #1

2024-07-18 Thread Rui Fan
+1(binding) - Reviewed the flink-web PR (Left some comments) - Checked Github release tag - Verified signatures - Verified sha512 (hashsums) - The source archives don't contain any binaries - Build the source with Maven 3 and java8 (Checked the license as well) - Start the cluster locally with jdk

Re: [DISCUSS] FLIP-XXX: Aligning timeout logic in the AdaptiveScheduler's WaitingForResources and Executing states

2024-07-18 Thread Zdenek Tison
Thanks, Mathias, for your opinions. I see two scenarios where different values for starting and rescaling would be appropriate: 1) Flink serverless providers may prefer the fastest possible job startup time, which can also be achieved by setting a smaller value for the stabilization timeout, such