Re: [ANNOUNCE] Release 1.20.0, release candidate #0

2024-07-08 Thread weijie guo
Thanks Santwana and Martijn for the testing! > I've closed my blocker ticket; after force killing all my Flink Java processes, it did work when I restarted it. That would be great! Best regards, Weijie Martijn Visser 于2024年7月9日周二 03:04写道: > Hi all, > > I've closed my blocker ticket; after f

[jira] [Created] (FLINK-35792) Sorting by proctime does not work in rank

2024-07-08 Thread xuyang (Jira)
xuyang created FLINK-35792: -- Summary: Sorting by proctime does not work in rank Key: FLINK-35792 URL: https://issues.apache.org/jira/browse/FLINK-35792 Project: Flink Issue Type: Bug Compo

Re: [DISCUSS] CLI action deprecation process

2024-07-08 Thread Xintong Song
I think there are three questions to be anwsered, and here are my two cents. 1) How do we define the compatibility guarantees for cli interfaces? I'd be +1 for reusing the existing @Public and @PublicEvolving annotations, as suggested by Ferenc. Having multiple sets of rules in the same project m

[jira] [Created] (FLINK-35791) Add database and table infos to Kafka json output.

2024-07-08 Thread LvYanquan (Jira)
LvYanquan created FLINK-35791: - Summary: Add database and table infos to Kafka json output. Key: FLINK-35791 URL: https://issues.apache.org/jira/browse/FLINK-35791 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-35790) Update docs for new schema definition in CTAS and RTAS

2024-07-08 Thread Jira
Sergio Peña created FLINK-35790: --- Summary: Update docs for new schema definition in CTAS and RTAS Key: FLINK-35790 URL: https://issues.apache.org/jira/browse/FLINK-35790 Project: Flink Issue Ty

[jira] [Created] (FLINK-35789) Allow WATERMARK & PRIMARY KEY in CREATE TABLE AS (CTAS)

2024-07-08 Thread Jira
Sergio Peña created FLINK-35789: --- Summary: Allow WATERMARK & PRIMARY KEY in CREATE TABLE AS (CTAS) Key: FLINK-35789 URL: https://issues.apache.org/jira/browse/FLINK-35789 Project: Flink Issue T

Re: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-07-08 Thread Kevin Lam
Hi David, Any updates on the Kafka Message Header support? I am also interested in supporting headers with the Flink SQL Formats: https://lists.apache.org/thread/spl88o63sjm2dv4l5no0ym632d2yt2o6 On Fri, Jun 14, 2024 at 6:10 AM David Radley wrote: > Hi everyone, > I have talked with Chesnay and

Re: [VOTE] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-07-08 Thread Alexey Leonov-Vendrovskiy
Thank you all! The voting is closed now. I've posted results [1]. [1] https://lists.apache.org/thread/mx3qd5qgyjtqk66p6sv639g777lm0jc6 Thanks, Alexey On Wed, Jul 3, 2024 at 6:39 PM Yuepeng Pan wrote: > +1 (non-binding) > > Best regards, > > Yuepeng Pan > > > > > > At 2024-07-03 01:46:13, "Serg

[RESULT][VOTE] FLIP-456: CompiledPlan support for Batch Execution Mode

2024-07-08 Thread Alexey Leonov-Vendrovskiy
Hello everyone, The vote [1] for FLIP-456 [2] is over. The number of required binding votes (3) was reached (total: 5, binding: 3, non-binding: 2). No objections were raised. Binding: - Martijn Visser - Sergey Nuyanzin - Timo Walther Non-binding: - Jim Hughes - Yuepeng Pan PRs

Re: [ANNOUNCE] Release 1.20.0, release candidate #0

2024-07-08 Thread Martijn Visser
Hi all, I've closed my blocker ticket; after force killing all my Flink Java processes, it did work when I restarted it. Thanks, Martijn On Mon, Jul 8, 2024 at 6:13 PM Santwana Verma wrote: > Hi everyone, > > Thanks for creating the release candidate. I've successfully validated the > release

Re: [ANNOUNCE] Release 1.20.0, release candidate #0

2024-07-08 Thread Santwana Verma
Hi everyone, Thanks for creating the release candidate. I've successfully validated the release candidate locally with the DataStream API. 1. I created a DataStream API based job, which read and deserialized JSON strings from an input Kafka topic using flink-connector-kafka, transformed the data,

[jira] [Created] (FLINK-35788) Deprecate old InputFormat and SinkFunction

2024-07-08 Thread Jira
João Boto created FLINK-35788: - Summary: Deprecate old InputFormat and SinkFunction Key: FLINK-35788 URL: https://issues.apache.org/jira/browse/FLINK-35788 Project: Flink Issue Type: Sub-task

Re: Potential Kafka Connector FLIP: Large Message Handling

2024-07-08 Thread Kevin Lam
Hi Fabian, Awesome, this project looks great! Thanks for sharing. It would work well with KafkaSource and the DataStream API as you've mentioned. We have something similar internally, but where we are encountering difficulty is integrating it with the Flink SQL Kafka DynamicTable Source and Sinks.

[jira] [Created] (FLINK-35787) DefaultSlotStatusSyncer might bring down JVM (exit code 239 instead of a proper shutdown)

2024-07-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35787: - Summary: DefaultSlotStatusSyncer might bring down JVM (exit code 239 instead of a proper shutdown) Key: FLINK-35787 URL: https://issues.apache.org/jira/browse/FLINK-3578

Re: Potential Kafka Connector FLIP: Large Message Handling

2024-07-08 Thread Fabian Paul
Hi Kevin, I worked on a project [1] in the past that had a similar purpose. You should be able to use a similar approach with the existing KafkaSource by implementing your own KafkaRecordDeserializationSchema that hides the logic of pulling the records from blob storage from the connector. You can

[jira] [Created] (FLINK-35786) NPE in BlobServer / shutdownHook

2024-07-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-35786: - Summary: NPE in BlobServer / shutdownHook Key: FLINK-35786 URL: https://issues.apache.org/jira/browse/FLINK-35786 Project: Flink Issue Type: Bug

Re: Potential Kafka Connector FLIP: Large Message Handling

2024-07-08 Thread Kevin Lam
Hi all, Thanks for the responses. Grace those are indeed both challenges, thanks for flagging them. Regarding expiry, we could consider having a Mark and Sweep garbage collection system. A service can consume the topics with large messages, and track references. When there are no references left

Re: [ANNOUNCE] Release 1.20.0, release candidate #0

2024-07-08 Thread Martijn Visser
Hi all, I've found that no query works in the SQL Client for RC0. See https://issues.apache.org/jira/browse/FLINK-35785 Best regards, Martijn On Fri, Jun 28, 2024 at 6:09 PM weijie guo wrote: > Hi everyone, > > > The release candidate #0(i.e. RC0) for Apache Flink 1.20.0 has been > created. >

[jira] [Created] (FLINK-35785) Executing query in SQL client results in "java.lang.ClassNotFoundException: org.apache.flink.core.execution.RestoreMode"

2024-07-08 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-35785: -- Summary: Executing query in SQL client results in "java.lang.ClassNotFoundException: org.apache.flink.core.execution.RestoreMode" Key: FLINK-35785 URL: https://issues.apache.o

[jira] [Created] (FLINK-35784) The cp file-merging directory not properly registered in SharedStateRegistry

2024-07-08 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-35784: --- Summary: The cp file-merging directory not properly registered in SharedStateRegistry Key: FLINK-35784 URL: https://issues.apache.org/jira/browse/FLINK-35784 Project: F

[jira] [Created] (FLINK-35783) Flink CDC Could not start the yaml Job

2024-07-08 Thread layhuts (Jira)
layhuts created FLINK-35783: --- Summary: Flink CDC Could not start the yaml Job Key: FLINK-35783 URL: https://issues.apache.org/jira/browse/FLINK-35783 Project: Flink Issue Type: Bug Compon

[jira] [Created] (FLINK-35782) Flink connector jdbc works wrong when using sql gateway

2024-07-08 Thread Yi Cai (Jira)
Yi Cai created FLINK-35782: -- Summary: Flink connector jdbc works wrong when using sql gateway Key: FLINK-35782 URL: https://issues.apache.org/jira/browse/FLINK-35782 Project: Flink Issue Type: Bug

Re: [DISCUSS] CLI action deprecation process

2024-07-08 Thread Muhammet Orazov
Hey Ferenc, Yes correct. My thoughts were based on finding tradeoff between following fully deprecation process and leaner one for CLIs. Since cli are not like APIs, I think users would be aware of deprecation only when were remove the commands. That is they try to run their jobs with upgrade an

Re: Potential Kafka Connector FLIP: Large Message Handling

2024-07-08 Thread Martijn Visser
Hi Kevin, I just want to double check, were you planning to send this proposal to the Kafka Dev mailing list? Because I don't see directly how this affects Flink :) Best regards, Martijn On Mon, Jul 8, 2024 at 8:05 AM Grace Grimwood wrote: > Hi Kevin, > > Thanks for starting this thread. > >