Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-06 Thread Ran Tao
thanks Lau. The vote will last for at least 72 hours (03/09, 19:30 UTC+8). It needs consensus approval, requiring 3 binding +1 votes and no binding vetoes. Best Regards, Ran Tao Jacky Lau 于2023年3月7日周二 15:11写道: > Thanks Ran. > +1 (non-binding) > > Regards, > Jacky Lau > > Ran Tao 于2023年3月6日周

Re: [DISCUSS] FLIP-298: Unifying the Implementation of SlotManager

2023-03-06 Thread Yangze Guo
Hi Weihua, Thanks for driving this. As Xintong mentioned, this was a technical debt from FLIP-56. The latest version of FLIP sounds good, +1 from my side. As a contributor to this component, I'm willing to assist with the review process. Feel free to reach me if you need help. Best, Yangze Guo

[jira] [Created] (FLINK-31355) CommonDataStreamTests.test_execute_and_collect failed

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31355: - Summary: CommonDataStreamTests.test_execute_and_collect failed Key: FLINK-31355 URL: https://issues.apache.org/jira/browse/FLINK-31355 Project: Flink Issue

[jira] [Created] (FLINK-31354) NettyClientServerSslTest.testValidSslConnectionAdvanced timed out

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31354: - Summary: NettyClientServerSslTest.testValidSslConnectionAdvanced timed out Key: FLINK-31354 URL: https://issues.apache.org/jira/browse/FLINK-31354 Project: Flink

[jira] [Created] (FLINK-31353) Merge SnapshotEnumerator into TableScan

2023-03-06 Thread yuzelin (Jira)
yuzelin created FLINK-31353: --- Summary: Merge SnapshotEnumerator into TableScan Key: FLINK-31353 URL: https://issues.apache.org/jira/browse/FLINK-31353 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-31352) OverAggregateITCase.testWindowAggregationSumWithoutOrderBy times out on CI

2023-03-06 Thread Jira
David Morávek created FLINK-31352: - Summary: OverAggregateITCase.testWindowAggregationSumWithoutOrderBy times out on CI Key: FLINK-31352 URL: https://issues.apache.org/jira/browse/FLINK-31352 Project:

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Aitozi
> we can initiate corresponding support issues for specific connectors to follow up on support after finalizing the API changes Make sense to me! Best, Aitozi. Lincoln Lee 于2023年3月7日周二 15:05写道: > Thanks Jingsong & Hang, > > Using Optional as the return value is a good idea. Previously, I > hop

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-06 Thread Jacky Lau
Thanks Ran. +1 (non-binding) Regards, Jacky Lau Ran Tao 于2023年3月6日周一 19:32写道: > Hi Everyone, > > > I want to start the vote on FLIP-297: Improve Auxiliary Sql Statements [1]. > The FLIP was discussed in this thread [2]. > > > The goal of the FLIP is to improve flink auxiliary sql statements(com

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Lincoln Lee
Thanks Jingsong & Hang, Using Optional as the return value is a good idea. Previously, I hoped to keep the style of the LookupTableSource.LookupContext#getKeys as consistent as possible, but the getKeys is actually non-empty when used, so I support updating to Optional. I'll update the flip doc a

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Lincoln Lee
Hi Aitozi, Thanks for your feedback! Yes, including HBase and JDBC connector, they can be considered for support in the next step (JDBC as as a standard protocol supported not only in traditional databases, but also in more and more new types of storage). Considering the ongoing externalizing of

Re: Regarding new command to download jars in flink cluster

2023-03-06 Thread Surendra Singh Lilhore
Hi David, Sorry I missed your reply. >>>Since you're mentioning docker, I assume you're deploying your application to k8s. Is that correct? yes, I am using k8s. >>>For handcrafted Kubernetes deployments, you can simply download the jar into the user lib folder in an init container [1]. You can

Re: [DISCUSS] FLIP-298: Unifying the Implementation of SlotManager

2023-03-06 Thread Weihua Hu
Hi, @David @Matthias There are a few days after hearing your thoughts. I would like to know if there are any other concerns about this FLIP. Best, Weihua On Mon, Mar 6, 2023 at 7:53 PM Weihua Hu wrote: > > Thanks Shammon, > > I've updated FLIP to add this redundant Task Manager limitation. >

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-06 Thread Kui Yuan
Hi Jing, Thanks for the reminder. The aim of this flip is letting the sql users to use those features in the Datastream API, we don't intend to extend flip-217. In my opinion, the watermark alignment with only one source can be configured by the options given in flip, and if the source connector

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Hang Ruan
Hi, Lincoln, Thanks for bringing this up. It looks good to me. I also agree with Jingsong's suggestion. Best, Hang Jingsong Li 于2023年3月7日周二 11:15写道: > Wow, we have 300 FLIPs... > > Thanks Lincoln, > > Have you considered returning an Optional? > > Empty array looks a little weird to me. > > Be

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Jingsong Li
Wow, we have 300 FLIPs... Thanks Lincoln, Have you considered returning an Optional? Empty array looks a little weird to me. Best, Jingsong On Tue, Mar 7, 2023 at 10:32 AM Aitozi wrote: > > Hi Lincoln, > Thank you for sharing this FLIP. Overall, it looks good to me. I have > one question:

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Aitozi
Hi Lincoln, Thank you for sharing this FLIP. Overall, it looks good to me. I have one question: with the introduction of this interface, will any existing Flink connectors need to be updated in order to take advantage of its capabilities? For example, HBase. yuxia 于2023年3月7日周二 10:01写道: > Tha

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread yuxia
Thanks. It makes sense to me. Best regards, Yuxia - 原始邮件 - 发件人: "Lincoln Lee" 收件人: "dev" 发送时间: 星期一, 2023年 3 月 06日 下午 10:26:26 主题: Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert hi yuxia, Thanks for your feed

Re: help: [FLINK-31321] @flinkbot run azure does not work?

2023-03-06 Thread yuxia
Thanks Martijn for clarification. After I reabsed master, it works. Best regards, Yuxia - 原始邮件 - 发件人: "Martijn Visser" 收件人: "dev" 发送时间: 星期一, 2023年 3 月 06日 下午 5:58:38 主题: Re: help: [FLINK-31321] @flinkbot run azure does not work? Hi, Please make sure that you have rebased on the lates

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-06 Thread Jing Ge
Hi Kui, Thanks for pointing that out. I knew FLIP-217 which was done by an engineer working in my team. As far as I am concerned, your FLIP should answer the following questions: 1. How to enable the watermark alignment of a source splits with Flink SQL? e.g. which options should be used if only

[jira] [Created] (FLINK-31351) HiveServer2EndpointITCase.testExecuteStatementInSyncModeWithRuntimeException2 times out on CI

2023-03-06 Thread Jira
David Morávek created FLINK-31351: - Summary: HiveServer2EndpointITCase.testExecuteStatementInSyncModeWithRuntimeException2 times out on CI Key: FLINK-31351 URL: https://issues.apache.org/jira/browse/FLINK-31351

[jira] [Created] (FLINK-31350) Support Calcite's 1.30+ UnknownType

2023-03-06 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-31350: --- Summary: Support Calcite's 1.30+ UnknownType Key: FLINK-31350 URL: https://issues.apache.org/jira/browse/FLINK-31350 Project: Flink Issue Type: Sub-ta

[jira] [Created] (FLINK-31349) Adapt to Calcite's 1.30+ new ROW null semantics

2023-03-06 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-31349: --- Summary: Adapt to Calcite's 1.30+ new ROW null semantics Key: FLINK-31349 URL: https://issues.apache.org/jira/browse/FLINK-31349 Project: Flink Issue T

[jira] [Created] (FLINK-31348) Documentation fails to build due to unclosed shortcodes

2023-03-06 Thread Jira
David Morávek created FLINK-31348: - Summary: Documentation fails to build due to unclosed shortcodes Key: FLINK-31348 URL: https://issues.apache.org/jira/browse/FLINK-31348 Project: Flink Iss

[jira] [Created] (FLINK-31347) AdaptiveSchedulerClusterITCase.testAutomaticScaleUp times out

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31347: - Summary: AdaptiveSchedulerClusterITCase.testAutomaticScaleUp times out Key: FLINK-31347 URL: https://issues.apache.org/jira/browse/FLINK-31347 Project: Flink

Re: Large schemas lead to long DataStream-to-table transformation names

2023-03-06 Thread Xingcan Cui
Oh, I just realized that FLIP-195 has already solved this. We'll upgrade our Flink version to 1.15+. Thanks! On Mon, Mar 6, 2023 at 10:08 AM Xingcan Cui wrote: > Hi Jark, > > Yes. I believe field names of the table would be enough to describe the > conversion operator. I'll try to improve this.

Re: Large schemas lead to long DataStream-to-table transformation names

2023-03-06 Thread Xingcan Cui
Hi Jark, Yes. I believe field names of the table would be enough to describe the conversion operator. I'll try to improve this. Best, Xingcan On Sun, Mar 5, 2023 at 9:18 PM Jark Wu wrote: > Hi Xingcan, > > I think `physicalDataType.toString()` is indeed verbose in this case. > Normal table sca

[jira] [Created] (FLINK-31346) IO scheduler does not throw TimeoutException if numRequestedBuffers is greater than 0

2023-03-06 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-31346: -- Summary: IO scheduler does not throw TimeoutException if numRequestedBuffers is greater than 0 Key: FLINK-31346 URL: https://issues.apache.org/jira/browse/FLINK-31346 Pro

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread Lincoln Lee
hi yuxia, Thanks for your feedback and tracking the issue of update statement! I've updated the FLIP[1] and also the poc[2]. Since the bug and flip are orthogonal, we can focus on finalizing the api changes first, and then work on the flip implementation and bugfix separately, WDYT? [1] https://c

Re: [DISCUSS] String literal behavior in Flink

2023-03-06 Thread Aitozi
Hi, Jark FYI, this feature has already been supported in Calcite 1.33.0 [1]. Therefore, I believe we can use it directly after upgrading the Calcite version. Best, Aitozi. [1]: https://issues.apache.org/jira/browse/CALCITE-5305 Aitozi 于2023年3月6日周一 16:48写道: > Thanks, will give it a try > >

[jira] [Created] (FLINK-31345) Trim autoscaler configMap to not exceed 1mb size limit

2023-03-06 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-31345: -- Summary: Trim autoscaler configMap to not exceed 1mb size limit Key: FLINK-31345 URL: https://issues.apache.org/jira/browse/FLINK-31345 Project: Flink

[VOTE] FLIP-299 Pub/Sub Lite Connector

2023-03-06 Thread Daniel Collins
Hello all, This is the vote thread for FLIP-299 https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=240885055 to add a Pub/Sub Lite connector. I would like to add this to the same repo and maven artifacts as the flink pubsub connector, per a suggestion on the DISCUSS thread, but there

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-06 Thread yuxia
Hi, Lincoln. Thanks for bringing this up. +1 for this FLIP, it's helpful for external storage system to implement partial update. The FLIP looks good to me. I only want to add one comment, update statement also doesn't support updating nested column, I have created FLINK-31344[1] to track it. Ma

[jira] [Created] (FLINK-31344) Support to update nested columns in update statement

2023-03-06 Thread luoyuxia (Jira)
luoyuxia created FLINK-31344: Summary: Support to update nested columns in update statement Key: FLINK-31344 URL: https://issues.apache.org/jira/browse/FLINK-31344 Project: Flink Issue Type: Impr

[jira] [Created] (FLINK-31343) Remove JMH dependency in flink-table-store-micro-benchmark

2023-03-06 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-31343: Summary: Remove JMH dependency in flink-table-store-micro-benchmark Key: FLINK-31343 URL: https://issues.apache.org/jira/browse/FLINK-31343 Project: Flink Is

Re: [DISCUSS] FLIP-298: Unifying the Implementation of SlotManager

2023-03-06 Thread Weihua Hu
Thanks Shammon, I've updated FLIP to add this redundant Task Manager limitation. Best, Weihua On Mon, Mar 6, 2023 at 5:07 PM Shammon FY wrote: > Hi weihua > > Can you add content related to `heterogeneous resources` to this FLIP? We > can record it and consider it in the future. It may be us

[VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-06 Thread Ran Tao
Hi Everyone, I want to start the vote on FLIP-297: Improve Auxiliary Sql Statements [1]. The FLIP was discussed in this thread [2]. The goal of the FLIP is to improve flink auxiliary sql statements(compared with sql standard or other mature engines). The vote will last for at least 72 hours (0

Re: Re: obtain the broadcast stream information in sink

2023-03-06 Thread Weihua Hu
AFAIK, we can not get the broadcast state in sink. Maybe you can enrich records with broadcast information, and then get the information from each record in the Sink function. Best, Weihua On Mon, Mar 6, 2023 at 6:20 PM zhan...@eastcom-sw.com < zhan...@eastcom-sw.com> wrote: > > The sinks

[jira] [Created] (FLINK-31342) SQLClientSchemaRegistryITCase timed out when starting the test container

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31342: - Summary: SQLClientSchemaRegistryITCase timed out when starting the test container Key: FLINK-31342 URL: https://issues.apache.org/jira/browse/FLINK-31342 Project: F

[jira] [Created] (FLINK-31341) OutOfMemoryError in Kafka e2e tests

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31341: - Summary: OutOfMemoryError in Kafka e2e tests Key: FLINK-31341 URL: https://issues.apache.org/jira/browse/FLINK-31341 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-31340) HiveTableSourceStatisticsReportTest failed with multiple test failures

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31340: - Summary: HiveTableSourceStatisticsReportTest failed with multiple test failures Key: FLINK-31340 URL: https://issues.apache.org/jira/browse/FLINK-31340 Project: Fli

Re: obtain the broadcast stream information in sink

2023-03-06 Thread Weihua Hu
Hi, Could you describe your usage scenario in detail? Why do you need to get the broadcast stream in sink? And could you split an operator from the sink to deal with broadcast stream? Best, Weihua On Mon, Mar 6, 2023 at 10:57 AM zhan...@eastcom-sw.com < zhan...@eastcom-sw.com> wrote: > > h

[jira] [Created] (FLINK-31339) PlannerScalaFreeITCase.testImperativeUdaf

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31339: - Summary: PlannerScalaFreeITCase.testImperativeUdaf Key: FLINK-31339 URL: https://issues.apache.org/jira/browse/FLINK-31339 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-31338) support infer parallelism for flink table store

2023-03-06 Thread Jun Zhang (Jira)
Jun Zhang created FLINK-31338: - Summary: support infer parallelism for flink table store Key: FLINK-31338 URL: https://issues.apache.org/jira/browse/FLINK-31338 Project: Flink Issue Type: Improv

Re: help: [FLINK-31321] @flinkbot run azure does not work?

2023-03-06 Thread Martijn Visser
Hi, Please make sure that you have rebased on the latest changes. This was already tracked and fixed with https://issues.apache.org/jira/browse/FLINK-30972 Best regards, Martijn On Mon, Mar 6, 2023 at 10:12 AM yuxia wrote: > Hi, I have created FLINK-31334[1] to track it. > [1] https://issues.

[jira] [Created] (FLINK-31337) EmbeddedDataStreamBatchTests.test_keyed_co_broadcast_side_output

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31337: - Summary: EmbeddedDataStreamBatchTests.test_keyed_co_broadcast_side_output Key: FLINK-31337 URL: https://issues.apache.org/jira/browse/FLINK-31337 Project: Flink

[jira] [Created] (FLINK-31336) interval type process has problem in table api and sql

2023-03-06 Thread jackylau (Jira)
jackylau created FLINK-31336: Summary: interval type process has problem in table api and sql Key: FLINK-31336 URL: https://issues.apache.org/jira/browse/FLINK-31336 Project: Flink Issue Type: Bu

Re: [DISCUSS] FLIP-296: Watermark options for table API & SQL

2023-03-06 Thread Kui Yuan
Hi Jark, Thanks for your kind reply. I think your questioning is right, and my initial meaning is that we don't need to emit every watermark on each event. Sorry for my previous not-rigorous descrptions. The reason we add the 'on-event.gap' option is that we need to verify the watermark alignment

[jira] [Created] (FLINK-31335) using sql-gateway to submit job to yarn cluster, sql-gateway don't support kerberos

2023-03-06 Thread felixzh (Jira)
felixzh created FLINK-31335: --- Summary: using sql-gateway to submit job to yarn cluster, sql-gateway don't support kerberos Key: FLINK-31335 URL: https://issues.apache.org/jira/browse/FLINK-31335 Project: Fl

Re: help: [FLINK-31321] @flinkbot run azure does not work?

2023-03-06 Thread yuxia
Hi, I have created FLINK-31334[1] to track it. [1] https://issues.apache.org/jira/browse/FLINK-31334 Best regards, Yuxia - 原始邮件 - 发件人: "felixzh" 收件人: "dev" 发送时间: 星期一, 2023年 3 月 06日 上午 10:18:12 主题: Re:Re: help: [FLINK-31321] @flinkbot run azure does not work? http://security.ubuntu.com/

[jira] [Created] (FLINK-31334) E2e ci fail with unsupported file exception

2023-03-06 Thread luoyuxia (Jira)
luoyuxia created FLINK-31334: Summary: E2e ci fail with unsupported file exception Key: FLINK-31334 URL: https://issues.apache.org/jira/browse/FLINK-31334 Project: Flink Issue Type: Bug

Re: [DISCUSS] FLIP-298: Unifying the Implementation of SlotManager

2023-03-06 Thread Shammon FY
Hi weihua Can you add content related to `heterogeneous resources` to this FLIP? We can record it and consider it in the future. It may be useful for some scenarios, such as the combination of streaming and ML. Best, Shammon On Mon, Mar 6, 2023 at 1:39 PM weijie guo wrote: > Hi Weihua, > > Th

[jira] [Created] (FLINK-31333) KafkaPartitionSplitReaderTest.testPendingRecordsGauge failed to create topic

2023-03-06 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31333: - Summary: KafkaPartitionSplitReaderTest.testPendingRecordsGauge failed to create topic Key: FLINK-31333 URL: https://issues.apache.org/jira/browse/FLINK-31333 Projec

Re: [DISCUSS] String literal behavior in Flink

2023-03-06 Thread Aitozi
Thanks, will give it a try Best, Aitozi. Jark Wu 于2023年3月6日周一 15:11写道: > Hi Aitozi, > > I would suggest trying to contribute it to the upstream project Calcite > first. > > Best, > Jark > > > 2023年3月6日 11:51,Aitozi 写道: > > > > Hi Jark, > > > > Thank you for your helpful suggestion. It appears

[jira] [Created] (FLINK-31332) Limit the use of ExecutionConfig on JdbcOutputFormat

2023-03-06 Thread Jira
João Boto created FLINK-31332: - Summary: Limit the use of ExecutionConfig on JdbcOutputFormat Key: FLINK-31332 URL: https://issues.apache.org/jira/browse/FLINK-31332 Project: Flink Issue Type: Im

[jira] [Created] (FLINK-31331) Flink 1.16 should implement new LookupFunction

2023-03-06 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-31331: Summary: Flink 1.16 should implement new LookupFunction Key: FLINK-31331 URL: https://issues.apache.org/jira/browse/FLINK-31331 Project: Flink Issue Type: Bu

[jira] [Created] (FLINK-31330) Batch shuffle may deadlock for operator with priority input

2023-03-06 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-31330: -- Summary: Batch shuffle may deadlock for operator with priority input Key: FLINK-31330 URL: https://issues.apache.org/jira/browse/FLINK-31330 Project: Flink Issu