[jira] [Created] (FLINK-27598) Improve the exception message when mixing use Python UDF and Pandas UDF

2022-05-12 Thread Dian Fu (Jira)
Dian Fu created FLINK-27598: --- Summary: Improve the exception message when mixing use Python UDF and Pandas UDF Key: FLINK-27598 URL: https://issues.apache.org/jira/browse/FLINK-27598 Project: Flink

[jira] [Created] (FLINK-27597) Get statistic for Hive paritioned table

2022-05-12 Thread luoyuxia (Jira)
luoyuxia created FLINK-27597: Summary: Get statistic for Hive paritioned table Key: FLINK-27597 URL: https://issues.apache.org/jira/browse/FLINK-27597 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27596) [Umbrella] Optimize Hive source

2022-05-12 Thread luoyuxia (Jira)
luoyuxia created FLINK-27596: Summary: [Umbrella] Optimize Hive source Key: FLINK-27596 URL: https://issues.apache.org/jira/browse/FLINK-27596 Project: Flink Issue Type: Improvement Co

[jira] [Created] (FLINK-27595) Helm: We should make security context configurable

2022-05-12 Thread Xin Hao (Jira)
Xin Hao created FLINK-27595: --- Summary: Helm: We should make security context configurable Key: FLINK-27595 URL: https://issues.apache.org/jira/browse/FLINK-27595 Project: Flink Issue Type: Improvem

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-12 Thread Александр Смирнов
Hi Jark, Qingsheng and Leonard! Glad to see that we came to a consensus on almost all points! However I'm a little confused whether InputFormat is deprecated or not. Am I right that it will be so in the future, but currently it's not? Actually I also think that for the first version it's OK to us

Re: [DISCUSS] FLIP-91: Support SQL Client Gateway

2022-05-12 Thread Jark Wu
Hi Shengkai, One comment on the configure_session VS initialize_session, I think configure_session is better than initialize_session, because "initialize" sounds like this method can only be invoked once. But I can see this method is useful to be invoked several times to configure the session duri

Re: [DISCUSS] FLIP-91: Support SQL Client Gateway

2022-05-12 Thread Jark Wu
Hi Timo, I agree with Shengkai that we should keep Gateway in the Flink main repo. Here are my thoughts: 1) It is not feasible and maintainable to invoke different versions of Flink dependencies in one JVM. I'm afraid it would be a nightmare when the code is messed up with all the reflections. 2

[ANNOUNCE] Problems with build due to tests using too much disk space

2022-05-12 Thread Martijn Visser
Hi everyone, There are currently problems with builds on master, release-1.15 and release-1.14. This is tracked under https://issues.apache.org/jira/browse/FLINK-24433 The E2E tests are failing due to running out of disk space. I have noticed something in one of the bash based E2E tests (where a

[jira] [Created] (FLINK-27594) Only recover JM deployment if HA metadata available

2022-05-12 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-27594: -- Summary: Only recover JM deployment if HA metadata available Key: FLINK-27594 URL: https://issues.apache.org/jira/browse/FLINK-27594 Project: Flink Issue Type: B

[DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

2022-05-12 Thread Lincoln Lee
Dear Flink developers, I would like to open a discussion on FLIP 232 [1], for an extension of AsyncWaitOperator to support retry for user's asyncFunction. To do so, new user interface will added to define the trigger condition for retry and when should retry. Internally, a delayed retry mechanis

Batching in kinesis sink

2022-05-12 Thread Zain Haider Nemati
Hi, I am using a kinesis sink with flink 1.13. The amount of data is in millions and it choke the 1MB cap for kinesis data streams. Is there any way to send data to kinesis sink in batches of less than 1MB? or any other workaround

[jira] [Created] (FLINK-27593) Parquet/Orc format reader optimization

2022-05-12 Thread dalongliu (Jira)
dalongliu created FLINK-27593: - Summary: Parquet/Orc format reader optimization Key: FLINK-27593 URL: https://issues.apache.org/jira/browse/FLINK-27593 Project: Flink Issue Type: Bug Co

RE: Re: JobManager High Availability

2022-05-12 Thread Tigran Manasyan
Hi everyone! I have encountered similar behavior in the case of native k8s HA with multiple JobManagers. Therefore, I have a question - were there any plans to add the ability not to restart the job during the change of the JobManagers leader? Or are there certain insurmountable obstacles preventi

Re: [DISCUSS] FLIP-91: Support SQL Client Gateway

2022-05-12 Thread Shengkai Fang
Hi Timo. Thanks for your feedback! > It would be great if you can copy/paste some of the tricky design decisions into the FLIP. Yes. I have already added a section about the `Rejected Alternatives`. It includes the discuss about the - TableInfo and FunctionInfo VS CatalogTable and CatalogFunctio

[jira] [Created] (FLINK-27592) WebUI - Status CANCELING not showing in overview

2022-05-12 Thread Peter Schrott (Jira)
Peter Schrott created FLINK-27592: - Summary: WebUI - Status CANCELING not showing in overview Key: FLINK-27592 URL: https://issues.apache.org/jira/browse/FLINK-27592 Project: Flink Issue Type

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-12 Thread Jark Wu
Hi Alex, Thanks for summarizing your points. In the past week, Qingsheng, Leonard, and I have discussed it several times and we have totally refactored the design. I'm glad to say we have reached a consensus on many of your points! Qingsheng is still working on updating the design docs and maybe

[jira] [Created] (FLINK-27591) Improve the plan for batch queries when statistics is unavailable

2022-05-12 Thread godfrey he (Jira)
godfrey he created FLINK-27591: -- Summary: Improve the plan for batch queries when statistics is unavailable Key: FLINK-27591 URL: https://issues.apache.org/jira/browse/FLINK-27591 Project: Flink

[jira] [Created] (FLINK-27590) Provide endpoint with information about leader Jobmanager in HA setups via Zookeeper

2022-05-12 Thread Dhruv Patel (Jira)
Dhruv Patel created FLINK-27590: --- Summary: Provide endpoint with information about leader Jobmanager in HA setups via Zookeeper Key: FLINK-27590 URL: https://issues.apache.org/jira/browse/FLINK-27590 Pr

[jira] [Created] (FLINK-27589) Fix incorrect implementation of TableStoreDateObjectInspector

2022-05-12 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27589: --- Summary: Fix incorrect implementation of TableStoreDateObjectInspector Key: FLINK-27589 URL: https://issues.apache.org/jira/browse/FLINK-27589 Project: Flink

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-12 Thread Robert Metzger
+1 on setting up our own Slack instance (PMC owned) +1 for having a separate discussion about setting up a discussion forum (I like the idea of using GH discussions) Besides, we still need to investigate how > http://apache-airflow.slack-archives.org works, I think > a slack of our own can be easi

[jira] [Created] (FLINK-27588) Update broadcast state related docs for PyFlink

2022-05-12 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-27588: - Summary: Update broadcast state related docs for PyFlink Key: FLINK-27588 URL: https://issues.apache.org/jira/browse/FLINK-27588 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-27587) Support keyed co-broadcast processing

2022-05-12 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-27587: - Summary: Support keyed co-broadcast processing Key: FLINK-27587 URL: https://issues.apache.org/jira/browse/FLINK-27587 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27586) Support non-keyed co-broadcast processing

2022-05-12 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-27586: - Summary: Support non-keyed co-broadcast processing Key: FLINK-27586 URL: https://issues.apache.org/jira/browse/FLINK-27586 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27585) JdbcOutpuFormat fails to flush data on checkpoint and causes data loss

2022-05-12 Thread Paul Lin (Jira)
Paul Lin created FLINK-27585: Summary: JdbcOutpuFormat fails to flush data on checkpoint and causes data loss Key: FLINK-27585 URL: https://issues.apache.org/jira/browse/FLINK-27585 Project: Flink

[jira] [Created] (FLINK-27584) Support broadcast state in PyFlink DataStream API

2022-05-12 Thread Dian Fu (Jira)
Dian Fu created FLINK-27584: --- Summary: Support broadcast state in PyFlink DataStream API Key: FLINK-27584 URL: https://issues.apache.org/jira/browse/FLINK-27584 Project: Flink Issue Type: New Featu

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-12 Thread Jingsong Li
Thank Jark~ Hi all, I just created a vote thread [1]. Feel free to -1 if you think there is something wrong with the design. [1] https://lists.apache.org/thread/lg5txz95mgko4mp6fqcwt1dd1hbjctjy Best, Jingsong On Thu, May 12, 2022 at 4:14 PM Jark Wu wrote: > Thank Jingsong for the explanation

[VOTE] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-12 Thread Jingsong Li
Hi, everyone Thanks all for your attention to FLIP-226: Introduce Schema Evolution on Table Store [1] and participation in the discussion in the mail thread [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] htt

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-12 Thread Jark Wu
Hi, I would +1 to create Apache Flink Slack for the lower barriers to entry as Jingsong mentioned. Besides, we still need to investigate how http://apache-airflow.slack-archives.org works, I think a slack of our own can be easier to set up the archive. Regarding Discourse vs Slack, I think they a

[jira] [Created] (FLINK-27583) Improve the plan for TPC-DS queries

2022-05-12 Thread godfrey he (Jira)
godfrey he created FLINK-27583: -- Summary: Improve the plan for TPC-DS queries Key: FLINK-27583 URL: https://issues.apache.org/jira/browse/FLINK-27583 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-27582) modify the number of parameters

2022-05-12 Thread zeliu (Jira)
zeliu created FLINK-27582: - Summary: modify the number of parameters Key: FLINK-27582 URL: https://issues.apache.org/jira/browse/FLINK-27582 Project: Flink Issue Type: Bug Components: API

[jira] [Created] (FLINK-27581) modify the number of parameters

2022-05-12 Thread zeliu (Jira)
zeliu created FLINK-27581: - Summary: modify the number of parameters Key: FLINK-27581 URL: https://issues.apache.org/jira/browse/FLINK-27581 Project: Flink Issue Type: Bug Components: API

[jira] [Created] (FLINK-27580) Implement filter pushdown for TableStoreHiveStorageHandler

2022-05-12 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27580: --- Summary: Implement filter pushdown for TableStoreHiveStorageHandler Key: FLINK-27580 URL: https://issues.apache.org/jira/browse/FLINK-27580 Project: Flink Issu

[jira] [Created] (FLINK-27579) The param client.timeout can not be set by dynamic properties when stopping the job

2022-05-12 Thread Liu (Jira)
Liu created FLINK-27579: --- Summary: The param client.timeout can not be set by dynamic properties when stopping the job Key: FLINK-27579 URL: https://issues.apache.org/jira/browse/FLINK-27579 Project: Flink

Re: [DISCUSS] FLIP-226: Introduce Schema Evolution on Table Store

2022-05-12 Thread Jark Wu
Thank Jingsong for the explanation. I don't have other concerns. Best, Jark On Thu, 12 May 2022 at 09:53, Jingsong Li wrote: > Hi all, > > If there are no more comments, I'm going to start a vote. > > Best, > Jingsong > > On Tue, May 10, 2022 at 10:37 AM Jingsong Li > wrote: > > > Hi Jark, > >

[jira] [Created] (FLINK-27578) Elasticsearch6SinkE2ECase.testScaleUp fails after "Exhausted retry attempts"

2022-05-12 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27578: -- Summary: Elasticsearch6SinkE2ECase.testScaleUp fails after "Exhausted retry attempts" Key: FLINK-27578 URL: https://issues.apache.org/jira/browse/FLINK-27578 Proj

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-12 Thread Александр Смирнов
Hi Martijn! Got it. Therefore, the realization with InputFormat is not considered. Thanks for clearing that up! Best regards, Smirnov Alexander чт, 12 мая 2022 г. в 14:23, Martijn Visser : > > Hi, > > With regards to: > > > But if there are plans to refactor all connectors to FLIP-27 > > Yes, FL

[jira] [Created] (FLINK-27577) Add windowAll operator

2022-05-12 Thread zhangjingcun (Jira)
zhangjingcun created FLINK-27577: Summary: Add windowAll operator Key: FLINK-27577 URL: https://issues.apache.org/jira/browse/FLINK-27577 Project: Flink Issue Type: New Feature Affects Ve

Re: [Discuss] Creating an Apache Flink slack workspace

2022-05-12 Thread Martijn Visser
Hi, I would +1 setting up our own Slack. It will allow us to provide the best experience for those in the community who want to use Slack. More than happy to help with setting up community guidelines on how to use Slack. Best regards, Martijn On Thu, 12 May 2022 at 05:22, Yun Tang wrote: > Hi

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-05-12 Thread Martijn Visser
Hi, With regards to: > But if there are plans to refactor all connectors to FLIP-27 Yes, FLIP-27 is the target for all connectors. The old interfaces will be deprecated and connectors will either be refactored to use the new ones or dropped. The caching should work for connectors that are using