Re: [VOTE] FLIP-295: Support lazy initialization of catalogs and persistence of catalog configurations

2023-06-16 Thread Jark Wu
+1 (binding) Best, Jark > 2023年6月15日 19:15,Shammon FY 写道: > > Thanks Feng for the FLIP. > > +1(non-binding) > > Best, > Shammon FY > > On Thu, Jun 15, 2023 at 6:30 PM Leonard Xu wrote: > >> Thanks Feng for driving this FLIP forward. >> >> +1(binding) >> >> Best, >> Leonard >> >>> On Jun

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-16 Thread Becket Qin
Hi Chesnay, I think that there are two things we are discussing here: 1. The API stability story we WANT to have. 2. The API stability guarantees we CAN have. We can only design for what we want. Good API stability with affordable maintenance overhead does demand careful design from the high leve

[VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

2023-06-16 Thread Joao Boto
Hi all, Thank you to everyone for the feedback on FLIP-287[1]. Based on the discussion thread [2], we have come to a consensus on the design and are ready to take a vote to contribute this to Flink. I'd like to start a vote for it. The vote will be open for at least 72 hours(excluding weekends, u

Re: [VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

2023-06-16 Thread Lijie Wang
+1 (binding) Thanks for driving it, Joao. Best, Lijie Joao Boto 于2023年6月16日周五 15:53写道: > Hi all, > > Thank you to everyone for the feedback on FLIP-287[1]. Based on the > discussion thread [2], we have come to a consensus on the design and are > ready to take a vote to contribute this to Flink

[jira] [Created] (FLINK-32365) orc format get table statistics slow

2023-06-16 Thread hanjie (Jira)
hanjie created FLINK-32365: -- Summary: orc format get table statistics slow Key: FLINK-32365 URL: https://issues.apache.org/jira/browse/FLINK-32365 Project: Flink Issue Type: Improvement Co

Re: [DISCUSS] FLIP-321: Introduce an API deprecation process

2023-06-16 Thread Becket Qin
Hi Xintong, Thanks for the explanation. Please see the replies inline below. I agree. And from my understanding, demoting a Public API is also a kind of > such change, just like removing one, which can only happen with major > version bumps. I'm not proposing to allow demoting Public APIs anytime

Re: [DISCUSS] FLIP-324: Introduce Runtime Filter for Flink Batch Jobs

2023-06-16 Thread Stefan Richter
Hi, Thanks for the proposal of this feature! I have a question about the filter build and a some suggestions for potential improvements. First, I wonder why you suggest to run the filter builder as separate operator with parallelism 1. I’d suggest to integrate the filter distributed build with

TAC Applications for Community Over Code North America and Asia now open

2023-06-16 Thread Gavin McDonald
Hi All, (This email goes out to all our user and dev project mailing lists, so you may receive this email more than once.) The Travel Assistance Committee has opened up applications to help get people to the following events: *Community Over Code Asia 2023 - * *August 18th to August 20th in Bei

[jira] [Created] (FLINK-32366) YarnConfigurationITCase.testFlinkContainerMemory fails with ConnectException (msg: Connection refused)

2023-06-16 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32366: - Summary: YarnConfigurationITCase.testFlinkContainerMemory fails with ConnectException (msg: Connection refused) Key: FLINK-32366 URL: https://issues.apache.org/jira/browse/FLINK

[jira] [Created] (FLINK-32367) lead function second param leat to ClassCastException

2023-06-16 Thread zhou (Jira)
zhou created FLINK-32367: Summary: lead function second param leat to ClassCastException Key: FLINK-32367 URL: https://issues.apache.org/jira/browse/FLINK-32367 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-32368) KubernetesTestFixture doesn't implement the checkAndUpdateConfigMapFunction properly

2023-06-16 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32368: - Summary: KubernetesTestFixture doesn't implement the checkAndUpdateConfigMapFunction properly Key: FLINK-32368 URL: https://issues.apache.org/jira/browse/FLINK-32368

[jira] [Created] (FLINK-32369) Setup cron build

2023-06-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32369: Summary: Setup cron build Key: FLINK-32369 URL: https://issues.apache.org/jira/browse/FLINK-32369 Project: Flink Issue Type: Sub-task Compo

[jira] [Created] (FLINK-32370) JDBC SQl gateway e2e test is unstable

2023-06-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32370: Summary: JDBC SQl gateway e2e test is unstable Key: FLINK-32370 URL: https://issues.apache.org/jira/browse/FLINK-32370 Project: Flink Issue Type: Tec

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

2023-06-16 Thread Etienne Chauchot
Hi Robert, Thanks for your feedback. I don't know the scheduler part well enough yet and I'm taking this ticket as a learning workshop. Regarding your comments: 1. Taking a look at the AdaptiveScheduler class which takes all its configuration from the JobManagerOptions, and also to be consis

Re: [DISCUSS] SqlClient gateway mode: support for URLs

2023-06-16 Thread Alexander Fedulov
One more requirement surfaced in the scope of introducing the above mentioned improvements - the need for SQL Gateway requests to be authenticated. I propose to introduce an environmental variable, FLINK_REST_CLIENT_HEADERS that could contain a list of HTTPS headers ( including authentication cook

Re: [DISCUSS] FLIP-322 Cooldown period for adaptive scheduler

2023-06-16 Thread Chesnay Schepler
1) Options specific to the adaptive scheduler should start with "jobmanager.adaptive-scheduler". 2) There isn't /really /a notion of a "scaling event". The scheduler is informed about new/lost slots and job failures, and reacts accordingly by maybe rescaling the job. (sure, you can think of th

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

2023-06-16 Thread Piotr Nowojski
Hi Dong, > Suppose there are 1000 subtask and each subtask has 1% chance of being > "backpressured" at a given time (due to random traffic spikes). Then at any > given time, the chance of the job > being considered not-backpressured = (1-0.01)^1000. Since we evaluate the > backpressure metric once

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

2023-06-16 Thread Jing Ge
+1 (binding) Best regards, Jing On Thu, Jun 15, 2023 at 7:55 PM Mason Chen wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-246 [1]. Based on the > discussion thread [2], we have come to a consensus on the design and are > ready to take a vote to contribute this to Flink. > >

Re: [VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

2023-06-16 Thread Jing Ge
+1(binding) Best Regards, Jing On Fri, Jun 16, 2023 at 10:10 AM Lijie Wang wrote: > +1 (binding) > > Thanks for driving it, Joao. > > Best, > Lijie > > Joao Boto 于2023年6月16日周五 15:53写道: > > > Hi all, > > > > Thank you to everyone for the feedback on FLIP-287[1]. Based on the > > discussion thre

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

2023-06-16 Thread Dong Lin
Hi Piotr, Thanks for the reply. Please see my comments inline. On Fri, Jun 16, 2023 at 11:59 PM Piotr Nowojski wrote: > Hi Dong, > > > Suppose there are 1000 subtask and each subtask has 1% chance of being > > "backpressured" at a given time (due to random traffic spikes). Then at > any > > giv

[jira] [Created] (FLINK-32371) Bump snappy-java to 1.1.10.1

2023-06-16 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-32371: --- Summary: Bump snappy-java to 1.1.10.1 Key: FLINK-32371 URL: https://issues.apache.org/jira/browse/FLINK-32371 Project: Flink Issue Type: Improvement

[NOTICE] Experimental Java 17 support now available on master

2023-06-16 Thread Chesnay Schepler
The master branch now builds and runs with Java 17 out-of-the-box. Notes: - a nightly cron build was set up, but I expect it to fail because some tests only run in the branches and thus weren't covered by my testing. I'll try to stabilize things over the weekend. - The Python Kafka connector is

Re: [VOTE] FLIP-246: Dynamic Kafka Source (originally Multi Cluster Kafka Source)

2023-06-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) +1 for either DynamicKafkaSource or DiscoveringKafkaSource Cheers, Gordon On Thu, Jun 15, 2023, 10:56 Mason Chen wrote: > Hi all, > > Thank you to everyone for the feedback on FLIP-246 [1]. Based on the > discussion thread [2], we have come to a consensus on the design and are > r

Re: [VOTE] FLIP-287: Extend Sink#InitContext to expose TypeSerializer, ObjectReuse and JobID

2023-06-16 Thread Tzu-Li (Gordon) Tai
+1 (binding) On Fri, Jun 16, 2023, 09:53 Jing Ge wrote: > +1(binding) > > Best Regards, > Jing > > On Fri, Jun 16, 2023 at 10:10 AM Lijie Wang > wrote: > > > +1 (binding) > > > > Thanks for driving it, Joao. > > > > Best, > > Lijie > > > > Joao Boto 于2023年6月16日周五 15:53写道: > > > > > Hi all, > >