[jira] [Created] (FLINK-32614) avro mappings aren't always named as pojos

Martin Sillence created FLINK-32614: --- Summary: avro mappings aren't always named as pojos Key: FLINK-32614 URL: https://issues.apache.org/jira/browse/FLINK-32614 Project: Flink Issue Type:

Re: [DISCUSS][2.0] FLIP-337: Remove JarRequestBody#programArgs

We'll log a warn message when it is used and maybe hide it from the docs. Archunit rule doesn't really work here because it's not annotated with stability annotations (as it shouldn't since the classes aren't really user-facing). On 17/07/2023 21:56, Jing Ge wrote: Hi Chesnay, I am trying t

[DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

The endpoint hasn't been working for years and was only kept to inform users about it. Let's finally remove it. https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint

Re:Re: [DISCUSS] Release 2.0 Work Items

Thanks Xintong Song and Matthias for the insightful discussion! I have double-checked the jira tickets that belong to the "Need action in 1.18" section and have some inputs to share. For FLINK-4675: The argument StreamExecutionEnvironment in WindowAssigner.getDefaultTrigger() is not used

[DISCUSS][2.0] FLIP-341: Remove MetricGroup methods accepting an int as a name

The MetricGroup interface contains methods to create groups and metrics using an int as a name. The original intention was to allow pattern like |group.addGroup("subtaskIndex").addGroup(0)| , but this didn't really work out, with |addGroup(String, String)|  serving this use case much better.

Re: [DISCUSS] Release 2.0 Work Items

On 18/07/2023 10:33, Wencong Liu wrote: For FLINK-6912: There are three implementations of RichFunction that actually use the Configuration parameter in RichFunction#open: 1. ContinuousFileMonitoringFunction#open: It uses the configuration to configure the FileInputFormat. [1] 2.

FLIP-401: Remove brackets around keys returned by MetricGroup#getAllVariables

MetricGroup#getAllVariables returns all variables associated with the metric, for example: | = abcde| | = ||0| The keys are surrounded by brackets for no particular reason. In virtually every use-case for this method the user is stripping the brackets from keys, as done in: * our datadog r

Re: [DISCUSS][2.0] FLIP-337: Remove JarRequestBody#programArgs

Something to note is that the UI is using this parameter, and would have to be changed to the new one. Since we want to avoid having to split arguments ourselves, this may imply changes to the UI. On 18/07/2023 10:18, Chesnay Schepler wrote: We'll log a warn message when it is used and maybe

Re: FLIP-342: Remove brackets around keys returned by MetricGroup#getAllVariables

The FLIP number was changed to 342. On 18/07/2023 10:56, Chesnay Schepler wrote: MetricGroup#getAllVariables returns all variables associated with the metric, for example: | = abcde| | = ||0| The keys are surrounded by brackets for no particular reason. In virtually every use-case for this m

Re: [DISCUSS][2.0] FLIP-337: Remove JarRequestBody#programArgs

+1 to this. Nice to simplify the REST API! Regards, Hong > On 18 Jul 2023, at 10:00, Chesnay Schepler wrote: > > Something to note is that the UI is using this parameter, and would have to > be changed to the new one. > > Since we want to avoid having to split arguments ourselves, this may

Re:Re: [DISCUSS] Release 2.0 Work Items

Hi Chesnay, Thanks for the reply. I think it is reasonable to remove the configuration argument in AbstractUdfStreamOperator#open if it is consistently empty. I'll propose a discuss about the specific actions in FLINK-6912 at a later time. Best, Wencong Liu At 2023-07-18 16:38:59, "C

[jira] [Created] (FLINK-32615) Cache AutoscalerInfo for each resource

Gyula Fora created FLINK-32615: -- Summary: Cache AutoscalerInfo for each resource Key: FLINK-32615 URL: https://issues.apache.org/jira/browse/FLINK-32615 Project: Flink Issue Type: Improvement

Re: [DISCUSS][2.0] FLIP-337: Remove JarRequestBody#programArgs

got it thanks! For @Deprecated, I meant to force using like: @Deprecated(since = "1.18", forRemoval = true) Best regards, Jing On Tue, Jul 18, 2023 at 11:06 AM Hong Teoh wrote: > +1 to this. Nice to simplify the REST API! > > > Regards, > Hong > > > > On 18 Jul 2023, at 10:00, Chesnay Schepler

[jira] [Created] (FLINK-32616) FlinkStatement#executeQuery resource leaks when the input sql is not query

Shengkai Fang created FLINK-32616: - Summary: FlinkStatement#executeQuery resource leaks when the input sql is not query Key: FLINK-32616 URL: https://issues.apache.org/jira/browse/FLINK-32616 Project:

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 Best, Xintong On Tue, Jul 18, 2023 at 4:25 PM Chesnay Schepler wrote: > The endpoint hasn't been working for years and was only kept to inform > users about it. Let's finally remove it. > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-340%3A+Remove+rescale+REST+endpoint > >

[jira] [Created] (FLINK-32617) FlinkResultSetMetaData throw exception for most methods

Shengkai Fang created FLINK-32617: - Summary: FlinkResultSetMetaData throw exception for most methods Key: FLINK-32617 URL: https://issues.apache.org/jira/browse/FLINK-32617 Project: Flink Iss

Re: [DISCUSS][2.0] FLIP-341: Remove MetricGroup methods accepting an int as a name

+1 in general. I think the list of affected public interfaces in the FLIP is not accurate. - `#counter(int, Counter)` is missed - `#meter(int)` should be `#meter(int, Meter)` - `#group(int)` should be `#addGroup(int)` Best, Xintong On Tue, Jul 18, 2023 at 4:39 PM Chesnay Schepler

Re: FLIP-342: Remove brackets around keys returned by MetricGroup#getAllVariables

+1 Best, Xintong On Tue, Jul 18, 2023 at 5:02 PM Chesnay Schepler wrote: > The FLIP number was changed to 342. > > On 18/07/2023 10:56, Chesnay Schepler wrote: > > MetricGroup#getAllVariables returns all variables associated with the > > metric, for example: > > > > | = abcde| > > | = ||0| >

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 On Tue, 18 Jul 2023 at 12:12, Xintong Song wrote: > +1 > > Best, > > Xintong > > > > On Tue, Jul 18, 2023 at 4:25 PM Chesnay Schepler > wrote: > > > The endpoint hasn't been working for years and was only kept to inform > > users about it. Let's finally remove it. > > > > > > > https://cwiki

[jira] [Created] (FLINK-32618) Remove the dependency of the flink-core in the flink-sql-jdbc-driver-bundle

Shengkai Fang created FLINK-32618: - Summary: Remove the dependency of the flink-core in the flink-sql-jdbc-driver-bundle Key: FLINK-32618 URL: https://issues.apache.org/jira/browse/FLINK-32618 Project

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 On Tue, Jul 18, 2023 at 12:29 PM Gyula Fóra wrote: > > +1 > > On Tue, 18 Jul 2023 at 12:12, Xintong Song wrote: > > > +1 > > > > Best, > > > > Xintong > > > > > > > > On Tue, Jul 18, 2023 at 4:25 PM Chesnay Schepler > > wrote: > > > > > The endpoint hasn't been working for years and was only

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 On Tue, Jul 18, 2023 at 1:05 PM Maximilian Michels wrote: > +1 > > On Tue, Jul 18, 2023 at 12:29 PM Gyula Fóra wrote: > > > > +1 > > > > On Tue, 18 Jul 2023 at 12:12, Xintong Song > wrote: > > > > > +1 > > > > > > Best, > > > > > > Xintong > > > > > > > > > > > > On Tue, Jul 18, 2023 at 4:2

Re: FLIP-342: Remove brackets around keys returned by MetricGroup#getAllVariables

+1 On Tue, Jul 18, 2023 at 12:24 PM Xintong Song wrote: > +1 > > Best, > > Xintong > > > > On Tue, Jul 18, 2023 at 5:02 PM Chesnay Schepler > wrote: > > > The FLIP number was changed to 342. > > > > On 18/07/2023 10:56, Chesnay Schepler wrote: > > > MetricGroup#getAllVariables returns all varia

Re: [DISCUSS][2.0] FLIP-341: Remove MetricGroup methods accepting an int as a name

Good catch; i've fixed the list. On 18/07/2023 12:20, Xintong Song wrote: +1 in general. I think the list of affected public interfaces in the FLIP is not accurate. - `#counter(int, Counter)` is missed - `#meter(int)` should be `#meter(int, Meter)` - `#group(int)` should be `#addGr

[jira] [Created] (FLINK-32619) ConfigOptions to support fallback configuration

Hong Liang Teoh created FLINK-32619: --- Summary: ConfigOptions to support fallback configuration Key: FLINK-32619 URL: https://issues.apache.org/jira/browse/FLINK-32619 Project: Flink Issue T

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

Thanks Dong! Piotrek wt., 18 lip 2023 o 06:04 Dong Lin napisał(a): > Hi all, > > I have updated FLIP-309 as suggested by Piotr to include a reference to > FLIP-328 in the future work section. > > Piotra, Stephan, and I discussed offline regarding the choice > between execution.checkpointing.max

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

+1 (binding) Piotrek wt., 18 lip 2023 o 08:51 Jing Ge napisał(a): > +1(binding) > > Best regards, > Jing > > On Tue, Jul 18, 2023 at 8:31 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1(binding) > > > > Best, > > Rui Fan > > > > > > On Tue, Jul 18, 2023 at 12:04 PM Dong Lin wrote: > > > > >

[jira] [Created] (FLINK-32620) Migrate DiscardingSink to sinkv2

Weijie Guo created FLINK-32620: -- Summary: Migrate DiscardingSink to sinkv2 Key: FLINK-32620 URL: https://issues.apache.org/jira/browse/FLINK-32620 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-32621) Add metrics for DataGeneratorSource

Weijie Guo created FLINK-32621: -- Summary: Add metrics for DataGeneratorSource Key: FLINK-32621 URL: https://issues.apache.org/jira/browse/FLINK-32621 Project: Flink Issue Type: Technical Debt

[jira] [Created] (FLINK-32622) Do not add mini-batch assigner operator if it is useless

Fang Yong created FLINK-32622: - Summary: Do not add mini-batch assigner operator if it is useless Key: FLINK-32622 URL: https://issues.apache.org/jira/browse/FLINK-32622 Project: Flink Issue Type

Re: [DISCUSS][2.0] FLIP-341: Remove MetricGroup methods accepting an int as a name

+1 On Tue, Jul 18, 2023 at 1:11 PM Chesnay Schepler wrote: > Good catch; i've fixed the list. > > On 18/07/2023 12:20, Xintong Song wrote: > > +1 in general. > > > > I think the list of affected public interfaces in the FLIP is not > accurate. > > > > - `#counter(int, Counter)` is missed > >

[jira] [Created] (FLINK-32623) Rest api doesn't return minimum resource requirements correctly

Gyula Fora created FLINK-32623: -- Summary: Rest api doesn't return minimum resource requirements correctly Key: FLINK-32623 URL: https://issues.apache.org/jira/browse/FLINK-32623 Project: Flink

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

+1 (binding) Best, Jark On Tue, 18 Jul 2023 at 20:30, Piotr Nowojski wrote: > +1 (binding) > > Piotrek > > wt., 18 lip 2023 o 08:51 Jing Ge napisał(a): > > > +1(binding) > > > > Best regards, > > Jing > > > > On Tue, Jul 18, 2023 at 8:31 AM Rui Fan <1996fan...@gmail.com> wrote: > > > > > +1(bi

Re: FLIP-401: Remove brackets around keys returned by MetricGroup#getAllVariables

Hi Chesnay, +1 Sounds good to me! -Max On Tue, Jul 18, 2023 at 10:59 AM Chesnay Schepler wrote: > > MetricGroup#getAllVariables returns all variables associated with the > metric, for example: > > | = abcde| > | = ||0| > > The keys are surrounded by brackets for no particular reason. > > In vir

[jira] [Created] (FLINK-32624) TieredStorageConsumerClientTest.testGetNextBufferFromRemoteTier failed on CI

lincoln lee created FLINK-32624: --- Summary: TieredStorageConsumerClientTest.testGetNextBufferFromRemoteTier failed on CI Key: FLINK-32624 URL: https://issues.apache.org/jira/browse/FLINK-32624 Project: F

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

+1 binding Thanks Dong for continuous driving. Best, Jingsong On Tue, Jul 18, 2023 at 10:04 PM Jark Wu wrote: > > +1 (binding) > > Best, > Jark > > On Tue, 18 Jul 2023 at 20:30, Piotr Nowojski wrote: > > > +1 (binding) > > > > Piotrek > > > > wt., 18 lip 2023 o 08:51 Jing Ge napisał(a): > > >

[jira] [Created] (FLINK-32625) MiniClusterTestEnvironment supports customized MiniClusterResourceConfiguration

Zili Chen created FLINK-32625: - Summary: MiniClusterTestEnvironment supports customized MiniClusterResourceConfiguration Key: FLINK-32625 URL: https://issues.apache.org/jira/browse/FLINK-32625 Project: Fl

[jira] [Created] (FLINK-32626) Get Savepoint REST API doesn't distinguish non-existent job from non-existent savepoint

Austin Cawley-Edwards created FLINK-32626: - Summary: Get Savepoint REST API doesn't distinguish non-existent job from non-existent savepoint Key: FLINK-32626 URL: https://issues.apache.org/jira/browse/FLI

Re: [DISCUSS] Flink Kubernetes Operator cleanup procedure

Hi Daren, The behavior is consistent with the regular FlinkDeployment where the cleanup will also cancel any running jobs. Are you intending to recover jobs from another session cluster? -Max On Mon, Jul 17, 2023 at 4:48 PM Wong, Daren wrote: > > Hi devs, > > I would like to enquire about the c

Keying a data stream by tenant and performing Flink ML tasks on each sub-stream ?

Hello everyone, Please forgive me if this is not the correct mailing list for the issue below. Here is my use case: I have an event data stream which I need to "keyby" a certain field (tenant id) and then for each tenant's events I need to independently perform ML clustering using FlinkML's Onlin

Flink 1.15.1 Troubleshooting following exception

Hi, we are trying to determine how fix the following exception. This is an issue that is repeatedly happening for us. We have tried looking online for some solutions. One thread suggested setting idleTimeout but this doesn’t seem supported in Flink Source code. https://github.com/netty/netty/

[DISCUSSION] Revival of FLIP-154 (SQL Implicit Type Coercion)

Hello everyone I'd like to revive FLIP-154[1] a bit. I failed with finding any discussion/vote thread about it (please point me to that if it is present somewhere) Also FLIP itself looks abandoned (no activity for a couple of years), however it seems to be useful. I did a bit investigation abou

[jira] [Created] (FLINK-32627) Add support for dynamic time window function

张一帆 created FLINK-32627: --- Summary: Add support for dynamic time window function Key: FLINK-32627 URL: https://issues.apache.org/jira/browse/FLINK-32627 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

+1 for interval-during-backlog best， leonard > On Jul 14, 2023, at 11:38 PM, Piotr Nowojski wrote: > > Hi All, > > We had a lot of off-line discussions. As a result I would suggest dropping > the idea of introducing an end-to-end-latency concept, until > we can properly implement it, which wi

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

Thanks Dong for the continuous work. +1(binding) Best, Leonard > On Jul 18, 2023, at 10:16 PM, Jingsong Li wrote: > > +1 binding > > Thanks Dong for continuous driving. > > Best, > Jingsong > > On Tue, Jul 18, 2023 at 10:04 PM Jark Wu wrote: >> >> +1 (binding) >> >> Best, >> Jark >> >>

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 Thanks, Zhu Jing Ge 于2023年7月18日周二 19:09写道： > > +1 > > On Tue, Jul 18, 2023 at 1:05 PM Maximilian Michels wrote: > > > +1 > > > > On Tue, Jul 18, 2023 at 12:29 PM Gyula Fóra wrote: > > > > > > +1 > > > > > > On Tue, 18 Jul 2023 at 12:12, Xintong Song > > wrote: > > > > > > > +1 > > > > > >

Re: Re: [DISCUSS] Release 2.0 Work Items

I went through the remaining Jira tickets with 2.0.0 fix-version and are not included in FLINK-3975. I skipped the 3 umbrella tickets below and their subtasks, which are newly created for the 2.0 work items. - FLINK-32377 Breaking REST API changes - FLINK-32378 Breaking Metrics system chang

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

+1 (non-binding) Thanks for driving. Best, Hang Leonard Xu 于2023年7月19日周三 10:42写道： > Thanks Dong for the continuous work. > > +1(binding) > > Best, > Leonard > > > On Jul 18, 2023, at 10:16 PM, Jingsong Li > wrote: > > > > +1 binding > > > > Thanks Dong for continuous driving. > > > > Best, >

Re: [DISCUSS][2.0] FLIP-340: Remove rescale REST endpoint

+1 Zhu Zhu 于2023年7月19日周三 10:53写道： > +1 > > Thanks, > Zhu > > Jing Ge 于2023年7月18日周二 19:09写道： > > > > +1 > > > > On Tue, Jul 18, 2023 at 1:05 PM Maximilian Michels > wrote: > > > > > +1 > > > > > > On Tue, Jul 18, 2023 at 12:29 PM Gyula Fóra wrote: > > > > > > > > +1 > > > > > > > > On Tue, 18

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Hi Xintong, Thanks for driving this topic. Regarding the Table API deprecation, I can provide some details to help with the process. 1. This sheet summarizes the user-facing classes an

Re: [DISUCSS] Deprecate multiple APIs in 1.18

Thanks for the beautiful sheets, Jane. 1. This sheet < > https://docs.google.com/spreadsheets/d/1dZBNHLuAHYJt3pFU8ZtfUzrYyf2ZFQ6wybDXGS1bHno/edit?usp=sharing> > summarizes the user-facing classes and methods that need to be deprecated > under the flink-table module, some of which are marked with a

Re: [VOTE] FLIP-309: Support using larger checkpointing interval when source is processing backlog

+1（binding） Best, Guowei On Wed, Jul 19, 2023 at 11:18 AM Hang Ruan wrote: > +1 (non-binding) > > Thanks for driving. > > Best, > Hang > > Leonard Xu 于2023年7月19日周三 10:42写道： > > > Thanks Dong for the continuous work. > > > > +1(binding) > > > > Best, > > Leonard > > > > > On Jul 18, 2023, at 10

[jira] [Created] (FLINK-32628) build_wheels_on_macos fails on AZP

Sergey Nuyanzin created FLINK-32628: --- Summary: build_wheels_on_macos fails on AZP Key: FLINK-32628 URL: https://issues.apache.org/jira/browse/FLINK-32628 Project: Flink Issue Type: Bug