Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-10 Thread Jark Wu
Thank you Feng, Feel free to start a FLIP proposal if you are interested. Looking forward to it! Best, Jark > 2023年2月10日 15:44,Feng Jin 写道: > > @Shengkai >> About the catalog jar hot updates > > Currently we do not have a similar requirement, but if the catalog > management interface is open

[jira] [Created] (FLINK-31009) Add recordCount to snapshot meta

2023-02-10 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-31009: Summary: Add recordCount to snapshot meta Key: FLINK-31009 URL: https://issues.apache.org/jira/browse/FLINK-31009 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-31010) Add Transformer and Estimator for GBTClassifier and GBTRegressor

2023-02-10 Thread Fan Hong (Jira)
Fan Hong created FLINK-31010: Summary: Add Transformer and Estimator for GBTClassifier and GBTRegressor Key: FLINK-31010 URL: https://issues.apache.org/jira/browse/FLINK-31010 Project: Flink Iss

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-10 Thread Feng Jin
I am very happy to do it, please help me to add editing permission, my jira id is hackergin Thanks Best, Feng On Fri, Feb 10, 2023 at 4:02 PM Jark Wu wrote: > > Thank you Feng, > > Feel free to start a FLIP proposal if you are interested. Looking forward to > it! > > Best, > Jark > > > 2023年2

[jira] [Created] (FLINK-31011) upgrade hiverunner version

2023-02-10 Thread Jun Zhang (Jira)
Jun Zhang created FLINK-31011: - Summary: upgrade hiverunner version Key: FLINK-31011 URL: https://issues.apache.org/jira/browse/FLINK-31011 Project: Flink Issue Type: Improvement Compon

[jira] [Created] (FLINK-31012) Update docs for files table in table store

2023-02-10 Thread Shammon (Jira)
Shammon created FLINK-31012: --- Summary: Update docs for files table in table store Key: FLINK-31012 URL: https://issues.apache.org/jira/browse/FLINK-31012 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-31013) SessionWindowAggregation cannot get trigger window using event time

2023-02-10 Thread Jane Chan (Jira)
Jane Chan created FLINK-31013: - Summary: SessionWindowAggregation cannot get trigger window using event time Key: FLINK-31013 URL: https://issues.apache.org/jira/browse/FLINK-31013 Project: Flink

Re: [DISCUSS] FLIP-291: Externalized Declarative Resource Management

2023-02-10 Thread feng xiangyu
Hi David, Thanks for creating this flip. I think this work it is very useful, especially in autoscaling scenario. I would like to share some questions from my view. 1, Is it possible for this REST API to declare TM resources in the future? I'm asking because we are building the autoscaling featu

[jira] [Created] (FLINK-31014) Expose exceptions in connector testing framework

2023-02-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31014: - Summary: Expose exceptions in connector testing framework Key: FLINK-31014 URL: https://issues.apache.org/jira/browse/FLINK-31014 Project: Flink Issue Type

[jira] [Created] (FLINK-31015) SplitReader doesn't extend AutoCloseable but implements close

2023-02-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31015: - Summary: SplitReader doesn't extend AutoCloseable but implements close Key: FLINK-31015 URL: https://issues.apache.org/jira/browse/FLINK-31015 Project: Flink

Re: [Discuss] :Introduce Catalog dynamic registration in flink catalog manager.

2023-02-10 Thread Jark Wu
I have assigned permission to you. Best, Jark > 2023年2月10日 17:26,Feng Jin 写道: > > I am very happy to do it, please help me to add editing permission, my > jira id is hackergin > > Thanks > > Best, > Feng > > On Fri, Feb 10, 2023 at 4:02 PM Jark Wu wrote: >> >> Thank you Feng, >> >> Feel

[jira] [Created] (FLINK-31016) "Run kubernetes pyflink application test" fails due to Rust package manager not being listed in the PATH

2023-02-10 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-31016: - Summary: "Run kubernetes pyflink application test" fails due to Rust package manager not being listed in the PATH Key: FLINK-31016 URL: https://issues.apache.org/jira/browse/FLI

[jira] [Created] (FLINK-31017) Early-started partial match timeout not yield completed matches

2023-02-10 Thread Juntao Hu (Jira)
Juntao Hu created FLINK-31017: - Summary: Early-started partial match timeout not yield completed matches Key: FLINK-31017 URL: https://issues.apache.org/jira/browse/FLINK-31017 Project: Flink Is

[jira] [Created] (FLINK-31018) SQL Client -j option does not load user jars to classpath.

2023-02-10 Thread Krzysztof Chmielewski (Jira)
Krzysztof Chmielewski created FLINK-31018: - Summary: SQL Client -j option does not load user jars to classpath. Key: FLINK-31018 URL: https://issues.apache.org/jira/browse/FLINK-31018 Project:

Re: Flink Operator - Supporting Recovery from Snapshot

2023-02-10 Thread Kevin Lam
Thanks for the response Gyula! Those caveats make sense, and I see, there's a bit of a complexity to consider if the feature is implemented. I do think it would be useful, so would also love to hear what others think! On Wed, Feb 8, 2023 at 3:47 AM Gyula Fóra wrote: > Hi Kevin! > > Thanks for st

Re: Flink Operator - Supporting Recovery from Snapshot

2023-02-10 Thread Yaroslav Tkachenko
Hi Kevin! In my case, I automated this workflow by first deleting the current Flink deployment and then creating a new one. So, if the initialSavepointPath is different it'll use it for recovery. This approach is indeed irreversible, but so far it's been working well. On Fri, Feb 10, 2023 at 8:1

Re: Flink Operator - Supporting Recovery from Snapshot

2023-02-10 Thread Kevin Lam
Hey Yaroslav! Awesome, good to know that approach works well for you. I think our plan as of now is to do the same--delete the current FlinkDeployment when deploying from a specific snapshot. It'll be a separate workflow from normal deployments to take advantage of the operator otherwise. Thanks!

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

2023-02-10 Thread Martijn Visser
Hi all, I wanted to get back on a couple of comments and have a proposal at the end for the next steps: @Steven Wu If I look back at the original discussion of FLIP-191 and also the thread that you're referring to, it appears from the discussion with Yun Gao that the solution was in near-sight, b

[jira] [Created] (FLINK-31019) Migrate FileSystemTableSink to FileSink

2023-02-10 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-31019: -- Summary: Migrate FileSystemTableSink to FileSink Key: FLINK-31019 URL: https://issues.apache.org/jira/browse/FLINK-31019 Project: Flink Issue Type: Impro

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Martijn Visser
@Qingsheng what are your next steps for this proposal? On Thu, Jan 19, 2023 at 9:14 AM Mason Chen wrote: > Hi all, > > Sorry to come into the discussion late--I saw the thread earlier. > > I'm also +1 for the change in general. I think most users have this turned > on by default since the overhe

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Martijn Visser
Oh and Mason, definitely interesting! :) On Fri, Feb 10, 2023 at 9:51 PM Martijn Visser wrote: > @Qingsheng what are your next steps for this proposal? > > On Thu, Jan 19, 2023 at 9:14 AM Mason Chen wrote: > >> Hi all, >> >> Sorry to come into the discussion late--I saw the thread earlier. >> >

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
Joined the discussion a bit late, but just want to add in my +1 as well :-) Historically, when dynamic partition discovery was implemented in the earlier versions of the FlinkKafkaConsumer, it was implemented such that multiple source subtasks would in parallel query Kafka brokers for topic/partit

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
re: @Mason about TimestampOffsetInitializer default offset reset strategy Should the OffsetsInitializer even be respected for partitions from new discoveries? By "new discoveries", I mean partitions that are discovered outside of the initial metadata pull on job startup. I had the impression that

Re: [DISCUSS] Enabling dynamic partition discovery by default in Kafka source

2023-02-10 Thread Tzu-Li (Gordon) Tai
re: @Mason about a pluggable error handling strategy +1 to that, we've had users ask for that in the past for other connectors as well, e.g. Elasticsearch. As long as the exposure doesn't lend itself to potentially breaking processing semantics, I don't see why not! On Fri, Feb 10, 2023 at 1:54 P

[jira] [Created] (FLINK-31020) Provide read-only mode for flink web UI

2023-02-10 Thread Omkar Deshpande (Jira)
Omkar Deshpande created FLINK-31020: --- Summary: Provide read-only mode for flink web UI Key: FLINK-31020 URL: https://issues.apache.org/jira/browse/FLINK-31020 Project: Flink Issue Type: New

[jira] [Created] (FLINK-31021) JavaCodeSplitter doesn't split static method properly

2023-02-10 Thread Xingcan Cui (Jira)
Xingcan Cui created FLINK-31021: --- Summary: JavaCodeSplitter doesn't split static method properly Key: FLINK-31021 URL: https://issues.apache.org/jira/browse/FLINK-31021 Project: Flink Issue Typ