Re:Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Godfrey Thanks for your reply and suggest! I adjusted the style of FLIP. There are still two questions that need to be discussed: 1. The new API we introduced to CTAS, in Table or Tableenvalonment? 2. The Drop Table Operation in JM is everyone's consensus, more details design at runtime need t

Re:Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Jark Thanks for your reply and suggest! >With this interface, we can easily support LEVEL-2 semantics by calling >`Catalog#dropTable` in the >`JobListener#onJobFailed`. We can also support LEVEL-3 by introducing >`StagingTableCatalog` like Spark, >calling `StagedTable#commitStagedChanges()` in

[jira] [Created] (FLINK-27752) Change 'path' to 'root-path' in table store

Jingsong Lee created FLINK-27752: Summary: Change 'path' to 'root-path' in table store Key: FLINK-27752 URL: https://issues.apache.org/jira/browse/FLINK-27752 Project: Flink Issue Type: Impro

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #1

Thanks Gyula for sharing the feedback. I have created two tickets[1][2] to fix the problems and will integrate them in the next release candidate. I am not closing this vote and waiting for other verification comments. [1]. https://issues.apache.org/jira/browse/FLINK-27746 [2]. https://issues.ap

[jira] [Created] (FLINK-27753) Scala suffix checks should check for maven errors

Chesnay Schepler created FLINK-27753: Summary: Scala suffix checks should check for maven errors Key: FLINK-27753 URL: https://issues.apache.org/jira/browse/FLINK-27753 Project: Flink Iss

Re: All flink-ci fail

See https://issues.apache.org/jira/browse/FLINK-27751 On 24/05/2022 08:12, rui fan wrote: Hi masters, All compile_ci of flink-ci[1] failed today, seemingly for the same reason: 05:03:16,982 ERROR org.apache.flink.tools.ci.suffixcheck.ScalaSuffixChecker [] - Violations found: Scala-free mod

[jira] [Created] (FLINK-27754) Getting Flink config fails if multiple files in /lib match flink-dist-*

Rudi Kershaw created FLINK-27754: Summary: Getting Flink config fails if multiple files in /lib match flink-dist-* Key: FLINK-27754 URL: https://issues.apache.org/jira/browse/FLINK-27754 Project: Flin

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.0.0, release candidate #1

Hi Yang, Thanks for the work! I successfully verified these items: 1. Verify that the checksums and GPG files are intact 2. Verify that the source distributions do not contain any binaries 3. Build the source distribution to ensure all source files 4. Validate the Maven artifacts do not contain any

[jira] [Created] (FLINK-27755) Introduce a filesystem catalog for table store

Jingsong Lee created FLINK-27755: Summary: Introduce a filesystem catalog for table store Key: FLINK-27755 URL: https://issues.apache.org/jira/browse/FLINK-27755 Project: Flink Issue Type: Ne

Re: All flink-ci fail

Hi Chesnay Schepler Thanks for your information. Best wishes fanrui On Tue, May 24, 2022 at 4:21 PM Chesnay Schepler wrote: > See https://issues.apache.org/jira/browse/FLINK-27751 > > On 24/05/2022 08:12, rui fan wrote: > > Hi masters, > > > > All compile_ci of flink-ci[1] failed today, seemin

[jira] [Created] (FLINK-27756) Fix Intermittingly failing test in `AsyncSinkWriterTest`

Ahmed Hamdy created FLINK-27756: --- Summary: Fix Intermittingly failing test in `AsyncSinkWriterTest` Key: FLINK-27756 URL: https://issues.apache.org/jira/browse/FLINK-27756 Project: Flink Issue

Re: Re: Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi Mang and Godfrey, I'm okay with `Table#saveAs(tablePath)`, but it would be better to consider overriding table options and create or replace table. So we may also need to provide methods for the returned object: - #option(key, value) - #create() - #createIfNotExist() - #createOrReplace() //

Re: Access to confluence

Hi Alexander, I have assigned the permission to you. Best, Jark On Mon, 23 May 2022 at 22:38, Александр Смирнов wrote: > Hi Jark and other PMC members! > > Сould you give me access to the editing of the confluence page with > FLIP-221 [1]? I would like to make small changes that were mentioned

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

Hi Lincoln, I have taken a look at FLIP-234, and I agree with you that the connectors can provide both async and sync runtime providers simultaneously instead of one of them. At that point, "lookup.async" looks redundant. If this option is planned to be removed in the long term, I think it makes s

[GitHub] [flink-connector-rabbitmq] MartijnVisser commented on pull request #1: [FLINK-20628] RabbitMQ Connector using FLIP-27 Source API

MartijnVisser commented on PR #1: URL: https://github.com/apache/flink-connector-rabbitmq/pull/1#issuecomment-1135867243 @pscls I think you've done a good job already with the root-pom; it looks like the one we currently have for Elasticsearch. I've just approved the run, so we can also se

Re: [DISCUSS] Docker images for SNAPSHOT

Hi everyone, Glad to inform you that the Github Actions workflow [1][2] that publishes SNAPSHOT Flink images to GHCR [3] is now up and running. Best, Alexander Fedulov [1] https://github.com/apache/flink-docker/pull/116/files [2] https://github.com/apache/flink-docker/actions/runs/2376596377 [3]

[jira] [Created] (FLINK-27757) Elasticsearch connector should not use flink-table-planner but flink-table-planner-loader

Martijn Visser created FLINK-27757: -- Summary: Elasticsearch connector should not use flink-table-planner but flink-table-planner-loader Key: FLINK-27757 URL: https://issues.apache.org/jira/browse/FLINK-27757

Re: About Native Deployment's Autoscaling implementation

Hi Gyula, This seems very promising for initial scaling. We are using Flink Kubernetes Operators. Most probably we are very early adapters for it :) Let me try it. Get back to you soon. My plan is building a general purpose CPU and backlog/throughput base autoscaling for Flink. I can create a Cus

[jira] [Created] (FLINK-27758) [JUnit5 Migration] Module: flink-table-runtime

Sergey Nuyanzin created FLINK-27758: --- Summary: [JUnit5 Migration] Module: flink-table-runtime Key: FLINK-27758 URL: https://issues.apache.org/jira/browse/FLINK-27758 Project: Flink Issue Ty

Re: About Native Deployment's Autoscaling implementation

Hi Talat! It would be great to have a HPA that works based on some flink throughput/backlog metrics. I wonder how you are going to access the Flink metrics in the HPA, we might need some integration with the k8s metrics system. In any case whether we need a FLIP or not depends on the complexity, i

[jira] [Created] (FLINK-27759) Rethink how to get the git commit id for docker image in Flink Kubernetes operator

Yang Wang created FLINK-27759: - Summary: Rethink how to get the git commit id for docker image in Flink Kubernetes operator Key: FLINK-27759 URL: https://issues.apache.org/jira/browse/FLINK-27759 Project:

Re:Re: [DISCUSS] FLIP-218: Support SELECT clause in CREATE TABLE(CTAS)

Hi, Martijn Thanks for your reply! I looked at the SQL standard, CTAS is part of the SQL standard. Feature T172 is "AS subquery clause in table definition". -- Best regards, Mang Zhang At 2022-05-04 21:49:00, "Martijn Visser" wrote: >Hi everyone, > >Can we identify if this proposed syntax

Re: About Native Deployment's Autoscaling implementation

Thanks for the interesting discussion. Compared with reactive mode, leveraging the flink-kubernetes-operator to do the job restarting/upgrading is another solution for auto-scaling. Given that fully restarting a Flink application on K8s is not too slow, this is a reasonable way. Really hope we cou

[jira] [Created] (FLINK-27760) NPE is thrown when executing PyFlink jobs in batch mode

Dian Fu created FLINK-27760: --- Summary: NPE is thrown when executing PyFlink jobs in batch mode Key: FLINK-27760 URL: https://issues.apache.org/jira/browse/FLINK-27760 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-27761) Add database and table field in canal-json sink

Zhiwen Sun created FLINK-27761: -- Summary: Add database and table field in canal-json sink Key: FLINK-27761 URL: https://issues.apache.org/jira/browse/FLINK-27761 Project: Flink Issue Type: Impro

Re: [DISCUSS] FLIP-235: Hybrid Shuffle Mode

Hi All, Thank you for your attention and feedback. Do you have any other comments? If there are no other questions, I'll vote on FLIP-235 tomorrow. Best regards, Weijie Aitozi 于2022年5月20日周五 13:22写道： > Hi Xintong > Thanks for your detailed explanation, I misunderstand the spill > behavior

Issue in seeing job manager and task manager logs in history sever

Hi Everyone, I am Sriram Ganesh part of the Microsoft Flink platform team. I recently went through the history server setup in Flink. I found an issue in the history server that was not showing the job manager and task manager logs. We are running clusters in session mode and using native Kuberne

[jira] [Created] (FLINK-27762) Kafka WakeUpException during handling splits changes

zoucan created FLINK-27762: -- Summary: Kafka WakeUpException during handling splits changes Key: FLINK-27762 URL: https://issues.apache.org/jira/browse/FLINK-27762 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-27763) Remove netty bundling&relocation in flink-streaming-kinesis-tests

Chesnay Schepler created FLINK-27763: Summary: Remove netty bundling&relocation in flink-streaming-kinesis-tests Key: FLINK-27763 URL: https://issues.apache.org/jira/browse/FLINK-27763 Project: Fl

[jira] [Created] (FLINK-27764) SetOperatorsITCase tests failed

Huang Xingbo created FLINK-27764: Summary: SetOperatorsITCase tests failed Key: FLINK-27764 URL: https://issues.apache.org/jira/browse/FLINK-27764 Project: Flink Issue Type: Bug Com

Re: [DISCUSS] FLIP-232: Add Retry Support For Async I/O In DataStream API

Hi everyone, Gen Luo, Yun Gao and I had a long offline discussion about the implementation of the recovery part. The key point was should we store the retry state and do the recovery after the job restart? We reached a consensus not to store the retry state for now, which is the clearest for user