[jira] [Created] (FLINK-25069) YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP

2021-11-26 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25069: - Summary: YARNHighAvailabilityITCase.testJobRecoversAfterKillingTaskManager failed on AZP Key: FLINK-25069 URL: https://issues.apache.org/jira/browse/FLINK-25069 Pro

Re: [VOTE][FLIP-195] Improve the name and structure of vertex and operator name for job

2021-11-26 Thread wenlong.lwl
Hi everyone, I'm happy to announce that we have unanimously approved this FLIP. There are 12 approved votes, 6 of which are binding: Terry Wang (non-binding) Aitozi (non-binding) Martijn Visser (non-binding) Jing Zhang (binding) Jark Wu (binding) Lijie Wang (non-binding) Godfrey He (binding) Ser

[jira] [Created] (FLINK-25070) FLIP-195: Improve the name and structure of vertex and operator name for job

2021-11-26 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-25070: --- Summary: FLIP-195: Improve the name and structure of vertex and operator name for job Key: FLINK-25070 URL: https://issues.apache.org/jira/browse/FLINK-25070 Project: F

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Dawid Wysakowicz
Hi all, I updated the FLIP with a few clarifications: 1. I added a description how would we trigger a "full snapshot" in the changelog state backend * (We would go for this option in the 1st version). Trigger a snapshot of the base state backend in the 1st checkpoint, which

[jira] [Created] (FLINK-25071) Enable ParquetFileSystemITCase.testLimitableBulkFormat

2021-11-26 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-25071: Summary: Enable ParquetFileSystemITCase.testLimitableBulkFormat Key: FLINK-25071 URL: https://issues.apache.org/jira/browse/FLINK-25071 Project: Flink Issue

Re: [DISCUSS] Drop Zookeeper 3.4

2021-11-26 Thread Till Rohrmann
According to this SO question [1], it seems that Zk 3.5 clients cannot talk to 3.4 servers. I also tried it out with a local deployment and Flink was not able to start. Newer Zk versions can talk to older Zk servers if no new APIs are used [2]. [1] https://stackoverflow.com/a/61630617/4815083 [2]

Re: [DISCUSS] Deprecate Java 8 support

2021-11-26 Thread Nico Kruber
+1 for the deprecation but let me raise one more point that we have discovered in our training exercises: All our training jobs that aim for high throughput run slower on Java 11 than on Java 8! That may not be a general problem for the various applications in the wild, but it could be and this

[jira] [Created] (FLINK-25072) Introduce description for operator

2021-11-26 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-25072: --- Summary: Introduce description for operator Key: FLINK-25072 URL: https://issues.apache.org/jira/browse/FLINK-25072 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-25073) Introduce Tree Mode

2021-11-26 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-25073: --- Summary: Introduce Tree Mode Key: FLINK-25073 URL: https://issues.apache.org/jira/browse/FLINK-25073 Project: Flink Issue Type: Sub-task Reporter:

[jira] [Created] (FLINK-25074) Simplify name of window operators in DS by moving details to description

2021-11-26 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-25074: --- Summary: Simplify name of window operators in DS by moving details to description Key: FLINK-25074 URL: https://issues.apache.org/jira/browse/FLINK-25074 Project: Flink

[jira] [Created] (FLINK-25075) Remove reflection to instantiate PlannerExpressionParser

2021-11-26 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-25075: --- Summary: Remove reflection to instantiate PlannerExpressionParser Key: FLINK-25075 URL: https://issues.apache.org/jira/browse/FLINK-25075 Project: Flink

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Konstantin Knauf
Hi Dawid, sounds good, specifically 2., too. Best, Konstantin On Fri, Nov 26, 2021 at 9:25 AM Dawid Wysakowicz wrote: > Hi all, > > I updated the FLIP with a few clarifications: > >1. I added a description how would we trigger a "full snapshot" in the >changelog state backend >

[jira] [Created] (FLINK-25076) Simplify name of SQL operators

2021-11-26 Thread Wenlong Lyu (Jira)
Wenlong Lyu created FLINK-25076: --- Summary: Simplify name of SQL operators Key: FLINK-25076 URL: https://issues.apache.org/jira/browse/FLINK-25076 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-25077) Postgresql connector fails in case column with nested arrays

2021-11-26 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-25077: --- Summary: Postgresql connector fails in case column with nested arrays Key: FLINK-25077 URL: https://issues.apache.org/jira/browse/FLINK-25077 Project: Flink

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Roman Khachatryan
Hi, Thanks for updating the FLIP Dawid There seems to be a consensus in the discussion, however, I couldn't find stop-with-savepoint in the document. A few minor things: - maybe include "checkpoint" in mode names, i.e. --no-claim-checkpoint? - add an explicit option to preserve the current behav

[jira] [Created] (FLINK-25078) An error is reported when the flink SQL connects to Phoenix

2021-11-26 Thread xinli liang (Jira)
xinli liang created FLINK-25078: --- Summary: An error is reported when the flink SQL connects to Phoenix Key: FLINK-25078 URL: https://issues.apache.org/jira/browse/FLINK-25078 Project: Flink Is

[jira] [Created] (FLINK-25079) Add assertj assertions for table types

2021-11-26 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-25079: --- Summary: Add assertj assertions for table types Key: FLINK-25079 URL: https://issues.apache.org/jira/browse/FLINK-25079 Project: Flink Issue Ty

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Dawid Wysakowicz
- maybe include "checkpoint" in mode names, i.e. --no-claim-checkpoint? I don't think this is a good idea. The modes apply to both savepoints and checkpoints, plus it's much longer to type in. (minor) - add an explicit option to preserve the current behavior (no claim and no duplicate

Re: [DISCUSS] Releasing Flink 1.14.1

2021-11-26 Thread David Morávek
Hi Martijn, * https://issues.apache.org/jira/browse/FLINK-24543 - Zookeeper connection > issue causes inconsistent state in Flink -> I think this depends on the > outcome of dropping Zookeeper 3.4 as was proposed on the Dev mailing list > We already have an approved patch for master, I'll be prep

Re: [DISCUSS] Releasing Flink 1.14.1

2021-11-26 Thread Chesnay Schepler
FLINK-25022: I will open a PR later today, and it should be easy to backport. FLINK-25027: Unlikely to make it for 1.14.1; I also wouldn't consider it a blocker On 24/11/2021 19:40, Martijn Visser wrote: Hi all, I would like to start a discussion on releasing Flink 1.14.1. Flink 1.14 was rele

[jira] [Created] (FLINK-25080) FutureUtils is located in flink-core whereas the corresponding test is still located in flink-runtime

2021-11-26 Thread Matthias (Jira)
Matthias created FLINK-25080: Summary: FutureUtils is located in flink-core whereas the corresponding test is still located in flink-runtime Key: FLINK-25080 URL: https://issues.apache.org/jira/browse/FLINK-25080

Re: [DISCUSS] Creating an external connector repository

2021-11-26 Thread Till Rohrmann
Hi Arvid, Thanks for updating this thread with the latest findings. The described limitations for a single connector repo sound suboptimal to me. * Option 2. sounds as if we try to simulate multi connector repos inside of a single repo. I also don't know how we would share code between the differ

Re: [DISCUSS] Creating an external connector repository

2021-11-26 Thread Chesnay Schepler
For sharing workflows we should be able to use composite actions. We'd have the main definition files in the flink-connectors repo, that we also need to tag/release, which other branches/repos can then import. These are also versioned, so we don't have to worry about accidentally breaking stuff

[jira] [Created] (FLINK-25081) When chaining an operator of a side output stream, the num records sent displayed on the dashboard is incorrect

2021-11-26 Thread Lijie Wang (Jira)
Lijie Wang created FLINK-25081: -- Summary: When chaining an operator of a side output stream, the num records sent displayed on the dashboard is incorrect Key: FLINK-25081 URL: https://issues.apache.org/jira/browse/FL

Re: [DISCUSS] FLIP-189: SQL Client Usability Improvements

2021-11-26 Thread Daisy Tsang
Hi, would a command that shows the available table source/sink factories be implemented? Something that can show that a factory is loaded, etc. - Daisy On Thu, Nov 4, 2021 at 8:30 AM Sergey Nuyanzin wrote: > I've started a [VOTE] thread for this FLIP > https://lists.apache.org/thread/f14jjhrs

Re: [DISCUSS] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-11-26 Thread Till Rohrmann
Thanks for writing this FLIP Timo. I think this will be a very important improvement for Flink and our SQL user :-) Similar to Francesco I would like to understand the statement > For simplification of the design, we assume that upgrades use a step size of a single minor version. We don't guarant

Re: [DISCUSS] Deprecate Java 8 support

2021-11-26 Thread Becket Qin
Thanks for raising the discussion, Chesnay. I think it is OK to deprecate Java 8 to let the users know that Java 11 migration should be put into the schedule. However, According to some of the statistics of the Java version adoption[1], a large number of users are still using Java 8 in production.

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Till Rohrmann
Thanks for writing this FLIP Dawid. Just to clarify one thing for the support of forced full snapshots. If a state backend does not support this feature, then the user either has to copy the snapshot manually or resume using --claim mode, create a savepoint in canonical format and then change the s

Re: [DISCUSS] FLIP-193: Snapshots ownership

2021-11-26 Thread Dawid Wysakowicz
If they'd like to use the --no-claim mode that would be the way to go, yes. Two points to be on the same page here: * all Flink native state backends (RocksDB, HashMap, changelog) would already support --no-claim * if in the end we add the --legacy mode, users can also use that mode i

Re: [DISCUSS] FLIP-194: Introduce the JobResultStore

2021-11-26 Thread Till Rohrmann
Thanks for creating this FLIP Matthias, Mika and David. I think the JobResultStore is an important piece for fixing Flink's last high-availability problem (afaik). Once we have this piece in place, users no longer risk to re-execute a successfully completed job. I have one comment concerning brea

[jira] [Created] (FLINK-25082) RMQSource is not parallel

2021-11-26 Thread Mike Lynch (Jira)
Mike Lynch created FLINK-25082: -- Summary: RMQSource is not parallel Key: FLINK-25082 URL: https://issues.apache.org/jira/browse/FLINK-25082 Project: Flink Issue Type: Bug Components: C

[jira] [Created] (FLINK-25083) Flaky Test in RowTest

2021-11-26 Thread Minyang Ye (Jira)
Minyang Ye created FLINK-25083: -- Summary: Flaky Test in RowTest Key: FLINK-25083 URL: https://issues.apache.org/jira/browse/FLINK-25083 Project: Flink Issue Type: Improvement Component