Re: [VOTE] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-19 Thread Yun Gao
+1 (binding) Thanks Xuannan for driving this! Best, Yun -- From:David Morávek Send Time:2022 Jan. 20 (Thu.) 15:12 To:dev Subject:Re: [VOTE] FLIP-205: Support cache in DataStream for Batch Processing +1 (non-binding) D. On Thu

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Yun Gao
+1 Thanks Martijn for the efforts! Best, Yun -- From:Yun Tang Send Time:2022 Jan. 20 (Thu.) 15:22 To:dev Subject:Re: [VOTE] Creating external connector repositories +1 Thanks for driving this topic, Martijn Best, Yun Tang

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Yun Tang
+1 Thanks for driving this topic, Martijn Best, Yun Tang From: Jingsong Li Sent: Thursday, January 20, 2022 12:18 To: dev Subject: Re: [VOTE] Creating external connector repositories +1 Thanks for driving this Martijn! Best, Jingsong On Thu, Jan 20, 2022 at

Re: [VOTE] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-19 Thread David Morávek
+1 (non-binding) D. On Thu 20. 1. 2022 at 4:09, Xuannan Su wrote: > Hi devs, > > I would like to start the vote for FLIP-205 [1], which was discussed and > reached a consensus in the discussion thread [2]. > > The vote will be open for at least 72h, unless there is an objection or not > enough

Re: Re: [DISCUSS] Introduce Hash Lookup Join

2022-01-19 Thread Jingsong Li
Hi Jing, About `SHUFFLE_HASH(left_table, right_table)`, one case can be shared: SELECT * FROM left_t JOIN right_1 ON ... JOIN right_2 ON ... JOIN right_3 ON ... What if we want to use shuffle_hash for all three joints? SELECT /*+ SHUFFLE_HASH('left_t', 'right_1', 'right_2', 'right_3') */

Re: Re: [DISCUSS] Introduce Hash Lookup Join

2022-01-19 Thread Jingsong Li
Hi Jing, Sorry for the late reply! Is there a conclusion about naming here? (Maybe I missed something?) Use USE_HASH or some other names. Slightly confusing in the FLIP. And the problem of what to write inside the hint, as mentioned by lincoln. I think maybe we can list the grammars of other di

RE: Request for Flink 1.12.8 release

2022-01-19 Thread V N, Suchithra (Nokia - IN/Bangalore)
Hi Guowei, Flink 1.12.7 is already released. Request is for 1.12.8 release with log4j 2.17.1. You meant 1.12.8 here? Regards, Suchithra -Original Message- From: Guowei Ma Sent: Thursday, January 20, 2022 11:26 AM To: dev Subject: Re: Request for Flink 1.12.8 release Hi Suchithra I

Re: Request for Flink 1.12.8 release

2022-01-19 Thread Guowei Ma
Hi Suchithra I don't think there is a plan to release 1.12.7 for this. But I think you could build it from the source.[1] [1] https://github.com/apache/flink/tree/release-1.12#building-apache-flink-from-source Best, Guowei On Wed, Jan 19, 2022 at 7:11 PM V N, Suchithra (Nokia - IN/Bangalore) <

Re: Re: [DISCUSS] Introduce Hash Lookup Join

2022-01-19 Thread Lincoln Lee
Hi, Jing, Sorry for the late reply! The previous discussion for the hint syntax left a minor difference there: whether to use both sides of join table names or just one 'build' side table name only. I would prefer the later one. Users only need to pass the `build` side table(usually the smalle

[jira] [Created] (FLINK-25713) Support VectorNormalize in FlinkML

2022-01-19 Thread weibo zhao (Jira)
weibo zhao created FLINK-25713: -- Summary: Support VectorNormalize in FlinkML Key: FLINK-25713 URL: https://issues.apache.org/jira/browse/FLINK-25713 Project: Flink Issue Type: New Feature

[ANNOUNCE] Apache Flink 1.14.3 released

2022-01-19 Thread Thomas Weise
The Apache Flink community is very happy to announce the release of Apache Flink 1.14.3, which is the second bugfix release for the Apache Flink 1.14 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Jingsong Li
+1 Thanks for driving this Martijn! Best, Jingsong On Thu, Jan 20, 2022 at 12:17 PM Qingsheng Ren wrote: > > +1 (non-binding) > > Thanks for driving this Martijn! > > Best, > > Qingsheng > > > On Jan 20, 2022, at 8:55 AM, heng du wrote: > > > > +1(non-binding) > > > > Thomas Weise 于2022年1月20日

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Qingsheng Ren
+1 (non-binding) Thanks for driving this Martijn! Best, Qingsheng > On Jan 20, 2022, at 8:55 AM, heng du wrote: > > +1(non-binding) > > Thomas Weise 于2022年1月20日周四 05:02写道: > >> +1 (binding) >> >> >> On Wed, Jan 19, 2022 at 7:57 AM Till Rohrmann >> wrote: >> >>> +1 >>> >>> Cheers, >>>

Re: [VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Qingsheng Ren
+1 (non-binding) Thanks Martijn for driving this! Best, Qingsheng Ren > On Jan 19, 2022, at 9:47 PM, Martijn Visser wrote: > > Hi everyone, > > I'd like to start a vote on moving connectors from Flink to external > connector repositories [1]. > > The vote will be open for at least 72 hours

[VOTE] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-19 Thread Xuannan Su
Hi devs, I would like to start the vote for FLIP-205 [1], which was discussed and reached a consensus in the discussion thread [2]. The vote will be open for at least 72h, unless there is an objection or not enough votes. Best, Xuannan [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP

Re: [VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Jingsong Li
+1 Thanks Martijn for driving Best, Jingsong On Thu, Jan 20, 2022 at 5:06 AM Thomas Weise wrote: > > +1 (binding) > > I think it would be a good idea to test the new infrastructure with a > sample connector for the next release before moving existing connectors. > > > On Wed, Jan 19, 2022 at 7

Re: [DISCUSS] FLIP-208: Update KafkaSource to detect EOF based on de-serialized record

2022-01-19 Thread Dong Lin
Hi everyone, Thanks for all the comments. If there are no further comments, we plan to start the voting thread on 1/24. Cheers, Dong On Fri, Jan 14, 2022 at 8:26 AM Dong Lin wrote: > Hi Qingsheng, > > Thank you for extending this FLIP to support this feature with Table/SQL. > I also prefer the

Re: [DISCUSS] Merge flink-connector-testing into flink-connector-test-utils

2022-01-19 Thread Qingsheng Ren
Thank you all for the feedback! I created FLINK-25712 [1] for merging these two modules. [1] https://issues.apache.org/jira/browse/FLINK-25712 Best, Qingsheng > On Jan 19, 2022, at 4:14 PM, Martijn Visser wrote: > > Thanks for bringing this up, +1 to merge the modules to simplify the > la

[jira] [Created] (FLINK-25712) Merge flink-connector-testing into flink-connector-test-utils

2022-01-19 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-25712: - Summary: Merge flink-connector-testing into flink-connector-test-utils Key: FLINK-25712 URL: https://issues.apache.org/jira/browse/FLINK-25712 Project: Flink

[VOTE] FLIP-204: Introduce Hash Lookup Join

2022-01-19 Thread Jing Zhang
Hi community, I'd like to start a vote on FLIP-204: Introduce Hash Lookup Join [1] which has been discussed in the thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-204%3A+Introduce

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread heng du
+1(non-binding) Thomas Weise 于2022年1月20日周四 05:02写道: > +1 (binding) > > > On Wed, Jan 19, 2022 at 7:57 AM Till Rohrmann > wrote: > > > +1 > > > > Cheers, > > Till > > > > On Wed, Jan 19, 2022 at 4:01 PM Timo Walther wrote: > > > > > +1 (binding) > > > > > > Regards, > > > Timo > > > > > > On 19

[jira] [Created] (FLINK-25711) OOM / buffer-overflow on KafkaIO SpecificRecord

2022-01-19 Thread Jordan Moore (Jira)
Jordan Moore created FLINK-25711: Summary: OOM / buffer-overflow on KafkaIO SpecificRecord Key: FLINK-25711 URL: https://issues.apache.org/jira/browse/FLINK-25711 Project: Flink Issue Type: B

Re: [VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Thomas Weise
+1 (binding) I think it would be a good idea to test the new infrastructure with a sample connector for the next release before moving existing connectors. On Wed, Jan 19, 2022 at 7:56 AM Till Rohrmann wrote: > +1 (binding) > > Cheers, > Till > > On Wed, Jan 19, 2022 at 3:58 PM Timo Walther w

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Thomas Weise
+1 (binding) On Wed, Jan 19, 2022 at 7:57 AM Till Rohrmann wrote: > +1 > > Cheers, > Till > > On Wed, Jan 19, 2022 at 4:01 PM Timo Walther wrote: > > > +1 (binding) > > > > Regards, > > Timo > > > > On 19.01.22 14:47, Martijn Visser wrote: > > > Hi everyone, > > > > > > I'd like to start a vot

[jira] [Created] (FLINK-25710) Multiple Kafka IT cases fail with "ContainerLaunch Container startup failed"

2022-01-19 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-25710: - Summary: Multiple Kafka IT cases fail with "ContainerLaunch Container startup failed" Key: FLINK-25710 URL: https://issues.apache.org/jira/browse/FLINK-25710

[jira] [Created] (FLINK-25709) GlueSchemaRegistryAvroKinesisITCase fails on AZP

2022-01-19 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25709: - Summary: GlueSchemaRegistryAvroKinesisITCase fails on AZP Key: FLINK-25709 URL: https://issues.apache.org/jira/browse/FLINK-25709 Project: Flink Issue Type

[jira] [Created] (FLINK-25708) Upgrade Flink dependency to 1.14.3

2022-01-19 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25708: - Summary: Upgrade Flink dependency to 1.14.3 Key: FLINK-25708 URL: https://issues.apache.org/jira/browse/FLINK-25708 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-25707) AdaptiveScheduler never transitions executions into SCHEDULED state

2022-01-19 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25707: Summary: AdaptiveScheduler never transitions executions into SCHEDULED state Key: FLINK-25707 URL: https://issues.apache.org/jira/browse/FLINK-25707 Project:

Re: [VOTE] FLIP-201: Persist local state in working directory

2022-01-19 Thread Matthias Pohl
+1 (binding) Best, Matthias On Mon, Jan 10, 2022 at 2:53 PM Till Rohrmann wrote: > +1 (binding) > > Cheers, > Till > > On Mon, Jan 10, 2022 at 2:30 PM Etienne Chauchot > wrote: > > > +1 > > > > Best > > > > Etienne Chauchot > > > > Le 10/01/2022 à 10:22, Till Rohrmann a écrit : > > > Hi everyo

Re: [VOTE] Release flink-shaded 15.0, release candidate #2

2022-01-19 Thread Till Rohrmann
+1 (binding) * Checked checksums and signatures * Looked at the diff between this RC and 14.0 and everything looked good Cheers, Till On Wed, Jan 19, 2022 at 2:30 PM Piotr Nowojski wrote: > +1 (binding) > > I checked the diff between 14.0..15.0-rc2, and 15.0-rc1..15.0-rc2, and > didn't find a

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Till Rohrmann
+1 Cheers, Till On Wed, Jan 19, 2022 at 4:01 PM Timo Walther wrote: > +1 (binding) > > Regards, > Timo > > On 19.01.22 14:47, Martijn Visser wrote: > > Hi everyone, > > > > I'd like to start a vote on creating external repositories [1]. Since the > > thread is fairly long, I've also previously

Re: [VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Till Rohrmann
+1 (binding) Cheers, Till On Wed, Jan 19, 2022 at 3:58 PM Timo Walther wrote: > +1 (binding) > > I think this will speed up connector evolution and core development > experience. > > Regards, > Timo > > On 19.01.22 14:47, Martijn Visser wrote: > > Hi everyone, > > > > I'd like to start a vote o

Re: [VOTE] Creating external connector repositories

2022-01-19 Thread Timo Walther
+1 (binding) Regards, Timo On 19.01.22 14:47, Martijn Visser wrote: Hi everyone, I'd like to start a vote on creating external repositories [1]. Since the thread is fairly long, I've also previously summarised it [2]. The vote will be open for at least 72 hours unless there is an objection or

Re: [VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Timo Walther
+1 (binding) I think this will speed up connector evolution and core development experience. Regards, Timo On 19.01.22 14:47, Martijn Visser wrote: Hi everyone, I'd like to start a vote on moving connectors from Flink to external connector repositories [1]. The vote will be open for at lea

[jira] [Created] (FLINK-25706) Add idleness to Kafka Source documentation

2022-01-19 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-25706: -- Summary: Add idleness to Kafka Source documentation Key: FLINK-25706 URL: https://issues.apache.org/jira/browse/FLINK-25706 Project: Flink Issue Type: Im

[jira] [Created] (FLINK-25705) Translate "Metric Reporters" page of "Deployment" in to Chinese

2022-01-19 Thread ChengKai Yang (Jira)
ChengKai Yang created FLINK-25705: - Summary: Translate "Metric Reporters" page of "Deployment" in to Chinese Key: FLINK-25705 URL: https://issues.apache.org/jira/browse/FLINK-25705 Project: Flink

[VOTE] Moving connectors from Flink to external connector repositories

2022-01-19 Thread Martijn Visser
Hi everyone, I'd like to start a vote on moving connectors from Flink to external connector repositories [1]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. If the vote passes, I'll document the outcome on a new page under https://cwiki.apache.org/c

[VOTE] Creating external connector repositories

2022-01-19 Thread Martijn Visser
Hi everyone, I'd like to start a vote on creating external repositories [1]. Since the thread is fairly long, I've also previously summarised it [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. If the vote passes, I'll document the outcome on a n

Re: [VOTE] Release flink-shaded 15.0, release candidate #2

2022-01-19 Thread Piotr Nowojski
+1 (binding) I checked the diff between 14.0..15.0-rc2, and 15.0-rc1..15.0-rc2, and didn't find anything suspicious. Best, Piotrek wt., 18 sty 2022 o 14:37 Chesnay Schepler napisał(a): > Hi everyone, > Please review and vote on the release candidate #2 for the version 15.0, > as follows: > [

[jira] [Created] (FLINK-25704) Performance regression on 18.01.2022 in batch network benchmarks

2022-01-19 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-25704: -- Summary: Performance regression on 18.01.2022 in batch network benchmarks Key: FLINK-25704 URL: https://issues.apache.org/jira/browse/FLINK-25704 Project: Flink

[jira] [Created] (FLINK-25703) In Batch, I think .stagingPath should be created when the task is running to prevent permission issues caused by inconsistency of hadoop usernames

2022-01-19 Thread shizhengchao (Jira)
shizhengchao created FLINK-25703: Summary: In Batch, I think .stagingPath should be created when the task is running to prevent permission issues caused by inconsistency of hadoop usernames Key: FLINK-25703 URL:

[jira] [Created] (FLINK-25702) Remove the usage of Configurable

2022-01-19 Thread Jing Ge (Jira)
Jing Ge created FLINK-25702: --- Summary: Remove the usage of Configurable Key: FLINK-25702 URL: https://issues.apache.org/jira/browse/FLINK-25702 Project: Flink Issue Type: Improvement Re

[jira] [Created] (FLINK-25701) Add API annotation to some Kafka connector core classes and interface

2022-01-19 Thread Jing Ge (Jira)
Jing Ge created FLINK-25701: --- Summary: Add API annotation to some Kafka connector core classes and interface Key: FLINK-25701 URL: https://issues.apache.org/jira/browse/FLINK-25701 Project: Flink

Request for Flink 1.12.8 release

2022-01-19 Thread V N, Suchithra (Nokia - IN/Bangalore)
Hello, We are using Apache Flink 1.12 version. Due to log4j security vulnerabilities(CVE-2021-44228) we have upgraded to Flink 1.12.7 which contains the fix for CVE-2021-44228(Critical) and CVE-2021-45046(Critical). Later two more vulnerabilities are reported CVE-2021-45105(Moderate) and CVE-2

Re: How to use TableEnvironment in multiple thread environments

2022-01-19 Thread Till Rohrmann
Hi Chang, I think the TableEnvironment is not designed to be used by multiple threads. If you want to share state between the different instances, then you have to either duplicate this information or put a synchronization mechanism into place (e.g. by creating a thread safe CatalogManager or sync

[jira] [Created] (FLINK-25700) flinksql Cascading Window TVF Aggregation exception

2022-01-19 Thread simenliuxing (Jira)
simenliuxing created FLINK-25700: Summary: flinksql Cascading Window TVF Aggregation exception Key: FLINK-25700 URL: https://issues.apache.org/jira/browse/FLINK-25700 Project: Flink Issue Ty

[jira] [Created] (FLINK-25699) Use HashMap for MAP value constructors

2022-01-19 Thread Timo Walther (Jira)
Timo Walther created FLINK-25699: Summary: Use HashMap for MAP value constructors Key: FLINK-25699 URL: https://issues.apache.org/jira/browse/FLINK-25699 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-25698) Elasticsearch7DynamicSinkITCase.testWritingDocuments fails on AZP

2022-01-19 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-25698: - Summary: Elasticsearch7DynamicSinkITCase.testWritingDocuments fails on AZP Key: FLINK-25698 URL: https://issues.apache.org/jira/browse/FLINK-25698 Project: Flink

Re: [DISCUSS] FLIP-203: Incremental savepoints

2022-01-19 Thread David Morávek
> > For some users canonical savepoints are prohibitively expensive either to > take or to recover from. To a point where the system is unable to complete > them before some timeout/failure happens. I'd say the same logic applies for re-scaling the job from the RDB based checkpoint / savepoint. I

Re: Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing

2022-01-19 Thread Yun Gao
Hi, Thanks Xuannan for the clarification, I also have no other issues~ Best, Yun --Original Mail -- Sender:Xuannan Su Send Date:Wed Jan 19 11:35:13 2022 Recipients:Flink Dev Subject:Re: [DISCUSS] FLIP-205: Support cache in DataStream for Batch Processing Hi

Re: [DISCUSS] Merge flink-connector-testing into flink-connector-test-utils

2022-01-19 Thread Martijn Visser
Thanks for bringing this up, +1 to merge the modules to simplify the landscape for the users. Best regards, Martijn On Wed, 19 Jan 2022 at 04:30, Jingsong Li wrote: > +1 to merge the two modules. > > Merging them I don't see any problem, not merging them I feel will > only increase the complexi

Re: [DISCUSS] FLIP-203: Incremental savepoints

2022-01-19 Thread Piotr Nowojski
Hi David, I didn't mean "best effort". It's just that we would be relaying on a 3rd party system, which quoting [1]: > RocksDB goes to great lengths to ensure data remains both forward- and backward-compatible But it's still a 3rd party system that we do not control. It's the same with KafkaClie