[jira] [Created] (FLINK-19534) build_wheels: linux failed with "ValueError: Incompatible component merge"

2020-10-08 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-19534: -- Summary: build_wheels: linux failed with "ValueError: Incompatible component merge" Key: FLINK-19534 URL: https://issues.apache.org/jira/browse/FLINK-19534 Projec

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-08 Thread Till Rohrmann
This sounds good. Maybe there are others in the community who can help with the review before the Jark and Leonard are back. Cheers, Till On Wed, Oct 7, 2020 at 7:33 PM Dylan Forciea wrote: > Actually…. It looks like what I did covers both cases. I’ll see about > getting some unit tests and doc

[jira] [Created] (FLINK-19535) SourceCoordinator should avoid fail job multiple times.

2020-10-08 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-19535: Summary: SourceCoordinator should avoid fail job multiple times. Key: FLINK-19535 URL: https://issues.apache.org/jira/browse/FLINK-19535 Project: Flink Issue

flink 连hbase 报错 java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance

2020-10-08 Thread 张立志
java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance(IFLjava/util/List;Ljava/util/List;Ljava/util/List;Lorg/apache/hadoop/yarn/api/records/ResourceBlacklistRequest;)Lorg/apache/hadoop/yarn/api/protocolrecords/AllocateRequest; at org.apache.hadoo

Wrapping a Flink Function

2020-10-08 Thread Lorenzo Pirazzini
Hello, I'm having trouble finding a way to add logic to an existing SinkFunction. What I would like to do is wrap an input SinkFunction inside another one that will perform its logic and then perform some additional logic, e.g.: SinkFunction wrapFunction(SinkFunction function) { return new Si

[jira] [Created] (FLINK-19536) Fix inconsistent priority notification

2020-10-08 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-19536: --- Summary: Fix inconsistent priority notification Key: FLINK-19536 URL: https://issues.apache.org/jira/browse/FLINK-19536 Project: Flink Issue Type: Bug Affe

[jira] [Created] (FLINK-19537) Processed in-flight bytes metric is not accurate

2020-10-08 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-19537: -- Summary: Processed in-flight bytes metric is not accurate Key: FLINK-19537 URL: https://issues.apache.org/jira/browse/FLINK-19537 Project: Flink Issue Ty

[jira] [Created] (FLINK-19538) Support more complex routers in remote ingress definitions

2020-10-08 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-19538: Summary: Support more complex routers in remote ingress definitions Key: FLINK-19538 URL: https://issues.apache.org/jira/browse/FLINK-19538 Project: Flink Is

[jira] [Created] (FLINK-19539) JMXReporterFactoryTest fails on Azure

2020-10-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-19539: - Summary: JMXReporterFactoryTest fails on Azure Key: FLINK-19539 URL: https://issues.apache.org/jira/browse/FLINK-19539 Project: Flink Issue Type: B

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-08 Thread Dylan Forciea
I’ve updated the unit tests and documentation, and I was running the azure test pipeline as described in the instructions. However, it appears that what seems to be an unrelated test for the JMX code failed. Is this a matter of me not setting things up correctly? I wanted to ensure everything lo

[jira] [Created] (FLINK-19540) How to reduce or disable checkpoint logs in Flink

2020-10-08 Thread Xinhao Li (Jira)
Xinhao Li created FLINK-19540: - Summary: How to reduce or disable checkpoint logs in Flink Key: FLINK-19540 URL: https://issues.apache.org/jira/browse/FLINK-19540 Project: Flink Issue Type: Impro

Re: Wrapping a Flink Function

2020-10-08 Thread Teng Fei Liao
I think any solution here is inherently fragile since future versions of flink can have different abstract classes or interfaces you won't know it has to support. But for a given release, something you can consider is a wrapper class that extends/implements the ones you support. Then, during the me

[jira] [Created] (FLINK-19541) Improve hiveConfDir for HiveCatalog

2020-10-08 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-19541: Summary: Improve hiveConfDir for HiveCatalog Key: FLINK-19541 URL: https://issues.apache.org/jira/browse/FLINK-19541 Project: Flink Issue Type: Task

Re: flink 连hbase 报错 java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest.newInstance

2020-10-08 Thread Jark Wu
Hi, This exception is because you don't have proper HBase dependency in your cluster. Please refer to the HBase documentation about how to add HBase dependency jars to HADOOP_CLASSPATH. Flink will load all jars under HADOOP_CLASSPATH automatically. Btw, dev mailing list is used for discussing dev

[RESULT][VOTE] FLIP-144: Native Kubernetes HA for Flink

2020-10-08 Thread Yang Wang
Hi devs, I'm happy to announce that FLIP-144[1] is officially approved. The vote has been opened for more than 72h + weekends, and we have received 4 votes, 3 of which are binding, and no vetos. Thanks everyone for participating. * Till (binding) * tison (binding) * Xintong (binding) * Yang Wang

[jira] [Created] (FLINK-19542) Implement LeaderElectionService and LeaderRetrievalService based on Kubernetes API

2020-10-08 Thread Yang Wang (Jira)
Yang Wang created FLINK-19542: - Summary: Implement LeaderElectionService and LeaderRetrievalService based on Kubernetes API Key: FLINK-19542 URL: https://issues.apache.org/jira/browse/FLINK-19542 Project:

[jira] [Created] (FLINK-19543) Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API

2020-10-08 Thread Yang Wang (Jira)
Yang Wang created FLINK-19543: - Summary: Implement RunningJobsRegistry, JobGraphStore based on Kubernetes API Key: FLINK-19543 URL: https://issues.apache.org/jira/browse/FLINK-19543 Project: Flink

[jira] [Created] (FLINK-19544) Implement CheckpointRecoveryFactory based on Kubernetes API

2020-10-08 Thread Yang Wang (Jira)
Yang Wang created FLINK-19544: - Summary: Implement CheckpointRecoveryFactory based on Kubernetes API Key: FLINK-19544 URL: https://issues.apache.org/jira/browse/FLINK-19544 Project: Flink Issue

[jira] [Created] (FLINK-19545) Add e2e test for native Kubernetes HA

2020-10-08 Thread Yang Wang (Jira)
Yang Wang created FLINK-19545: - Summary: Add e2e test for native Kubernetes HA Key: FLINK-19545 URL: https://issues.apache.org/jira/browse/FLINK-19545 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-19546) Add document for native Kubernetes HA

2020-10-08 Thread Yang Wang (Jira)
Yang Wang created FLINK-19546: - Summary: Add document for native Kubernetes HA Key: FLINK-19546 URL: https://issues.apache.org/jira/browse/FLINK-19546 Project: Flink Issue Type: Sub-task

Re: autoCommit for postgres jdbc streaming in Table/SQL API

2020-10-08 Thread Jark Wu
Hi Dylan, Sorry for the late reply. We've just come back from a long holiday. Thanks for reporting this problem. First, I think this is a bug that `autoCommit` is false by default (JdbcRowDataInputFormat.Builder). We can fix the default to true in 1.11 series, and I think this can solve your prob

[jira] [Created] (FLINK-19547) Partial record clean Up after receiver task fails and restart

2020-10-08 Thread Yuan Mei (Jira)
Yuan Mei created FLINK-19547: Summary: Partial record clean Up after receiver task fails and restart Key: FLINK-19547 URL: https://issues.apache.org/jira/browse/FLINK-19547 Project: Flink Issue

[jira] [Created] (FLINK-19548) Report correct progress to yarn resource manager

2020-10-08 Thread fa zheng (Jira)
fa zheng created FLINK-19548: Summary: Report correct progress to yarn resource manager Key: FLINK-19548 URL: https://issues.apache.org/jira/browse/FLINK-19548 Project: Flink Issue Type: Improvem

[jira] [Created] (FLINK-19549) [CI] ##[error]The job running on agent Azure Pipelines 5 ran longer than the maximum time of 260 minutes.

2020-10-08 Thread Di Xu (Jira)
Di Xu created FLINK-19549: - Summary: [CI] ##[error]The job running on agent Azure Pipelines 5 ran longer than the maximum time of 260 minutes. Key: FLINK-19549 URL: https://issues.apache.org/jira/browse/FLINK-19549

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-08 Thread Benchao Li
Congratulations! Zhu Zhu 于2020年10月7日周三 下午8:05写道: > Thank you all! It's my great pleasure to work with you to > continuously improve Flink and grow the community! > > Thanks, > Zhu > > Matthias Pohl 于2020年10月6日周二 下午9:39写道: > > > Congratulations! :) > > > > Best, > > Matthias > > > > On Tue, Oct

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-08 Thread Yangze Guo
Congratulations! Zhu Zhu. Best, Yangze Guo On Fri, Oct 9, 2020 at 12:58 PM Benchao Li wrote: > > Congratulations! > > Zhu Zhu 于2020年10月7日周三 下午8:05写道: > > > Thank you all! It's my great pleasure to work with you to > > continuously improve Flink and grow the community! > > > > Thanks, > > Zhu >

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-08 Thread Danny Chan
Congrats, Zhu Zhu ~ Best, Danny Chan 在 2020年10月9日 +0800 PM1:05,dev@flink.apache.org,写道: > > Congrats, Zhu Zhu

[jira] [Created] (FLINK-19550) StreamingFileSink can't be interrupted when writing to S3

2020-10-08 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-19550: - Summary: StreamingFileSink can't be interrupted when writing to S3 Key: FLINK-19550 URL: https://issues.apache.org/jira/browse/FLINK-19550 Project: Flink

[DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-08 Thread Yun Gao
Hi, devs & users As discussed in FLIP-131 [1], Flink will make DataStream the unified API for processing bounded and unbounded data in both streaming and blocking modes. However, one long-standing problem for the streaming mode is that currently Flink does not support checkpoints after some tas

Re: [ANNOUNCE] New PMC member: Zhu Zhu

2020-10-08 Thread Yun Tang
Congratulations, Zhu! Best Yun Tang From: Danny Chan Sent: Friday, October 9, 2020 13:51 To: dev@flink.apache.org Subject: Re: [ANNOUNCE] New PMC member: Zhu Zhu Congrats, Zhu Zhu ~ Best, Danny Chan 在 2020年10月9日 +0800 PM1:05,dev@flink.apache.org,写道: > > Congrat

Re: [DISCUSS] FLIP-147: Support Checkpoints After Tasks Finished

2020-10-08 Thread Yun Gao
Hi, devs & users Very sorry for the spoiled formats, I resent the discussion as follows. As discussed in FLIP-131[1], Flink will make DataStream the unified API for processing bounded and unbounded data in both streaming and blocking modes. However, one long-standing problem for the streaming m

[jira] [Created] (FLINK-19551) Follow up improvements for shuffle service

2020-10-08 Thread Zhijiang (Jira)
Zhijiang created FLINK-19551: Summary: Follow up improvements for shuffle service Key: FLINK-19551 URL: https://issues.apache.org/jira/browse/FLINK-19551 Project: Flink Issue Type: Improvement