Re: [DISCUSS] FLIP-245: Source Supports Speculative Execution For Batch Job

[Guowei Ma]

Hi, Jing

Thanks a lot for writing this FLIP, which is very useful to Batch users.
Currently  I have only two small questions:

1. First of all, please complete the fault-tolerant processing flow in the
FLIP. (Maybe you've already considered it, but it's better to explicitly
give the specific solution in the FLIP.)
For example, how to handle Source `Reader` in case of error. As far as I
know, once the reader is unavailable, it will result in the inability to
allocate a new split, which may be unacceptable in the case of speculative
execution.

2. Secondly the FLIP only says that user-defined events are not supported,
but it does not explain how to deal with the existing
ReportedWatermarkEvent/ReaderRegistrationEvent. After all, in the case of
speculative execution, there may be two "same" tasks being executed at the
same time. If these events are repeated, whether they really have no effect
on the execution of the job, there is still a clear evaluation.

Best,
Guowei


On Fri, Jun 24, 2022 at 5:41 PM Jing Zhang  wrote:

> Hi all,
> One major problem of Flink batch jobs is slow tasks running on hot/bad
> nodes, resulting in very long execution time.
>
> In order to solve this problem, FLIP-168: Speculative Execution for Batch
> Job[1] is introduced and approved recently.
>
> Here, Zhu Zhu and I propose to support speculative execution of sources as
> one of follow up of FLIP-168. You could find more details in FLIP-245[2].
> Looking forward to your feedback.
>
> Best,
> Jing Zhang
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-168%3A+Speculative+Execution+for+Batch+Job#FLIP168:SpeculativeExecutionforBatchJob-NointegrationwithFlink'swebUI
>
> [2]
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-245%3A+Source+Supports+Speculative+Execution+For+Batch+Job
>

Re: [VOTE] Apache Flink ML Release 2.1.0, release candidate #1

[Dong Lin]

Thanks for creating the release!

+1 (non-binding)

Here is what I checked. All required checks are included.

- Verified that the checksums and GPG files match the corresponding release
files.
- Verified that the source distributions do not contain any binaries.
- Built the source distribution and ensured that all source files have
Apache headers.
- Verified that all POM files point to the same version.
- Browsed through JIRA release notes files and did not find anything
unexpected.
- Browsed through README.md files and did not find anything unexpected.
- Checked the source code tag "release-2.0.0-rc1" and did not find anything
unexpected.



On Tue, Jun 28, 2022 at 5:28 PM Zhipeng Zhang 
wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.1.0 of
> Apache Flink ML as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> **Testing Guideline**
>
> You can find here [1] a page in the project wiki on instructions for
> testing.
> To cast a vote, it is not necessary to perform all listed checks, but
> please
> mention which checks you have performed when voting.
>
> **Release Overview**
>
> As an overview, the release consists of the following:
> a) Flink ML source release to be deployed to dist.apache.org
> b) Flink ML Python source distributions to be deployed to PyPI
> c) Maven artifacts to be deployed to the Maven Central Repository
>
> **Staging Areas to Review**
>
> The staging areas containing the above mentioned artifacts are as follows,
> for your review:
> * All artifacts for a) and b) can be found in the corresponding dev
> repository at dist.apache.org [2],
> which are signed with the key with fingerprint
> 0789F389E67ADDFA034E603FABF0C46E59C8941C [3]
> * All artifacts for c) can be found at the Apache Nexus Repository [4]
>
> Other links for your review:
> * JIRA release notes [5]
> * Source code tag "release-2.0.0-rc1" [6]
> * PR to update the website Downloads page to include Flink ML links [7]
>
> **Vote Duration**
> The voting time will run for at least 72 hours.
> It is adopted by majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Yun and Zhipeng
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+ML+Release
> [2] https://dist.apache.org/repos/dist/dev/flink/flink-ml-2.1.0-rc1/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1516/
> [5]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12351141
> [6] https://github.com/apache/flink-ml/releases/tag/release-2.1.0-rc1
> [7] https://github.com/apache/flink-web/pull/556
>

[VOTE] FLIP-242: Introduce configurable RateLimitingStrategy for Async Sink

[Teoh, Hong]

Hi everyone,

Thanks for all the feedback so far. Based on the discussion [1], we seem to 
have consensus. So, I would like to start a vote on FLIP-242 [2].

The vote will last for at least 72 hours unless there is an objection or 
insufficient votes.

[1] https://lists.apache.org/thread/k5s970xlqoj7opx1pzbpylxcov0tp842
[2] 
https://cwiki.apache.org/confluence/display/FLINK/FLIP-242%3A+Introduce+configurable+RateLimitingStrategy+for+Async+Sink

Regards,
Hong

[jira] [Created] (FLINK-28303) Kafka SQL Connector loses data when restoring from a savepoint with a topic with empty partitions

[Robert Metzger (Jira)]

Robert Metzger created FLINK-28303:
--

 Summary: Kafka SQL Connector loses data when restoring from a 
savepoint with a topic with empty partitions
 Key: FLINK-28303
 URL: https://issues.apache.org/jira/browse/FLINK-28303
 Project: Flink
  Issue Type: Bug
  Components: Connectors / Kafka
Affects Versions: 1.14.4
Reporter: Robert Metzger


Steps to reproduce:
- Set up a Kafka topic with 10 partitions
- produce records 0-9 into the topic
- take a savepoint and stop the job
- produce records 10-19 into the topic
- restore the job from the savepoint.

The job will be missing usually 2-4 records from 10-19.

My assumption is that if a partition never had data (which is likely with 10 
partitions and 10 records), the savepoint will only contain offsets for 
partitions with data. 
While the job was offline (and we've written record 10-19 into the topic), all 
partitions got filled. Now, when Kafka comes online again, it will use the 
"latest" offset for those partitions, skipping some data.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] FLIP-242: Introduce configurable RateLimitingStrategy for Async Sink

[Piotr Nowojski]

Thanks for starting the voting thread.  +1 (binding) from my side.

Best,
Piotrek



śr., 29 cze 2022 o 17:32 Teoh, Hong 
napisał(a):

> Hi everyone,
>
> Thanks for all the feedback so far. Based on the discussion [1], we seem
> to have consensus. So, I would like to start a vote on FLIP-242 [2].
>
> The vote will last for at least 72 hours unless there is an objection or
> insufficient votes.
>
> [1] https://lists.apache.org/thread/k5s970xlqoj7opx1pzbpylxcov0tp842
> [2]
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-242%3A+Introduce+configurable+RateLimitingStrategy+for+Async+Sink
>
> Regards,
> Hong
>

[jira] [Created] (FLINK-28304) Rework Kubernetes tests to no longer use driver version "none"

[Martijn Visser (Jira)]

Martijn Visser created FLINK-28304:
--

 Summary: Rework Kubernetes tests to no longer use driver version 
"none"
 Key: FLINK-28304
 URL: https://issues.apache.org/jira/browse/FLINK-28304
 Project: Flink
  Issue Type: Technical Debt
  Components: Deployment / Kubernetes, Tests
Affects Versions: 1.16.0, 1.15.2, 1.14.6
Reporter: Martijn Visser


As a follow-up of FLINK-28269 we should reconsider moving away from the current 
implementation which uses driver version {{none}} to the default driver 
{{docker}}. That's currently also done for flink-kubernetes-operator. However, 
when trying that for Flink, the tests timed out. 

Even better would be to get completely rid of the Bash based Kubernetes tests, 
see FLINK-28277. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] FLIP-242: Introduce configurable RateLimitingStrategy for Async Sink

[Martijn Visser]

This is looking good! Thanks for the efforts. +1 (binding)

Op wo 29 jun. 2022 om 20:55 schreef Piotr Nowojski 

> Thanks for starting the voting thread.  +1 (binding) from my side.
>
> Best,
> Piotrek
>
>
>
> śr., 29 cze 2022 o 17:32 Teoh, Hong 
> napisał(a):
>
> > Hi everyone,
> >
> > Thanks for all the feedback so far. Based on the discussion [1], we seem
> > to have consensus. So, I would like to start a vote on FLIP-242 [2].
> >
> > The vote will last for at least 72 hours unless there is an objection or
> > insufficient votes.
> >
> > [1] https://lists.apache.org/thread/k5s970xlqoj7opx1pzbpylxcov0tp842
> > [2]
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-242%3A+Introduce+configurable+RateLimitingStrategy+for+Async+Sink
> >
> > Regards,
> > Hong
> >
>

Re: [VOTE] Apache Flink ML Release 2.1.0, release candidate #1

[Dong Lin]

Hi Zhipeng,

I just realized that these two PRs (link
 and link
) are not included in the
release branch, which means that the Flink ML 2.1 website will not include
the documentation (link
)
to run those examples. Though they can still follow the documentation on
the latest website (which is nightly-built and non-stable), it is probably
more user-friendly to backport those examples and documentation to the 2.1
release branch.

Sorry for the inconvenience. Could you help make another release candidate
to fix this issue?

Thanks,
Dong


On Tue, Jun 28, 2022 at 5:28 PM Zhipeng Zhang 
wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.1.0 of
> Apache Flink ML as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> **Testing Guideline**
>
> You can find here [1] a page in the project wiki on instructions for
> testing.
> To cast a vote, it is not necessary to perform all listed checks, but
> please
> mention which checks you have performed when voting.
>
> **Release Overview**
>
> As an overview, the release consists of the following:
> a) Flink ML source release to be deployed to dist.apache.org
> b) Flink ML Python source distributions to be deployed to PyPI
> c) Maven artifacts to be deployed to the Maven Central Repository
>
> **Staging Areas to Review**
>
> The staging areas containing the above mentioned artifacts are as follows,
> for your review:
> * All artifacts for a) and b) can be found in the corresponding dev
> repository at dist.apache.org [2],
> which are signed with the key with fingerprint
> 0789F389E67ADDFA034E603FABF0C46E59C8941C [3]
> * All artifacts for c) can be found at the Apache Nexus Repository [4]
>
> Other links for your review:
> * JIRA release notes [5]
> * Source code tag "release-2.0.0-rc1" [6]
> * PR to update the website Downloads page to include Flink ML links [7]
>
> **Vote Duration**
> The voting time will run for at least 72 hours.
> It is adopted by majority approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Yun and Zhipeng
>
> [1]
>
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+ML+Release
> [2] https://dist.apache.org/repos/dist/dev/flink/flink-ml-2.1.0-rc1/
> [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> [4]
> https://repository.apache.org/content/repositories/orgapacheflink-1516/
> [5]
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12351141
> [6] https://github.com/apache/flink-ml/releases/tag/release-2.1.0-rc1
> [7] https://github.com/apache/flink-web/pull/556
>

[jira] [Created] (FLINK-28305) SQL Client end-to-end test failed with ElasticsearchException

[Huang Xingbo (Jira)]

Huang Xingbo created FLINK-28305:


 Summary: SQL Client end-to-end test failed with 
ElasticsearchException
 Key: FLINK-28305
 URL: https://issues.apache.org/jira/browse/FLINK-28305
 Project: Flink
  Issue Type: Bug
  Components: Table SQL / Client
Affects Versions: 1.14.5
Reporter: Huang Xingbo



{code:java}
2022-06-29T19:14:39.0384925Z Jun 29 19:14:38 java.lang.RuntimeException: An 
error occurred in ElasticsearchSink.
2022-06-29T19:14:39.0385547Z Jun 29 19:14:38at 
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427)
 ~[?:?]
2022-06-29T19:14:39.0386327Z Jun 29 19:14:38at 
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkAsyncErrorsAndRequests(ElasticsearchSinkBase.java:432)
 ~[?:?]
2022-06-29T19:14:39.0387078Z Jun 29 19:14:38at 
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.invoke(ElasticsearchSinkBase.java:329)
 ~[?:?]
2022-06-29T19:14:39.0388071Z Jun 29 19:14:38at 
org.apache.flink.table.runtime.operators.sink.SinkOperator.processElement(SinkOperator.java:65)
 ~[flink-table_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0389189Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:233)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0390354Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:134)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0391515Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:105)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0392712Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0393822Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:495)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0394861Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:203)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0395885Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:806)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0396858Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:758) 
~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0397824Z Jun 29 19:14:38at 
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0398882Z Jun 29 19:14:38at 
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:937) 
~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0400103Z Jun 29 19:14:38at 
org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) 
~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0401000Z Jun 29 19:14:38at 
org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) 
~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0401575Z Jun 29 19:14:38at 
java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_332]
2022-06-29T19:14:39.0402051Z Jun 29 19:14:38Suppressed: 
java.lang.RuntimeException: An error occurred in ElasticsearchSink.
2022-06-29T19:14:39.0402682Z Jun 29 19:14:38at 
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.checkErrorAndRethrow(ElasticsearchSinkBase.java:427)
 ~[?:?]
2022-06-29T19:14:39.0403421Z Jun 29 19:14:38at 
org.apache.flink.streaming.connectors.elasticsearch.ElasticsearchSinkBase.close(ElasticsearchSinkBase.java:366)
 ~[?:?]
2022-06-29T19:14:39.0404395Z Jun 29 19:14:38at 
org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0405470Z Jun 29 19:14:38at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.close(AbstractUdfStreamOperator.java:114)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0406549Z Jun 29 19:14:38at 
org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.close(StreamOperatorWrapper.java:141)
 ~[flink-dist_2.11-1.14-SNAPSHOT.jar:1.14-SNAPSHOT]
2022-06-29T19:14:39.0407648Z Jun 29 19:14:38at 
org.apache.flin

Re: [VOTE] FLIP-241: Completed Jobs Information Enhancement

[Yang Wang]

+1 (binding)

Best,
Yang

Zhu Zhu  于2022年6月29日周三 14:31写道：

> +1 (binding)
>
> Thanks,
> Zhu
>
> Xintong Song  于2022年6月23日周四 17:01写道：
> >
> > +1 (binding)
> >
> > Best,
> >
> > Xintong
> >
> >
> >
> > On Thu, Jun 23, 2022 at 1:49 PM Yangze Guo  wrote:
> >
> > > +1 (binding)
> > >
> > > Best,
> > > Yangze Guo
> > >
> > > On Thu, Jun 23, 2022 at 1:07 PM junhan yang 
> > > wrote:
> > > >
> > > > Hi everyone,
> > > >
> > > > Thanks for the feedbacks on the discussion thread[1]. I would like to
> > > start
> > > > a vote thread here for FLIP-241: Completed Jobs Information
> > > Enhancement[2].
> > > >
> > > > The vote will last for at least 72 hours unless there is an
> objection, I
> > > > will try to close it by *next Tuesday* if we receive sufficient votes
> > > until
> > > > then.
> > > >
> > > > Thank you again for your participation in this FLIP discussion.
> > > >
> > > > [1] https://lists.apache.org/thread/qycqmxbh37b5qzs72y110rp8457kkxkb
> > > > [2] https://cwiki.apache.org/confluence/x/dRD1D
> > > >
> > > > Best regards,
> > > > Junhan
> > >
>

Re: [VOTE] Apache Flink ML Release 2.1.0, release candidate #1

[Zhipeng Zhang]

Hi Dong,

Thanks for the check. I will cancel this release and release RC2 today.

Dong Lin  于2022年6月30日周四 08:47写道：

> Hi Zhipeng,
>
> I just realized that these two PRs (link
>  and link
> ) are not included in the
> release branch, which means that the Flink ML 2.1 website will not include
> the documentation (link
> <
> https://nightlies.apache.org/flink/flink-ml-docs-master/docs/try-flink-ml/quick-start/
> >)
> to run those examples. Though they can still follow the documentation on
> the latest website (which is nightly-built and non-stable), it is probably
> more user-friendly to backport those examples and documentation to the 2.1
> release branch.
>
> Sorry for the inconvenience. Could you help make another release candidate
> to fix this issue?
>
> Thanks,
> Dong
>
>
> On Tue, Jun 28, 2022 at 5:28 PM Zhipeng Zhang 
> wrote:
>
> > Hi everyone,
> >
> > Please review and vote on the release candidate #1 for the version 2.1.0
> of
> > Apache Flink ML as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> > **Testing Guideline**
> >
> > You can find here [1] a page in the project wiki on instructions for
> > testing.
> > To cast a vote, it is not necessary to perform all listed checks, but
> > please
> > mention which checks you have performed when voting.
> >
> > **Release Overview**
> >
> > As an overview, the release consists of the following:
> > a) Flink ML source release to be deployed to dist.apache.org
> > b) Flink ML Python source distributions to be deployed to PyPI
> > c) Maven artifacts to be deployed to the Maven Central Repository
> >
> > **Staging Areas to Review**
> >
> > The staging areas containing the above mentioned artifacts are as
> follows,
> > for your review:
> > * All artifacts for a) and b) can be found in the corresponding dev
> > repository at dist.apache.org [2],
> > which are signed with the key with fingerprint
> > 0789F389E67ADDFA034E603FABF0C46E59C8941C [3]
> > * All artifacts for c) can be found at the Apache Nexus Repository [4]
> >
> > Other links for your review:
> > * JIRA release notes [5]
> > * Source code tag "release-2.0.0-rc1" [6]
> > * PR to update the website Downloads page to include Flink ML links [7]
> >
> > **Vote Duration**
> > The voting time will run for at least 72 hours.
> > It is adopted by majority approval, with at least 3 PMC affirmative
> votes.
> >
> > Thanks,
> > Yun and Zhipeng
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/Verifying+a+Flink+ML+Release
> > [2] https://dist.apache.org/repos/dist/dev/flink/flink-ml-2.1.0-rc1/
> > [3] https://dist.apache.org/repos/dist/release/flink/KEYS
> > [4]
> > https://repository.apache.org/content/repositories/orgapacheflink-1516/
> > [5]
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12351141
> > [6] https://github.com/apache/flink-ml/releases/tag/release-2.1.0-rc1
> > [7] https://github.com/apache/flink-web/pull/556
> >
>


-- 
best,
Zhipeng

Re: [VOTE] FLIP-241: Completed Jobs Information Enhancement

[junhan yang]

Hi all,

Thanks for participating in this voting process. Closing it now.

Best regards,
Junhan

Yang Wang  于2022年6月30日周四 10:15写道：

> +1 (binding)
>
> Best,
> Yang
>
> Zhu Zhu  于2022年6月29日周三 14:31写道：
>
> > +1 (binding)
> >
> > Thanks,
> > Zhu
> >
> > Xintong Song  于2022年6月23日周四 17:01写道：
> > >
> > > +1 (binding)
> > >
> > > Best,
> > >
> > > Xintong
> > >
> > >
> > >
> > > On Thu, Jun 23, 2022 at 1:49 PM Yangze Guo  wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Best,
> > > > Yangze Guo
> > > >
> > > > On Thu, Jun 23, 2022 at 1:07 PM junhan yang <
> yangjunhan1...@gmail.com>
> > > > wrote:
> > > > >
> > > > > Hi everyone,
> > > > >
> > > > > Thanks for the feedbacks on the discussion thread[1]. I would like
> to
> > > > start
> > > > > a vote thread here for FLIP-241: Completed Jobs Information
> > > > Enhancement[2].
> > > > >
> > > > > The vote will last for at least 72 hours unless there is an
> > objection, I
> > > > > will try to close it by *next Tuesday* if we receive sufficient
> votes
> > > > until
> > > > > then.
> > > > >
> > > > > Thank you again for your participation in this FLIP discussion.
> > > > >
> > > > > [1]
> https://lists.apache.org/thread/qycqmxbh37b5qzs72y110rp8457kkxkb
> > > > > [2] https://cwiki.apache.org/confluence/x/dRD1D
> > > > >
> > > > > Best regards,
> > > > > Junhan
> > > >
> >
>

[RESULT][VOTE] FLIP-241: Completed Jobs Information Enhancement

[junhan yang]

Hi everyone,

FLIP-241[1] has been accepted according to the votes. There have been 4
approving votes, 4 out of 4 are binding[2]:

   - Yangze Guo
   - Xintong Song
   - Zhu Zhu
   - Yang Wang

There is no disapproving vote.

[1]: https://cwiki.apache.org/confluence/x/dRD1D
[2]: https://lists.apache.org/thread/xg6mtqmjbr2dloms8hh4yh30mrlkoboj

Best regards,
Junhan

[jira] [Created] (FLINK-28306) Support get table statistic for Hive partition table

[luoyuxia (Jira)]

luoyuxia created FLINK-28306:


 Summary: Support get table statistic for Hive partition table
 Key: FLINK-28306
 URL: https://issues.apache.org/jira/browse/FLINK-28306
 Project: Flink
  Issue Type: Sub-task
Reporter: luoyuxia






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[RESULT][VOTE] FLIP-240: Introduce "ANALYZE TABLE" Syntax

[godfrey he]

Hi, everyone.

FLIP-240: Introduce "ANALYZE TABLE" Syntax[1] has been accepted.

There are 3 binding votes, 2 non-binding votes[2].
- Jingsong Li(binding)
- Shengkai Fang(binding)
- Roc Marshal(non-binding)
- Leonard Xu(binding)
- yuxia(non-binding)

None against.

Thanks again for every one who concerns on this FLIP.


[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217386481
[2] https://lists.apache.org/thread/lltcom64dzzf324fpl19nfzxb56k37wd


Best,
Godfrey

[jira] [Created] (FLINK-28307) FLIP-241: Completed Jobs Information Enhancement

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28307:
--

 Summary: FLIP-241: Completed Jobs Information Enhancement
 Key: FLINK-28307
 URL: https://issues.apache.org/jira/browse/FLINK-28307
 Project: Flink
  Issue Type: New Feature
  Components: Runtime / REST, Runtime / Web Frontend
Reporter: Yangze Guo
 Fix For: 1.16.0


Streaming and Batch users have different interests in probing a job. While 
streaming users mainly care about the instant status of a running job (tps, 
delay, backpressure, etc.), batch users care more about the overall job status 
during the entire execution (queueing / execution time, total data amount, 
etc.).

As Flink grows into a unified streaming & batch processor and is adopted by 
more and more batch users, the experiences in inspecting completed jobs has 
become more important than ever.

We compared Flink with other popular batch processors, and spotted several 
potential improvements. Most of these changes involves WebUI & REST API 
changes, which should be discussed and voted on as FLIPs. However, creating 
separated FLIPs for each of the improvement might be overkill, because changes 
needed by each improvement are quite small. Thus, we include all these 
potential improvements in this one FLIP.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28308) Introduce accumulated time metrics that a running task is busy / idle / back-pressured

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28308:
--

 Summary: Introduce accumulated time metrics that a running task is 
busy / idle / back-pressured
 Key: FLINK-28308
 URL: https://issues.apache.org/jira/browse/FLINK-28308
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / REST
Reporter: Yangze Guo
Assignee: Yangze Guo
 Fix For: 1.16.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28309) Introduce metrics of the duration that a task stays in each status

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28309:
--

 Summary: Introduce metrics of the duration that a task stays in 
each status
 Key: FLINK-28309
 URL: https://issues.apache.org/jira/browse/FLINK-28309
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Metrics, Runtime / REST
Reporter: Yangze Guo
Assignee: Yangze Guo
 Fix For: 1.16.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28310) Introduce aggregating task metrics

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28310:
--

 Summary: Introduce aggregating task metrics
 Key: FLINK-28310
 URL: https://issues.apache.org/jira/browse/FLINK-28310
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / Metrics, Runtime / REST
Reporter: Yangze Guo
Assignee: Yangze Guo
 Fix For: 1.16.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28311) Introduce REST APIs for the environmental information

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28311:
--

 Summary: Introduce REST APIs for the environmental information
 Key: FLINK-28311
 URL: https://issues.apache.org/jira/browse/FLINK-28311
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / REST
Reporter: Yangze Guo
Assignee: Yangze Guo
 Fix For: 1.16.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28312) Introduce REST APIs for log URL retrieval

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28312:
--

 Summary: Introduce REST APIs for log URL retrieval
 Key: FLINK-28312
 URL: https://issues.apache.org/jira/browse/FLINK-28312
 Project: Flink
  Issue Type: Sub-task
Reporter: Yangze Guo






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28313) Add history server flag to DashboardConfiguration

[Yangze Guo (Jira)]

Yangze Guo created FLINK-28313:
--

 Summary: Add history server flag to DashboardConfiguration
 Key: FLINK-28313
 URL: https://issues.apache.org/jira/browse/FLINK-28313
 Project: Flink
  Issue Type: Sub-task
  Components: Runtime / REST
Reporter: Yangze Guo
Assignee: Yangze Guo
 Fix For: 1.16.0






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28314) Introduce "Cluster Environment" tab under history server Web UI

[Junhan Yang (Jira)]

Junhan Yang created FLINK-28314:
---

 Summary: Introduce "Cluster Environment" tab under history server 
Web UI
 Key: FLINK-28314
 URL: https://issues.apache.org/jira/browse/FLINK-28314
 Project: Flink
  Issue Type: Sub-task
Affects Versions: 1.16.0
Reporter: Junhan Yang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28315) [UI] Introduce aggregate stats in tables of the subtasks and taskmanagers

[Junhan Yang (Jira)]

Junhan Yang created FLINK-28315:
---

 Summary: [UI] Introduce aggregate stats in tables of the subtasks 
and taskmanagers
 Key: FLINK-28315
 URL: https://issues.apache.org/jira/browse/FLINK-28315
 Project: Flink
  Issue Type: Sub-task
Affects Versions: 1.16.0
Reporter: Junhan Yang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28316) [UI] add external taskmanager log links in the vertex drawer under history server

[Junhan Yang (Jira)]

Junhan Yang created FLINK-28316:
---

 Summary: [UI] add external taskmanager log links in the vertex 
drawer under history server
 Key: FLINK-28316
 URL: https://issues.apache.org/jira/browse/FLINK-28316
 Project: Flink
  Issue Type: Sub-task
Reporter: Junhan Yang






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28317) Throw unsupport exception when RexSubQuery contains LogicalUnion and correlate variables

[luoyuxia (Jira)]

luoyuxia created FLINK-28317:


 Summary: Throw unsupport exception when RexSubQuery contains 
LogicalUnion and correlate variables
 Key: FLINK-28317
 URL: https://issues.apache.org/jira/browse/FLINK-28317
 Project: Flink
  Issue Type: Sub-task
Reporter: luoyuxia






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (FLINK-28318) table idle state ttl configure does not have check when window size bigger then table state ttl

[yutao (Jira)]

yutao created FLINK-28318:
-

 Summary: table idle  state ttl configure does not have check when 
window  size bigger then table state ttl
 Key: FLINK-28318
 URL: https://issues.apache.org/jira/browse/FLINK-28318
 Project: Flink
  Issue Type: Improvement
  Components: Table SQL / Planner, Table SQL / Runtime
Affects Versions: 1.14.4, 1.13.5, 1.12.4
 Environment: flinksql 1.12.4
Reporter: yutao


example: my tumbling window size is 1 day ,but  i config  table.exec.state.ttl 
is 1 hour, if count(1) group by somthing  and  some key more than a hour 
comming element ,we can get a error vlaue.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)