[
https://issues.apache.org/jira/browse/FLINK-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620523#comment-14620523
]
ASF GitHub Bot commented on FLINK-2292:
---------------------------------------
GitHub user mxm opened a pull request:
https://github.com/apache/flink/pull/896
[FLINK-2292][FLINK-1573] add live per-task accumulators
This refactors the accumulators to accumulate per task execution. The
accumulators are reported from the task managers periodically to the job
manager via the Heartbeat message. If the execution contains chained
tasks, the accumulators are chained as well. The final accumulator
results are reported via the UpdateTaskExecutionState message.
The accumulators are now saved in the Execution within the
ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
removed for now. In the future, we might introduce some caching for the
web frontend visualization.
Two types of accumulators are available:
- external (user-defined via the RuntimeContext)
- internal (flink metrics defined in the invocables)
The internal (built-in) metrics are targeted at users who want to
monitor their programs, e.g. through the job manager's web frontend.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mxm/flink live-accumulators
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/896.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #896
----
commit 7cec1236f087e72b40022bf02a6dbb12d74acbac
Author: Maximilian Michels <m...@apache.org>
Date: 2015-07-08T07:23:42Z
[FLINK-2292][FLINK-1573] add live per-task accumulators
This refactors the accumulators to accumulate per task execution. The
accumulators are reported from the task managers periodically to the job
manager via the Heartbeat message. If the execution contains chained
tasks, the accumulators are chained as well. The final accumulator
results are reported via the UpdateTaskExecutionState message.
The accumulators are now saved in the Execution within the
ExecutionGraph. This makes the AccumulatorManager obsolete. It has been
removed for now. In the future, we might introduce some caching for the
web frontend visualization.
Two types of accumulators are available:
- external (user-defined via the RuntimeContext)
- internal (flink metrics defined in the invocables)
The internal (built-in) metrics are targeted at users who want to
monitor their programs, e.g. through the job manager's web frontend.
----
> Report accumulators periodically while job is running
> -----------------------------------------------------
>
> Key: FLINK-2292
> URL: https://issues.apache.org/jira/browse/FLINK-2292
> Project: Flink
> Issue Type: Sub-task
> Components: JobManager, TaskManager
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Fix For: 0.10
>
>
> Accumulators should be sent periodically, as part of the heartbeat that sends
> metrics. This allows them to be updated in real time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
