[
https://issues.apache.org/jira/browse/KAFKA-14112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elkhan Eminov updated KAFKA-14112:
----------------------------------
Description:
The offset lag is the difference of the last replicated record's (LRO) source
offset and the end offset of the source (LEO).
The offset lag is a difference (LRO-LEO), but its constituents calculated at
different points of time and place
* LEO shall be calculated during source task's poll loop (ready to get it from
the consumer)
* LRO shall be kept in an in-memory "cache", that is updated during the task's
producer callback
LRO is initialized when task is started, from the offset store. The difference
shall be calculated when the freshest LEO acquired
in the poll loop. The calculated amount shall be defined as a MirrorMaker
metric.
This would describe to amount of "to be replicated" number of records for a
certain topic-partition.
was:
The offset lag is the difference of the last replicated record's source offset
and the end offset of the source.
The offset lag is a difference (LRO-LEO), but its constituents calculated at
different points of time and place
* LEO shall be calculated during source task's poll loop (ready to get it from
the consumer)
* LRO shall be kept in an in-memory "cache", that is updated during the task's
producer callback
LRO is initialized when task is started, from the offset store. The difference
shall be calculated when the freshest LEO acquired
in the poll loop. The calculated amount shall be defined as a MirrorMaker
metric.
This would describe to amount of "to be replicated" number of records for a
certain topic-partition.
> Expose replication-offset-lag Mirror metric
> -------------------------------------------
>
> Key: KAFKA-14112
> URL: https://issues.apache.org/jira/browse/KAFKA-14112
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Elkhan Eminov
> Assignee: Elkhan Eminov
> Priority: Minor
>
> The offset lag is the difference of the last replicated record's (LRO) source
> offset and the end offset of the source (LEO).
> The offset lag is a difference (LRO-LEO), but its constituents calculated at
> different points of time and place
> * LEO shall be calculated during source task's poll loop (ready to get it
> from the consumer)
> * LRO shall be kept in an in-memory "cache", that is updated during the
> task's producer callback
> LRO is initialized when task is started, from the offset store. The
> difference shall be calculated when the freshest LEO acquired
> in the poll loop. The calculated amount shall be defined as a MirrorMaker
> metric.
> This would describe to amount of "to be replicated" number of records for a
> certain topic-partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
