[
https://issues.apache.org/jira/browse/KAFKA-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068575#comment-14068575
]
Jay Kreps edited comment on KAFKA-1546 at 7/21/14 2:41 PM:
-----------------------------------------------------------
I think I was being a little vague. What I was trying to say is this. When each
fetch is serviced we check
{code}
if(!fetchedData.readToEndOfLog)
this.lagBegin = System.currentTimeMillis()
else
this.lagBegin = -1
{code}
Then the liveness criteria is
{code}
partitionLagging = this.lagBegin > 0 && System.currentTimeMillis() -
this.lagBegin > REPLICA_LAG_TIME_MS
{code}
was (Author: jkreps):
I think I was being a little vague. What I was trying to say is this. When each
fetch is serviced we check
{code}
if(fetchedData.size < maxSize)
this.lagBegin = System.currentTimeMillis()
else
this.lagBegin = -1
{code}
Then the liveness criteria is
{code}
partitionLagging = this.lagBegin > 0 && System.currentTimeMillis() -
this.lagBegin > REPLICA_LAG_TIME_MS
{code}
> Automate replica lag tuning
> ---------------------------
>
> Key: KAFKA-1546
> URL: https://issues.apache.org/jira/browse/KAFKA-1546
> Project: Kafka
> Issue Type: Improvement
> Components: replication
> Affects Versions: 0.8.0, 0.8.1, 0.8.1.1
> Reporter: Neha Narkhede
> Labels: newbie++
>
> Currently, there is no good way to tune the replica lag configs to
> automatically account for high and low volume topics on the same cluster.
> For the low-volume topic it will take a very long time to detect a lagging
> replica, and for the high-volume topic it will have false-positives.
> One approach to making this easier would be to have the configuration
> be something like replica.lag.max.ms and translate this into a number
> of messages dynamically based on the throughput of the partition.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
