There is "replica.lag.max.messages" (defaults to 4k) and "replica.lag.time.max.ms" (defaults to 10s). If a replica is behind the leader by either that many messages or hasn't sent any fetch requests within the lag config, then it falls out of ISR.
However, as mentioned earlier it is best to avoid a cross-DC replication set up. You should set the above configs to be reasonable values (the defaults should be reasonable, but it depends on your volume) and if you see replicas going out of ISR then address the underlying issue(s) - e.g., depending on what the issue is it could be enough to increase the number of replica fetcher threads. Increasing the replica message and time lag may help reduce ISR churns but it does not address the issue (which is replicas being unable to keep up). On Sat, Jun 29, 2013 at 10:47 AM, Yu, Libo <libo...@citi.com> wrote: > Thanks for your answer, Joel. What parameters can I tune in this case to > increase the lag? > By default, under what kind of scenario will a replica it fall out of the > "in-sync-replica" > Set? > > Regards, > > Libo > > > -----Original Message----- > From: Joel Koshy [mailto:jjkosh...@gmail.com] > Sent: Friday, June 28, 2013 9:04 PM > To: users@kafka.apache.org > Subject: Re: how brokers sync with each other > > I'm assuming this is somewhat related to your previous question on cross-DC > replication. This is not an ideal set up as mentioned there. > If the replica lags then it will fall out of the "in-sync-replica" > set. You could tune parameters that effectively allow a high (but bounded > over time) lag between the replica and the leader. There is no buffer per se > - the replicas just keep fetching from the leader (which would have appended > the messages to its log). > > If your producer has acks set to -1 (which means wait until all replicas in > ISR have received the message) it would slow down your producer considerably. > (You could use acks = 1 in that case though but with weaker guarantees - > i.e., your message could be lost in failure scenarios). > > Joel > > > On Fri, Jun 28, 2013 at 5:41 PM, Yu, Libo <libo...@citi.com> wrote: >> Hi, >> >> Assume A and B are two brokers in a kafka server. And there is long >> network Latency between A and B. For a partition with two >> replications, one replication Is assigned to A and the other is >> assigned to B. Number of acknowledge is set to one. Assume the partition is >> handled by broker A. >> >> After a message is accepted and saved by A, ack will be sent and the >> message will be sent to broker B at the same time. But due to long >> network latency between A and B, it is possible that B lags behind A. >> In this case, is there a buffer on broker A that holds all messages to >> be written to B? Is there any limit on the lag between A and B? >> >> Compare with the case when there is no network latency between A and >> B, any difference in publishing speed? >> >> Thanks >> >> Libo >>
