We have JMX beans that report #messages per topic? Does the total count match btw the two clusters?
Thanks, Jun On Thu, Aug 22, 2013 at 2:14 PM, Rajasekar Elango <rela...@salesforce.com>wrote: > Hi, > > We are using mirrormaker to replicate data between two kafka clusters. I am > seeing huge difference in size of log in data dir between the broker in > source cluster vs broker in destination cluster: > > For eg: Size of ~/data/Topic-0/ is about 910 G in source broker, but only > its only 25G in destination broker. I see segmented log files (~500 M) is > created for about every 2 or 3 mins in source brokers, but I see segmented > log files is created for about every 25 mins in destination broker. > > I verified mirrormaker is doing fine using consumer offset checker, not > much lag, offsets are incrementing. I also verified that topics/partitions > are not under replicated in both source and target cluster. What is the > reason for this difference in disk usage? > > > -- > Thanks, > Raja. >
