Hi, We are using mirrormaker to replicate data between two kafka clusters. I am seeing huge difference in size of log in data dir between the broker in source cluster vs broker in destination cluster:
For eg: Size of ~/data/Topic-0/ is about 910 G in source broker, but only its only 25G in destination broker. I see segmented log files (~500 M) is created for about every 2 or 3 mins in source brokers, but I see segmented log files is created for about every 25 mins in destination broker. I verified mirrormaker is doing fine using consumer offset checker, not much lag, offsets are incrementing. I also verified that topics/partitions are not under replicated in both source and target cluster. What is the reason for this difference in disk usage? -- Thanks, Raja.
