Iftach, is it possible the source topic is compressed? Ryanne
On Wed, Jul 1, 2020, 8:39 AM Iftach Ben-Yosef <iben-yo...@outbrain.com> wrote: > Hello everyone. > > I'm testing mm2 for our cross dc topic replication. We used to do it using > mm1 but faced various issues. > > So far, mm2 is working well, but I have 1 issue which I can't really > explain; the destination topic is larger than the source topic. > > For example, We have 1 topic which on the source cluster is around > 2.8-2.9TB with retention.ms=86400000 > > I added to our mm2 cluster the "sync.topic.configs.enabled=false" config, > and edited the retention.ms of the destination topic to be 57600000. Other > than that, I haven't touched the topic created by mm2 on the destination > cluster. > > By logic I'd say that if I shortened the retention on the destination, the > topic size should decrease, but in practice, I see that it is larger than > the source topic (it's about 4.6TB). > This same behaviour is seen on all 3 topics which I am currently mirroring > (all 3 from different source clusters, into the same destination clusters) > > Does anyone have any idea as to why mm2 acts this way for me? > > Thanks, > Iftach > > -- > The above terms reflect a potential business arrangement, are provided > solely as a basis for further discussion, and are not intended to be and > do > not constitute a legally binding obligation. No legally binding > obligations > will be created, implied, or inferred until an agreement in final form is > executed in writing by all parties involved. > > > This email and any > attachments hereto may be confidential or privileged. If you received > this > communication by mistake, please don't forward it to anyone else, please > erase all copies and attachments, and please let me know that it has gone > to the wrong person. Thanks. >
