Re: Proposing KIP-712: Shallow Mirroring

2021-02-05 Thread Ning Zhang
Hello Henry, This is a very interesting proposal. https://issues.apache.org/jira/browse/KAFKA-10728 reflects the similar concern of re-compressing data in mirror maker. Probably one thing may need to clarify is: how "shallow" mirroring is only applied to mirrormaker use case, if the changes ne

Proposing KIP-712: Shallow Mirroring

2021-02-04 Thread Henry Cai
Dear Community members, We are proposing a new feature to improve the performance of Kafka mirror maker: https://cwiki.apache.org/confluence/display/KAFKA/KIP-712%3A+Shallow+Mirroring The current Kafka MirrorMaker process (with the underlying Consumer and Producer library) uses significant CPU cy