With the completion of KAFKA-17046 <https://issues.apache.org/jira/browse/KAFKA-17046>, the netty version has been upgraded to 4.1.111.Final in Kafka 3.9.0. This netty version has a known issue with data corruption, see netty issue <https://github.com/netty/netty/issues/14126> and grpc-java issue <https://github.com/grpc/grpc-java/issues/11284>. It is localized to grpc-java. This causes data corruption in any Kafka application that uses grpc-java (e.g., debezium-vitess-connector which runs on Kafka Connect). We should upgrade to a newer version to avoid this data corruption. I requested Jira access to open a ticket for this.
For anyone who is trying to resolve this, the workaround is manually removing these 4.1.111.Final netty dependencies in the /kafka/libs directory and installing another netty version, eg: ``` RUN rm -f /kafka/libs/netty-codec-4.1.111.Final.jar RUN curl -sfSL -o /kafka/libs/netty-codec-4.1.110.Final.jar https://repo1.maven.org/maven2/io/netty/netty-codec/4.1.110.Final/netty-codec-4.1.110.Final.jar ``` You need to repeat this for all of the netty packages.
