Thanks to Harper. The problem here was that why we lost the ISR? As we have a cluster with 7 nodes, and brought down 1 node.
On 2019/04/19 21:06:10, Harper Henn <harper.h...@datto.com> wrote: > The docs indicate that exception is thrown when the "number of insync > replicas for the partition is lower than min.insync.replicas". > > https://kafka.apache.org/20/javadoc/org/apache/kafka/common/errors/NotEnoughReplicasException.html > > When you took your broker offline, your messages couldn't be replicated to > enough brokers to satisfy your min.insync.replicas configuration. Did you > bring that broker back online yet? Changing your configuration for > min.insync.replicas sounds like it would fix your issue, but then your > messages won't be written to as many brokers and you're at a greater risk > of your data being unavailable if they fail. > > On Tue, Apr 16, 2019 at 9:28 PM wenxing zheng <wenxing.zh...@gmail.com> > wrote: > > > Dear all, > > > > *The problem* > > Our kafka cluster is running with 7 brokers and 3 days ago, we brought down > > node 8 for upgrading the HW. After that, we found the error log below > > saying: org.apache.kafka.common.errors.NotEnoughReplicasException: Number > > of insync replicas for partition. > > > > Before shutting down, our application worked well with enabling > > transaction. But after the operation, the application stopped working. > > > > Please kindly advice what would be the problem here. > > > > > > April 14th 2019, 17:03:52.421 > > [2019-04-14 17:03:45,045] TRACE [Controller id=1 epoch=42] Sending > > UpdateMetadata request PartitionState(controllerEpoch=42, leader=6, > > leaderEpoch=138, isr=[6], zkVersion=259, replicas=[8, 6, 7, 1, 2], > > offlineReplicas=[8]) to brokers Set(1, 5, 2, 6, 7, 4) for partition > > __transaction_state-25 (state.change.logger) > > April 14th 2019, 17:03:52.533 > > [2019-04-14 17:03:51,574] ERROR [ReplicaManager broker=6] Error processing > > append operation on partition __transaction_state-25 > > (kafka.server.ReplicaManager) > > org.apache.kafka.common.errors.NotEnoughReplicasException: Number of insync > > replicas for partition __transaction_state-25 is [1], below required > > minimum [3] > > *Server version* > > Kafka version: 2.0.1 > > Client version: 2.0.1 > > JDK: 1.8 > > > > Thank you very much, > > Regards, > > >
