Hi Liam, Thanks for your quick reply, and great detailed explanation!
When starting console producer with "--request-required-acks all", I do see "Messages are rejected since there are fewer in-sync replicas than required" in log. Best, Can Zhang On Wed, Jun 24, 2020 at 11:33 AM Liam Clarke-Hutchinson <liam.cla...@adscale.co.nz> wrote: > > Hi Can, > > acks is a producer setting only, setting it to all on the broker will have > no effect. The default acks for a producer is 1, which means so long as the > partition leader acknowledges the write, it's successful. You have three > replicas, two downed brokers leaves 1 replica (which become the leader if > not already, assuming it was in-sync prior), so the producer will receive > that 1 ack and consider the write completed. > > If you set acks to all on the producer, then it will throw an exception > when attempting a write when there's not enough insync replicas to > acknowledge the write. > > Hope that makes sense? > > Kind regards, > > Liam Clarke-Hutchinson > > On Wed, Jun 24, 2020 at 3:22 PM Can Zhang <c...@canx.me> wrote: > > > Hello, > > > > I'm doing some tests about behaviors of Kafka under faulty circumstances. > > > > Here are my configs(one of total 3, comments removed): > > > > broker.id=0 > > listeners=PLAINTEXT://:9092 > > advertised.listeners=PLAINTEXT://10.0.8.233:9092 > > num.network.threads=3 > > num.io.threads=8 > > socket.send.buffer.bytes=102400 > > socket.receive.buffer.bytes=102400 > > socket.request.max.bytes=104857600 > > log.dirs=/tmp/kafka-logs > > num.partitions=1 > > num.recovery.threads.per.data.dir=1 > > default.replication.factor=3 > > replication.factor=3 > > acks=all > > offsets.topic.replication.factor=3 > > transaction.state.log.replication.factor=3 > > transaction.state.log.min.isr=2 > > min.insync.replicas = 2 > > log.retention.hours=168 > > log.segment.bytes=1073741824 > > log.retention.check.interval.ms=300000 > > zookeeper.connect=localhost:2181 > > zookeeper.connection.timeout.ms=18000 > > group.initial.rebalance.delay.ms=3000 > > > > I created a topic with 1 partition and 3 replicas, and produce/consume > > with those shell tools shipped with Kafka. And I find even if I kill 2 > > processes of Kafka with `kill -9`, the topic is still writable. I > > believe this would cause potential data loss. > > > > I don't know if I misconfigured something, could someone review it for > > me? I'm testing with kafka_2.12-2.5.0 > > > > > > > > Best, > > Can Zhang > >
