Kirill Kulikov created KAFKA-8293:
-------------------------------------
Summary: Messages undelivered when small quotas applied
Key: KAFKA-8293
URL: https://issues.apache.org/jira/browse/KAFKA-8293
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 2.1.1
Reporter: Kirill Kulikov
I observe a strange Kafka behavior when using small quotas.
For ex. I set quotas for the consumer like
{code:java}
kafka-configs --zookeeper zookeeper:2181 --entity-type users --entity-name
kafka --alter --add-config 'producer_byte_rate=2048000,
consumer_byte_rate=256'{code}
If I send a small batch of messages as
{code:java}
kafka-producer-perf-test --producer.config /etc/kafka/consumer.properties
--producer-props acks=-1 compression.type=none bootstrap.servers=kafka:9093
--num-records 10 --record-size 20 --throughput 1000 --print-metrics --topic
test
{code}
they go through without problems.
But when the batch is bigger
{code:java}
kafka-producer-perf-test --producer.config /etc/kafka/consumer.properties
--producer-props acks=-1 compression.type=none bootstrap.servers=kafka:9093
--num-records 100 --record-size 20 --throughput 1000 --print-metrics --topic
test
{code}
... I do not get any messages on the consumer side *at all*.
On the other hand, if `kafka-producer-perf-test` throughput is limited like
{code:java}
kafka-producer-perf-test --producer.config /etc/kafka/consumer.properties
--producer-props acks=-1 compression.type=none bootstrap.servers=kafka:9093
--num-records 1000 --record-size 10 --throughput 10 --print-metrics --topic test
{code}
I can see only the first 20-30 messages in `kafka-console-consumer`. But then
it gets stuck (throttled perhaps) and other queued messages never come through.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
