[
https://issues.apache.org/jira/browse/KAFKA-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-736:
--------------------------------
Attachment: kafka-736-v3.patch
Found a bug in the v2 patch that caused the multiple producer throughput
degradation. The empty response was created with processor id 0. Now, when the
response is enqueued, we wake up this processor id instead of the request's
processor id. So, processor 0 always woke up on time, but not the rest of the
processors that led to their select operation block for the entire 300 ms.
New performance numbers make sense now and the v2/v3 design shows ~16 %
throughput degradation compared to the draft patch approach, which can be
explained.
Message size is 1K in all the tests
batch size 1, producer threads 1
kafka-736-v3 - 13 MB/s
kafka-736-draft - 30 MB/s
batch size 100, producer threads 1
kafka-736-v2 - 48.4 MB/s
kafka-736-draft - 61.5 MB/s
batch size 100, producer threads 20
kafka-736-v2 - 70 MB/s
kafka-736-draft - 81.6 MB/s
> Add an option to the 0.8 producer to mimic 0.7 producer behavior
> ----------------------------------------------------------------
>
> Key: KAFKA-736
> URL: https://issues.apache.org/jira/browse/KAFKA-736
> Project: Kafka
> Issue Type: Improvement
> Components: producer
> Affects Versions: 0.8
> Reporter: Neha Narkhede
> Assignee: Neha Narkhede
> Priority: Blocker
> Labels: p2, replication-performance
> Attachments: check-message-ordering.py, kafka-736-draft.patch,
> kafka-736-v1.patch, kafka-736-v2.patch, kafka-736-v3.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> I profiled a producer throughput benchmark between a producer and a remote
> broker. It turns out that the background send threads spends ~97% of its time
> waiting to read the acknowledgement from the broker.
> I propose we change the current behavior of request.required.acks=0 to mean
> no acknowledgement from the broker. This will mimic the 0.7 producer behavior
> and will enable tuning the producer for very high throughput.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
