[
https://issues.apache.org/jira/browse/KAFKA-1933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jun Rao updated KAFKA-1933:
---------------------------
Fix Version/s: (was: 0.8.3)
> Fine-grained locking in log append
> ----------------------------------
>
> Key: KAFKA-1933
> URL: https://issues.apache.org/jira/browse/KAFKA-1933
> Project: Kafka
> Issue Type: Improvement
> Components: log
> Reporter: Maxim Ivanov
> Assignee: Maxim Ivanov
> Priority: Minor
> Attachments: KAFKA-1933.patch, KAFKA-1933_2015-02-09_12:27:06.patch
>
>
> This patch adds finer locking when appending to log. It breaks
> global append lock into 2 sequential and 1 parallel phase.
> Basic idea is to allow every thread to "reserve" offsets in non
> overlapping ranges, then do compression in parallel and then
> "commit" write to log in the same order offsets where reserved.
> Results on a server with 16 cores CPU available:
> gzip: 564.0 sec -> 45.2 sec (12.4x speedup)
> LZ4: 56.7 sec -> 9.9 sec (5.7x speedup)
> Kafka was configured to run 16 IO threads, data was pushed using 32 netcat
> instances pushing in parallel batches of 200 msg 6.2 kb each (3264 MB in
> total)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
