[
https://issues.apache.org/jira/browse/KAFKA-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Jorgensen updated KAFKA-4141:
------------------------------------
Summary: 2x increase in cpu usage on new Producer API (was: 2x increase in
cp usage on new Producer API)
> 2x increase in cpu usage on new Producer API
> --------------------------------------------
>
> Key: KAFKA-4141
> URL: https://issues.apache.org/jira/browse/KAFKA-4141
> Project: Kafka
> Issue Type: Bug
> Reporter: Andrew Jorgensen
>
> We are seeing about a 2x increase in CPU usage for the new kafka producer
> compared to the 0.8.0.1 producer. We are currently using gzip compression.
> We recently upgraded our kafka server and producer to 0.10.0.1 from 0.8.1.1
> and noticed that the cpu usage for the new producer had increased pretty
> significantly compared to the old producer. This has caused us to need more
> resources to do the same amount of work we were doing before. I did some
> quick profiling and it looks like during sending half of the cpu cycles are
> sped in org.apache.kafka.common.record.Compressor.putRecord and the other
> half is in org.apache.kafka.common.record.Record.computeChecksum (both are
> around 5.8% of the cpu cycles for that method). I know its not apples to
> apples but the old producer did not seem to have this overhead or at least it
> was greatly reduced.
> Is this a known performance degradation compared to the old producer?
> Here is the trace from the old producer:
> !http://imgur.com/1xS34Dl.jpg!
> New producer:
> !http://imgur.com/0w0G5b1.jpg!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
