Allen,

Apache mailing lists don't allow attachments. Could you please link to a
pastebin or something?

Thanks,
Neha

On Thu, Nov 6, 2014 at 12:02 PM, Allen Wang <aw...@netflix.com.invalid>
wrote:

> After digging more into the stack trace got from flight recorder (which is
> attached), it seems that Kafka (0.8.1.1) can optimize the usage of Crc32.
> The stack trace shows that Crc32 is invoked twice from Log.append(). First
> is from the line number 231:
>
> val appendInfo = analyzeAndValidateMessageSet(messages)
>
> The second time is from line 252 in the same function:
>
> validMessages = validMessages.assignOffsets(offset, appendInfo.codec)
>
> If one of the Crc32 invocation can be eliminated, we are looking at saving
> at least 7% of CPU usage.
>
> Thanks,
> Allen
>
>
>
>
> On Wed, Nov 5, 2014 at 6:32 PM, Allen Wang <aw...@netflix.com> wrote:
>
>> Hi,
>>
>> Using flight recorder, we have observed high CPU usage of CRC32
>> (kafka.utils.Crc32.update()) on Kafka broker. It uses as much as 25% of CPU
>> on an instance. Tracking down stack trace, this method is invoked by
>> ReplicaFetcherThread.
>>
>> Is there any tuning we can do to reduce this?
>>
>> Also on the topic of CPU utilization, we observed that overall CPU
>> utilization is proportional to AllTopicsBytesInPerSec metric. Does this
>> metric include incoming replication traffic?
>>
>> Thanks,
>> Allen
>>
>>
>

Reply via email to