Allen, Apache mailing lists don't allow attachments. Could you please link to a pastebin or something?
Thanks, Neha On Thu, Nov 6, 2014 at 12:02 PM, Allen Wang <aw...@netflix.com.invalid> wrote: > After digging more into the stack trace got from flight recorder (which is > attached), it seems that Kafka (0.8.1.1) can optimize the usage of Crc32. > The stack trace shows that Crc32 is invoked twice from Log.append(). First > is from the line number 231: > > val appendInfo = analyzeAndValidateMessageSet(messages) > > The second time is from line 252 in the same function: > > validMessages = validMessages.assignOffsets(offset, appendInfo.codec) > > If one of the Crc32 invocation can be eliminated, we are looking at saving > at least 7% of CPU usage. > > Thanks, > Allen > > > > > On Wed, Nov 5, 2014 at 6:32 PM, Allen Wang <aw...@netflix.com> wrote: > >> Hi, >> >> Using flight recorder, we have observed high CPU usage of CRC32 >> (kafka.utils.Crc32.update()) on Kafka broker. It uses as much as 25% of CPU >> on an instance. Tracking down stack trace, this method is invoked by >> ReplicaFetcherThread. >> >> Is there any tuning we can do to reduce this? >> >> Also on the topic of CPU utilization, we observed that overall CPU >> utilization is proportional to AllTopicsBytesInPerSec metric. Does this >> metric include incoming replication traffic? >> >> Thanks, >> Allen >> >> >
