Sure. Here is the link to the screen shot of jmc with the JTR file loaded: http://picpaste.com/fligh-recorder-crc.png
On Thu, Nov 6, 2014 at 2:12 PM, Neha Narkhede <neha.narkh...@gmail.com> wrote: > Allen, > > Apache mailing lists don't allow attachments. Could you please link to a > pastebin or something? > > Thanks, > Neha > > On Thu, Nov 6, 2014 at 12:02 PM, Allen Wang <aw...@netflix.com.invalid> > wrote: > > > After digging more into the stack trace got from flight recorder (which > is > > attached), it seems that Kafka (0.8.1.1) can optimize the usage of Crc32. > > The stack trace shows that Crc32 is invoked twice from Log.append(). > First > > is from the line number 231: > > > > val appendInfo = analyzeAndValidateMessageSet(messages) > > > > The second time is from line 252 in the same function: > > > > validMessages = validMessages.assignOffsets(offset, appendInfo.codec) > > > > If one of the Crc32 invocation can be eliminated, we are looking at > saving > > at least 7% of CPU usage. > > > > Thanks, > > Allen > > > > > > > > > > On Wed, Nov 5, 2014 at 6:32 PM, Allen Wang <aw...@netflix.com> wrote: > > > >> Hi, > >> > >> Using flight recorder, we have observed high CPU usage of CRC32 > >> (kafka.utils.Crc32.update()) on Kafka broker. It uses as much as 25% of > CPU > >> on an instance. Tracking down stack trace, this method is invoked by > >> ReplicaFetcherThread. > >> > >> Is there any tuning we can do to reduce this? > >> > >> Also on the topic of CPU utilization, we observed that overall CPU > >> utilization is proportional to AllTopicsBytesInPerSec metric. Does this > >> metric include incoming replication traffic? > >> > >> Thanks, > >> Allen > >> > >> > > >
