[
https://issues.apache.org/jira/browse/KAFKA-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14132613#comment-14132613
]
vijay commented on KAFKA-1630:
------------------------------
Consumer Threads take too much CPU resource at this time.And this happens when
i push too much of messages to that topic.And i am also getting
OutOfMemoryError: java heap space in that app server. Can u tell me how much
message can be processed by a single consumer thread per second?
> ConsumerFetcherThread locked in Tomcat
> --------------------------------------
>
> Key: KAFKA-1630
> URL: https://issues.apache.org/jira/browse/KAFKA-1630
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.0
> Environment: linux redhat
> Reporter: vijay
> Assignee: Neha Narkhede
> Labels: performance
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> I am using high level consumer API for consuming messages from kafka.
> ConsumerFetcherThread gets locked. Kindly look in to the below stack trace
> ConsumerFetcherThread-SocialTwitterStream6_172.31.240.136-1410398702143-61a247c3-0-1"
> prio=10 tid=0x00007f294001e800 nid=0x1677 runnable [0x00007f297aae9000]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x00007f2a7c38eb40> (a sun.nio.ch.Util$1)
> - locked <0x00007f2a7c38eb28> (a java.util.Collections$UnmodifiableSet)
> - locked <0x00007f2a7c326f20> (a sun.nio.ch.EPollSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:193)
> - locked <0x00007f2a7c2163c0> (a java.lang.Object)
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:86)
> - locked <0x00007f2a7c229950> (a
> sun.nio.ch.SocketAdaptor$SocketInputStream)
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:200)
> - locked <0x00007f2a7c38ea50> (a java.lang.Object)
> at kafka.utils.Utils$.read(Utils.scala:395)
> at
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
> at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
> at
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
> at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
> at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:73)
> at
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:71)
> - locked <0x00007f2a7c38e9f0> (a java.lang.Object)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:110)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:110)
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:109)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)
> at
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:109)
> at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
> at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:108)
> at
> kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:94)
> at
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:86)
> at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
