Perhaps the default heap options aren't sufficient for your particular use of the tool. The consumer perf test defaults to 512MB. I'd simply try increasing the max heap usage: KAFKA_HEAP_OPTS="-Xmx1024M" to bump it up a bit.
-Ewen On Wed, Jan 11, 2017 at 2:59 PM, Check Peck <comptechge...@gmail.com> wrote: > I am trying to run kafka performance script on my linux box. Whenever I run > "kafka-consumer-perf-test.sh", I always get an error. In the same box, I am > running "kafka-producer-perf-test.sh" as well and that is not failing at > all. Looks like something is wrong with "kafka-consumer-perf-test.sh". > > I am running Kafka version 0.10.1.0. > > Command I ran: > ./bin/kafka-consumer-perf-test.sh --zookeeper 110.27.14.10:2181 --messages > 50 --topic test-topic --threads 1 > > Error I got: > [2017-01-11 22:34:09,785] WARN [ConsumerFetcherThread-perf- > consumer-14195_kafka-cluster-3098529006 <(309)%20852-9006> > -zeidk-1484174043509-46a51434-2-0], Error in fetch kafka.consumer. > ConsumerFetcherThread$FetchRequest@54fb48b6 (kafka.consumer. > ConsumerFetcherThread) > java.lang.OutOfMemoryError: Java heap space > at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) > at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) > at org.apache.kafka.common.network.NetworkReceive. > readFromReadableChannel(NetworkReceive.java:93) > at kafka.network.BlockingChannel.readCompletely( > BlockingChannel.scala:129) > at kafka.network.BlockingChannel.receive(BlockingChannel.scala: > 120) > at kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer. > scala:99) > at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$ > $sendRequest(SimpleConsumer.scala:83) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$ > apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:132) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$ > apply$mcV$sp$1.apply(SimpleConsumer.scala:132) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$ > apply$mcV$sp$1.apply(SimpleConsumer.scala:132) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp( > SimpleConsumer.scala:131) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply( > SimpleConsumer.scala:131) > at kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply( > SimpleConsumer.scala:131) > at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33) > at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:130) > at kafka.consumer.ConsumerFetcherThread.fetch( > ConsumerFetcherThread.scala:109) > at kafka.consumer.ConsumerFetcherThread.fetch( > ConsumerFetcherThread.scala:29) > at kafka.server.AbstractFetcherThread.processFetchRequest( > AbstractFetcherThread.scala:118) > at kafka.server.AbstractFetcherThread.doWork( > AbstractFetcherThread.scala:103) > at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:63) >
