[
https://issues.apache.org/jira/browse/KAFKA-2352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kostya Golikov updated KAFKA-2352:
----------------------------------
Attachment: producer.conf
consumer.conf
output.log
> Possible memory leak in MirrorMaker and/or new Producer
> -------------------------------------------------------
>
> Key: KAFKA-2352
> URL: https://issues.apache.org/jira/browse/KAFKA-2352
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8.2.1
> Reporter: Kostya Golikov
> Attachments: consumer.conf, output.log, producer.conf
>
>
> I've been playing around with Mirror Maker (version from trunk, dated July
> 7th) and got a few problems, most noticeable of which is that MirrorMaker
> exhausts it's memory pool, even though it's set to relatively huge value 132
> MB, and individual messages are around 2 KB. Batch size is set to just 2
> messages (full configs are attached).
> {code}
> [2015-07-21 15:19:52,915] FATAL [mirrormaker-thread-1] Mirror maker thread
> failure due to (kafka.tools.MirrorMaker$MirrorMakerThread)
> org.apache.kafka.clients.producer.BufferExhaustedException: You have
> exhausted the 134217728 bytes of memory you configured for the client and the
> client is configured to error rather than block when memory is exhausted.
> at
> org.apache.kafka.clients.producer.internals.BufferPool.allocate(BufferPool.java:124)
> at
> org.apache.kafka.clients.producer.internals.RecordAccumulator.append(RecordAccumulator.java:172)
> at
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:388)
> at
> kafka.tools.MirrorMaker$MirrorMakerProducer.send(MirrorMaker.scala:380)
> at
> kafka.tools.MirrorMaker$MirrorMakerThread$$anonfun$run$3.apply(MirrorMaker.scala:311)
> at
> kafka.tools.MirrorMaker$MirrorMakerThread$$anonfun$run$3.apply(MirrorMaker.scala:311)
> at scala.collection.Iterator$class.foreach(Iterator.scala:727)
> at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
> at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
> at kafka.tools.MirrorMaker$MirrorMakerThread.run(MirrorMaker.scala:311)
> {code}
> Am I doing something wrong? Any help in further diagnosing of this problem
> might be handy.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
