To follow up with my last email, I have been looking into socket.receive.buffer.byte as well as socket.send.buffer.bytes. Would it help to increase the buffer for OOM issue?
All help is appreciated! Thanks! -nick From: "McKoy, Nick" <nicholas.mc...@washpost.com<mailto:nicholas.mc...@washpost.com>> Date: Monday, April 18, 2016 at 3:41 PM To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" <users@kafka.apache.org<mailto:users@kafka.apache.org>> Subject: Out of memory - Java Heap space Hey all, I have a kafka cluster of 5 nodes that’s working really hard. CPU is around 40% idle daily. I looked at the file descriptor note on this documentation page http://docs.confluent.io/1.0/kafka/deployment.html#file-descriptors-and-mmap and decided to give it a shot on one instance in the cluster just to see how it performed. I increased this number to 1048576. I kept getting this error from the kafka logs: ERROR [ReplicaFetcherThread--1-6], Error due to (kafka.server.ReplicaFetcherThread) java.lang.OutOfMemoryError: Java heap space I increased heap to see if that would help and I kept seeing these errors. Could the file descriptor change have something related to this? — Nicholas McKoy Engineering – Big Data and Personalization Washington Post Media One Franklin Square, Washington, DC 20001 Email: nicholas.mc...@washpost.com<mailto:nicholas.mc...@washpost.com>
