Additionally, here's the producer config kafka.bootstrap.servers=x.x.x.x:9092,x.x.x.x:9092,x.x.x.x:9092 kafka.acks=0 kafka.key.serializer=org.apache.kafka.common.serialization.StringSerializer kafka.value.serializer=org.apache.kafka.common.serialization.StringSerializer kafka.max.block.ms=1000 kafka.request.timeout.ms=1000 kafka.max.in.flight.requests.per.connection=1 kafka.retries=0 kafka.compression.type=gzip kafka.security.protocol=SSL kafka.ssl.truststore.location=/data/kafka/kafka-server-truststore.jks kafka.ssl.truststore.password=XXXXXX kafka.linger.ms=300 logger.level=INFO
On Tue, Sep 18, 2018 at 5:36 PM Shantanu Deshmukh <shantanu...@gmail.com> wrote: > Hello, > > We have a 3 broker Kafka 0.10.1.0 deployment in production. There are some > applications which have Kafka Producers embedded in them which send > application logs to a topic. This topic has 10 partitions with replication > factor of 3. > > We are observing that memory usage on some of these application servers > keep shooting through the roof intermittently. After taking heapdump we > found out that top suspects were: > *---------------------* > > > *org.apache.kafka.common.network.Selector -* > > occupies *352,519,104 (24.96%)* bytes. The memory is accumulated in one > instance of *"byte[]"* loaded by *"<system class loader>"*. > > *org.apache.kafka.common.network.KafkaChannel -* > > occupies *352,527,424 (24.96%)* bytes. The memory is accumulated in one > instance of *"byte[]"* loaded by *"<system class loader>"* > > * --------------------- * > > Both of these were holding about 352MB of space. 3 such instances, so they > were consuming about 1.2GB of memory. > > Now regarding usage of producers. Not a huge amount of logs are being sent > to Kafka cluster. It is about 200 msgs/sec. Only one producer object is > being used throughout application. Async send function is used. > > What could be the cause of such huge memory usage? Is this some sort of > memory leak in this specific Kafka version? > >
