Kafka OOME: Direct buffer memory

2016-07-06 Thread Joseph Francis
We are running kafka 0.9.0.1 in production and saw these exceptions: [2016-06-23 22:55:10,239] INFO [KafkaApi-3] Closing connection due to error during produce request with correlation id 6 from client id kafka-python with ack=0 Topic and partition to exceptions: [xyx,8] -> kafka.common.Mes

Too many ISRs with spikes in metadata and produce times

2016-09-28 Thread Joseph Francis
We are running Kafka 0.9.0.1 in AWS and seeing quite frequent ISRs on many nodes. None of the nodes seems to be really struggling for server capacity. Is there any reason the occurrence of ISRs can cause metadata and produce response times on the brokers to spike up? Thanks, Joseph