Re: why more open files

2017-07-05 Thread Kaufman Ng
Keep in mind Kafka brokers can use many file descriptors/handles. You may need to increase the OS file descriptor limits. http://kafka.apache.org/documentation/#os "File descriptor limits: Kafka uses file descriptors for log segments and open connections. If a broker hosts many partitions, consid

Re: why more open files

2017-07-05 Thread M. Manna
This is quite vague. What commands have you executed? What do you refer to by open files? Is it the log partition or consumer offsets? On 5 Jul 2017 3:21 pm, "Satyavathi Anasuri" wrote: > Hi, >I have created a topic with 500 partitions in 3 node > cluster with replication f

why more open files

2017-07-05 Thread Satyavathi Anasuri
Hi, I have created a topic with 500 partitions in 3 node cluster with replication factor 3. kafka version is 0.11. I executed lsof command and it lists more 1 lakh open files. why these many open files and how to reduce it ?. reg's Satya.