Keep in mind Kafka brokers can use many file descriptors/handles. You may need to increase the OS file descriptor limits.
http://kafka.apache.org/documentation/#os "File descriptor limits: Kafka uses file descriptors for log segments and open connections. If a broker hosts many partitions, consider that the broker needs at least (number_of_partitions)*(partition_size/segment_size) to track all log segments in addition to the number of connections the broker makes. We recommend at least 100000 allowed file descriptors for the broker processes as a starting point." On Wed, Jul 5, 2017 at 10:24 AM, M. Manna <manme...@gmail.com> wrote: > This is quite vague. > > What commands have you executed? > > What do you refer to by open files? Is it the log partition or consumer > offsets? > > On 5 Jul 2017 3:21 pm, "Satyavathi Anasuri" <satyavath...@imimobile.com> > wrote: > > > Hi, > > I have created a topic with 500 partitions in 3 node > > cluster with replication factor 3. kafka version is 0.11. I executed lsof > > command and it lists more 1 lakh open files. why these many open files > and > > how to reduce it ?. > > reg's > > Satya. > > > -- Kaufman Ng +1 646 961 8063 Solutions Architect | Confluent | www.confluent.io
