Hi Caleb,
We already set the number of max open files to 100,000 before this error
happened.
Normally, the file description is about 20,000, but in some time, it suddenly
jump to so many count.
This is our monitor about Kafka FD info:
2017-05-17-05:04:19 FD_total_num:19261 FD_pair_num:15256 FD_ads_num:3191
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 149 REG 18877
2017-05-17-05:04:31 FD_total_num:19267 FD_pair_num:15259 FD_ads_num:3192
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 152 REG 18883
2017-05-17-05:04:44 FD_total_num:19272 FD_pair_num:15263 FD_ads_num:3197
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 148 REG 18892
2017-05-17-05:04:57 FD_total_num:19280 FD_pair_num:15268 FD_ads_num:3197
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 150 REG 18895
2017-05-17-05:05:09 FD_total_num:19277 FD_pair_num:15271 FD_ads_num:3197
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 152 REG 18898
2017-05-17-05:05:21 FD_total_num:19223 FD_pair_num:15217 FD_ads_num:3189
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 158 REG 18836
2017-05-17-05:05:34 FD_total_num:19235 FD_pair_num:15223 FD_ads_num:3189
FD_Type:TYPE 1 DIR 2 unix 2 sock 4 CHR 7 a_inode 73 FIFO 146 IPv4 158 REG 18842
On 13/05/2017, 9:57 AM, "Caleb Welton" <ca...@autonomic.ai> wrote:
You need to up your OS open file limits, something like this should work:
# /etc/security/limits.conf
* - nofile 65536
On Fri, May 12, 2017 at 6:34 PM, Yang Cui <y...@freewheel.tv> wrote:
> Our Kafka cluster is broken down by the problem “java.io.IOException: Too
> many open files” three times in 3 weeks.
>
> We encounter these problem on both 0.9.0.1 and 0.10.2.1 version.
>
> The error is like:
>
> java.io.IOException: Too many open files
> at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
> at sun.nio.ch.ServerSocketChannelImpl.accept(
> ServerSocketChannelImpl.java:422)
> at sun.nio.ch.ServerSocketChannelImpl.accept(
> ServerSocketChannelImpl.java:250)
> at kafka.network.Acceptor.accept(SocketServer.scala:340)
> at kafka.network.Acceptor.run(SocketServer.scala:283)
> at java.lang.Thread.run(Thread.java:745)
>
> Is someone encounter the similar problem?
>
>
>
