Without knowing the intricacies of Kafka, i think the default open file
descriptors is 1024 on unix. This can be changed by setting a higher ulimit
value ( typically 8192 but sometimes even 10 ).
Before modifying the ulimit I would recommend you check the number of
sockets stuck in TIME_WAIT mo
Hi Prakash,
How many open files do you expect a broker to be able to handle? It seems
like this broker is crashing at around 4100 or so open files.
Thanks,
Paul Lung
On 6/24/14, 11:08 PM, "Lung, Paul" wrote:
>Ok. What I just saw was that when the controller machine reaches around
>4100+ files,
Ok. What I just saw was that when the controller machine reaches around
4100+ files, it crashes. Then I think the controller bounced between 2
other machines, taking them down too, and the circled back to the original
machine.
Paul Lung
On 6/24/14, 10:51 PM, "Lung, Paul" wrote:
>The controller
The controller machine has 3500 or so, while the other machines have
around 1600.
Paul Lung
On 6/24/14, 10:31 PM, "Prakash Gowri Shankor"
wrote:
>How many files does each broker itself have open ? You can find this from
>'ls -l /proc//fd'
>
>
>
>
>On Tue, Jun 24, 2014 at 10:18 PM, Lung, Paul w
How many files does each broker itself have open ? You can find this from
'ls -l /proc//fd'
On Tue, Jun 24, 2014 at 10:18 PM, Lung, Paul wrote:
> Hi All,
>
>
> I just upgraded my cluster from 0.8.1 to 0.8.1.1. I’m seeing the following
> error messages on the same 3 brokers once in a while:
>
Hi All,
I just upgraded my cluster from 0.8.1 to 0.8.1.1. I’m seeing the following
error messages on the same 3 brokers once in a while:
[2014-06-24 21:43:44,711] ERROR Error in acceptor (kafka.network.Acceptor)
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChann
