Rajiv,
Which version of Kafka are you using? I just checked SimpleConsumer's code,
and in its close() function, disconnect() is called, which will close the
socket.
Guozhang
On Mon, Jan 26, 2015 at 2:36 PM, Rajiv Kurian <ra...@signalfuse.com> wrote:
> Meant to write a run loop.
>
> void run() {
> while (running) {
> if (simpleConsumer == null) {
> simpleConsumer = new SimpleConsumer(host, port,
> (int) kafkaSocketTimeout, kafkaRExeiveBufferSize, clientName);
> }
> try {
> // Do stuff with simpleConsumer.
> } catch (Exception e) {
> logger.error(e); // Assume UnresolvedAddressException.
> if (consumer != null) {
> simpleConsumer.close();
> simpleConsumer = null;
> }
> }
> }
> }
>
> On Mon, Jan 26, 2015 at 2:27 PM, Rajiv Kurian <ra...@signalfuse.com>
> wrote:
>
> > Here is my typical flow:
> > void run() {
> > if (simpleConsumer == null) {
> > simpleConsumer = new SimpleConsumer(host, port, (int)
> kafkaSocketTimeout,
> > kafkaRExeiveBufferSize, clientName);
> > }
> > try {
> > // Do stuff with simpleConsumer.
> > } catch (Exception e) {
> > if (consumer != null) {
> > simpleConsumer.close();
> > simpleConsumer = null;
> > }
> > }
> > }
> >
> > If there is a problem with the host name, or some DNS issues, we get an
> > UnresolvedAddressException as expected and attempt to close the
> > simpleConsumer. However this does not really get rid of the underlying
> > socket. So we end up leaking a FD every time this happens. Though this is
> > not a common case I think there needs to be a way on the SimpleConsumer
> to
> > get rid of all OS resources that it is holding onto. Right now if this
> > keeps happening the number of FDs being consumed by the process keeps
> > increasing till we hit the OS limits. As a user I cannot do anything else
> > but call simpleConsumer.close(). We need to be able to close the
> underlying
> > socketChannel/socket when this kind of an error happens.
> >
> > To reproduce, one can just run this code but just put in any garbage host
> > name, running lsof -p while running this will show that the open FDs
> > increases without limit.
> >
> > Thanks,
> > Rajiv
> >
> >
>
--
-- Guozhang
