Re: Consumer error : This consumer has already been closed

2016-10-25 Thread Gourab Chowdhury
I can see that you have done consumer.subscribe(Arrays.asList(topic)); I don't know if this is a problem but try consumer.subscribe(Collections. singletonList(this.topic)); You can refer to official example code of Kafka at github[1]. Hope that helps. [1] https://github.com/apache/kafka/blob/trun

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-24 Thread Gourab Chowdhury
No issues, I have solved the problem by changing the permission of /data Thanks, Gourab On Mon, Oct 24, 2016 at 2:20 PM, Gourab Chowdhury wrote: > Thanks for the reply, I tried changing the data directory as follows:- > dataDir=/data/zookeeper > > I also create a /data but still

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-24 Thread Gourab Chowdhury
WS or EBS it is just standard Linux than > everything under /tmp is deleted when Linux reboots. > > -hans > > /** > * Hans Jespersen, Principal Systems Engineer, Confluent Inc. > * h...@confluent.io (650)924-2670 > */ > > On Fri, Oct 21, 2016 at 10:38 AM, Gourab Ch

Re: Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Gourab Chowdhury
ose all data on the drive. > > On Fri, Oct 21, 2016 at 2:09 PM, Gourab Chowdhury > wrote: > > > I am running kafka in a EC2 server. While running if there is a sudden > > restart of server, and then we restart the kafka broker, we can't find > the > > deta

Kafka unable to find topic details after sudden EC2 server restart

2016-10-21 Thread Gourab Chowdhury
with this in in real life production? Regards, Gourab Chowdhury,

Re: To find the Lag of consumer offset using kafka client library

2016-09-29 Thread Gourab Chowdhury
eck this out: > > https://github.com/linkedin/Burrow <https://github.com/linkedin/Burrow> > > Regards > > Jan > > > On 29 Sep 2016, at 15:47, Gourab Chowdhury wrote: > > > > I can get the *Lag* of offsets with the following command:- > > > >

To find the Lag of consumer offset using kafka client library

2016-09-29 Thread Gourab Chowdhury
documentation other than https://github.com/apache/ kafka/tree/trunk/docs? I can't find much documentation of kafka. Thanks, Gourab Chowdhury, Software Engg. JunctionTV Inc.

Re: How to get the Global IP address of the Producer?

2016-07-28 Thread Gourab Chowdhury
that > > each producer includes its own IP address in the payload of the messages > it > > produces. > > > > -Michael > > > > > > On Tue, Jul 26, 2016 at 10:15 AM, Gourab Chowdhury > > > wrote: > > > >> Hi, > >> > >&

How to get the Global IP address of the Producer?

2016-07-26 Thread Gourab Chowdhury
Hi, I am working on a project that includes Apache Kafka. My consumer is a Apache Spark and I am try to log the Global IP from where the message was generated, to verify from which device/system? Ofcourse Kafka know the IP of the producer, since it can read the packets. Can it append it in the me