Offset in high level consumer

Hi, Could you tell me how to find the offset in a high level Java consumer ? Thanks Arathi

Re: Offset in high level consumer

wrote: > You can run the ConsumerOffsetChecker tool that ships with Kafka. > > Thanks, > Neha > > > On Wed, May 22, 2013 at 2:02 PM, arathi maddula >wrote: > > > Hi, > > > > Could you tell me how to find the offset in a high level Java consumer ? > > > > Thanks > > Arathi > > >

SimpleConsumer and message offsets

Hi, Iam using a program which reads data from a stream using SimpleConsumer. Is there a way for SimpleConsumer to remember the last offset read? I want the program to continue from the last offset read when it is restarted. Thanks in advance. -Arathi

Issue during commitOffsets using SimpleConsumer

Hi, I get the following error on running SimpleConsumer.commitOffsets(). Could you tell me what is the issue? *java.io.EOFException*: Received -1 when reading from channel, socket has likely been closed. at kafka.utils.Utils$.read(Utils.scala:375) at kafka.network.BoundedByteBufferReceive.read

High level zookeeper consumer life

Hi, I use a high level consumer inside a servlet which reads from Kafka stream. Every time I send an HTTP request, I use a different group ID. This results in lots of consumers on zookeeper node( ls /consumers on zookeeper client shows list of all consumers). Please help me clarify these: 1)Is the

Maximum no. of connections to kafka from a single machine

Hi, Is there a limit to the maximum no. of connections a high level consumer, running on a single machine, can create to Kafka? Thanks Arathi

Kafka compression issues

Hi, I use kafka 0.8. When I run the kafka console producer using ./kafka-console-producer.sh --topic test.compress.e --compress true --broker-list 127.0.0.1:9092 Iam able to see compressed messages in the log. But when I run a Java producer class using the following properties, no mess

Consumer offset oscillates between 2 values

Hi, I use a Java high level consumer to read from a topic with compression set to snappy. When I query the consumer offset for that group using zookeeper shell, the offset value oscillates between 2 values. I use kafka 0.8 and I never faced this issue before. This happens even when there are no hig

Messages TTL setting

Hi, We have a 3 node Kafka cluster. We want to increase the maximum amount of time for which messages are saved in Kafka data logs. Can we change the configuration on one node, stop it and start it and then change the configuration of the next node? Or should we stop all 3 nodes at a time, make co

Default compression code in Kafka topics

Hi, We have a cluster of Kafka servers. We want data of all topics on these servers to be compressed, Is there some configuration to achieve this? I was able to compress code by using compression.codec property in ProducerConfig in Kafka Producer. But I wanted to know if there is a way of enablin

Change kafka broker ids dynamically

Hi, Is it possible to change the broker.ids property for a node belonging to a Kafka cluster? For example, currently if I have brokers with ids 1,2,3. If I want to stop broker 1, can I change broker.id to 0 (with current id = 1) in server.properties and meta.properties files and then restart b

Closing socket for 1.2.4.5 because of error (kafka.network.Processor)

Hi, I found this error in my kafka.out. How can I find what is causing this error? Kafka jar : kafka_2.10-0.8.2.2.3.2.0-2950.jar Closing socket for /1.2.3.4 because of error (kafka.network.Processor) java.lang.NullPointerException at kafka.network.Processor.run(SocketServer.scala:404)

Regarding question Kafka metrics. Issue with unclean leader election rate

This is wrt question Kafka metrics. Issue with unclean leader election rate (http://www.marshut.com/inkitk/kafka-metrics-issue-with-unclean-leader-election-rate.html) We use Kafka 0.8.0

Kafka metrics. Issue with unclean leader election rate

Yes we use 0.8.0 release