Chris,
The last offset can be calculated by adding the file size to the name of
the last Kafka segment file. Then you can see if your offset is in the
range.
Thanks,
Jun
On Wed, Mar 13, 2013 at 2:53 PM, Christopher Alexander <
calexan...@gravycard.com> wrote:
> Thanks for the reply Phillip. I
Sadly, I don't have access to those logs anymore, I don't have access to
environment. Though I remember seeing some exception during offset writing,
most probably due to zookeeper connection issue.
What would be side effects of not being able to write the consumer offset,
beside seeing this except
If there is no partitions (only one partition), then two consumers exist.
Is it possible only one consumer to get data from kafka(two broker)?
I can`t understand ...
-Original Message-
From: Jun Rao [mailto:jun...@gmail.com]
Sent: Tuesday, December 18, 2012 1:39 AM
To: users@kafka.apach
OK, it depends. What that tells you is that offset 0 is available,
since those files are named after the very first offset available in
the file. However, we cannot be sure if 562949953452239 is there. It's
kind of a large number so I doubt it, but you never know. What size is
that file?
If you ha
Thanks for the reply Phillip. I am new to kafka so please bear with me if I say
something that's "noobish".
I am running in a localhost configuration for testing. If I checkout kafka logs:
> cd /tmp/kafka-logs
> ls mytopic-0 # my topic is present
> cd mytopic-0
> ls .kafka
>
Is offset 562949953452239, partition 0, actually available on the
Kafka broker? Have you checked?
Philip
On Wed, Mar 13, 2013 at 1:53 PM, Christopher Alexander
wrote:
> Hello All,
>
> I am using Node-Kafka to connect to Kafka 0.7.2. Things were working fine but
> we are experiencing repeated ex
Hello All,
I am using Node-Kafka to connect to Kafka 0.7.2. Things were working fine but
we are experiencing repeated exceptions of the following:
[2013-03-13 16:45:14,615] ERROR error when processing request
FetchRequest(topic:promoterregack, part:0 offset:562949953452239
maxSize:1048576) (ka
+1 works every time. We provided a nextOffset() API for convenience just in
case this changes in the future.
Thanks,
Neha
On Wed, Mar 13, 2013 at 12:42 PM, Chris Curtin wrote:
> Thanks Neha,
>
> I some how missed the 'nextOffset' when converting my logic. I'm assuming
> the +1 trick works by ch
Thanks Neha,
I some how missed the 'nextOffset' when converting my logic. I'm assuming
the +1 trick works by chance and I shouldn't assume the next Offset is +1?
(It is minor for me to fix, I'm just curious where +1 might not work.)
Thanks,
Chris
On Wed, Mar 13, 2013 at 3:14 PM, Neha Narkhede
In 0.8, the iterator over the data returned in the FetchResponse is over
MessageAndOffset. This class has a nextOffset() API, which is the offset of
the next message in the message set. So, the nextOffset() value returned on
the last message in the message should be used as the fetch offset in the
Hi,
I noticed the same thing. In 0.8.0 the offset passed to the fetch is where
you want to start, not where you left off. So the last offset read from the
previous batch is truly the 'last offset' so you need to save it and ask
for it +1. Otherwise you keep asking for that last offset, which is va
If you provide an empty list in the getMetaData api, you get the metadata
for all topics.
Thanks,
Jun
On Wed, Mar 13, 2013 at 12:38 AM, Snehalata Nagaje <
snehalata.nag...@harbingergroup.com> wrote:
> Hi,
>
>
>
> I am using kafka 0.8 version.
>
>
>
> I want to know the list of topics created in
Where are the brokers installed, on 1,2,3? The load on ZK is light, but ZK
clients are latency sensitive. So it's better to put ZK on different
servers from the brokers or at least put ZK data on different disks if ZK
shares servers with brokers.
Thanks,
Jun
On Wed, Mar 13, 2013 at 1:17 AM, Arj
I'm curious the use case, but I think you'll need to use a combination of
ZooKeeper meta data and the TopicMetadata API.
The list of known topics for the cluster is in ZooKeeper:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper
Also in ZooKeeper is the list of
Hi,
I have 6 systems A,B,C,1,2,3.
Kafka is installed in A(producer),B(producer),C(consumer) and they are
configured.
Now 1,2,3 have zookeeper installed in all and these three manage the kafka
cluster.
Is this the right way to cluster Kafka?
Am i right about the logic used? If not please explain m
Hi,
I am using kafka 0.8 version.
I want to know the list of topics created in kafka server along with it's
metadata.
Is there any API available to find out this?
There is API to fetch TopicMetadata, but this needs name of topic as input
parameters.
I need information for all to
16 matches
Mail list logo
