Hello,
We are using kafka version 0.8.1 and the python kafka client.
Everything has been working fine and suddenly this morning I saw
a OffsetOutOfRange on one of the partitions. (We have 20 partitions in our
kafka cluster)
We fixed it by seeking to the head offset and restarting the app.
I dug deeper and saw this during normal operation:
In the kafka broker log:
[2014-11-03 21:39:25,658] ERROR [KafkaApi-8] Error when processing fetch
request for partition [activity.stream,5] offset 7475239 from consumer with
correlation id 69 (kafka.server.KafkaApis)
kafka.common.OffsetOutOfRange
I dug deeper and saw this during normal operation:
In the kafka broker log:
[2014-11-03 21:39:25,658] ERROR [KafkaApi-8] Error when processing fetch
request for partition [activity.stream,5] offset 7475239 from consumer with
correlation id 69 (kafka.server.KafkaApis)
kafka.common.OffsetOutOfRange
Hello,
I understand what this error means, just not sure why I keep running into
it after 24-48 hrs of running fine consuming > 300 messages / second.
What happens when a kafka log rolls over and some old records are aged
out? I mean what happens to the offsets? We are using a python client w
ting their offset to, e.g. the head of
> the log.
>
> How frequent do your clients read / write the offsets in ZK?
>
> Guozhang
>
> On Thu, Nov 6, 2014 at 6:23 PM, Jimmy John wrote:
>
> > Hello,
> >
> > I understand what this error means, just not sure wh
Livefyre (http://web.livefyre.com/) uses kafka for the real time
notifications, analytics pipeline and as the primary mechanism for general
pub/sub.
thx...
jim
On Sat, Nov 8, 2014 at 7:41 AM, Gwen Shapira wrote:
> Done!
>
> Thank you for using Kafka and letting us know :)
>
> On Sat, Nov 8, 201
How do I configure the application kafka log dir?
Right now the default is /var/log/upstart/kafka.log . I want to point it to
a different mount dir... e.g.
/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
--logdir /mnt/kafka/kafka-app-logs
But the above gives me errors :
im
On Mon, Nov 17, 2014 at 11:29 AM, Harsha wrote:
> you can configure it under /opt/kafka/config/log4j.properties and look
> for kafka.log.dir
>
> On Mon, Nov 17, 2014, at 11:11 AM, Jimmy John wrote:
> > How do I configure the application kafka log dir?
> >
> > Right
Looks like there is already a tkt for this:
https://issues.apache.org/jira/browse/KAFKA-1204
thx
jim
On Mon, Nov 17, 2014 at 1:59 PM, Jimmy John wrote:
> I tried that but it did not work. Dug a little deeper and saw this line
> in bin/kafka-run-class.sh:
>
> KAFKA_LOG4J_OPTS="
