Hi Guozhang,
My kafka works in product environment, a large messages are produced or
consumed. So it is not easy to get the accurate offset through the GetOffset
tool when an OffsetOutOfRangeException happens.But in my application, I have
coded comparing the consuming offset with the latest offset got by
SimpleConsumer.getOffsetsBefore API.
I don't quite understand the kernel pagecache, so I wonder the delay
between writting and reading.
Thanks,
Yuanjia
From: Guozhang Wang
Date: 2014-12-03 03:23
To: [email protected]
Subject: Re: Pagecache cause OffsetOutOfRangeException
Yuanjia,
I am not sure that pagecache can be the cause of this, could you attach
your full stack trace and use the GetOffset tool Manikumar mentioned to
make sure the offset does exist in the broker?
Guozhang
On Tue, Dec 2, 2014 at 7:50 AM, Manikumar Reddy <[email protected]>
wrote:
> You can check the latest/earliest offsets of a given topic by running
> GetOffsetShell.
>
>
> https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-GetOffsetShell
>
> On Tue, Dec 2, 2014 at 2:05 PM, yuanjia8947 <[email protected]> wrote:
>
> > Hi all,
> > I'm using kafka 0.8.0 release now. And I often encounter the problem
> > OffsetOutOfRangeException when cosuming message by simple consumer API.
> > But I'm sure that the consuming offset is smaller than the latest offset
> > got from OffsetRequest.
> > Can it be caused by that new messages are wrote to kernel's pagecache and
> > not flush to the file yet,
> > while I'm consuming new messages from the file?
> > How fix it?
> >
> > Thanks,
> > liyuanjia
> >
> >
> >
> >
> >
> > liyuanjia
>
--
-- Guozhang
