OffsetOutOfRangeException will be returned when the requested partition's
offset range is [a, b] and the requested offset is either < a or > b; the
offset range will be change whenever:
1. new messages appended to the log, which increments a;
2. old messages get cleaned based on the log retention (controlled by
log.retention.{minutes,hours} and log.retention.bytes).
The pagecache here should not play role as the [a, b] values are kept at
broker's memory and the exception will only be thrown based on these two
values, not the data on disk.
Guozhang
On Tue, Dec 2, 2014 at 6:30 PM, YuanJia Li <yuanjia8...@163.com> wrote:
> Hi Guozhang,
> My kafka works in product environment, a large messages are produced
> or consumed. So it is not easy to get the accurate offset through the
> GetOffset tool when an OffsetOutOfRangeException happens.But in my
> application, I have coded comparing the consuming offset with the latest
> offset got by SimpleConsumer.getOffsetsBefore API.
> I don't quite understand the kernel pagecache, so I wonder the delay
> between writting and reading.
>
> Thanks,
> Yuanjia
>
>
>
>
>
> From: Guozhang Wang
> Date: 2014-12-03 03:23
> To: users@kafka.apache.org
> Subject: Re: Pagecache cause OffsetOutOfRangeException
> Yuanjia,
>
> I am not sure that pagecache can be the cause of this, could you attach
> your full stack trace and use the GetOffset tool Manikumar mentioned to
> make sure the offset does exist in the broker?
>
> Guozhang
>
> On Tue, Dec 2, 2014 at 7:50 AM, Manikumar Reddy <ku...@nmsworks.co.in>
> wrote:
>
> > You can check the latest/earliest offsets of a given topic by running
> > GetOffsetShell.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-GetOffsetShell
> >
> > On Tue, Dec 2, 2014 at 2:05 PM, yuanjia8947 <yuanjia8...@163.com> wrote:
> >
> > > Hi all,
> > > I'm using kafka 0.8.0 release now. And I often encounter the problem
> > > OffsetOutOfRangeException when cosuming message by simple consumer API.
> > > But I'm sure that the consuming offset is smaller than the latest
> offset
> > > got from OffsetRequest.
> > > Can it be caused by that new messages are wrote to kernel's pagecache
> and
> > > not flush to the file yet,
> > > while I'm consuming new messages from the file?
> > > How fix it?
> > >
> > > Thanks,
> > > liyuanjia
> > >
> > >
> > >
> > >
> > >
> > > liyuanjia
> >
>
>
>
> --
> -- Guozhang
>
--
-- Guozhang
