Hi, Jun The getOffsetBefore API only return the offset array, i can't find anywhere to get the error code.
Thanks. On Sat, Dec 13, 2014 at 1:02 AM, Jun Rao <j...@confluent.io> wrote: > > Yes, for regular consumers, we always return the high watermark as the > latest offset. Did you check the error code in the getOffsetBefore request? > > Thanks, > > Jun > > On Tue, Dec 9, 2014 at 12:31 AM, 向河林 <xian...@mediav.com> wrote: > > > Thanks Guozhang， > > > > I checked the server log, and I am 90% sure that no leader movement > > happened at that moment. > > > > Could it be that the data was in the memory but yet not flushed to the > > disk? > > I read the doc in the wiki > > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Replication, > > and said > > > > The high watermark (HW) is the offset of the last committed message. > > Each log is periodically synced to disks. Data before the flushed > > offset is guaranteed to be persisted on disks. As we will see, the > > flush offset can be before or after HW. > > > > since the https://issues.apache.org/jira/browse/KAFKA-501 said > > > > For regular consumers, getOffset should return highWatermark as the > > latest offset. > > > > Thanks again. > > > > > > On Tue, Dec 9, 2014 at 5:28 AM, Guozhang Wang <wangg...@gmail.com> > wrote: > > > > > Helin, > > > > > > Is there a leader movement just before the get latest offset call? If > > your > > > follower is not synced and it then becomes the leader due to some > reason, > > > it will not have the complete partition data. > > > > > > Guozhang > > > > > > On Mon, Dec 8, 2014 at 3:02 AM, Helin Xiang <xkee...@gmail.com> wrote: > > > > > > > 1 additional information we found in the kafka’s application log > since > > > the > > > > MAGIC time: > > > > > > > > 2014-12-04 09:59:36,726 [kafka-scheduler-2] INFO > > > > kafka.cluster.Partition - Partition [a.s.3,26] on broker 5: > Shrinking > > > > ISR for partition [a.s.3,26] from 5,4 to 5 > > > > 2014-12-04 09:59:36,728 [kafka-scheduler-2] ERROR > kafka.utils.ZkUtils$ > > > > - Conditional update of path > > > > /brokers/topics/a.s.3/partitions/26/state with data > > > > > > {"controller_epoch":2,"leader":5,"version":1,"leader_epoch":4,"isr":[5]} > > > > and expected version 675 failed due to > > > > org.apache.zookeeper.KeeperException$BadVersionException: > > > > KeeperErrorCode = BadVersion for > > > > /brokers/topics/a.s.3/partitions/26/state > > > > > > > > > > > > > > > > On Mon, Dec 8, 2014 at 6:59 PM, Helin Xiang <xkee...@gmail.com> > wrote: > > > > > > > > > Hi, > > > > > > > > > > We have currently upgraded our kafka cluster from 0.7.2 to 0.8.1.1. > > > > > > > > > > In one of our application, we want to get all partitions' latest > > > offsets, > > > > > so we use getoffsetbefore java API (latest). > > > > > > > > > > We believe at some time, 1 of the partition's latest offset we got > is > > > > much > > > > > smaller than its real latest offset,(we saw in the application's > log > > > that > > > > > the partition's offset is much smaller than other partitions'). > Since > > > the > > > > > real data file of that partition was already deleted, we cannot > prove > > > our > > > > > guess, but we found some clue in the kafka's application log which > > > helps > > > > us > > > > > to conclude that the partition's latest offset at that moment did > > have > > > a > > > > > much larger number. > > > > > > > > > > some additional useful information: the partition have 1 additional > > > > > replica(follower), and at that time, it was not synced with the > > leader > > > > > partition(far away behind the leader). > > > > > > > > > > Does any one have the same issue? In what condition could lead to > > this > > > > > situation? > > > > > > > > > > Thanks. > > > > > > > > > > -- > > > > > > > > > > > > > > > *Best Regards向河林* > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > *Best Regards向河林* > > > > > > > > > > > > > > > > -- > > > -- Guozhang > > > > > > > > > > > -- > > > > *向河林* > > > > > > *MV AD **聚效广告* *上海* · 北京 · 广州 · 杭州 > > _______________________________________________________________ > > > > 上海市闸北区天目中路585号新梅大厦4楼 200070 > > > > MOB：18621368491 > > > > TEL：021-52559088（分机：8133） > > > > EMAIL：xian...@mediav.com <ya...@mediav.com> > > > > HTTP：www.mediav.com > > > > > > -------------------------------CONFIDENTIAL > ------------------------------ > > ------------ > > > > 本邮件载有秘密信息，请您恪守保密义务，勿向第三人透露。谢谢合作。 > > > > This email communication is confidential. Recipient(s) named above > is(are) > > obligated to maintain secrecy and is(are) not permitted to disclose the > > contents of this communication to others. Thank you. > > > -- *Best Regards向河林*
