I think you are just missing the —execute flag.
-hans
> On Oct 25, 2017, at 1:24 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> I wonder if you have hit KAFKA-5600.
>
> Is it possible that you try out 0.11.0.1 ?
>
> Thanks
>
>> On Wed, Oct 25, 2017 at 1:15 PM, Dan Markhasin <minimi...@gmail.com> wrote:
>>
>> I am using 0.11.0.0.
>>
>> There is no difference configuration-wise - both have 10 partitions and 2
>> replicas. There are no errors in the logs, but looking in the data folder
>> it seems like Kafka is not updating the timeindex file for data1_log -
>> notice how the timeindex file for the current log segment is not being
>> updated.
>>
>> bash-4.2$ pwd
>> /kafka/data/data1_log-1
>> bash-4.2$ ls -ltr | tail
>> -rw-rw-r-- 1 ibiuser it 1073731573 Oct 25 01:21 00000000000337554984.log
>> -rw-rw-r-- 1 ibiuser it 943616 Oct 25 01:21 00000000000337554984.index
>> -rw-rw-r-- 1 ibiuser it 1073734199 Oct 25 13:38 00000000000339816017.log
>> -rw-rw-r-- 1 ibiuser it 10485756 Oct 25 13:38
>> 00000000000341934289.timeindex
>> -rw-rw-r-- 1 ibiuser it 10 Oct 25 13:38
>> 00000000000341934289.snapshot
>> -rw-rw-r-- 1 ibiuser it 0 Oct 25 13:38
>> 00000000000339816017.timeindex
>> -rw-rw-r-- 1 ibiuser it 566712 Oct 25 13:38 00000000000339816017.index
>> -rw-rw-r-- 1 ibiuser it 17 Oct 25 20:23 leader-epoch-checkpoint
>> -rw-rw-r-- 1 ibiuser it 10485760 Oct 25 23:03 00000000000341934289.index
>> -rw-rw-r-- 1 ibiuser it 461590419 Oct 25 23:04 00000000000341934289.log
>>
>> For comparison, the beats topic:
>>
>> bash-4.2$ cd ../beats-1
>> bash-4.2$ ls -ltr
>> total 3212088
>> -rw-rw-r-- 1 ibiuser it 17 Oct 25 00:23 leader-epoch-checkpoint
>> -rw-rw-r-- 1 ibiuser it 10 Oct 25 20:04
>> 00000000000188672034.snapshot
>> -rw-rw-r-- 1 ibiuser it 2773008 Oct 25 20:04
>> 00000000000185224087.timeindex
>> -rw-rw-r-- 1 ibiuser it 1073741779 Oct 25 20:04 00000000000185224087.log
>> -rw-rw-r-- 1 ibiuser it 1967440 Oct 25 20:04 00000000000185224087.index
>> -rw-rw-r-- 1 ibiuser it 10485760 Oct 25 23:03 00000000000188672034.index
>> -rw-rw-r-- 1 ibiuser it 10485756 Oct 25 23:04
>> 00000000000188672034.timeindex
>> -rw-rw-r-- 1 ibiuser it 50166645 Oct 25 23:04 00000000000188672034.log
>>
>>
>> To give some context to why I'm even trying to reset the offsets, we had
>> encountered a strange situation earlier today:
>>
>> 1) One of the brokers had a hardware failure, and had to be rebuilt from
>> scratch (data partition was gone)
>> 2) When it went down, we noticed a spike in lag in one particular consumer
>> group - it seems to have reset its offset to an earlier point in time (but
>> not the earliest offset of the topic); I have read other messages on this
>> mailing list of users who experienced the same behavior with 0.11.0.0
>> 3) The broker was reinstalled and rejoined the cluster with the same
>> broker.id (but with no data on it) - it rebalanced and eventually all
>> replicas became synced and the cluster was functioning normally.
>> 4) I then decided to bounce the same broker again to see if I can reproduce
>> the issue I saw in #2 - and as soon as the broker was restarted, the exact
>> same consumer group had its offset reset again and was lagging with
>> millions of records behind the current offset.
>> 5) I then tried to manually reset the consumer group's offset to a few
>> minutes before I restarted the broker, only to discover this strange
>> behavior where no matter which datetime value I provided, it kept resetting
>> to the latest offset.
>>
>>
>>> On 25 October 2017 at 22:48, Ted Yu <yuzhih...@gmail.com> wrote:
>>>
>>> Do you mind providing a bit more information ?
>>>
>>> Release of Kafka you use
>>>
>>> Any difference between data1_log and the other, normal topic ?
>>>
>>> Probably check the broker log where data1_log is hosted - see if there is
>>> some clue.
>>>
>>> Thanks
>>>
>>> On Wed, Oct 25, 2017 at 12:11 PM, Dan Markhasin <minimi...@gmail.com>
>>> wrote:
>>>
>>>> I'm trying to use the kafka-consumer-groups.sh tool in order to rewind
>> a
>>>> consumer group's offset, however it seems to be returning the latest
>>> offset
>>>> regarding of the requested offset.
>>>>
>>>> You can see in the below example that two consecutive commands to reset
>>> the
>>>> offset to a specific point in time return different (increasing)
>> offsets,
>>>> which are actually the latest offsets for the topic.
>>>>
>>>> - The consumer group ("test_consumer") is a console consumer that was
>>>> started with --from-beginning and terminated after a few seconds, just
>>>> enough for it to commit its offsets.
>>>> - The topic data1_log is very busy with thousands of incoming messages
>>> per
>>>> second
>>>> - The datetime value provided is approx. 5 hours earlier than the
>> current
>>>> UTC time
>>>>
>>>> [admin@broker01] ~> /kafka/latest/bin/kafka-consumer-groups.sh
>>>> --bootstrap-server localhost:9092 --reset-offsets --group test_consumer
>>>> --topic data1_log --to-datetime '2017-10-25T13:40:00.000'
>>>> Note: This will only show information about consumers that use the Java
>>>> consumer API (non-ZooKeeper-based consumers).
>>>>
>>>>
>>>> TOPIC PARTITION NEW-OFFSET
>>>> data1_log 8 301485420
>>>> data1_log 1 342788637
>>>> data1_log 7 287621428
>>>> data1_log 3 268612266
>>>> data1_log 0 201860717
>>>> data1_log 9 202749553
>>>> data1_log 4 188974032
>>>> data1_log 6 234308481
>>>> data1_log 2 263507741
>>>> data1_log 5 232707238
>>>>
>>>> [admin@broker01] ~> /kafka/latest/bin/kafka-consumer-groups.sh
>>>> --bootstrap-server localhost:9092 --reset-offsets --group test_consumer
>>>> --topic data1_log --to-datetime '2017-10-25T13:40:00.000'
>>>> Note: This will only show information about consumers that use the Java
>>>> consumer API (non-ZooKeeper-based consumers).
>>>>
>>>>
>>>> TOPIC PARTITION NEW-OFFSET
>>>> data1_log 8 301485491
>>>> data1_log 1 342788779
>>>> data1_log 7 287621534
>>>> data1_log 3 268612364
>>>> data1_log 0 201860796
>>>> data1_log 9 202749620
>>>> data1_log 4 188974068
>>>> data1_log 6 234308564
>>>> data1_log 2 263507823
>>>> data1_log 5 232707293
>>>>
>>>> This issue seems to be topic-specific - there is a different topic
>> (also
>>>> very active) where the same command consistently returns the correct
>>>> offsets fixed in the time for the requested datetime.
>>>>
>>>> What could be the issue here?
>>>>
>>>> Thanks,
>>>> Dan
>>>>
>>>
>>
