[
https://issues.apache.org/jira/browse/KAFKA-9527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277626#comment-17277626
]
Matthias J. Sax commented on KAFKA-9527:
----------------------------------------
I guess the problem is, that we don't have any timestamp information from the
time index, and thus we cannot get the offset we want to commit. – Silently
ignoring seems the worst option from the three you suggested, because in the
end, we don't know where the application will effectively start to consume –
either there is an exiting committed offset or we might fall back to
`auto.offset.reset`. Note, that we cannot make any assumption what the time gap
between resetting and restarting is, and thus, new data might be written into
the topic in the meantime. Thus, I think we should at least inform the user
that no offset was committed for some partitions (and report the corresponding
partitions), thus, option 3 might be the best? Option 1 would also be ok IMHO.
With one is better seems to depend how "robust" the tool should be (don't have
strong opinion about it). The point is that both options allow the user to
react without the danger of unexpected behavior on restart of the application.
Btw: I would assume that this issue not only affects the "stream reset tool"
but also the "consumer group" tool that offset a similar feature (cf. KIP-122
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-122%3A+Add+Reset+Consumer+Group+Offsets+tooling]
and KIP-171
[https://cwiki.apache.org/confluence/display/KAFKA/KIP-171+-+Extend+Consumer+Group+Reset+Offset+for+Stream+Application])
Not sure what behavior [~jbfletch] had in mind? Maybe [~jeqo] (who proposed the
feature originally) has some input?
Btw: the ticket is assigned to [~jbfletch] so she should first agree to hand it
off to you before you start working on it.
> Application Reset Tool Returns NPE when --to-timestamp or --by-duration are
> run on --input-topics with empty partitions
> ------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-9527
> URL: https://issues.apache.org/jira/browse/KAFKA-9527
> Project: Kafka
> Issue Type: Bug
> Components: streams, tools
> Affects Versions: 2.3.0
> Reporter: jbfletch
> Assignee: jbfletch
> Priority: Minor
>
> When running the streams application reset tool with --by-duration or
> --to-timestamp if any partitions for a given input topic are empty a NPE is
> thrown. I tested this with a topic with 3 partitions, I received a NPE until
> all 3 partitions had at least one message. The behavior was the same for
> both --to-timestamp and --by-duration.
> Error below:
> Reset-offsets for input topics [sample-cdc-topic]Reset-offsets for input
> topics [sample-cdc-topic]Following input topics offsets will be reset to (for
> consumer group des-demo-stream)ERROR:
> java.lang.NullPointerExceptionjava.lang.NullPointerException at
> kafka.tools.StreamsResetter.resetToDatetime(StreamsResetter.java:496) at
> kafka.tools.StreamsResetter.maybeReset(StreamsResetter.java:426) at
> kafka.tools.StreamsResetter.maybeResetInputAndSeekToEndIntermediateTopicOffsets(StreamsResetter.java:374)
> at kafka.tools.StreamsResetter.run(StreamsResetter.java:164) at
> kafka.tools.StreamsResetter.run(StreamsResetter.java:131) at
> kafka.tools.StreamsResetter.main(StreamsResetter.java:678)
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
