[
https://issues.apache.org/jira/browse/KAFKA-2500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739223#comment-14739223
]
Jason Gustafson commented on KAFKA-2500:
----------------------------------------
[~willf] If I understand correctly, the basic requirement for a consistent
snapshot on a compacted topic is to read beyond the current cleaner point.
Since the high watermark will always be beyond that, it's just as good to read
up to there. Is that about right?
[~aauradkar] I talked with [~jkreps] briefly about it and we felt a little
inclined to push it to the next release. I think we mainly want to make sure
that enough time has been given to think through the API before committing
ourselves to supporting it. What do you think? Also, feel free to assign this
issue to someone on your team if you already have a good idea about it.
> Make logEndOffset available in the 0.8.3 Consumer
> -------------------------------------------------
>
> Key: KAFKA-2500
> URL: https://issues.apache.org/jira/browse/KAFKA-2500
> Project: Kafka
> Issue Type: Sub-task
> Components: consumer
> Affects Versions: 0.8.3
> Reporter: Will Funnell
> Assignee: Jason Gustafson
> Priority: Critical
> Fix For: 0.8.3
>
>
> Originally created in the old consumer here:
> https://issues.apache.org/jira/browse/KAFKA-1977
> The requirement is to create a snapshot from the Kafka topic but NOT do
> continual reads after that point. For example you might be creating a backup
> of the data to a file.
> This ticket covers the addition of the functionality to the new consumer.
> In order to achieve that, a recommended solution by Joel Koshy and Jay Kreps
> was to expose the high watermark, as maxEndOffset, from the FetchResponse
> object through to each MessageAndMetadata object in order to be aware when
> the consumer has reached the end of each partition.
> The submitted patch achieves this by adding the maxEndOffset to the
> PartitionTopicInfo, which is updated when a new message arrives in the
> ConsumerFetcherThread and then exposed in MessageAndMetadata.
> See here for discussion:
> http://search-hadoop.com/m/4TaT4TpJy71
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
