[
https://issues.apache.org/jira/browse/CALCITE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16863628#comment-16863628
]
Andrei Sereda commented on CALCITE-3073:
----------------------------------------
I think it is similar issue to {{_id}} column in elastic. See discussion on
[dev
list|https://lists.apache.org/thread.html/91ac1a5215e22afa9abee924426e05cfbcf257aae6858741308ed7b8@%3Cdev.calcite.apache.org%3E].
Can this be available only if requested explicitly ?
> Support to consume from timestamp in KafkaAdapter
> -------------------------------------------------
>
> Key: CALCITE-3073
> URL: https://issues.apache.org/jira/browse/CALCITE-3073
> Project: Calcite
> Issue Type: Improvement
> Components: kafka-adapter
> Reporter: Xu Mingmin
> Assignee: Matt Wang
> Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> Currently the KafkaAdapter consumes data from default
> offset(latest/earliest/last_offset) and runs forever.
> In other words, if the app runs at first time and user want to consume the
> past data, user must set the value of
> '{color:#707070}*auto.offset.reset'*{color} parameter to *earliest*.
> {quote}{{auto.offset.reset：What to do when there is no initial offset in
> Kafka or if the current offset does not exist any more on the server (e.g.
> because that data has been deleted):}}
> * earliest: automatically reset the offset to the earliest offset
> * latest: automatically reset the offset to the latest offset
> * none: throw exception to the consumer if no previous offset is found for
> the consumer's group
> * anything else: throw exception to the consumer.{quote}
> for example, suppose data in Kafka is retained for 7 days and you just want
> to read from the data of yesterday, if you could not control the start
> timestamp, you can only read from the earliest offset, it's very inefficient.
> If supporting to consume from special timestamp in KafkaAdapter will be a
> good idea for some cases.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
