Hi Dong, Currently we are storing historical timestamp in the message.
What we are trying to achieve is to make it possible to do Kafka lookup by timestamp. Ideally I would do `offsetsForTimes` to find articles published in 1910s (if we are storing articles on the log). So first two suggestions aren't really covering our use-case. We could create a new timestamp type like "HistoricalTimestamp" or "MaybeNegativeTimestamp". And the only difference between this one and CreateTime is that it could be negative. I tend to use CreateTime for this purpose because it's easier to understand from user perspective as a timestamp which publisher can set. Thanks, Konstantin > On Dec 5, 2017, at 3:47 PM, Dong Lin <lindon...@gmail.com> wrote: > > Hey Konstantin, > > Thanks for the KIP. I have a few questions below. > > Strictly speaking Kafka actually allows you to store historical data. And > user are free to encode arbitrary timestamp field in their Kafka message. > For example, your Kafka message can currently have Json or Avro format and > you can put a timestamp field there. Do you think that could address your > use-case? > > Alternatively, KIP-82 introduced Record Header in Kafka and you can also > define your customized key/value pair in the header. Do you think this can > address your use-case? > > Also, currently there are two types of timestamp according to KIP-32. If > the type is LogAppendTime then the timestamp value is the time when broker > receives the message. If the type is CreateTime then the timestamp value is > determined when producer produces message. With these two definitions, the > timestamp should always be positive. We probably need a new type here if we > can not put timestamp in the Record Header or the message payload. Does > this sound reasonable? > > Thanks, > Dong > > > > On Tue, Dec 5, 2017 at 8:40 AM, Konstantin Chukhlomin <chuhlo...@gmail.com> > wrote: > >> Hi all, >> >> I have created a KIP to support negative timestamp: >> https://cwiki.apache.org/confluence/display/KAFKA/KIP- >> 228+Negative+record+timestamp+support <https://cwiki.apache.org/ >> confluence/display/KAFKA/KIP-228+Negative+record+timestamp+support> >> >> Here are proposed changes: https://github.com/apache/ >> kafka/compare/trunk...chuhlomin:trunk <https://github.com/apache/ >> kafka/compare/trunk...chuhlomin:trunk> >> >> I'm pretty sure that not cases are covered, so comments and suggestions >> are welcome. >> >> Thank you, >> Konstantin
