[
https://issues.apache.org/jira/browse/KAFKA-4785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-4785:
-----------------------------------
Description:
Users can specify what timestamp extractor should be used to decode the
timestamp of input topic records. As long as RecordMetadataTimestamp or
WallclockTime is use this is fine.
However, for custom timestamp extractors it might be invalid to apply this
custom extractor to records received from internal repartitioning topics. The
reason is that Streams sets the current "stream time" as record metadata
timestamp explicitly before writing to intermediate repartitioning topics
because this timestamp should be use by downstream subtopologies. A custom
timestamp extractor might return something different breaking this assumption.
Thus, for reading data from intermediate repartitioning topic, the configured
timestamp extractor should not be used, but the record's metadata timestamp
should be extracted as record timestamp.
was:
Users can specify what timestamp extractor should be used to decode the
timestamp of input topic records. As long as RecordMetadataTimestamp or
WallclockTime is use this is fine.
However, for custom timestamp extractors it might be invalid to apply this
custom extractor to records received from internal repartitioning topics. The
reason is that Streams sets the current "stream time" as record metadata
timestamp explicitly before writing to intermediate repartitioning topics
because this timestamp should be use by downstream subtopologies. A custom
timestamp extractor might return something different breaking this assumption.
> Records from internal repartitioning topics should always use
> RecordMetadataTimestampExtractor
> ----------------------------------------------------------------------------------------------
>
> Key: KAFKA-4785
> URL: https://issues.apache.org/jira/browse/KAFKA-4785
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 0.10.2.0
> Reporter: Matthias J. Sax
>
> Users can specify what timestamp extractor should be used to decode the
> timestamp of input topic records. As long as RecordMetadataTimestamp or
> WallclockTime is use this is fine.
> However, for custom timestamp extractors it might be invalid to apply this
> custom extractor to records received from internal repartitioning topics. The
> reason is that Streams sets the current "stream time" as record metadata
> timestamp explicitly before writing to intermediate repartitioning topics
> because this timestamp should be use by downstream subtopologies. A custom
> timestamp extractor might return something different breaking this assumption.
> Thus, for reading data from intermediate repartitioning topic, the configured
> timestamp extractor should not be used, but the record's metadata timestamp
> should be extracted as record timestamp.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
