[
https://issues.apache.org/jira/browse/KAFKA-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823413#comment-15823413
]
Ewen Cheslack-Postava commented on KAFKA-3522:
----------------------------------------------
[~imandhan] There's a KIP proposal for exactly once semantics, so now might be
a good time to restart this discussion. Although I'm not clear how they
interact since this seems tied to local state that can easily be rebuilt,
perhaps [~guozhang] can expand on his thoughts. Regardless, I dropped priority
here because the proposed solution is easily added incrementally by first
introducing the extra on-disk info (in any upcoming version) and then adding
support for a newer RocksDB format in a later version and making the upgrade
path required clear to users via release notes (and possibly configs, depending
on what support for different versions looked like in RocksDB).
> Consider adding version information into rocksDB storage format
> ---------------------------------------------------------------
>
> Key: KAFKA-3522
> URL: https://issues.apache.org/jira/browse/KAFKA-3522
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Guozhang Wang
> Assignee: Ishita Mandhan
> Labels: architecture
> Fix For: 0.10.2.0
>
>
> Kafka Streams does not introduce any modifications to the data format in the
> underlying Kafka protocol, but it does use RocksDB for persistent state
> storage, and currently its data format is fixed and hard-coded. We want to
> consider the evolution path in the future we we change the data format, and
> hence having some version info stored along with the storage file / directory
> would be useful.
> And this information could be even out of the storage file; for example, we
> can just use a small "version indicator" file in the rocksdb directory for
> this purposes. Thoughts? [~enothereska] [~jkreps]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
