[
https://issues.apache.org/jira/browse/KAFKA-12453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318193#comment-17318193
]
Matthias J. Sax commented on KAFKA-12453:
-----------------------------------------
{quote} # Why does the input topic need to be configured with log
compaction?{quote}
If you enable topology optimization, and you use `builder.table()` the table's
state would be recovered from the _input_ topic. Thus, if you don't configure
the input topic with compaction but retention time, you might lose state.
It might actually make sense, to add a new section about topology optimization
instead of just updating
[https://kafka.apache.org/27/documentation/streams/developer-guide/config-streams.html#topology-optimization]
– the config section should briefly describe config IMHO – we can add a link
to the new section to the short description, though.
So maybe we should add a new `topology_optimization.html` page?
Similar for the JavaDocs. It might be too much content to add it all there?
> Guidance on whether a topology is eligible for optimisation
> -----------------------------------------------------------
>
> Key: KAFKA-12453
> URL: https://issues.apache.org/jira/browse/KAFKA-12453
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Patrick O'Keeffe
> Priority: Major
>
> Since the introduction of KStream.toTable() in Kafka 2.6.x, the decision
> about whether a topology is eligible for optimisation is no longer a simple
> one, and is related to whether toTable() operations are preceded by key
> changing operators.
> This decision requires expert level knowledge, and there are serious
> implications associated with getting it wrong in terms of fault tolerance
> Some ideas spring to mind around how to guide developers to make the correct
> decision:
> # Topology.describe() could indicate whether this topology is eligible for
> optimisation
> # Topologies could be automatically optimised - note this may have an impact
> at deployment time, in that an application reset may be required. The
> developer would need to made aware of this and adjust the deployment plan
> accordingly
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
