[
https://issues.apache.org/jira/browse/KAFKA-20230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Wittwer updated KAFKA-20230:
------------------------------------
Description:
Prior to KAFKA-7663, one would be required to maintain a separate stream
processor or latent sub-topology to populate a source topic to be used by
addGlobalStore because the Processor provided in addGlobalStore was not used
during the restore phase, instead projecting the raw topic into memory.
Kafka-7663 made it such that the supplied Processor is reliably used not only
on initial ingestion, but also on the restore. This gives the benefit of being
able to re-key into a global state store for use in process() steps.
The DSL method globalTable() now lags behind addGlobalStore in this respect.
Proposal:
Allow for an overload of globalTable(...), that accepts an optional
KeyValueMapper to be provided which would override the default
ProcessorSupplier that is instantiated on the internalStreamsBuilder
[here|https://github.com/apache/kafka/blob/84f810e49b9eb1ca554b95059f6ab1c720d37b0a/streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/TableSourceNode.java#L95]
Proposal
was:
KAFKA-7663 introduced the ability for a addGlobalStore to provide a Processor
which is reliably used not only on initial ingestion, but also on the restore
thread. This gives a great benefit of being able to re-key into a global state
store for use in process() steps. The DSL method globalTable() now lags behind
addGlobalStore in this respect.
Proposal
> globaltable processor
> ---------------------
>
> Key: KAFKA-20230
> URL: https://issues.apache.org/jira/browse/KAFKA-20230
> Project: Kafka
> Issue Type: Improvement
> Reporter: Brandon Wittwer
> Priority: Major
>
> Prior to KAFKA-7663, one would be required to maintain a separate stream
> processor or latent sub-topology to populate a source topic to be used by
> addGlobalStore because the Processor provided in addGlobalStore was not used
> during the restore phase, instead projecting the raw topic into memory.
> Kafka-7663 made it such that the supplied Processor is reliably used not only
> on initial ingestion, but also on the restore. This gives the benefit of
> being able to re-key into a global state store for use in process() steps.
> The DSL method globalTable() now lags behind addGlobalStore in this respect.
>
> Proposal:
> Allow for an overload of globalTable(...), that accepts an optional
> KeyValueMapper to be provided which would override the default
> ProcessorSupplier that is instantiated on the internalStreamsBuilder
> [here|https://github.com/apache/kafka/blob/84f810e49b9eb1ca554b95059f6ab1c720d37b0a/streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/TableSourceNode.java#L95]
>
> Proposal
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
