[
https://issues.apache.org/jira/browse/KAFKA-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15236121#comment-15236121
]
Greg Fodor edited comment on KAFKA-3543 at 4/11/16 10:40 PM:
-------------------------------------------------------------
Also note that in my case flatMap() is not sufficient for my needs, since I
need to able to use state stores.
was (Author: gfodor):
Also note that in my case the iflatMap() is not sufficient for my needs, since
I need to able to use state stores.
> Allow a variant of transform() which can emit multiple values
> -------------------------------------------------------------
>
> Key: KAFKA-3543
> URL: https://issues.apache.org/jira/browse/KAFKA-3543
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Affects Versions: 0.10.0.0
> Reporter: Greg Fodor
> Assignee: Guozhang Wang
>
> Right now it seems that if you want to apply an arbitrary transformation to a
> stream, you either have to use a TransformerSupplier or ProcessorSupplier
> sent to transform() or process(). The custom processor will allow you to emit
> multiple new values, but the process() method currently terminates that
> branch of the topology so you can't apply additional data flow. transform()
> lets you continue the data flow, but forces you to emit a single value for
> every input value.
> (It actually doesn't quite force you to do this, since you can hold onto the
> ProcessorContext and emit multiple, but that's probably not the ideal way to
> do it :))
> It seems desirable to somehow allow a transformation that emits multiple
> values per input value. I'm not sure of the best way to factor this inside of
> the current TransformerSupplier/Transformer architecture in a way that is
> clean and efficient -- currently I'm doing the workaround above of just
> calling forward() myself on the context and actually emitting dummy values
> which are filtered out downstream.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
