[ https://issues.apache.org/jira/browse/KAFKA-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812918#comment-15812918 ]
Matthias J. Sax commented on KAFKA-4125: ---------------------------------------- I think we need a KIP for this -- we did a lot of API changes/improvements recently and we got lot's of helpful feedback via KIP. (\cc [~guozhang] WDYT ?) Writing the KIP should not be a big deal in this case: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals Let me know if you need any help with it (matth...@confluent.io) and I can guide you through the process (in case we need a KIP -- let's see what [~guozhang] thinks). > Provide low-level Processor API meta data in DSL layer > ------------------------------------------------------ > > Key: KAFKA-4125 > URL: https://issues.apache.org/jira/browse/KAFKA-4125 > Project: Kafka > Issue Type: Sub-task > Components: streams > Reporter: Matthias J. Sax > Assignee: Guozhang Wang > Priority: Minor > > For Processor API, user can get meta data like record offset, timestamp etc > via the provided {{Context}} object. It might be useful to allow uses to > access this information in DSL layer, too. > The idea would be, to do it "the Flink way", ie, by providing > RichFunctions; {{mapValue()}} for example. > Is takes a {{ValueMapper<V1, V2>}} that only has method > {noformat} > V2 apply(V1 value); > {noformat} > Thus, you cannot get any meta data within apply (it's completely "blind"). > We would add two more interfaces: {{RichFunction}} with a method > {{open(Context context)}} and > {noformat} > RichValueMapper<V1, V2> extends ValueMapper<V1, V2>, RichFunction > {noformat} > This way, the user can chose to implement Rich- or Standard-function and > we do not need to change existing APIs. Both can be handed into > {{KStream.mapValues()}} for example. Internally, we check if a Rich > function is provided, and if yes, hand in the {{Context}} object once, to > make it available to the user who can now access it within {{apply()}} -- or > course, the user must set a member variable in {{open()}} to hold the > reference to the Context object. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
