[
https://issues.apache.org/jira/browse/NIFI-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221832#comment-17221832
]
ASF subversion and git services commented on NIFI-7953:
-------------------------------------------------------
Commit c610aab3cb017d7030381f2715de446923870966 in nifi's branch
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=c610aab ]
NIFI-7953: Updated
ConsumeKafka_2_0/ConsumeKafkaRecord_2_0/ConsumeKafka_2_6/ConsumeKafkaRecord_2_6
to allow separating records by key
> Allow ConsumeKafka_2_* / ConsumeKafkaRecord_2_* to separate Records based on
> Message Key
> ----------------------------------------------------------------------------------------
>
> Key: NIFI-7953
> URL: https://issues.apache.org/jira/browse/NIFI-7953
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The Kafka consumer processors ignore the Message Key when using a demarcator
> or when using the Record variant, because adding a FlowFile attribute for the
> message key doesn't make sense, since the key is likely different per record.
> However, a fairly common use case is to use the Key only for partitioning and
> many Kafka Messages may exist for the same key. For example, this is commonly
> used for CDC type of use cases where the key may just be a table name.
> For such cases, it would be helpful to allow the consumer processors to avoid
> adding two messages to the same FlowFile if their keys are different. I.e.,
> all records in a given FlowFile will have the same key. The key can then be
> added as a FlowFile attribute.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
