[ 
https://issues.apache.org/jira/browse/NIFI-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18147522#comment-18147522
 ] 

ASF subversion and git services commented on NIFI-16605:
--------------------------------------------------------

Commit ffa7504314c2f07e3a0de2a7899fd23d0c1c41a0 in nifi's branch 
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=ffa7504314c ]

NIFI-16605 Add per-event end attributes to ConsumeAzureEventHub (#11944)

> Add per-event end position attributes to ConsumeAzureEventHub
> -------------------------------------------------------------
>
>                 Key: NIFI-16605
>                 URL: https://issues.apache.org/jira/browse/NIFI-16605
>             Project: Apache NiFi
>          Issue Type: Improvement
>          Components: Extensions
>            Reporter: David Handermann
>            Assignee: David Handermann
>            Priority: Major
>          Time Spent: 50m
>  Remaining Estimate: 0h
>
> ConsumeAzureEventHub currently writes {{eventhub.enqueued.timestamp}}, 
> {{eventhub.offset}}, and {{eventhub.sequence}} from Azure 
> {{LastEnqueuedEventProperties}}, which describe the newest event in the 
> partition rather than the consumed event. This change should keep those 
> existing attributes unchanged and add 
> {{eventhub.event.end.enqueued.timestamp}}, {{eventhub.event.end.offset}}, and 
> {{eventhub.event.end.sequence}} from EventData. For one FlowFile per event, 
> those names come from that event. For a record-oriented batch, they come from 
> the last successfully parsed event in the batch. Parse-failure FlowFiles 
> should receive a dedicated attribute map so they do not share state with the 
> combined success FlowFile.
> The processor documentation already describes the current names as applying 
> to the consumed message, and GetAzureEventHub already writes per-event 
> positions, so ConsumeAzureEventHub has no stable way to expose an event’s own 
> sequence, offset, or enqueue time. Using the {{eventhub.event.end}} prefix 
> for both output modes keeps a single attribute contract, preserves existing 
> flows that depend on last-enqueued partition values, and provides the last 
> consumed event’s position for ordering, correlation, and checkpointing.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to