kevdoran commented on code in PR #11944:
URL: https://github.com/apache/nifi/pull/11944#discussion_r4621716919
##########
nifi-extension-bundles/nifi-azure-bundle/nifi-azure-processors/src/main/java/org/apache/nifi/processors/azure/eventhub/ConsumeAzureEventHub.java:
##########
@@ -122,12 +123,42 @@
"Cluster state is used to store partition ownership and checkpoint
information when component state is configured as the checkpointing strategy.")
@TriggerSerially
@WritesAttributes({
- @WritesAttribute(attribute = "eventhub.enqueued.timestamp",
description = "The time (in milliseconds since epoch, UTC) at which the message
was enqueued in the event hub"),
- @WritesAttribute(attribute = "eventhub.offset", description = "The
offset into the partition at which the message was stored"),
- @WritesAttribute(attribute = "eventhub.sequence", description = "The
sequence number associated with the message"),
- @WritesAttribute(attribute = "eventhub.name", description = "The name
of the event hub from which the message was pulled"),
- @WritesAttribute(attribute = "eventhub.partition", description = "The
name of the partition from which the message was pulled"),
- @WritesAttribute(attribute = "eventhub.property.*", description = "The
application properties of this message. IE: 'application' would be
'eventhub.property.application'")
+ @WritesAttribute(
+ attribute = "eventhub.enqueued.timestamp",
+ description = "ISO-8601 Instant string of the Enqueued
Timestamp for the last event in the partition when the events were received"
+ ),
+ @WritesAttribute(
+ attribute = "eventhub.offset",
+ description = "Offset of the last event in the partition when
the events were received"
+ ),
+ @WritesAttribute(
+ attribute = "eventhub.sequence",
+ description = "Sequence Number of the last event in the
partition when the events were received"
+ ),
+ @WritesAttribute(
+ attribute = "eventhub.event.end.enqueued.timestamp",
+ description = "ISO-8601 Instant string of the Enqueued
Timestamp for the last event received"
+ ),
+ @WritesAttribute(
+ attribute = "eventhub.event.end.offset",
+ description = "Offset of the last event received"
+ ),
+ @WritesAttribute(
+ attribute = "eventhub.event.end.sequence",
+ description = "Sequence Number of the last event received"
+ ),
Review Comment:
In record mode only the end position is exposed. Consumers that derive
per-record positions as first offset + record index can't get the first
position, and using `end.sequence` there makes later, smaller batches fall at
or below the committed offset, so they are skipped. Would a matching
`eventhub.event.start.*` set from the first written event be worth adding?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]