[
https://issues.apache.org/jira/browse/KAFKA-13183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahesh Sambaram reassigned KAFKA-13183:
---------------------------------------
Assignee: Mahesh Sambaram
> Dropping nul key/value records upstream to repartiton topic not tracked via
> metrics
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-13183
> URL: https://issues.apache.org/jira/browse/KAFKA-13183
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: Mahesh Sambaram
> Priority: Major
>
> For joins and aggregation, we consider records with null key or value as
> invalid, and drop them. Inside the aggregate and join processors, we record
> dropped record with a corresponding metric (cf `droppedRecrodSensor`).
> However, we also apply an upstream optimization if we need to repartition
> data. As we know that the downstream aggregation / join will drop those
> records anyway, we drop them _before_ we write them into the repartition
> topic (we still need the drop logic in the processor for the case we don't
> have a repartition topic).
> We add a `KStreamFilter` (cf `KStreamImpl#createRepartiitonSource()`)
> upstream but this filter does not update the corresponding metric to record
> dropped records.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
