Admaing opened a new pull request, #29414:
URL: https://github.com/apache/flink/pull/29414
## What is the purpose of the change
`SinkUpsertMaterializer`, `SinkUpsertMaterializerV2`,
`WatermarkCompactingSinkMaterializer` and `KeyedLookupJoinWrapper` only log
when a retraction finds no matching row (or when the state was already cleared
by TTL), so wrong results caused by non-deterministic updates or a too short
TTL are invisible in metrics. `LateralSnapshotJoinOperator` already exposes
`numUnmatchedBuildRetractions` for exactly this situation.
This adds the same counter to those four operators.
## Brief change log
- `SinkUpsertMaterializer`, `SinkUpsertMaterializerV2` and
`WatermarkCompactingSinkMaterializer` register a `numUnmatchedBuildRetractions`
counter and increment it in the retraction path when no accumulated row matches
- `KeyedLookupJoinWrapper` registers the same counter and increments it
when a retraction finds no state for its key
- Each operator exposes its counter to tests through a
`@VisibleForTesting` accessor
## Verifying this change
This change added tests and can be verified as follows:
- `SinkUpsertMaterializerTest` covers both the upsert-key and the
no-upsert-key variants: a retraction of an unaccumulated row increments the
counter, a retraction of an accumulated row does not
- `WatermarkCompactingSinkMaterializerTest` covers the same for the
watermark compacting materializer
- `KeyedLookupJoinHarnessTest` covers the lookup join wrapper
- Removing the four `inc()` calls makes all new tests fail with `expected:
1L but was: 0L`
- `./mvnw test -pl flink-table/flink-table-runtime
-Dtest='SinkUpsertMaterializerTest,WatermarkCompactingSinkMaterializerTest,KeyedLookupJoinHarnessTest'`
passes: 238 tests, 0 failures
A note on the TTL case: in these four operators "a retraction finds no
matching row" and "the state was cleared by TTL" are the *same* code path
(`state.value() == null` and `index == -1`), so they cannot be told apart at
runtime and are counted by this single counter. `LateralSnapshotJoinOperator`
tracks them with two separate counters (`numUnmatchedBuildRetractions` and
`numStateTtlEvictions`); splitting them here would need information these
operators do not have.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no — a
counter increment on an already existing error path
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: DeepSeek Harness (deepseek-v4.1-flash)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
