Abdgali Yerkingaliyev created KAFKA-21207:
---------------------------------------------
Summary: ReplaceField reads excluded or missing rename sources in
schemaful records
Key: KAFKA-21207
URL: https://issues.apache.org/jira/browse/KAFKA-21207
Project: Kafka
Issue Type: Bug
Components: connect
Environment: Apache Kafka trunk
62f0fef0e81a8e24f6d174f9cf4b640620696e7c
Reporter: Abdgali Yerkingaliyev
ReplaceField can read the wrong source field when transforming a Struct. A
rename mapping remains in the global reverseRenames map even when its source
field is excluded or absent from the input schema. If the rename target is an
existing retained field, the schemaful path reads the excluded/missing source
instead of that retained field.
Reproduction
1. Create an input Struct with INT32 fields a and b, with values a=1 and b=2.
2. Configure ReplaceField with exclude=a and renames=a:b.
3. Apply the transformation.
Expected: Struct{b=2}, with only the original b field retained.
Actual: Struct{b=1}; the excluded a value is copied into b.
A related case uses a schema containing only b, with renames=a:b. The transform
should retain b, since a is absent. Instead, it attempts to read a and throws
DataException. Incompatible source/target field types can also make the
excluded-source case fail with DataException instead of silently changing the
value.
Cause and proposed fix
makeUpdatedSchema filters input fields and applies forward renames, but
applyWithSchema iterates output fields and uses a reverse map built from all
configured renames. These operations disagree when a rename source did not
contribute to the output schema. The focused fix is to iterate input schema
fields, apply the same filter, and copy each retained value to its
forward-renamed output field.
Verification
Reproduced on trunk commit 62f0fef0e81a8e24f6d174f9cf4b640620696e7c. With the
proposed fix, all 301 connect-transforms module tests pass, including 19
ReplaceField tests; style checks pass. Released versions have not been tested.
Related work
KAFKA-10640 and https://github.com/apache/kafka/pull/9493 concern broader
recursive transformations. That PR is closed and unmerged and also proposed
iterating input fields. This report is limited to incorrect source-field
selection in the existing schemaful ReplaceField behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
