rameshkrishnan muthusamy created KAFKA-9326:
-----------------------------------------------
Summary: Kafka Connect JSON Converter results in data loss when
additional fields apart from provided JSON schema is present in the payload
Key: KAFKA-9326
URL: https://issues.apache.org/jira/browse/KAFKA-9326
Project: Kafka
Issue Type: Bug
Components: KafkaConnect
Affects Versions: 2.2.1
Reporter: rameshkrishnan muthusamy
JSON Convertor in Kafka Connect validates the payload against the schema
provided by the user. In case there is an additional field in the payload which
is not mentioned in the schema the JSON convertor does not fail and returns the
value without the additional column.
e.g) message has additional column "new_col"
{
"schema": {
"type": "struct",
"fields": [
{
"type": "string",
"optional": true,
"field": "type"
},
{
"type": "string",
"optional": false,
"field": "data"
}
],
"optional": false,
"name": "test"
},
"payload": {
"type": "testpayload",
"data": "test_data",
"new_col": 1
}
}
The output of JSON convertor "Struct\{type=testpayload,data=test_data}" should
ideally fail assuming fail fast rather than skipping the field in the output
payload.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
