SEPURI-SAI-KRISHNA commented on issue #12966:
URL: https://github.com/apache/seatunnel/issues/12966#issuecomment-6410260961
Both noted, @SEZ9, and I will hold myself to them. First a correction,
because I gave you a wrong fact yesterday.
**Correction.** I said there is "no `compatible_kafka_connect_json` coverage
anywhere in `connector-kafka-e2e`". That is wrong.
`KafkaFormatIT.testFormatCompatibleCheck` already drives this format end to
end: it publishes `/compatible/compatible_data.txt` to topic
`jdbc_source_record`, runs
`/compatibleFormatIT/kafkasource_jdbc_record_to_pgsql.conf` with `format =
COMPATIBLE_KAFKA_CONNECT_JSON`, and `checkCompatibleFormat()` asserts exactly 3
rows in Postgres. I should have grepped before telling you otherwise.
This changes the follow-up for the better, and in the direction the
contributing guide asks for. Rather than a new test class and a greenfield
conf, the work extends `KafkaFormatIT`, reusing the existing topic, the
existing envelope shape from `compatible_data.txt` and the existing Postgres
assertion path.
**On covering the same path.** I will show it rather than assert it. The
guard is `if (msg.value() == null) { return; }` in
`CompatibleKafkaConnectDeserializationSchema`, placed before
`convertToSinkRecord`, so the unit tests and the IT must hit that same branch
or one of them is testing nothing. The demonstration is the mutation argument I
used for the zero-length case: removing the guard has to fail both the unit
tests and the new IT, and restoring it has to bring both back. If the IT still
passes with the guard removed, the config is not exercising the path, and I
will say so instead of shipping it.
**One real obstacle, stated now.** The fixture loop is `while ((line =
br.readLine()) != null)`, publishing each line as the record value, so a
tombstone cannot be expressed in `compatible_data.txt` at all. It needs an
explicit `ProducerRecord<>(topic, key, null)` alongside the file-driven
records. That is the part worth reviewing closely, since it touches shared IT
plumbing used by the canal, ogg, maxwell and debezium lanes too.
**On linking.** I will post the PR number here as soon as it is open, so
this issue references #12967 and the IT together.
I still cannot run a Kafka IT locally, so the first honest signal will be CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
