DanielLeens opened a new pull request, #12970:
URL: https://github.com/apache/seatunnel/pull/12970

   ### Purpose of this pull request
   
   Yes, I am contributing these DB2 CDC correctness fixes to Apache SeaTunnel.
   
   This is a focused follow-up to the DB2 source introduced in #10780 (feature 
issue #10735) and the CDC history restoration introduced in #7463.
   
   An active ASN Capture registration can contain both `CD_OLD_SYNCHPOINT` and 
an advancing `CD_NEW_SYNCHPOINT`. Debezium 1.9.8 maps those fields to a change 
table's stop/start LSNs, so SeaTunnel's reader skips committed changes at or 
beyond the old synchronization point. The connector now maps the raw 
registration at its JDBC boundary: OLD is the start, and an active registration 
has no stop. The existing streaming reader can still assign a stop when 
replacing an older capture instance. Reading raw capture names also avoids 
quoting mixed-case names twice; no third-party class is replaced.
   
   Saved DB2 history encodes two-part `schema.table` identifiers. The shared 
serializer previously ignored its existing `useCatalogBeforeSchema` argument 
and restored them as `catalog.table`. The serializer now honors that argument, 
and the JDBC restoration entry point derives it from the authoritative split 
table ID, preserving MySQL's catalog-first history. Vitess keeps its existing 
schema-first call; a regression test protects the real decoder's namespace.
   
   The DB2-specific follow-up also normalizes unquoted configured table names 
before catalog lookup/filtering, preserves custom keys and 
`snapshotSplitColumn`, forwards the existing `enable_concurrent_read` setting, 
handles absent catalog key metadata/source structs, and closes the auxiliary 
connection even if closing the primary connection fails.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes. Active DB2 captures continue producing changes after their 
synchronization points advance, and saved schema history is restored into the 
correct DB2 namespace. Lowercase unquoted table configuration matches DB2's 
uppercase metadata, and `enable_concurrent_read = false` reaches the reader 
configuration.
   
   Existing option names, aliases, defaults, `table-pattern` expressions, 
quoted spelling, public APIs, dependency versions, offset/state fields and 
checkpoint writing formats are retained. The Apache snapshot backfill 
implementation and `exactly_once` setting are retained. No POM, connector 
registration, distribution, workflow or unrelated connector implementation is 
changed.
   
   ### How was this patch tested?
   
   Added 16 unit regression cases across DB2 and Vitess, plus three DB2 
container scenarios alongside the original test:
   
   - Capture mapping: non-null OLD point, unavailable OLD point, mixed-case 
capture name, and explicit successor stop assignment.
   - History restore: production JDBC registration after JSON history and 
binary split round-trip, without live-schema fallback; empty/null catalog DB2 
IDs, catalog-first MySQL IDs, three-part IDs and quoted dotted/non-ASCII 
identifiers. Assertions compare catalog/schema separately because combined-name 
equality can hide the defect.
   - Factory and lifecycle: normalized capture filtering, preserved key/split 
configuration and URL alias, unchanged patterns/quoted spelling, forwarded 
snapshot mode, absent key metadata/source structs, and auxiliary connection 
cleanup after a primary close failure.
   - Vitess: FIELD history and catalog bootstrap both restore the schema-first 
identity used by the actual Debezium decoder.
   - DB2 E2E: original snapshot/INSERT/DELETE reconciliation, lowercase 
configuration, two-table savepoint/restore with stopped-job writes and resumed 
changes, and adding/registering a new table while the original job is stopped. 
The active registration now has a non-null OLD point, so the original stop-LSN 
interpretation makes the reconciliation fail.
   
   Local validation is intentionally limited to formatting under the 
contributor's Apache SeaTunnel validation policy:
   
   ```text
   JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_171.jdk/Contents/Home 
./mvnw spotless:apply spotless:check -pl 
seatunnel-connectors-v2/connector-cdc/connector-cdc-base,seatunnel-connectors-v2/connector-cdc/connector-cdc-db2,seatunnel-connectors-v2/connector-cdc/connector-cdc-vitess,seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-db2-e2e
 -nsu -Dmaven.gitcommitid.skip=true
   git diff --check
   ```
   
   Compilation, unit tests, packaging and DB2 E2E were **not run locally**; 
they require GitHub CI on this PR head. No existing build/deployment artifacts 
were reused, and no service or Docker container was started. 
`seatunnel-engine-ui` was excluded because this change does not touch it. 
Independent static review found no source-level blocker, which is not a 
runtime-validation or merge conclusion.
   
   The DB2 E2E uses a synthetic ASNCDC catalog; it does not validate the real 
IBM ASN Capture agent or IBM DDL-history replay. The existing original 
assertions and DependencyJar driver setup are preserved.
   
   ### Check list
   
   - [x] English and Chinese DB2 CDC documentation updated.
   - [x] Unit regressions and DB2 E2E scenarios added.
   - [x] Relevant modules formatted and checked with Spotless.
   - [x] No new dependency, JAR binary, module, POM or registration change.
   - [ ] Compilation, unit tests and DB2 E2E validated by GitHub CI on the 
final head.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to