LeonYoah opened a new pull request, #12912:
URL: https://github.com/apache/seatunnel/pull/12912
Background
A long-running Postgres CDC job on Zeta failed with
java.net.SocketException: Connection reset
at
io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.probeConnectionIfNeeded
Caused by: PSQLException: An I/O error occurred while sending to the backend
Debezium probes the JDBC data connection every status.update.interval.ms. In
this deployment, the data connection carries almost no traffic during
steady-state streaming and was periodically reset by the network. The old probe
allowed the exception to escape, which became a RetriableException, failed the
source task, and — with job.retry.times = 0 — left Zeta with no
checkpoint-restore retry, so the whole job failed.
This contribution to Apache SeaTunnel does three things: it exposes the
options the deployment actually needed, it reconnects the probe connection once
instead of failing immediately, and it documents how longer outages are
recovered (Zeta checkpoint retry, same as Flink CDC).
Changes
Commit Summary
62cd7e07c Add top-level sslmode, publication.name,
publication.autocreate.mode, and heartbeat.interval.ms options for Postgres
CDC. Legacy debezium {} passthrough remains unchanged.
e3f2b0be5 Reconnect the JDBC data connection once when the WAL-streaming
probe fails (close → connect → re-probe). If reconnect also fails, the
exception still propagates so the engine can restart from checkpoint.
e4d9f1905 Unit tests for the probe reconnect path, plus FAQ docs (EN/ZH) on
connection-reset recovery.
Behavior
Transient reset: probe fails → close/reconnect/re-probe → streaming
continues without a task restart.
Prolonged outage or walsender loss: reconnect fails, task fails, Zeta
restores from the last checkpoint via job.retry.times /
job.retry.interval.seconds.
Checkpoint restore does not re-run the snapshot; post-recovery rows are
consumed from the slot LSN.
job.retry.times = 0 removes the safety net for streaming CDC and is not
recommended.
Validation
Reproduced the exact stack trace on the parent commit using a real TCP RST
(iptables --reject-with tcp-reset) against the JDBC data backend.
On this branch, 3/3 transient resets recovered in place.
25s database outage and walsender termination with job.retry.times = 10: job
recovered from checkpoint, no snapshot re-run, post-recovery INSERTs captured.
With job.retry.times = 0, the job stayed FAILED as expected.
Aligns with Flink CDC, which also does not reconnect inside the connector
and relies on checkpoint + restart.
spotless:apply, -DskipTests verify, and the new unit tests pass.
close: #12887
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
