Re: [PR] [FLINK-38742][cdc-pipeline/postgres] Fix timestamp with time zone [flink-cdc]

Tue, 26 May 2026 02:34:33 -0700 


leonardBang commented on PR #4181:
URL: https://github.com/apache/flink-cdc/pull/4181#issuecomment-4542892253

   > > I am not sure if server-time-zone affects TIMESTAMPTZ.
   > 
   > @Fluder-Paradyne No influence. I have see the code of debezium. In 
Debezium's PostgreSQL connector, timestamp with time zone values are always 
normalized to UTC+0 regardless of the data source path. Here's why:
   > 
   > ### Streaming Path (pgoutput logical replication)
   > When consuming WAL via the pgoutput plugin, the value goes through 
`AbstractColumnValue.asOffsetDateTimeAtUtc()` :
   > 
   > ```java
   > return DateTimeFormat.get()
   >     .timestampWithTimeZoneToOffsetDateTime(asString())   // Step 1: parse 
raw text with original offset
   >     .withOffsetSameInstant(ZoneOffset.UTC);              // Step 2: 
normalize to UTC+0
   > ```
   > 
   > The pgoutput protocol delivers column values as UTF-8 text strings (e.g., 
"2019-10-02T13:51:30.123456+02:00"). The parser first produces an 
OffsetDateTime with the original timezone offset from the WAL entry, then 
explicitly converts it to UTC via .withOffsetSameInstant(ZoneOffset.UTC). This 
guarantees the OffsetDateTime entering convertTimestampWithZone() is always at 
offset zero.
   > 
   > ### Snapshot Path (JDBC)
   > During initial snapshots, the JDBC driver returns java.sql.Timestamp (a 
subclass of java.util.Date). In convertTimestampWithZone():
   > 
   > ```java
   > if (data instanceof java.util.Date) {
   >     data = OffsetDateTime.ofInstant(((Date) data).toInstant(), 
ZoneOffset.UTC);
   > }
   > ```
   > 
   > <img alt="image" width="796" height="538" 
src="https://private-user-images.githubusercontent.com/125648852/598008296-0af0542e-52d1-46cc-942d-3225c2906b64.png?jwt=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Nzk3Nzg3MDMsIm5iZiI6MTc3OTc3ODQwMywicGF0aCI6Ii8xMjU2NDg4NTIvNTk4MDA4Mjk2LTBhZjA1NDJlLTUyZDEtNDZjYy05NDJkLTMyMjVjMjkwNmI2NC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjYwNTI2JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI2MDUyNlQwNjUzMjNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lZTM4ZTVlZjY1ZDc5NDAwMGE5MjFmMGMxNWM2ZmY3MGNmN2FhMGYzZGVjZDUxY2MyYThlNDc1Mzk0YmFjYzRjJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZyZXNwb25zZS1jb250ZW50LXR5cGU9aW1hZ2UlMkZwbmcifQ.z2Gp0Bkuow4plre2ULSmrFTsIa7XeRxjqcjtgOzZ748";>
   > java.util.Date.toInstant() yields an absolute point in time (epoch-based), 
and OffsetDateTime.ofInstant(..., ZoneOffset.UTC) renders it explicitly at 
UTC+0.
   > 
   > Thus, using timestamp ltz is ok because time offset is normalized to UTC+0 
by debezium. The only concern is compatibility, but it should be fine. Although 
the downstream jobs created previously used timestamp with time zone instead of 
timestamp with local time zone, no data has been written yet. Users will need 
to delete the downstream jobs and re-submit them. @leonardBang ,WDYT?
   
   Thanks @loserwang1024 for the detailed analysis, I think there're no normal 
PG CDC job with timestamptz type in users' existing pipeline as the error will 
happen immediately once the DataChangEvent arrives. 
   
   +1 to map PG’s timestamptz type to Flink CDC timestamp_ltz type.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to