cj-zhukov opened a new pull request, #351:
URL: https://github.com/apache/datafusion-spark/pull/351
## Which issue does this PR close?
- Closes #337.
## Rationale for this change
`to_json` rounds a time zone offset to the minute. With session time zone
`America/Los_Angeles`, Spark writes `TIMESTAMP '1800-01-01 00:00:00'` as
`1800-01-01T00:00:00.000-07:52:58`, while `to_json` writes `-07:53`.
Follow-up to #86, raised by @andygrove in the review.
## What changes are included in this PR?
- Format the offset with `%::z` and drop a trailing `:00`. Offsets with
seconds keep them; whole-minute offsets and `Z` for UTC are unchanged.
## Are these changes tested?
- Unit test `test_timestamp_offset_keeps_its_seconds`.
- Two new cases in `json/to_json.slt` (Los Angeles in 1800 and in 2024).
- Checked against Spark 4.2.0 with the PySpark runner in
`crates/sqllogictest/scripts`: both new cases pass.
## Are there any user-facing changes?
`to_json` now matches Spark for time zone offsets that have a seconds part
(historical local mean time offsets). No API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]