andygrove opened a new pull request, #6337: URL: https://github.com/apache/datafusion-comet/pull/6337
## Which issue does this PR close? No issue closes. Part of #6335, the timezone EPIC from the same audit. ## Rationale for this change The contributor guide doesn't describe how Comet handles timezones, and the rules are only implicit in the code. The most important one is that every `TimestampType` value inside a native plan is labelled `Timestamp(Microsecond, "UTC")`, with the session timezone applied per expression rather than at the boundary. The same class of bug keeps coming back when a native expression breaks that rule: #2720 and #2649 in the past, #5556 as a workaround, and #6327, #6328 and #6330 from this audit. A single page that states the model, and the ways it goes wrong, should make those easier to catch in review. ## What changes are included in this PR? A new page, `docs/source/contributor-guide/timezones.md`, listed under Project Architecture after Arrow FFI. It covers: - Spark's model: what each temporal type stores, where the session timezone applies and where it doesn't, and how `ResolveTimeZone` and `Cast.needsTimeZone` decide whether an expression carries a timezone - how Comet represents timestamps, where the `"UTC"` label is set, and what breaks when a value carries a different label, including why tests that only project a result miss it - how the session timezone reaches native code: serde, `array_with_timezone`, DST resolution, timezone ID parsing, and chrono-tz's tzdata versus the JVM's - Parquet and Iceberg scans, including how each Parquet timestamp encoding maps to Spark's types - the codegen dispatcher, guidelines, and a testing checklist ## How are these changes tested? This is a docs-only change. `prettier --check` passes on both files. A local Sphinx build gives the same warnings as `main`, so the new page's links and toctree entry resolve. I checked the statements about current behavior against the code on `main` and against the repros in #6335. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
