zclllyybb commented on issue #65741: URL: https://github.com/apache/doris/issues/65741#issuecomment-4998168536
Breakwater-GitHub-Analysis-Slot: slot_a66f91e21aeb This content is generated by AI for reference only. Initial assessment: this looks like a real Arrow Flight SQL type-mapping bug for `DATETIMEV2`, not only a Python client display issue. I checked current `upstream/master` because the issue does not specify an affected Doris version. The relevant path is: - `CAST(... AS DATETIME(6))` is a `DATETIMEV2(6)` result. - In `be/src/format/arrow/arrow_row_batch.cpp`, `convert_to_arrow_type()` currently handles `TYPE_TIMESTAMPTZ` and `TYPE_DATETIMEV2` in the same branch and creates `arrow::TimestampType(unit, timezone)`. The nearby TODO already notes that these types may need to be distinguished. - The Arrow Flight result sink builds that schema with `state->timezone()` in `be/src/exec/operator/result_sink_operator.cpp`, so the Arrow field receives non-empty timezone metadata. - `DataTypeDateTimeV2SerDe::write_column_to_arrow()` then reads `timestamp_type->timezone()` and, when it is non-empty, serializes the value using the session timezone. With a UTC session/default timezone, PyArrow/ADBC will naturally return a timezone-aware Python value such as `2026-07-02 01:36:22.069504+00:00`. That behavior is probably wrong for ordinary Doris `DATETIME` / `DATETIMEV2`: it is a timezone-naive wall-clock type, while an Arrow timestamp with timezone has instant semantics. The timezone-aware Arrow representation is appropriate for `TIMESTAMPTZ`, but should not be reused for `DATETIMEV2`. Suggested fix direction: - Split `TYPE_DATETIMEV2` and `TYPE_TIMESTAMPTZ` in Arrow schema conversion. - For `TYPE_DATETIMEV2`, emit an Arrow timestamp with the correct unit but without timezone metadata, or another explicitly timezone-naive representation. - Keep timezone-aware Arrow metadata for `TYPE_TIMESTAMPTZ`. - Add coverage that checks the Arrow schema, not only the rendered row values. The existing `regression-test/suites/arrow_flight_sql_p0/test_select.groovy` covers `datetime(6)` result values, but it appears to compare converted output and does not assert that the Arrow field timezone is empty. Missing information that would help confirm the exact user-facing scope: - Doris build/version, since the issue body currently omits it. - The session value of `@@time_zone` when the query is run. - The exact printed `Arrow field:` line, plus PyArrow / ADBC driver versions. - Whether the same query through the MySQL protocol returns the expected timezone-naive text. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
