morningman opened a new pull request, #66970:
URL: https://github.com/apache/doris/pull/66970
### What problem does this PR solve?
Related PR: #66873
Problem Summary:
Two unrelated `external_table_p0` failures, both in test code only. No BE/FE
code is touched.
#### 1. `remote_doris` DATETIME baselines are stale after #66873
#66873 changed the Arrow Flight result schema so `DATETIMEV2` is exposed as
a **timezone-naive** Arrow timestamp. Both sides of the conversion agree —
`write_column_to_arrow` and `read_column_from_arrow` both decode an empty Arrow
timezone as UTC — so a cross-cluster `type=doris` read now round-trips the
whole DATETIME domain.
Under the previous timezone-aware encoding, `0000-01-01 00:00:00` was
written as an epoch below the minimum representable value once the session
timezone was applied, `from_unixtime` rejected it, and the value came back as
an invalid datetime (rendered empty).
`test_remote_doris_all_types_select` and `test_remote_doris_statistics` are
the only suites that store that boundary value in a `DATETIME` column read over
an arrow-flight Doris catalog, so they are the only ones whose baselines still
record the old broken output:
```
test_remote_doris_all_types_select.out
scalar c_datetime : → 0000-01-01T00:00
array a_datetime : [""] → ["0000-01-01 00:00:00"]
test_remote_doris_statistics.out
c_datetime min/max : 2023-10-01 12:34:56 / (empty)
→ 0000-01-01 00:00:00 / 9999-12-31 23:59:59
```
The new values are exactly the ones the test inserts, and the refreshed
baseline is now **identical cell for cell** to
`test_query_remote_doris_as_olap_table_select`, which reads the same table
definition and the same rows with `use_arrow_flight = false`. Making the two
transports agree is what #66873 set out to do, so that equality is the check
that these values are right.
This is a master-wide failure, not specific to any PR: across ~60 recent
`Doris_External_Regression` builds, every build whose base contains 910b741ba32
fails these two suites, and every build with an older base passes them. Both
are currently muted in TeamCity, so the mutes should be removed once this lands.
#### 2. `test_iceberg_write_merge_duplicate_source_negative` is flaky (13 of
50 recent runs)
```
mc: <ERROR> Unable to initialize new alias from the provided credentials.
Get "http://172.17.0.6:19001/probe-.../?location=": dial tcp
172.17.0.6:19001: i/o timeout
```
The suite counted data objects by starting a throwaway `minio/mc` container
per call. MinIO runs on the isolated `doris--iceberg` compose network
(`168.38.0.0/24`) and only publishes its API on the host, while the ad-hoc
container joins the default bridge — so every call had to hairpin from that
bridge back to the published port. That path times out while the agent starts
and stops other containers, and with four calls per run the suite failed about
a quarter of the time. MinIO itself was healthy in every failing run, and this
was the only suite in the repo using that pattern.
Count the objects with the S3 SDK in the suite's own JVM instead, against
the same endpoint the catalog is already configured with — the same approach
`test_iceberg_variant_read.groovy` already uses. Two incidental bugs go away
with it:
- the listing now paginates over continuation tokens, so a table with more
than 1000 objects can no longer be undercounted;
- the prefix is anchored at `.../data/` rather than `.../data`, so a sibling
directory sharing the prefix is not swept in.
### Release note
None
### Check List (For Author)
- Test
- [x] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01TGba4CrDYYiDtuW7kuKFpL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
