morningman opened a new pull request, #69849:
URL: https://github.com/apache/doris/pull/69849
### What problem does this PR solve?
Issue Number: None
Related PR: #69601, #64699
Problem Summary:
**In short.** #69601 made FE reject `format=avro` while it parses file
format properties. This PR adds end-to-end coverage for that: a regression
suite that runs `s3()` and `hdfs()` with `format=avro` and checks that they
fail during analysis, and a unit test for the two table-function constructors.
Test only, no code change.
**Background.** Master no longer reads standalone Avro files: #61141 removed
the BE reader, #65733 the scanner module, and #69601 made
`FileFormatProperties.createFileFormatProperties` throw `Avro file reads are no
longer supported`. Before #69601, FE accepted the format and the query failed
later in BE with `Not supported file format in fetch table schema: 12`.
branch-4.2 still reads Avro through the JNI scanner (#64699 fixed a crash in
it), so users who upgrade from 4.x with Avro table functions or loads get this
error.
**What was missing.** #69601's unit test calls `createFileFormatProperties`
directly. Nothing checks that a table function reaches that check before it
validates storage properties or lists files. If a constructor validated the
location first, `format=avro` against an unreachable location would report a
storage error instead, and no test would notice.
**What this PR adds.**
- `external_table_p0/tvf/test_tvf_avro_format_rejected`: `s3()` and `hdfs()`
with `format` set to `avro` and to `AVRO`, against a made-up bucket, made-up
credentials and an HDFS address nothing listens on. Each call must fail with
`Avro file reads are no longer supported`.
-
`UnsupportedExternalReadsTest#testAvroTableFunctionsAreRejectedBeforeStorageAccess`:
constructs `S3TableValuedFunction` and `HdfsTableValuedFunction` with
`format=avro` / `AVRO` and checks the same message.
**Results.** On master `2a4191c1dc9` with this change, all four statements
of the suite fail during analysis with `Can not build s3(): Avro file reads are
no longer supported` / `Can not build hdfs(): ...`, and the unit test passes.
Before #69601 the same `s3()` call would have gone on to validate and list the
made-up location, so the suite pins the order as well as the message.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- `run-regression-test.sh --run -d external_table_p0/tvf -s
test_tvf_avro_format_rejected`: passes on a local cluster whose FE is built
from this branch.
- [x] Unit Test
- `run-fe-ut.sh --run
org.apache.doris.datasource.UnsupportedExternalReadsTest`: 3 of 3 pass; fe-core
checkstyle passes.
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
