morningman opened a new pull request, #66369:
URL: https://github.com/apache/doris/pull/66369
### What problem does this PR solve?
Related PR: #64304 (catalog SPI)
Problem Summary:
A connector never decides which columns to read — it renders whatever list
`ConnectorScanRequest.getColumns()` carries. The jdbc connector turns that list
verbatim into the remote `SELECT` list and falls back to `SELECT *` when it is
empty (`JdbcQueryBuilder#buildQuery`). So column pruning for every
plugin-driven external scan rests entirely on
`PluginDrivenScanNode#buildColumnHandles()`, which intersects the connector's
column handles with this scan's tuple slots.
Pruning does work today — `select k8 from test1` (12 columns) already emits
`SELECT \`k8\` FROM \`doris_test\`.\`test1\`` — but the method that decides it
had **no direct coverage**, and its failure mode (projecting more columns than
the query needs) is a pure performance regression that no result-comparing test
can observe. The existing jdbc explain assertions all pass a column list and
assert those same columns are present; none of them can fail on an over-wide
projection they did not anticipate.
This PR closes two gaps.
**1. The projection decision itself (`fe-core`)**
`PluginDrivenScanNodeColumnPruningTest` drives the real
`buildColumnHandles()` and pins:
- only tuple-slot columns are projected (3-column table, 1 requested →
exactly 1 handle);
- the order follows the slot order, not the connector's handle-map order —
the connector renders this list positionally;
- slots with no backing column, and slots with no matching handle, are
skipped rather than leaking into the list;
- an empty tuple projects nothing — the sole input that reaches the
connector's `SELECT *` fallback.
Every assertion was mutation-checked against the production method:
returning `allHandles.values()` kills 4 of the 5, and making the unmatched-slot
path fail loud unconditionally kills the 5th.
**2. `count(*)`, the one shape whose projection would otherwise go empty
(`external_table_p0`)**
`count(*)` asks for no column of its own. The engine keeps a single smallest
slot (`PhysicalPlanTranslator#updateScanSlotsMaterialization`) instead of
letting the tuple go empty — and an empty tuple is exactly what makes the jdbc
connector emit `SELECT *`, i.e. a full 12-column read of `test1` just to count
rows. Nothing pinned that. The new explain assertion in
`test_mysql_jdbc_catalog` asserts the remote select list stays one column wide
and is not `*`; it deliberately does **not** pin which column wins, since that
is `getSmallestSlot`'s business and tracks type widths.
Test-only change — no production code is touched.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [x] Regression test
- [x] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- Behavior changed:
- [x] No.
- [ ] Yes.
- Does this need documentation?
- [x] No.
- [ ] Yes.
### Check List (For Reviewer who merge this PR)
- [x] Confirm the release note
- [x] Confirm test cases
- [x] Confirm document
- [x] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
