David Li created ARROW-17229:
--------------------------------
Summary: [C++] ReadRel is translated to a source node that emits
unexpected fields
Key: ARROW-17229
URL: https://issues.apache.org/jira/browse/ARROW-17229
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: David Li
Currently, a Substrait plan with a RelRoot containing a ReadRel will contain
extra, unexpected fields, namely {{__fragment_index}} et. al. Right now they
are always included by default. There are a few things to be done:
* ReadRel's {{base_schema}} could be converted into a
{{ScanOptions.dataset_schema}} to limit the fields read. (Also see ARROW-15585,
these fields should be used for pushdown projection)
* The scanner always adds these extra fields - maybe it should be opt-in instead
* There's no way to manually insert a Project to "fix" things because as
implemented, it can only add new columns
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
