tokoko commented on code in PR #12800:
URL: https://github.com/apache/datafusion/pull/12800#discussion_r1792393774
##########
datafusion/substrait/src/logical_plan/consumer.rs:
##########
@@ -990,24 +978,46 @@ pub async fn from_substrait_rel(
fn ensure_schema_compatability(
table: DataFrame,
substrait_schema: DFSchema,
-) -> Result<DataFrame> {
- let df_schema = table.schema().to_owned().strip_qualifiers();
+) -> Result<LogicalPlan> {
Review Comment:
the whole flow goes like this:
- `from_substrait_named_struct` gets substrait schema from the plan
(base_schema)
- `apply_projection` applies MaskExpression from the plan to that schema.
We're effectively combining base_schema and projection into a single schema
object that we can use later to add projections to DF TableScan in one go.
- once we have that schema, `ensure_schema_compatability` does two things.
1) it checks that all requested columns exist and data types match up and 2) it
adds projection to TableScan if not all columns from the table were requested.
How about I rename `apply_projection` to `apply_projection_to_schema` and
`ensure_schema_compatability` to `adjust_with_requested_schema`?
@Blizzara any better ideas?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
