Sevenannn commented on code in PR #12979:
URL: https://github.com/apache/datafusion/pull/12979#discussion_r1807113509
##########
datafusion/expr/src/logical_plan/builder.rs:
##########
@@ -1402,7 +1402,12 @@ pub fn validate_unique_names<'a>(
pub fn union(left_plan: LogicalPlan, right_plan: LogicalPlan) ->
Result<LogicalPlan> {
// Temporarily use the schema from the left input and later rely on the
analyzer to
// coerce the two schemas into a common one.
- let schema = Arc::clone(left_plan.schema());
+
+ // Functional Dependencies doesn't preserve after UNION operation
+ let schema = (**left_plan.schema()).clone();
+ let schema =
+
Arc::new(schema.with_functional_dependencies(FunctionalDependencies::empty())?);
+
Review Comment:
Hi @berkaysynnada, thanks for the review! I don’t think any FD persists when
performing UNION on 2 tables.
A simple example would be UNION table t1 with another table t2 which only
has 1 row, there always exists such data in t2 which could break the FDs in t1
/ t2 after the UNION.
In this case, clearing FDs would be the right fix since we don’t want FDs to
get wrongly retained and affect later plans, e.g. aggregation.
Please let me know if you have any further questions regarding this PR,
thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
