findepi commented on issue #14352: URL: https://github.com/apache/datafusion/issues/14352#issuecomment-2621637814
This is actually two separate bugs: - initial plan for the `Union` already has the field marked as not-nullable, because the plan builder simply takes first input schema as the union schema (thus ignoring actual nullability). This alone _maybe_ is corrected somewhere later, as the query without `b` projection does not reproduce the bug. - even if the plan is constructed correctly (by whatever means), the column pruning optimizer will do the same, replace union schema with the first input's schema (thus ignoring actual nullability). Both bugs are being fixed in https://github.com/apache/datafusion/pull/14356. The second led me to believe the idea that we can "reconstruct schema", while very convenient, is not solid, thus https://github.com/apache/datafusion/issues/14357. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
