ksbeyer opened a new pull request, #52449: URL: https://github.com/apache/spark/pull/52449
### What changes were proposed in this pull request? Improve how metadata is propagated during analysis and rewrites. - applyColumnMetadata: tried to avoid an Alias but the caused incorrect propagation. - collapseAlias and trimNonTopLevelAliases: now preserves the meaning of stacked aliases. - removeRedundantAlias: will remove a redundant alias when the only extra key is AUTO_GENERATED_ALIAS. - other minor cleanup This PR improves metadata propagation but it doesn't solve all the problems. A thorough review is recommended. Also recommended is to make and Alias unresolved if its inferring metadata and once its child is resolved, the ineferred metadata should be made explicit. Then the metadata would be significantly more stable. ### Why are the changes needed? Wrong metadata is inferred for DML and probably other places. ### Does this PR introduce _any_ user-facing change? Changing metadata can change DML behavior. Not removing aliases can cause some Project operators to remain. ### How was this patch tested? Tests were added with the previous PR on this topic [SPARK-52772]. ### Was this patch authored or co-authored using generative AI tooling? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
