chenkovsky commented on PR #14057: URL: https://github.com/apache/datafusion/pull/14057#issuecomment-2644409227
> Do you know the rationale for spark that why it doesn't consider projected columns as normal column? If my memory is correct, it's also designed for dataframe api user. for `withColumn` clause, it's a project. dataframe users often chain withColumn operators. ```python df.withColumn(...).withColumn(...) ``` If project cannot propagate metadata/system columns, it's very hard to use this chain. I cannot tell are there any other differences between spark standard and #14362 's standard because most sqls will have at least one project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
