chenkovsky commented on PR #14362: URL: https://github.com/apache/datafusion/pull/14362#issuecomment-2624968962
i dont mean that this way cannot reach the target. but when i use spark style api i dont need to take care it. it's battery included. no need to call extra api to do it. please correct me if I'm wrong. the benefit of this approch is 1. dont need to change api the drawbacks are 1. table provider developers have to take care of that and call an api to merge system field and normal field. 2. hashmap performance is not good. 3. hashmap need more effort to take care. for example, maybe we have another method that can also change hashmap later. we have to make sure that method wont change this special key value. Another thing I haven't checked for this approach is that system column doesn't only apply on scan. it can also be applied on [Projection](https://github.com/apache/spark/blob/19fbeaa04abe71529a707e07a36adf14098e7741/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala#L95) [Join](https://github.com/apache/spark/blob/19fbeaa04abe71529a707e07a36adf14098e7741/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala#L648) [SubqueryAlias](https://github.com/apache/spark/blob/19fbeaa04abe71529a707e07a36adf14098e7741/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala#L1657) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
