the-sakthi opened a new pull request, #49755: URL: https://github.com/apache/spark/pull/49755
Cherry-pick https://github.com/apache/spark/pull/46583 to branch-3.5 Original JIRA: SPARK-48276 ### What changes were proposed in this pull request? Original PR proposed these changes: - Add the missing `__repr__` method for `SQLExpression` - Also adjust the output of `lit(None): None -> NULL` to be more consistent with the Spark Classic I had to make very minor modifications to fix unclean cherry-pick: - The UT added in the original PR needed an import. ### Why are the changes needed? - In Spark 3.5, when PySpark is launched with a remote `Spark Connect` configuration, calls to `pyspark.sql.functions.expr` incorrectly return `Column<None>` instead of the expected expression. This change addresses the issue to ensure proper expression resolution in Spark Connect mode. - As per original PR: [Bug fix] All expressions should implement the `__repr__` method. ### Does this PR introduce _any_ user-facing change? Yes, this PR ensures that `pyspark.sql.functions.expr` correctly resolves expressions in Spark Connect mode. Previously, it returned `Column<None>`, but now it behaves correctly. ### How was this patch tested? Manually tested. Also original PR added a UT. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
