dongjoon-hyun commented on code in PR #50159:
URL: https://github.com/apache/spark/pull/50159#discussion_r1983844231
##########
python/pyspark/sql/connect/client/core.py:
##########
@@ -666,7 +666,7 @@ def __init__(
elif user_id is not None:
self._user_id = user_id
else:
- self._user_id = os.getenv("USER", None)
+ self._user_id = os.getenv("SPARK_USER", os.getenv("USER", None))
Review Comment:
Thank you for review, @viirya .
It's because `user.name` is one of JVM's `System.getProperty` values which
Python doesn't have. So, in Python environment, `USER` was used inevitably.
Technically, the following is Apache Spark `getCurrentUserName` JVM logic and
`user.name` is JVM's user concept. AFAIK, both `user.name` and `USER` serves
the same concept in both language environment.
https://github.com/apache/spark/blob/44e751f243a5a7be8e64c306e40ddc1502f72710/core/src/main/scala/org/apache/spark/util/Utils.scala#L2441-L2455
For the second question, `core.py` change is @HyukjinKwon 's request.
Initially, I only fixed Java/Scala client to support `SPARK_USER` like
`getCurrentUserName`, but Python Connect client also needs that of course. So,
I addressed the comment.
- https://github.com/apache/spark/pull/50159#discussion_r1980500141
##########
python/pyspark/sql/connect/client/core.py:
##########
@@ -666,7 +666,7 @@ def __init__(
elif user_id is not None:
self._user_id = user_id
else:
- self._user_id = os.getenv("USER", None)
+ self._user_id = os.getenv("SPARK_USER", os.getenv("USER", None))
Review Comment:
Thank you for review, @viirya .
It's because `user.name` is one of JVM's `System.getProperty` values which
Python doesn't have. So, in Python environment, `USER` was used inevitably.
Technically, the following is Apache Spark `getCurrentUserName` JVM logic and
`user.name` is JVM's user concept. AFAIK, both `user.name` and `USER` serves
the same concept in both language environments.
https://github.com/apache/spark/blob/44e751f243a5a7be8e64c306e40ddc1502f72710/core/src/main/scala/org/apache/spark/util/Utils.scala#L2441-L2455
For the second question, `core.py` change is @HyukjinKwon 's request.
Initially, I only fixed Java/Scala client to support `SPARK_USER` like
`getCurrentUserName`, but Python Connect client also needs that of course. So,
I addressed the comment.
- https://github.com/apache/spark/pull/50159#discussion_r1980500141
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
