[
https://issues.apache.org/jira/browse/SPARK-61168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-61168:
---------------------------------
Fix Version/s: connect-rust-4.3.0
> Log the gRPC error of the call as the exception of the query context loggers,
> in the Python client as PySpark does
> ------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-61168
> URL: https://issues.apache.org/jira/browse/SPARK-61168
> Project: Spark
> Issue Type: Bug
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Priority: Major
> Fix For: connect-rust-4.3.0
>
>
> PySpark Connect builds the exception of an error of the server as it handles
> the gRPC error of the call (a grpc _MultiThreadedRendezvous), and raises it
> from None. So when the error has a DataFrame or SQL query context, the
> DataFrameQueryContextLogger and SQLQueryContextLogger of pyspark.logger log
> that gRPC error as the exception of their record: its class, its str (of the
> status and the details, the message of the server) and its traceback.
> The Python client builds the exception in Rust, where no Python exception is
> being handled, so the loggers log an exception of class UnknownException, of
> msg "None" and no stacktrace. Apache Spark's
> test_dataframe_query_context_logger of test_parity_logger.py fails on this
> client (err_msg not in log_json["exception"]["msg"]).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
