Hyukjin Kwon created SPARK-61066:
------------------------------------
Summary: Raise the errors of join types and dict aggregations, and
return the status of a streaming query, in the Rust client as PySpark does
Key: SPARK-61066
URL: https://issues.apache.org/jira/browse/SPARK-61066
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 5.0.0
Reporter: Hyukjin Kwon
The Python client of the Rust Spark Connect client differs from PySpark 4.3.0:
df.join with an unsupported join type raises a local ValueError where PySpark
raises AnalysisException UNSUPPORTED_JOIN_TYPE; df.agg({}) returns an empty row
where PySpark raises AssertionError; dict aggregations with non-str values
raise TypeError where PySpark raises AssertionError; and StreamingQuery.status
returns an object rather than PySpark's dict, and lastProgress and
recentProgress JSON strings rather than StreamingQueryProgress.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
