Hyukjin Kwon created SPARK-61144:
------------------------------------
Summary: Repartition by range without a number of partitions in
repartitionByRange of columns, in the Python client as PySpark does
Key: SPARK-61144
URL: https://issues.apache.org/jira/browse/SPARK-61144
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
repartitionByRange("id"), repartitionByRange(col("id")) and
repartitionByRange("id", "id") fail with "requirement failed: Number of
partitions (0) must be positive." since 2f173672, because the Python client
sends a partition count of 0 for these forms and the core always sends it.
PySpark sends no partition count in these forms, so the server picks it and
returns the rows. The fix sends no count in these forms (the core's
repartition_by_range_opt takes an Option).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
