Anna Liang created SPARK-60285:
----------------------------------
Summary: Add unit tests for ReadLimitRegistry
Key: SPARK-60285
URL: https://issues.apache.org/jira/browse/SPARK-60285
Project: Spark
Issue Type: Improvement
Components: PySpark, Tests
Affects Versions: 5.0.0
Reporter: Anna Liang
Add unit tests for ReadLimitRegistry in pyspark.sql.datasource_internal.
The existing Python tests (test_python_streaming_datasource) never rebuild
ReadMinRows, ReadMaxFiles or ReadMaxBytes through the registry, and the
missing/unregistered type error paths are not covered. Coverage reports show
ReadLimitRegistry as uncovered.
The new tests rebuild all five builtin ReadLimit types from their dict form
and verify the PySparkException raised for a missing or unregistered type.
Running test_python_streaming_datasource raises coverage of
datasource_internal.py from 39% to 57%.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
