zhengruifeng opened a new pull request, #49863: URL: https://github.com/apache/spark/pull/49863
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html 2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html 3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'. 4. Be sure to keep the PR description updated to reflect all changes. 5. Please write your PR title to summarize what this PR proposes. 6. If possible, provide a concise example to reproduce the issue for a faster review. 7. If you want to add a new configuration, please read the guideline first for naming configurations in 'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'. 8. If you want to add or modify an error type or message, please read the guideline first in 'common/utils/src/main/resources/error/README.md'. --> ### What changes were proposed in this pull request? Pin `plotly==5.24.1` ### Why are the changes needed? the latest plotlly 6.0 has causes many plot-related test failures ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? manually checked with ``` python/run-tests -k --python-executables python3 --testnames 'pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot' ``` before: ``` (spark_312) ➜ spark git:(pin_plotly) python/run-tests -k --python-executables python3 --testnames 'pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot' Running PySpark tests. Output is in /Users/ruifeng.zheng/Dev/spark/python/unit-tests.log Will test against the following Python executables: ['python3'] Will test the following Python tests: ['pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot'] python3 python_implementation is CPython python3 version is: Python 3.12.9 Starting test(python3): pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot (temp output: /Users/ruifeng.zheng/Dev/spark/python/target/4d10075d-bb7b-4d4b-b17d-edbef2f22227/python3__pyspark.sql.tests.connect.test_parity_frame_plot_plotly_FramePlotPlotlyParityTests.test_pie_plot__6qxzu16x.log) Running tests... ---------------------------------------------------------------------- WARNING: Using incubator modules: jdk.incubator.vector Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). /Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/conf.py:64: UserWarning: Failed to set spark.connect.execute.reattachable.senderMaxStreamDuration to Some(1s) due to [CANNOT_MODIFY_CONFIG] Cannot modify the value of the Spark config: "spark.connect.execute.reattachable.senderMaxStreamDuration". See also 'https://spark.apache.org/docs/latest/sql-migration-guide.html#ddl-statements'. SQLSTATE: 46110 warnings.warn(warn) /Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/connect/conf.py:64: UserWarning: Failed to set spark.connect.execute.reattachable.senderMaxStreamSize to Some(123) due to [CANNOT_MODIFY_CONFIG] Cannot modify the value of the Spark config: "spark.connect.execute.reattachable.senderMaxStreamSize". See also 'https://spark.apache.org/docs/latest/sql-migration-guide.html#ddl-statements'. SQLSTATE: 46110 warnings.warn(warn) test_pie_plot (pyspark.sql.tests.connect.test_parity_frame_plot_plotly.FramePlotPlotlyParityTests.test_pie_plot) ... FAIL (1.760s) ====================================================================== FAIL [1.760s]: test_pie_plot (pyspark.sql.tests.connect.test_parity_frame_plot_plotly.FramePlotPlotlyParityTests.test_pie_plot) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/tests/plot/test_frame_plot_plotly.py", line 318, in test_pie_plot self._check_fig_data(fig["data"][0], **expected_fig_data_sales) File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/sql/tests/plot/test_frame_plot_plotly.py", line 81, in _check_fig_data self.assertEqual(converted_values, expected_value) AssertionError: Lists differ: [1517356800000000000, 1519776000000000000,[37 chars]0000] != [datetime.datetime(2018, 1, 31, 0, 0), dat[105 chars], 0)] First differing element 0: datetime.datetime(2018, 1, 31, 0, 0) - [1517356800000000000, - 1519776000000000000, - 1522454400000000000, - 1525046400000000000] + [datetime.datetime(2018, 1, 31, 0, 0), + datetime.datetime(2018, 2, 28, 0, 0), + datetime.datetime(2018, 3, 31, 0, 0), + datetime.datetime(2018, 4, 30, 0, 0)] ---------------------------------------------------------------------- Ran 1 test in 5.573s FAILED (failures=1) Generating XML reports... Generated XML report: target/test-reports/TEST-pyspark.sql.tests.connect.test_parity_frame_plot_plotly.FramePlotPlotlyParityTests-20250210120410.xml Had test failures in pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot with python3; see logs. ``` after: ``` (spark_312) ➜ spark git:(pin_plotly) python/run-tests -k --python-executables python3 --testnames 'pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot' Running PySpark tests. Output is in /Users/ruifeng.zheng/Dev/spark/python/unit-tests.log Will test against the following Python executables: ['python3'] Will test the following Python tests: ['pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot'] python3 python_implementation is CPython python3 version is: Python 3.12.9 Starting test(python3): pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot (temp output: /Users/ruifeng.zheng/Dev/spark/python/target/28a7ea8a-6249-4b16-9831-d785b1af2254/python3__pyspark.sql.tests.connect.test_parity_frame_plot_plotly_FramePlotPlotlyParityTests.test_pie_plot__eezgr0hf.log) Finished test(python3): pyspark.sql.tests.connect.test_parity_frame_plot_plotly FramePlotPlotlyParityTests.test_pie_plot (5s) Tests passed in 5 seconds ``` ### Was this patch authored or co-authored using generative AI tooling? no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org