RE: [FYI] Known `Spark Connect` Test Suite Flakiness

I have worked on tests related to “interrupt”. Not sure about SPARK-50888: My findings: 1. These test failures only occur in the GitHub CI. 2. The failure is due to the thread pool we created in CI having only two threads, while our tests require three concurrent threads to run. To workaround th

RE: [DISCUSS] Ongoing projects for Spark 4.0

I am working on a project https://issues.apache.org/jira/browse/SPARK-48918 which targets Spark 4.0. Some items on the list would need to go into this new branch. Now three PRs are ongoing: https://github.com/apache/spark/pull/49373 https://github.com/apache/spark/pull/49339 https://github.com/a