Hi all, Thanks for your feedback! Regarding https://github.com/apache/spark/pull/50187 , I don't think it's a 4.0 blocker as it's a CI issue for the examples module. Other than that, all other issues have been resolved and I'll cut the next RC after https://github.com/apache/spark/pull/50197 is merged, which updates the to-be-released stored procedure DS v2 API.
Thanks, Wenchen On Mon, Mar 24, 2025 at 7:36 PM Hyukjin Kwon <gurwls...@apache.org> wrote: > -1. > > Scala and PySpark shell are broken by > https://github.com/apache/spark/pull/49971. Reverted it for now. > > On Mon, 24 Mar 2025 at 12:36, Yang Jie <yangji...@apache.org> wrote: > >> -1, >> >> The pull request at https://github.com/apache/spark/pull/49604 >> introduced a connection-related example module and successfully merged it >> into branch-4.0, which is a commendable achievement. >> >> However, it is recommended that this example module should rely on the >> snapshot version of its corresponding development branch. Subsequently, >> after the release of a new version, it should be updated to depend on the >> latest released version of that branch, rather than a historical or preview >> release version (for instance, in the v4.0.0 tag, it should depend on Spark >> 4.0.0 instead of 4.0.0-preview2). It appears that the pull request at >> https://github.com/apache/spark/pull/50187 is currently discussing and >> addressing this issue; however, it is still in progress. I firmly believe >> that this issue should be resolved before the release of Apache Spark 4.0.0. >> >> Best regards, >> Jie Yang >> >> On 2025/03/23 01:26:43 Ángel Álvarez Pascua wrote: >> > Great to hear that! >> > >> > Fortunately, I was waiting for someone to answer and hadn't looked at it >> > yet. >> > >> > Seems a quite straightforward solution for an issue that shouldn't have >> to >> > exist if the right unit test had been implemented. What do you think >> about >> > adding a test to check this issue doesn't happen again? >> > >> > >> > >> > I'm planning to write an article about Spark Connect, so ... thanks >> again >> > for providing the project to start playing around. >> > >> > El dom, 23 mar 2025, 2:01, Bobby <wbo4...@gmail.com> escribió: >> > >> > > Thx @Ángel <angel.alvarez.pas...@gmail.com> , I had a PR >> > > https://github.com/apache/spark/pull/50334 to fix it. Please help >> review. >> > > >> > > Ángel Álvarez Pascua <angel.alvarez.pas...@gmail.com> 于2025年3月22日周六 >> > > 16:27写道： >> > > >> > >> Is anyone looking into this issue? If not, I'd like to try fixing it. >> > >> I've never tried out Spark Connect, so... 2x1! (way better than >> spending >> > >> the weekend binge-watching on Netflix 😅🤣). >> > >> >> > >> @Bobby <wbo4...@gmail.com> , thanks a lot, not only for reporting >> the >> > >> issue but also for providing a time-saving project for testing. >> > >> >> > >> El sáb, 22 mar 2025 a las 1:11, Bobby (<wbo4...@gmail.com>) >> escribió: >> > >> >> > >>> I think https://issues.apache.org/jira/browse/SPARK-51537 should be >> > >>> fixed before 4.0.0 release, or else, the executor side will not use >> global >> > >>> jars added by `--jars` in the connect mode, which could result in >> > >>> deserialization exceptions. >> > >>> >> > >> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >>
