Hi all, Thank you for reviewing and merging PR #48791 <https://github.com/apache/spark/pull/48791>, which enables running spark.ml on Connect starting from Spark 4.0. With this PR, it becomes extremely easy to support third-party Spark ML libraries on Connect.
I also want to highlight PR #49503 <https://github.com/apache/spark/pull/49503>, which adds support for the Connect ML backend. This PR allows users to leverage GPU-accelerated spark-ml libraries to speed up the built-in spark.ml algorithms without requiring any changes to user code. You can find the full story at SPARK-50812 <https://issues.apache.org/jira/browse/SPARK-50812>. Thank you again for your feedback and for merging these PRs! Best Regards, Bobby Wang Mich Talebzadeh <mich.talebza...@gmail.com> 于2024年11月13日周三 21:17写道： > OK I added a comment to PR > > HTH, > > Mich Talebzadeh, > > Architect | Data Engineer | Data Science | Financial Crime > PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial College > London <https://en.wikipedia.org/wiki/Imperial_College_London> > London, United Kingdom > > > view my Linkedin profile > <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> > > > https://en.everybodywiki.com/Mich_Talebzadeh > > > > *Disclaimer:* The information provided is correct to the best of my > knowledge but of course cannot be guaranteed . It is essential to note > that, as with any advice, quote "one test result is worth one-thousand > expert opinions (Werner <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von > Braun <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". > > > On Wed, 13 Nov 2024 at 09:17, Bobby <wbo4...@gmail.com> wrote: > >> Hi all, >> >> Finally, I got the draft PR up https://github.com/apache/spark/pull/48791 >> last week, but I would like to send it out here and hope to get >> your feedback and reviews. >> >> Thx very much. >> >> Mich Talebzadeh <mich.talebza...@gmail.com> 于2024年10月15日周二 15:47写道： >> >>> +1 >>> >>> It will be a desirable feature >>> >>> Mich Talebzadeh, >>> >>> Architect | Data Engineer | Data Science | Financial Crime >>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>> College London <https://en.wikipedia.org/wiki/Imperial_College_London> >>> London, United Kingdom >>> >>> >>> view my Linkedin profile >>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>> >>> >>> https://en.everybodywiki.com/Mich_Talebzadeh >>> >>> >>> >>> *Disclaimer:* The information provided is correct to the best of my >>> knowledge but of course cannot be guaranteed . It is essential to note >>> that, as with any advice, quote "one test result is worth one-thousand >>> expert opinions (Werner >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>> >>> >>> On Tue, 15 Oct 2024 at 08:24, Bobby <wbo4...@gmail.com> wrote: >>> >>>> Thank you for your kind response. I will prepare a formal PR for Spark. >>>> >>>> >>>> >>>> Niranjan Jayakar <n...@databricks.com> 于2024年10月11日周五 22:45写道： >>>> >>>>> +1 >>>>> >>>>> On Thu, Oct 10, 2024 at 5:28 PM Xiao Li <gatorsm...@gmail.com> wrote: >>>>> >>>>>> Thank you for working on this! >>>>>> >>>>>> Xiao >>>>>> >>>>>> Martin Grund <mar...@databricks.com.invalid> 于2024年10月10日周四 03:01写道： >>>>>> >>>>>>> >>>>>>> Hi Bobby, >>>>>>> >>>>>>> Awesome to see the proposal! I'm very much looking forward to the >>>>>>> contributions! >>>>>>> >>>>>>> Martin >>>>>>> >>>>>>> On Thu, Oct 10, 2024 at 4:16 AM Ángel < >>>>>>> angel.alvarez.pas...@gmail.com> wrote: >>>>>>> >>>>>>>> You have my vote (btw, great idea, ML is so sexy nowadays 😉) >>>>>>>> >>>>>>>> El jue, 10 oct 2024 a las 3:19, Bobby (<wbo4...@gmail.com>) >>>>>>>> escribió: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'd like to start a discussion about support spark.ml on Connect. >>>>>>>>> With this feature, Users don't need to change their code to run Spark >>>>>>>>> ML >>>>>>>>> cases on Connect. >>>>>>>>> >>>>>>>>> Please refer to the JIRA: >>>>>>>>> https://issues.apache.org/jira/browse/SPARK-49907 >>>>>>>>> >>>>>>>>> Thx, >>>>>>>>> Bobby Wang >>>>>>>>> >>>>>>>>
