I thought that too ... until I read the message from Matei Zaharia: "Votes to deprecate both SparkR and GraphX have passed. These components will officially be deprecated in Spark 4."
Didn't know in open source you could deprecate things that have been there years so lightly without carrying out any impact analysis and in the middle of an active (and interesting, btw) discussion. El mar, 12 nov 2024, 21:59, Russell Jurney <russell.jur...@gmail.com> escribió: > That is unfortunate. I saw someone volunteer to review my PRs. I thought > there was a holdout? > > On Tue, Nov 12, 2024 at 12:56 PM Ángel <angel.alvarez.pas...@gmail.com> > wrote: > >> Nope. didn't miss that, in fact, I mentioned that graphframes used GraphX >> under the hood. >> >> The thing is ... even though we were trying to get maintainers the >> deprecation of GraphX passed suddenly in the middle of that discussion. >> >> El mar, 12 nov 2024, 21:47, Russell Jurney <russell.jur...@gmail.com> >> escribió: >> >>> I guess you missed where Reynold Xin suggested we instead bring >>> GraphFrames into Spark and others agreed? >>> >>> On Tue, Nov 12, 2024 at 12:08 PM Ángel <angel.alvarez.pas...@gmail.com> >>> wrote: >>> >>>> You only have to look at the subject of this thread of mails. It says >>>> nothing about graphframes. I thought we were "fighting" against deprecating >>>> GraphX because it seemed not have any maintainers in quite a few time. >>>> Maybe I got it wrong. >>>> >>>> El mar, 12 nov 2024, 19:12, Russell Jurney <russell.jur...@gmail.com> >>>> escribió: >>>> >>>>> Not sure what you mean? GraphX is the core Apache Spark technology >>>>> underneath GraphFrames - parts of GraphFrames use it. `git grep -i graphx >>>>> | >>>>> wc -l` shows 147 hits for `graphx` in GraphFrames master branch as of now. >>>>> >>>>> I started out getting familiar with the GraphFrames codebase with some >>>>> low hanging fixes for housekeeping - GraphFrames hasn't been maintained, >>>>> the docs are outdated and most examples didn't work. The friends graph >>>>> didn't even load in all of the examples, as the Python >>>>> graphframes.examples.Graphs interface changed to be object oriented. I am >>>>> now a maintainer of the Github repo thanks to Weichen Xu, who is reviewing >>>>> my PRs along with Reynold Xin (very nice, thanks!). I am getting familiar >>>>> with GraphFrames internals before looking at what it takes to get it into >>>>> Apache Spark proper. I was just letting people know that I am actively >>>>> taking on the work that we are discussing. >>>>> >>>>> Next, I am also going to get to know GraphX in the next couple of >>>>> weeks to better understand what moving GraphFrames into Spark would look >>>>> like. >>>>> >>>>> In a week or so, I will announce the hackathon. Need to figure out a >>>>> time that works for the people who've already expressed interest. >>>>> >>>>> Thanks, >>>>> Russell Jurney >>>>> >>>>> >>>>> >>>>> >>>>> On Tue, Nov 12, 2024 at 6:48 AM Ángel <angel.alvarez.pas...@gmail.com> >>>>> wrote: >>>>> >>>>>> But the goal wasn't to fix bugs in GraphX? What has that to do with >>>>>> graphframes? >>>>>> >>>>>> El mar, 12 nov 2024, 12:58, Russell Jurney <russell.jur...@gmail.com> >>>>>> escribió: >>>>>> >>>>>>> I started working on GraphFrames this weekend, got it building and >>>>>>> started with some docs PRs. A lot of the example code no longer worked, >>>>>>> so >>>>>>> I fixed it. I'm updating the docs to indicate our plan to integrate it >>>>>>> with >>>>>>> Apache Spark. >>>>>>> >>>>>>> I'll announce a hackathon in the next week or so :) >>>>>>> >>>>>>> Russell >>>>>>> >>>>>>> On Wed, Oct 16, 2024 at 6:53 PM Russell Jurney < >>>>>>> russell.jur...@gmail.com> wrote: >>>>>>> >>>>>>>> For starters I created a ticket. I'm going to work on the project a >>>>>>>> bit and then name a date and time. >>>>>>>> >>>>>>>> https://github.com/graphframes/graphframes/issues/460 >>>>>>>> >>>>>>>> On Tue, Oct 15, 2024 at 7:48 PM Ángel < >>>>>>>> angel.alvarez.pas...@gmail.com> wrote: >>>>>>>> >>>>>>>>> We could create a prioritized list of the most important bugs to >>>>>>>>> fix first and distribute the tasks among us. We can also share the >>>>>>>>> knowledge we gain from resolving them. >>>>>>>>> btw, what happened to the (great) hackathon idea? any date/s in >>>>>>>>> mind? >>>>>>>>> >>>>>>>>> El mié, 16 oct 2024 a las 3:53, Russell Jurney (< >>>>>>>>> russell.jur...@gmail.com>) escribió: >>>>>>>>> >>>>>>>>>> I've never used Visual Studio Code for Java / Scala, but I find >>>>>>>>>> IntelliJ works really well. >>>>>>>>>> >>>>>>>>>> On Tue, Oct 15, 2024 at 4:58 PM Mich Talebzadeh < >>>>>>>>>> mich.talebza...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> VSC is also another good candidate heavily used in the banks. >>>>>>>>>>> Anyway you parted with money so all the best. >>>>>>>>>>> >>>>>>>>>>> cheers >>>>>>>>>>> >>>>>>>>>>> Mich Talebzadeh, >>>>>>>>>>> >>>>>>>>>>> Architect | Data Engineer | Data Science | Financial Crime >>>>>>>>>>> PhD <https://en.wikipedia.org/wiki/Doctor_of_Philosophy> Imperial >>>>>>>>>>> College London >>>>>>>>>>> <https://en.wikipedia.org/wiki/Imperial_College_London> >>>>>>>>>>> London, United Kingdom >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> view my Linkedin profile >>>>>>>>>>> <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://en.everybodywiki.com/Mich_Talebzadeh >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Disclaimer:* The information provided is correct to the best >>>>>>>>>>> of my knowledge but of course cannot be guaranteed . It is >>>>>>>>>>> essential to >>>>>>>>>>> note that, as with any advice, quote "one test result is worth >>>>>>>>>>> one-thousand expert opinions (Werner >>>>>>>>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>Von Braun >>>>>>>>>>> <https://en.wikipedia.org/wiki/Wernher_von_Braun>)". >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>
