I think there are a lot of interesting and relatively isolated components of the project, it might make sense to write per-transform one pagers for isolated things like the most useful pieces (just basically copying the documentation and justifying the API) instead of doing a one-shot import or having it live forever in an external project.
-Daniel On Fri, Sep 8, 2023 at 11:10 AM Kenneth Knowles <k...@apache.org> wrote: > I agree with everyone about "not everything has to be in the Beam repo". I > really like the idea of having a clearer "ecosystem" section of the > website, which is sort of started at > https://beam.apache.org/community/integrations/ but that is not very > prominent. > > Agree with John though. The transforms in Asgarde could potentially be > used in Beam. Potentially best accomplished by just adding them as > transforms to the core Java SDK? > > Kenn > > On Wed, Sep 6, 2023 at 1:46 PM John Casey via dev <dev@beam.apache.org> > wrote: > >> Agreed on documentation and on keeping it in a separate repo. >> >> We have a few pretty significant beam extensions (scio and Dataflow >> Templates also come to mind) that Beam should highlight, but are separate >> repos for their own governance, contributions, and release reasons. >> >> The difference with Asgarde is that we might want to use it in Beam >> itself, which makes it more reasonable to include in the main repo. >> >> On Tue, Sep 5, 2023 at 8:36 PM Robert Bradshaw via dev < >> dev@beam.apache.org> wrote: >> >>> I think this is a great library. I'm on the fence of whether it makes >>> sense to include with Beam proper vs. be a library that builds on top of >>> Beam. (Would there be benefits of tighter integration? There is the >>> maintenance/loss of governance issue.) I am definitely not on the side that >>> the entire Beam ecosystem needs to be distributed/maintained by Beam >>> itself. >>> >>> Regardless of the direction we go, I think it could make a lot of sense >>> to put pointers to it in our documentation. >>> >>> >>> On Tue, Sep 5, 2023 at 7:21 AM Danny McCormick via dev < >>> dev@beam.apache.org> wrote: >>> >>>> I think my only concerns here are around the toil we'll be taking on, >>>> and will we be leaving the asgarde project in a better or worse place. >>>> >>>> From a release standpoint, we would need to release it with the same >>>> cadence as Beam. Adding asgarde into our standard release process seems >>>> fairly straightforward, though, so I'm not too worried about it - looks >>>> like it's basically (1) add a commit like this >>>> <https://github.com/tosun-si/asgarde/commit/432de527d67dc71f06507328319b466b6d0fb56a>, >>>> (2) run this workflow >>>> <https://github.com/tosun-si/asgarde/blob/main/.github/workflows/publish-project.yml>, >>>> and (3) tag/mark the release as released on GitHub. >>>> >>>> In terms of bug fixes and improvements, though, I'm a little worried >>>> that we might be leaving things in a worse state since Mazlum has been the >>>> only contributor thus far, and he would lose some governance (and possibly >>>> the ability to commit code on his own). An extra motivated community member >>>> or two could change the math a bit, but I'm not sure if there are actually >>>> clear advantages to including it in Apache other than visibility. Would >>>> adding links to our docs calling Asgarde out as an option accomplish the >>>> same purpose? >>>> >>>> > Let's be careful about whether these tests are included in our >>>> presubmits. Contrib code with flaky tests has been a major pain point in >>>> the past. >>>> >>>> +1 - I think if we do this I'd vote that it be in a separate repo ( >>>> github.com/apache/beam-asgarde made sense to me). >>>> >>>> --------------------------------------- >>>> >>>> Overall, I'm probably a slight -1 to adding this to the Apache >>>> workspace, but +1 to at least adding links from the Beam docs to Asgarde. >>>> >>>> Thanks, >>>> Danny >>>> >>>> >>>> >>>> On Tue, Sep 5, 2023 at 12:03 AM Reuven Lax via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>>> Let's be careful about whether these tests are included in our >>>>> presubmits. Contrib code with flaky tests has been a major pain point in >>>>> the past. >>>>> >>>>> On Sat, Sep 2, 2023 at 12:02 PM Austin Bennett <aus...@apache.org> >>>>> wrote: >>>>> >>>>>> Wanting us to not miss this. @Mazlum TOSUN <mazlum.to...@gmail.com> is >>>>>> happy to donate Asgarde to our project. >>>>>> >>>>>> It looks like he'd need a SGA and CCLA [ 1 ] on file; anything else? >>>>>> >>>>>> I recalled the donation of Euphoria [ 2 ] , so I looked at those >>>>>> threads [ 3 ] for insights into the process. It didn't look like there >>>>>> was a needed VOTE, so mostly a matter of ensuring necessary signatures, >>>>>> and >>>>>> ideally some sort of consensus [ or non-opposition ] to the donation. >>>>>> >>>>>> >>>>>> [ 1 ] https://www.apache.org/licenses/contributor-agreements.html >>>>>> [ 2 ] https://beam.apache.org/documentation/sdks/java/euphoria/ >>>>>> [ 3 ] >>>>>> https://lists.apache.org/thread/xzlx4rm2tvc36mmwvhyvtdvsw7bnjscp >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Jun 15, 2023 at 7:05 AM Kerry Donny-Clark via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> This looks like an excellent contribution. I can easily understand >>>>>>> the motivation, and I think Beam would benefit from a higher level >>>>>>> abstraction for error handling. >>>>>>> Kerry >>>>>>> >>>>>>> On Wed, Jun 14, 2023, 6:31 PM Austin Bennett <aus...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Beam Devs, >>>>>>>> >>>>>>>> @Mazlum <https://www.linkedin.com/in/mazlum-tosun-900b1812/> was >>>>>>>> suggested to consider donating Asgarde >>>>>>>> <https://github.com/tosun-si/asgarde> to Beam for Java/Kotlin >>>>>>>> error handling to Beam [ see: >>>>>>>> https://2022.beamsummit.org/sessions/error-handling-asgarde/ for >>>>>>>> last year's Beam Summit talk ], he is also the author of Pasgard >>>>>>>> <https://github.com/tosun-si/pasgarde>e [ for Python ] and Milgard >>>>>>>> [ for a simplified Kotlin API ]. >>>>>>>> >>>>>>>> Would Asgarde be a good contribution, something the Beam community >>>>>>>> would be willing to accept? I imagine we might want it to live at >>>>>>>> github.com/apache/beam-asgarde ? Or perhaps there is a good place >>>>>>>> in github.com/apache/beam ?? >>>>>>>> >>>>>>>> Especially once/if officially part of Beam, I imagine we'd add >>>>>>>> follow-up items like getting onto the website/docs, and related. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Austin >>>>>>>> >>>>>>>> >>>>>>>> P.S. This might warrant separate/additional conversations for his >>>>>>>> other libraries, but let's focus any discussion on Asgarde for now? >>>>>>>> >>>>>>>
