Tangentially related: Long ago, attaching an issue to a release was a mandatory step as part of closing. Now I think it is not. Is it automatically happening? It looks like we have 820 with no milestone https://github.com/apache/beam/issues?q=is%3Aissue+no%3Amilestone+is%3Aclosed
Kenn On Tue, Oct 24, 2023 at 1:25 PM Chamikara Jayalath via dev < dev@beam.apache.org> wrote: > +1 for going by the commits since this is what matters at the end of the > day. Also, many issues may not get tagged correctly for a given release due > to either the contributor not tagging the issue or due to commits for the > issue spanning multiple Beam releases. > > For example, > > For all commits in a given release RC: > * If we find a Github issue for the commit: add a notice to the Github > issue > * Else: add the notice to a generic issue for the release including tags > for the commit ID, PR author, and the committer who merged the PR. > > Thanks, > Cham > > > > > On Mon, Oct 23, 2023 at 11:49 AM Danny McCormick via dev < > dev@beam.apache.org> wrote: > >> I'd probably vote to include both the issue filer and the contributor. It >> is pretty equally straightforward - one way to do this would be using all >> issues related to that release's milestone and extracting the issue author >> and the issue closer. >> >> This does leave out the (unfortunately sizable) set of contributions that >> don't have an associated issue; if we're worried about that, we could >> always fall back to anyone with a commit in the last release who doesn't >> have an associated issue (aka what I thought we were initially proposing >> and what I think Airflow does today). >> >> I'm pretty much +1 on any sort of automation here, and it certainly can >> come in stages :) >> >> On Mon, Oct 23, 2023 at 1:50 PM Johanna Öjeling via dev < >> dev@beam.apache.org> wrote: >> >>> Yes that's a good point to include also those who created the issue. >>> >>> On Mon, Oct 23, 2023, 19:18 Robert Bradshaw via dev <dev@beam.apache.org> >>> wrote: >>> >>>> On Mon, Oct 23, 2023 at 7:26 AM Danny McCormick via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> So to summarize, I think there's broad consensus (or at least lazy >>>>> consensus) around the following: >>>>> >>>>> - (1) Updating our release email/guidelines to be more specific about >>>>> what we mean by release validation/how to be helpful during this process. >>>>> This includes both encouraging validation within each user's own code base >>>>> and encouraging people to document/share their process of validation and >>>>> link it in the release spreadsheet. >>>>> - (2) Doing something like what Airflow does (#29424 >>>>> <https://github.com/apache/airflow/issues/29424>) and creating an >>>>> issue asking people who have contributed to the current release to help >>>>> validate their changes. >>>>> >>>>> I'm also +1 on doing both of these. The first bit (updating our >>>>> guidelines) is relatively easy - it should just require updating >>>>> https://github.com/apache/beam/blob/master/contributor-docs/release-guide.md#vote-and-validate-the-release-candidate >>>>> . >>>>> >>>>> I took a look at the second piece (copying what Airflow does) to see >>>>> if we could just copy their automation, but it looks like it's tied >>>>> to airflow breeze >>>>> <https://github.com/apache/airflow/blob/main/dev/breeze/src/airflow_breeze/provider_issue_TEMPLATE.md.jinja2> >>>>> (their repo-specific automation tooling), so we'd probably need to build >>>>> the automation ourselves. It shouldn't be terrible, basically we'd want a >>>>> GitHub Action that compares the current release tag with the last release >>>>> tag, grabs all the commits in between, parses them to get the author, and >>>>> creates an issue with that data, but it does represent more effort than >>>>> just updating a markdown file. There might even be an existing Action that >>>>> can help with this, I haven't looked too hard. >>>>> >>>> >>>> I was thinking along the lines of a script that would scrape the issues >>>> resolved in a given release and add a comment to them noting that the >>>> change is in release N and encouraging (with clear instructions) how this >>>> can be validated. Creating a "validate this release" issue with all >>>> "contributing" participants could be an interesting way to do this as well. >>>> (I think it'd be valuable to get those who filed the issue, not just those >>>> who fixed it, to validate.) >>>> >>>> >>>>> As our next release manager, I'm happy to review PRs for either of >>>>> these if anyone wants to volunteer to help out. If not, I'm happy to >>>>> update >>>>> the guidelines, but I probably won't have time to add the commit >>>>> inspection >>>>> tooling (I'm planning on throwing any extra time towards continuing to >>>>> automate release candidate creation which is currently a more impactful >>>>> problem IMO). I would very much like it if both of these things happened >>>>> though :) >>>>> >>>>> Thanks, >>>>> Danny >>>>> >>>>> On Mon, Oct 23, 2023 at 10:05 AM XQ Hu <x...@google.com> wrote: >>>>> >>>>>> +1. This is a great idea to try. @Danny McCormick >>>>>> <dannymccorm...@google.com> FYI as our next release manager. >>>>>> >>>>>> On Wed, Oct 18, 2023 at 2:30 PM Johanna Öjeling via dev < >>>>>> dev@beam.apache.org> wrote: >>>>>> >>>>>>> When I have contributed to Apache Airflow, they have tagged all >>>>>>> contributors concerned in a GitHub issue when the RC is available and >>>>>>> asked >>>>>>> us to validate it. Example: #29424 >>>>>>> <https://github.com/apache/airflow/issues/29424>. >>>>>>> >>>>>>> I found that to be an effective way to notify contributors of the RC >>>>>>> and nudge them to help out. In the issue description there is a >>>>>>> reference >>>>>>> to the guidelines on how to test the RC and a note that people are >>>>>>> encouraged to vote on the mailing list (which could admittedly be more >>>>>>> highlighted because I did not pay attention to it until now and was >>>>>>> unaware >>>>>>> that contributors had a vote). >>>>>>> >>>>>>> It might be an idea to consider something similar here to increase >>>>>>> the participation? >>>>>>> >>>>>>> On Tue, Oct 17, 2023 at 7:01 PM Jack McCluskey via dev < >>>>>>> dev@beam.apache.org> wrote: >>>>>>> >>>>>>>> I'm +1 on helping explain what we mean by "validate the RC" since >>>>>>>> we're really just asking users to see if their existing use cases work >>>>>>>> along with our typical slate of tests. I don't know if offloading that >>>>>>>> work >>>>>>>> to our active validators is the right approach though, >>>>>>>> documentation/screen >>>>>>>> share of their specific workflow is definitely less useful than having >>>>>>>> a >>>>>>>> more general outline of how to install the RC and things to look out >>>>>>>> for >>>>>>>> when testing. >>>>>>>> >>>>>>>> On Tue, Oct 17, 2023 at 12:55 PM Austin Bennett <aus...@apache.org> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Great effort. I'm also interested in streamlining releases -- so >>>>>>>>> if there are alot of manual tests that could be automated, would be >>>>>>>>> great >>>>>>>>> to discover and then look to address. >>>>>>>>> >>>>>>>>> On Tue, Oct 17, 2023 at 8:47 AM Robert Bradshaw via dev < >>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>> >>>>>>>>>> +1 >>>>>>>>>> >>>>>>>>>> I would also strongly suggest that people try out the release >>>>>>>>>> against their own codebases. This has the benefit of ensuring the >>>>>>>>>> release >>>>>>>>>> won't break your own code when they go out, and stress-tests the new >>>>>>>>>> code >>>>>>>>>> against real-world pipelines. (Ideally our own tests are all >>>>>>>>>> passing, and >>>>>>>>>> this validation is automated as much as possible (though ensuring it >>>>>>>>>> matches our documentation and works in a clean environment still has >>>>>>>>>> value), but there's a lot of code and uses out there that we don't >>>>>>>>>> have >>>>>>>>>> access to during normal Beam development.) >>>>>>>>>> >>>>>>>>>> On Tue, Oct 17, 2023 at 8:21 AM Svetak Sundhar via dev < >>>>>>>>>> dev@beam.apache.org> wrote: >>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> I’ve participated in RC testing for a few releases and have >>>>>>>>>>> observed a bit of a knowledge gap in how releases can be tested. >>>>>>>>>>> Given that >>>>>>>>>>> Beam encourages contributors to vote on RC’s regardless of tenure, >>>>>>>>>>> and that >>>>>>>>>>> voting on an RC is a relatively low-effort, high leverage way to >>>>>>>>>>> influence >>>>>>>>>>> the release of the library, I propose the following: >>>>>>>>>>> >>>>>>>>>>> During the vote for the next release, voters can document the >>>>>>>>>>> process they followed on a separate document, and add the link on >>>>>>>>>>> column G >>>>>>>>>>> here >>>>>>>>>>> <https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=437054928>. >>>>>>>>>>> One step further, could be a screencast of running the test, and >>>>>>>>>>> attaching >>>>>>>>>>> a link of that. >>>>>>>>>>> >>>>>>>>>>> We can keep repeating this through releases until we have >>>>>>>>>>> documentation for many of the different tests. We can then add >>>>>>>>>>> these docs >>>>>>>>>>> into the repo. >>>>>>>>>>> >>>>>>>>>>> I’m proposing this because I’ve gathered the following feedback >>>>>>>>>>> from colleagues that are tangentially involved with Beam: They are >>>>>>>>>>> interested in participating in release validation, but don’t know >>>>>>>>>>> how to >>>>>>>>>>> get started. Happy to hear other suggestions too, if there are any >>>>>>>>>>> to >>>>>>>>>>> address the above. >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Svetak Sundhar >>>>>>>>>>> >>>>>>>>>>> Data Engineer >>>>>>>>>>> s <nellywil...@google.com>vetaksund...@google.com >>>>>>>>>>> >>>>>>>>>>>
