Chia-Ping, thanks for bringing this up. I assumed that was done by a Jira + GitHub integration, not Jenkins. I'll ask the Infra team for suggestions.
A new GitHub Action is an option, but maybe there's some prior art we can use here. -David On Sun, Sep 29, 2024 at 1:42 PM Chia-Ping Tsai <chia7...@apache.org> wrote: > hi David > > There is no Jenkins job that automatically adds the PR link to the Jira > issue. Should we consider adding a GitHub action to handle this, or is it > sufficient for the committer to manually add the link when resolving the > Jira issue? > > Best, > Chia-Ping > > On 2024/09/26 21:37:17 David Arthur wrote: > > Josep, I've filed KAFKA-17628 and submitted a PR to partially automate > the > > workflow approval. With this change, we just need a committer to add a > > label one time to a PR, then it will get auto-approved. > > > > I've also wondered about a "triage" or "new" label that is automatically > > added to new PRs. This would make for an easy filter for committers to > use > > when seeing what needs attention. The trouble with this is we would then > > need to remove the label. > > > > -David > > > > > > On Thu, Sep 26, 2024 at 1:27 PM Chia-Ping Tsai <chia7...@apache.org> > wrote: > > > > > > I'm not sure caching is even that useful beyond the time > > > > between the branch point and the .0 release (since the rate of change > > > slows > > > > way down after a release). > > > > > > I try to keep us optimistic. 🙂 > > > > > > With the restore keys provided by setup-gradle, CI will always find a > > > cache to restore. While some task outputs might not be reusable, at > least > > > we avoid downloading all dependencies again. > > > > > > By the way, the bulk of the heavy dependencies comes from different > > > versions of rocksdb required by the upgrade-system-tests-xxx. > > > > > > In short, the cache remains valuable even for branches with slower > changes. > > > > > > Best, > > > Chia-Ping > > > > > > On 2024/09/26 14:13:55 David Arthur wrote: > > > > We can probably get the new CI working on older release branches, it > will > > > > just take a bit of effort. As a start, we can just disable the build > > > cache > > > > for these builds. I'm not sure caching is even that useful beyond the > > > time > > > > between the branch point and the .0 release (since the rate of change > > > slows > > > > way down after a release). There is also a 10Gb limit for our total > cache > > > > items, which we are pretty close to already. > > > > > > > > On Thu, Sep 26, 2024 at 9:51 AM Chia-Ping Tsai <chia7...@gmail.com> > > > wrote: > > > > > > > > > It seems we need to promote approve-workflows.py to all committers > 😀 > > > > > > > > > > Josep Prat <josep.p...@aiven.io.invalid> 於 2024年9月26日 週四 下午9:42寫道： > > > > > > > > > > > I see you have the python script under "committer-tools", I > guess I > > > might > > > > > > need to get used to call that script instead of going to the > "pulls" > > > > > page. > > > > > > > > > > > > Best, > > > > > > > > > > > > On Thu, Sep 26, 2024 at 3:36 PM Josep Prat <josep.p...@aiven.io> > > > wrote: > > > > > > > > > > > >> Hi David, > > > > > >> I think we need a way to flag in the PR list ( > > > > > >> github.com/apache/kafka/pulls) the ones that are waiting for a > > > > > committer > > > > > >> to approve the workflows. As an example: > > > > > >> [image: image.png] > > > > > >> This PR has a green checkmark where the check status usually > goes. > > > But > > > > > if > > > > > >> one navigates to the PR in question, one can see that the CI > tasks > > > > > didn't > > > > > >> start and wait for a committer to approve and run. > > > > > >> [image: image.png] > > > > > >> Do you have another way to identify these PRs? Or should we > maybe > > > work > > > > > on > > > > > >> auto labelling PRs from non-committers (the ones that would wait > > > for CI > > > > > to > > > > > >> run). > > > > > >> > > > > > >> On Thu, Sep 26, 2024 at 11:00 AM Josep Prat < > josep.p...@aiven.io> > > > > > wrote: > > > > > >> > > > > > >>> That's what I feared > > > > > >>> > > > > > >>> On Thu, Sep 26, 2024 at 10:31 AM Chia-Ping Tsai < > > > chia7...@gmail.com> > > > > > >>> wrote: > > > > > >>> > > > > > >>>> hi Josep > > > > > >>>> > > > > > >>>> > Do you see any potential impact if we backport the change to > > > those? > > > > > >>>> > > > > > >>>> In my opinion, the main concern is that non-trunk PRs can't > > > > > effectively > > > > > >>>> leverage the cache, meaning they require more time and > resources > > > to > > > > > run > > > > > >>>> CI. > > > > > >>>> Additionally, github-ci is triggered by trunk branch only, > and we > > > have > > > > > >>>> not > > > > > >>>> tested it on non-trunk branch yet. Given that 3.9.0 and 3.8.1 > > > releases > > > > > >>>> are > > > > > >>>> processing, we could continue using Jenkins CI to avoid the > > > additional > > > > > >>>> overhead of backporting. > > > > > >>>> > > > > > >>>> By the way, we'll eventually need to backport GitHub CI to the > > > > > non-trunk > > > > > >>>> branches once the 4.1 branch is created. > > > > > >>>> > > > > > >>>> Best, > > > > > >>>> Chia-Ping > > > > > >>>> > > > > > >>>> > > > > > >>>> > > > > > >>>> Chia-Ping Tsai <chia7...@gmail.com> 於 2024年9月26日 週四 下午4:15寫道： > > > > > >>>> > > > > > >>>> > Thanks to David for providing us with an improved CI! > > > > > >>>> > > > > > > >>>> > Cheers, > > > > > >>>> > Chia-Ping > > > > > >>>> > > > > > > >>>> > David Arthur <mum...@gmail.com> 於 2024年9月26日 週四 上午8:51寫道： > > > > > >>>> > > > > > > >>>> >> Today, we disabled the Jenkins build on trunk. With this > > > change, we > > > > > >>>> should > > > > > >>>> >> now be expecting all green status checks on PRs before > > > merging. Of > > > > > >>>> course, > > > > > >>>> >> flaky tests still exist, but generally speaking we should > have > > > > > green > > > > > >>>> >> builds > > > > > >>>> >> (see KIP-1090 for some plans on flaky tests). > > > > > >>>> >> > > > > > >>>> >> Any committer or "collaborator" (as defined in .asf.yaml) > is > > > able > > > > > to > > > > > >>>> >> manually re-run a GitHub Action via the UI. > > > > > >>>> >> > > > > > >>>> >> For non-committers, someone must approve the workflow. > There > > > is a > > > > > >>>> >> "approve-workflows.py" script in committer-tools to help > with > > > this. > > > > > >>>> I'm > > > > > >>>> >> still investigating options to improve this. > > > > > >>>> >> > > > > > >>>> >> We will keep the Jenkins build enabled for 3.9 and other > > > release > > > > > >>>> branches. > > > > > >>>> >> > > > > > >>>> >> Cheers, > > > > > >>>> >> David A > > > > > >>>> >> > > > > > >>>> > > > > > > >>>> > > > > > >>> > > > > > >>> > > > > > >>> -- > > > > > >>> [image: Aiven] <https://www.aiven.io> > > > > > >>> > > > > > >>> *Josep Prat* > > > > > >>> Open Source Engineering Director, *Aiven* > > > > > >>> josep.p...@aiven.io | +491715557497 > > > > > >>> aiven.io <https://www.aiven.io> | > > > > > >>> <https://www.facebook.com/aivencloud> > > > > > >>> <https://www.linkedin.com/company/aiven/> > > > > > >>> <https://twitter.com/aiven_io> > > > > > >>> *Aiven Deutschland GmbH* > > > > > >>> Alexanderufer 3-7, 10117 Berlin > > > > > >>> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > > > > > >>> Anna Richardson, Kenneth Chen > > > > > >>> Amtsgericht Charlottenburg, HRB 209739 B > > > > > >>> > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> [image: Aiven] <https://www.aiven.io> > > > > > >> > > > > > >> *Josep Prat* > > > > > >> Open Source Engineering Director, *Aiven* > > > > > >> josep.p...@aiven.io | +491715557497 > > > > > >> aiven.io <https://www.aiven.io> | > > > > > >> <https://www.facebook.com/aivencloud> > > > > > >> <https://www.linkedin.com/company/aiven/> > > > > > >> <https://twitter.com/aiven_io> > > > > > >> *Aiven Deutschland GmbH* > > > > > >> Alexanderufer 3-7, 10117 Berlin > > > > > >> Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > > > > > >> Anna Richardson, Kenneth Chen > > > > > >> Amtsgericht Charlottenburg, HRB 209739 B > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > [image: Aiven] <https://www.aiven.io> > > > > > > > > > > > > *Josep Prat* > > > > > > Open Source Engineering Director, *Aiven* > > > > > > josep.p...@aiven.io | +491715557497 > > > > > > aiven.io <https://www.aiven.io> | > > > > > > <https://www.facebook.com/aivencloud> > > > > > > <https://www.linkedin.com/company/aiven/> < > > > > > https://twitter.com/aiven_io> > > > > > > *Aiven Deutschland GmbH* > > > > > > Alexanderufer 3-7, 10117 Berlin > > > > > > Geschäftsführer: Oskari Saarenmaa, Hannu Valtonen, > > > > > > Anna Richardson, Kenneth Chen > > > > > > Amtsgericht Charlottenburg, HRB 209739 B > > > > > > > > > > > > > > > > > > > > > > > -- > > > > David Arthur > > > > > > > > > > > > > -- > > David Arthur > > > -- David Arthur
