1) The committer/PMC/Triage member will remove the needs-triage label. This is not really an additional step. We are already relabeling when we triage an issue. The removal of the label doesn't have to happen on the first touchdown. Sometimes the triager doesn't have the full knowledge so tagging another member of the community or needs to ask followup questions. In my perspective triage is done once issue is understood, reproducible and just waiting for someone to pick it up (usually this is also the stage where we add the good first issue / area labels)
2) Procedures take time to be fully adopted. From past experience eventually everyone is aligned with new policies. Even if we get it wrong in specific places it's very easy to correct it. Dashboard can be really nice. 3) There is not much we can do. The next step after triage is to open PR. This depends on someone who will pick up the issue. We can measure time since creation/last action but also break by reported version. On Mon, Feb 13, 2023 at 11:38 AM Jarek Potiuk <ja...@potiuk.com> wrote: > Yes. I agree it is a good first step. Let's just not stop on that. > Once we have it, I think starting measuring "responsiveness" is > crucial. > > Also - even if it is the first, step, it has to be well defined. > Adding such labels should be accompanied with some way of explaining > and educating those who would use it how to deal with it. Because > setting it is one thing and important is what happens next. Few > questions: > > 1) Who and when should I remove it ? I believe it adds extra > responsibility on those who look at the issue and respond to the user, > to remove it when the issue has been "triaged" already - is that the > idea - to do it always as an extra manual step when we respond to such > issue (sounds like extra small but regular burden). Maybe the bot > could automatically remove the label when a maintainer responded to > the issue? We could do this later, but I am curious what you think > there. > > 2) Do you think we should have some "dashboard" showing the issues to > be triaged? Or you think just "label needs-triage" will be enough and > every one of the maintainers will know and should simply look at those > issues that "needs triage"? > > 3) what should we do about issues that have been triaged and the user > "responded" (and no-one will follow up - this happens). Are we going > to track them too or is it something to tackle next ? > > J. > > > > On Mon, Feb 13, 2023 at 10:02 AM Elad Kalif <elad...@apache.org> wrote: > > > > > Setting the label does not mean that someone will have eyes on it. > > > > True. but that is just about creating a work queue so when someone does > spend time on triage the issues can be found easily. > > This will also address your other points of needing data. By having the > label can measure several metrics regarding waiting for triage time (script > that checks open issues with the labels (daily?) and possibly push > notification to the issue-triage channel in slack or some other channel?) > > > > There are many further improvements we can do. For example setup > https://github.com/google/triage-party tool > > > > On Sun, Feb 12, 2023 at 10:29 PM Jarek Potiuk <ja...@potiuk.com> wrote: > >> > >> Maybe I am not getting the full extent of the proposal and maybe I > >> "hijacked" it a bit, but my comment was really related to (4) - new > >> issues. My mistake. Let me comment on those. > >> > >> 1) 2) 3) -> This is good as a cleanup. We can do this as a manually > >> run bulk process to add such comments now for all such old issues and > >> run them periodically (every few months) if needed. That will be > >> simpler. I am fine with that. We can even initially run it manually > >> and if we find it useful we can turn it into a bot. But I think we > >> should not have to do it again and this is what I mostly commented on. > >> > >> > In item (4) I suggested adding a needs-triage label to make sure that > any issue we get will have at least 1 committer/PMC/Triage member eyes on > it. > >> > >> Setting the label does not mean that someone will have eyes on it. > >> It's a good starting point though. I think measuring and incentivising > >> responsiveness for new issues is a key. And if we make sure that we > >> respond to issues in a timely manner, the current stale-bot is enough. > >> It will be closing only issues/PRs which are "pending response". All > >> the other issues will be either acknowledged by a maintainer and at > >> least vaguely planned to work on, or converted to a discussion, or > >> fixed or marked as "good first issue" for anyone to pick up. > >> > >> J. > >> > >> > >> > >> On Sun, Feb 12, 2023 at 8:37 PM Elad Kalif <elad...@apache.org> wrote: > >> > > >> > I'm not sure if the scenario you are worried about can happen? > >> > > >> > In item (4) I suggested adding a needs-triage label to make sure that > any issue we get will have at least 1 committer/PMC/Triage member eyes on > it. > >> > In this step the issue can be accepted and then this label is > replaced by (reported_version) or rejected and be closed/converted to > discussion. > >> > If a year passed and no one did anything with the issue the > automation will simply ask the user to let us know if the issue is still > happening on newer Airflow version. > >> > The issue may have already been solved and we just didn't notice. > Assuming the user won't comment in a defined time frame then it will close > the issue (if someone in the future will say we did wrong we can always > reopen) > >> > This is basically what happens today just by manually process. > >> > > >> > - What happens if the user replies that it's reproducible? > >> > We will replace the previous reported_version with a new one (for > example: reported_version:2.0 -> reported_version:2.5) this will bump the > issue to the latest bug lists. > >> > > >> > > >> > On Sun, Feb 12, 2023 at 5:48 PM Jarek Potiuk <ja...@potiuk.com> > wrote: > >> >> > >> >> TL;DR; I think we should first solve the issue of improving our > >> >> "responsiveness" as committers first. I believe once we solve it, the > >> >> stale bot closing issue will be a useful and non-offensive tool. > >> >> > >> >> (Sorry for a loong email, but I have been thinking a lot about it > and > >> >> I had many observations that came from responding and triaging our > >> >> issues and PRs and discussions - this took likely 30%/40% of my time > >> >> over the last few months). > >> >> > >> >> Yes. I also have some serious doubts about closing issues "blindly" > by > >> >> one criteria only by time of inactivity. I think this is just wrong > >> >> and we should not do it. > >> >> > >> >> I agree with Ash that this is really infuriating, when I opened an > >> >> issue, then 3 months have passed and it has been closed due to > >> >> inactivity. This is simply offensive - no matter if it's a bug or > >> >> issue or PR. But I think this is not a problem that we have to deal > >> >> better with the issues as maintainers, not that the stale bot is a > >> >> problem. > >> >> > >> >> But this is only one side of the story - If you have a stale issue > >> >> first and closed, when the issue is "pending response" from the user > - > >> >> I have absolutely no problem with that. If the user who opened it is > >> >> asked for extra information and has not found a time to provide it - > >> >> there is absolutely no reason it should take the mental space of the > >> >> maintainers and we should close it automatically. We can always > >> >> re-open it if the user comes back with more information. > >> >> > >> >> > >> >> But coming back to closing issues without reaction from anyone. If > >> >> this kind of closing happens that we should be ashamed of that means > >> >> something else. That means that we as maintainers have done a bad job > >> >> in triaging this issue. This is really an indication that no-one - > >> >> neither regular contributors (which happens) nor maintainers (which > >> >> should look at it if no contributor does) found a time to read, > >> >> analyse and respond to an issue. No matter what response it will be. > >> >> ANY response from a maintainer (won'tfix, asking for more > information, > >> >> asking others to provide more community to provide more evidence if > >> >> the issue is impossible to diagnose, convert to a discussion) is > >> >> better than silence. Way, way better. > >> >> > >> >> From my point of view, I think the real problem we have is that we > >> >> often have issues open for weeks or monhts without ANY interaction - > >> >> or there is no interaction after the user provided some kind of > >> >> response, additional information etc. Every now and then I do a > >> >> "streak" where I try to provide A response to EVERY issue and PRs > >> >> opened and not responded to for the last few weeks. And there are a > >> >> number of those for issues or PRs that are even 3-4 weeks without any > >> >> answer. > >> >> > >> >> And I am as guilty as everyone else here, but I have a feeling that > if > >> >> we collectively as maintainers spend quite a good chunk of our time > >> >> triaging and responding to issues in due time. I think if we end up > >> >> with a situation where a user raises an issue or PR or provides a > >> >> feedback/new iteration etc. and there is absolutely no response for > >> >> more than a week - this is an indication we have a huge problem. And > >> >> the worst types of thoe are where someone "requests changes", those > >> >> changes are applied and the user pings the reviewer and there are > >> >> weeks of no response (even to multiple pings). Those happen rarely in > >> >> our and I think they are a bit even disrespectful to the users who > had > >> >> "done their part". > >> >> > >> >> And I believe (I have no stats, just gut feeling) that we have that > to > >> >> some extent - for features, bugs, PRs, discussions. > >> >> > >> >> If this happens a lot, then this is I think even equally offensive > (or > >> >> even more) as closing stale issues. I think out of many stats, > >> >> "average response time" to an issue is absolutely most important to > >> >> see how good the community is in handling issues and PRs. This should > >> >> be to both - new issues and PRs. but also to issues that have been > >> >> opened and not responded after the user provided a response back. > >> >> > >> >> Now - many of those are not "intentional" and absolutely no "bad > will" > >> >> - and it is mostly because we do not realise that we have a problem. > >> >> > >> >> We are all humans and have our daily issues and jobs and a lot of > what > >> >> we do for our issues is done in our free time. But maybe we can > >> >> automate and improve that part - which in turn will make our stale > bot > >> >> far "nicer" as it will only have to deal with the case where the > >> >> "user" has not provided necessary input and the maintainers looked > and > >> >> responded to it. > >> >> > >> >> I do not have a very concrete proposal, but some vague ideas how this > >> >> could be improved: > >> >> > >> >> * Maybe we should start with building some simple stats and seeing > our > >> >> responsiveness and find out if we really have a problem there. I am > >> >> sure there must be some tools for that and we might write ours if > >> >> needed - I remember we discussed similar issues in the past > >> >> > >> >> * Then maybe we can figure out a way to share the burden of reviews > >> >> between more committers somehow. For example identify issues and PRs > >> >> that have not been responded or followed up for some time and make > >> >> some way to incentivise and involve committers to provide feedback to > >> >> those > >> >> > >> >> * The stats could help us to understand if we are falling behind and > >> >> maybe we could have some weekly summary of stats that would help us > >> >> with understanding if we should do something and up-end our efforts > in > >> >> triaging > >> >> > >> >> I think - if we do that then the only thing that Stale bot will be > >> >> doing is closing issues and PRs which had not received an input from > >> >> the user. Which is perfectly fine IMHO. > >> >> > >> >> On Sun, Feb 12, 2023 at 10:56 AM Ash Berlin-Taylor <a...@apache.org> > wrote: > >> >> > > >> >> > Got it, yes that makes sense to me! > >> >> > > >> >> > On 12 February 2023 09:36:44 GMT, Elad Kalif <elad...@apache.org> > wrote: > >> >> >> > >> >> >> Thanks for the comments my replies are in blue for all points > raised. > >> >> >> > >> >> >> > We have currently more than 700 issues and many of them have > had no activity since a year. What will we do with those issues? > >> >> >> > >> >> >> Half of the open issues are feature requests thus will not be > impacted. The thing I'm trying to resolve here is to know if the old issue > is still reproducible on main/latest version. If so the issue will be > tagged appropriately and will be kept open if the author does not respond. > We can assume the issue is no longer relevant and close it. > >> >> >> > >> >> >> > Why close only stale issues not stale PR's? > >> >> >> > >> >> >> We already have that. Stale bot works for PR (excluding ones with > pinned label) > >> >> >> > >> >> >> > There is nothing I find more infuriating and demoralising when > dealing with an open source project (and big ones like Kubernetes are the > worst offenders at this) where I find a bug or feature request is closed > simply due to lack of traction. > >> >> >> > >> >> >> I understand and share your concerns. First, this suggestion is > just about bugs not about features. The automation calls for action from > the author to recheck the issue. > >> >> >> This is something I'm doing today manually by going issue by > issue and commenting the exact same thing "Is this issue happens in latest > airflow version?" The auto close part is something that happens today when > we add the pending-response label. My goal is to make sure that the list of > open bugs we have is relevant. I'm not against larger intervals should we > decide for it. To clarify I'm not suggesting to close bug reports because > lack of attraction I'm suggesting to close reports that are not on recent > versions of Airflow. In practice I don't see people trying to reproduce > bugs reported on 2.0 in latest main - this simply doesn't happen so by > having this process we are asking the author to recheck his report. If the > issue is still reproducible then by letting us know that and by having the > proper labels it might get more attraction to it. > >> >> >> > >> >> >> > >> >> >> On Sun, Feb 12, 2023 at 10:15 AM Ash Berlin-Taylor < > a...@apache.org> wrote: > >> >> >>> > >> >> >>> I feel very strongly against automated closing of _issues_. > >> >> >>> > >> >> >>> There is nothing I find more infuriating and demoralising when > dealing with an open source project (and big ones like Kubernetes are the > worst offenders at this) where I find a bug or feature request is closed > simply due to lack of traction. > >> >> >>> > >> >> >>> I might be okay with a very long time (such as stale after 1 > year and close another year after that.) > >> >> >>> > >> >> >>> Ash > >> >> >>> > >> >> >>> On 12 February 2023 02:00:00 GMT, Pankaj Singh < > ags.pankaj1...@gmail.com> wrote: > >> >> >>>> > >> >> >>>> Hi Elad, > >> >> >>>> > >> >> >>>> Thanks for bringing this topic. > >> >> >>>> > >> >> >>>> I also feel we should have some automation to close the stale > issue. > >> >> >>>> > >> >> >>>> Few questions I have > >> >> >>>> - We have currently more than 700 issues and many of them have > had no activity since a year. What will we do with those issues? > >> >> >>>> - Why close only stale issues not stale PR's? > >> >> >>>> > >> >> >>>> > >> >> >>>> On Sun, Feb 12, 2023 at 1:23 AM Elad Kalif <elad...@apache.org> > wrote: > >> >> >>>>> > >> >> >>>>> Hi everyone, > >> >> >>>>> > >> >> >>>>> It's been a while since we talked about the issue triage > process. Currently our process involves a lot of manual work of pinging > issue authors and I'm looking to automate some of it. > >> >> >>>>> > >> >> >>>>> Here are my suggestions: > >> >> >>>>> > >> >> >>>>> 1. add a new bot automation to detect core bug issues > (kind:bug, area:code) that are over 1 year old without any activity. The > bot will add a comment asking the user to check the issue against the > latest Airflow version and assign a "pending-response" label. If the user > will not respond the issue will be marked stale and will be closed by our > current stale bot automation. I suggest 1 year here because in 1 year we > usually have 3 feature releases + many bug fixes which contain a lot of > fixes. We don't normally go back to check bugs on older versions unless > reporting as reproducible on the latest version. There can be 2 outcomes of > this: > >> >> >>>>> > >> >> >>>>> The author will comment and say it is reproducible in that > case we will assign the updated affected_version label and the issue will > be bumped up. > >> >> >>>>> The author will not comment. In that case we can assume the > problem is fixed/not relevant and the issue will be closed. > >> >> >>>>> > >> >> >>>>> 2. similar to (1) for providers with labels (kind:bug, > area:provider) and with a shortened time period of 6 months as providers > release frequently. > >> >> >>>>> > >> >> >>>>> 3. similar to (1) for airflow-client-python and > airflow-client-go with no labels and period of 6 months as well. > >> >> >>>>> > >> >> >>>>> 4. On another front, we sometimes miss the triage of new > issues. My suggestion is that any new issue opened will automatically have > a needs-triage label (this is practice several other projects use) That way > we can easily filter the list of issues that need first review. When > triaging the issue we will remove the label and assign proper ones (good > first issue, area, kind, etc..) > >> >> >>>>> > >> >> >>>>> What do others think? > >> >> >>>>> > >> >> >>>>> Elad > >> >> >>>>> > >> >> >>>>> >
