On Thu, Apr 2, 2020 at 7:48 PM Adam Williamson <adamw...@fedoraproject.org> wrote:
> On Thu, 2020-04-02 at 11:52 +0100, Leigh Griffin wrote: > > [up front: apologies for any weird formatting in this email, Evolution > is crashing on me like crazy while I try to edit it, so I'm sending it > from Roundcube which I don't normally do] > > > I'm also not entirely sure about Adam Saleh, but he does not have any > > > infra activity I can find since the end of January and lists himself as > > > working for Exponea on Github and his personal blog. > > > > > > > Adam is working in CPE. CPE isn't entirely about Infra, he is working > > on CI > > improvements at the moment alongside some others > > Okay, well, next time I see him I'll mention he might want to update the > Exponea references :P > > I just pinged him on it :) > But "CPE isn't entirely about Infra" is sort of the point here: my > initial assertion was that there are fewer people working on > *Fedora-relevant app development* than there were before. Rightly or wrongly our span of control is: - Infrastructure service and hardware keep alive - Service maintenance / bug fixes etc The above two are lights on work - Service creation Across 2 distros. The rough breakdowns are >50% of our resourcing just on lights on work. > I'm happy to > accept that there may be more people in the CPE team than there were at > some point in its past, but that's not what I'm actually interested in > here. CI is great (whichever CI you mean, I lose track sometimes :>) but > someone working on CI is not someone working on the Fedora app > infrastructure, or on sourcing/deploying/integrating/maintaining > outsourced replacements for it, which is the actual problem space here. > Anything we do benefits the infrastructure or the services that make it up, the CI work here is improvements that take a lot of the manual work out for us / the community and a lot of our initiatives are geared towards easing that pressure on us while offering a value add. > > > > If I'm missing anyone, please do correct me. > > > > Other developers in our pool right now are: > > - Ryan Lerch > > I mentioned already that Ryan could go in either list or neither - he > was around at both times but I wasn't sure whether to count him as a > developer or not. But he can't count to one list but not the other, as > he's been here all along. :) > > > - Michal Konecny > > - Mohan Boddu > > - Tomas Hrcka > > - Nils Philippsen > > - Vipul Siddharth > > - James Richardson > > > > We additionally have 2 new Ops folks joining us over the next 2 weeks. > > > > Across them, the majority are working on the Fedora aspects (Infra, > > Dev, > > Releng) of the CPE remit. > > So yeah, let's discount the releng folks first, because releng has > existed all along We can't really discount them when their workload extends beyond Tomas and Mohan, that workload hits Smooge, Kevin, Clement and others. Their work is now within the lights on category to help with the workload, look at improving how they operate and ultimately trying to not leave them drown. If they drown, we miss releases. This is a much bigger workload on the team than folks realise. > and - as I said - my original statement was not about > "people who are organizationally in the same team as the people who work > on Fedora app stuff" but "people who work on Fedora app stuff". So that > lets out Mohan and Tomas. > Granted, but I wanted to clarify the above. > > As for the others: so, I didn't count Michal at first as I could not > find any infra-related activity for him, but since you included him I > looked closer, and found he's just hiding himself really well - his > github account name, for some inaccountable reason, is "Zlopez", and his > profile doesn't have his real name in it. I really want to know the story behind that name, I must ask him! > So I finally got his logs: > > https://github.com/Zlopez > https://pagure.io/user/zlopez > > ...and yeah, there's some infra activity there, so add him to the list. > > Initially I was just looking at Github logs, as all the infra stuff I > could find was hosted there, and Nils' list is: > > https://github.com/nphilipp > > which was busy up to the end of December but looked extremely empty all > of this year, so I figured he'd switched track or role or something and > didn't count him. But now I look closer, it seems what happened is he's > been working almost exclusively on a thing called rpmautospec, which is > hosted on Pagure: > > https://pagure.io/Fedora-Infra/rpmautospec > > so, he's clearly working hard on something, although I'm not sure it's > directly a part of Fedora app/infra stuff - it's an automatic packaging > thingy, it looks like, I'm not sure what it's being used for. In fact, > it seems like pingou and Adam Saleh are doing quite a bit of work on > this thing too, so it's clearly sucking up quite a lot of developer > time. It's also a fairly new project, which seems interesting given that > "we don't have time to maintain all the projects we have already" is the > recurring refrain here. > It was a request in our priority queue which is driven by the CentOS Board and the Fedora Council. We deliver value for them and sometimes it involves creating new services, sometimes it involves modifying an existing service. We need to weigh up that value proposition for what it will deliver Vs what it will cost us longer term. The difference now is we have a minimum team of 3 people per initiative to ensure quality, resiliency in the solution and longer term maintenance that removes a single point of failure > > Here's Vipul's lists: > > https://github.com/siddharthvipul > https://pagure.io/user/siddharthvipul1 > > he seems to work exclusively on CentOS CI. Okay, Fedora *uses* CentOS > CI, but presumably back in the 2018 timeframe, someone (whether that's > Vipul or someone else) was working on CentOS CI who wasn't included in > my list, because I only listed people working on Fedora stuff. So this > still seems like a wash. > James I didn't count as he's listed as an intern. But here's his Github > log (I can't find him on Pagure): > > https://github.com/james02135 > > so I don't mean this as a knock at all, but he's obviously not > equivalent to one regular full-time dev, nor would we (I hope) expect > him to be. > > So if we include Ryan on both lists and add Michal, we get to this. > Previous: > > puiterwijk > Randy (bowlofeggs) > pingou > jcline > Ralph > jflory > abompard > rlerch > > Current: > > abompard > lrossetti/odra > pingou > scoady > cverna > mkonecny (Zlopez) > > If we also count rpmautospec, we add Nils and Adam to the current list: > It is part of the Fedora remit, so yes. > > abompard > lrossetti/odra > pingou > scoady > cverna > mkonecny (Zlopez) > nphilipp > asaleh > > and it looks like it's about even. But that's counting rpmautospec, > which seems to be an odd counterpoint to the overall CPE narrative that > "we have too many projects and we're trying to get rid of the > maintenance burden" - you already don't have resources to maintain the > things that Fedora very definitely needs and is using right now, but you > *do* have resources to dedicate some or all of the time of three > engineers to a brand new project which certainly looks cool but is > definitely a new invented-here thing that isn't absolutely essential to > Fedora's needs and isn't replacing an existing project? > Like I said, it came as a request for us to solve a problem. We aren't in the habit of inventing work to do for ourselves anymore, we can propose changes, we can propose ease of life improvements, they ultimately go to our stakeholders for consideration and we plan our quarter out. > > To be really clear: I don't want the takeaway from this to be "Adam is > very mad and doesn't want CPE to be allowed to work on cool new projects > any more". I like cool new projects! Cool new projects are great! I > write them myself sometimes! I'm just having trouble joining up the dots > here in terms of high-level strategy. > > > The number of active developers on Fedora initiatives has gone up > > drastically since I joined the team in 2019. You are possibly not > > seeing > > that as the team have moved from a model of siloed work on multiple > > apps, > > swimming against the tid working 16 hour days, to working on team > > oriented > > initiatives to add real value to the ecosystem. So the noise of working > > on > > multiple small things at once is not as loud as it was in 2018 which is > > giving that illusion. > > What I'm looking at is the commit logs. That's all that ultimately > matters. But see above revisions, of course. > I think that's a very narrow view of the world to base your assertions on commit logs only, I don't see the value in it. If your end argument here is that CPE do not spend enough time working on Fedora things then you are very mistaken. Almost 80% of our team capacity is on Fedora and our upcoming Q2 initiatives this is the same. > > [snip the stuff about whether we need elections apps and blahblah > because I don't have anything to add there] > > > > > Now when the CPE team goes and ask for more people because we > struggle > > > with > > > > current situation, I can only guess that these non critical > applications > > > > are mentioned. If I was putting my own money to sponsor a team to > help > > > > building a Linux distribution I would be asking why do we have to > > > develop a > > > > calendar application or why do we need a custom git forge. I > personally > > > > find really great that the different use cases and requirements for > the > > > use > > > > of Pagure were gathered and I am convinced that people working on > this > > > did > > > > their very best to find a use case to justify the investment needed > in > > > > Pagure but it seems that we don't have such a thing. > > > > > > I think other people are following up on this, but from following the > > > discussion, it appears to me that there seems to have been a large slip > > > somewhere along the line from Ben submitting a list of ~50 Fedora > > > submissions, to Leigh suggesting that (after those were, mysteriously, > > > "summarized" somehow) > > > > Can you help me understand what the mystery is about? We took in 300+ > > requirements that we distilled down into the generic list that we came > > up > > with, many of which are buckets / Epics. Every single requirement was > > analysed. I have said this multiple times but please, if this is still > > a > > mystery to you, let me know how I can help clarify it. > > The specific gap I am talking about is the gap between this list > submitted by Ben Cotton on behalf of Fedora: > > > https://lists.fedoraproject.org/archives/list/council-disc...@lists.fedoraproject.org/thread/OEPDGVKYAJIQ6YYZU5J3XT3LHXFROFI5/ The thread you reference is not the list that was submitted. The first post on that is not the final list, the final list was a result of the debates and discussions that occurred on that thread and was submitted directly to CPE. To be clear, we did not pull our Fedora requirements from that list you are referencing. > > > and the 'summarized' list you have pointed to here: > > https://hackmd.io/@My419-DISUGgo4gcmO1D6A/HJB74lcL8 > > Now, right off the bat, we have a huge problem. The "summarized" list > claims this: > > "after duplications and similar in theme requirements were merged > together, we were left with the following unique User Story list:" > > you've also phrased the same thing slightly differently on the mailing > list: > > "As a team we evaluated every single requirement (over 300 of them) and > the presentation in the combined User Story list is an amalgamation of > like for like User Stories to capture at a high level the spirit of the > requests." > However, the top three points on the Fedora list relate to F/OSS and > self-hosting principles. These points are entirely missing from the > "summarized" list. They were never formal requirements as submitted by Ben. I'm assuming you did not read Matthews reply <https://lists.fedoraproject.org/archives/list/council-disc...@lists.fedoraproject.org/message/E5R55AS3Z7QLGRXAT6RGQKJNVXLTAJHJ/> on the thread you linked which descoped om prem and OSS as a standard, which I am assuming Ben used as his basis to remove them from the formally submitted list. > They have not been "merged" with "duplications" or > "similar in theme requirements". The "spirit" of them has not been > "captured" in a "User Story". They are just *not there*. They have not > been summarized. They were dropped. So the claim is false, and there was > not just a summarization process here, there was some sort of filtering > process. Things from the stakeholder lists were outright left out of the > "summarization". I'll pick this up again later. > > Of the other 44 points on the Fedora list, 23 mention dist-git > specifically. The term "dist-git" is mentioned 40 times. Some of these > aren't truly "dist-git" specific, and are really just generic git or > forge requirements, but many of them *are* specific to dist-git and the > packager workflow integration; I'd count at least points 9, 17, 18, 19, > 20, 22, 23, 24, 25, 26, 28, 29, and maybe 37 in this bucket. In the > "summarized" list, the string "dist-git" does not appear once. I can buy > that a few of the requirements are *possibly* "summarized" into the > points about access control, but generic access control requirements do > not at all cover all the details of dist-git integration. Effectively, > all the details relating to dist-git and packager workflow interaction - > which constitute the large majority of significant requirements on the > Fedora list that wouldn't be satisfied by all of the candidates - are > lost in the summarized list. > If your entire focal point is on the summarised list that I presented as an amalgamation of the requirements then your entire argument is off. I honestly cannot say it in any other way shape or form, we evaluated every single requirement in making our decision. If it helps, I can update the original hackmd with the stories and add all the Fedora ones verbatim? It won't change what the analysis showed but it might help with your analysis here. Let me know if that works for you. > > You say that all the (original 300+) requirements were analyzed, but > when you have been asked for specific details on any of the issues > relating to dist-git / packager workflow integration, these are the > kinds of answers you have given: > > "The Fedora specific requirements (as I am on the Fedora lists here) had > very few unique needs such that both Gitlab and Pagure would have > satisfied their desire." > > If you read the two lists above and my notes, I do not see how you can > possibly claim that Fedora "had very few unique needs". I stand by that claim. > This was the > quote that first got me really worried that Fedora's needs had not been > properly considered and understood. (We can also note, of course, that > while both Gitlab CE and Pagure can potentially satisfy the "open > source" and "self-hosted" requirements, Gitlab Ultimate - to which the > decision appears to be weighted, a suggestion no-one has yet denied, > beyond a very mealy-mouth and deniable "no option is off the table yet" > - does not satisfy either). You later made more or less the same claim > again: > > "The needs of Fedora, CentOS, RHEL and the CPE team were weighed equally > in our decision. Fedora had very few specific needs in their > requirements where as some stakeholder groups had." > > which only makes me worry more. > Today, we are meeting Gitlab, first thing we are discussing is license options. Anything else is speculation on your part. > > [to a question about Bugzilla package workflow integration] "I don't > have an answer to this as we haven't done that deep level of tooling > analysis and integration analysis yet." > > How can you have considered Fedora's requirements with regard to tooling > and integration if you haven't "done a deep level of tooling analysis > and integration analysis yet"? > Key word is deep. If we done the depth of analysis that you are desiring here, we might have the decision around Christmas time. The level of due diligence was done. > > [when Till attempted to make similar points about the dist-git > requirements] "The User Stories are deliberately vague and that > represents around 10 unique requirements that boil down to having group > membership and management capabilities." > > "Having group memembership and management capabilities" is nowhere close > to covering the dist-git integration requirements. Pagure had these more > or less from day one, but we still had to spend months on engineering > the dist-git integration. You cannot count generic group membership and > ACL capabilities as "covering" these requirements, and if you did, that > was a fundamental misunderstanding of what Fedora was requesting. > > [when asked if you know how many existing integrations with Pagure there > are] "No. Our analysis will tell us that in due course" > > Once again - if you don't even know that, how can you possibly have > fully evaluated Fedora's detailed requirements regarding dist-git / > packager workflow integration? You defended this by saying "I do know of > many important integrations that I come into contact with during my day > to day job in the team, that doesn't mean I know every single one of > them. We analysed the critical path needs via the requirements and any > reference to tooling integrations (e.g. fedora messaging was reference > in multiple conversations) will be brought forward with us for deeper > technical conversations", but the extent to which the detail in Fedora's > list is lost in the "summarized" list gives me (and I suspect others) no > confidence at all that this analysis was done correctly. > > To put it slightly more generally: I think Fedora as a whole would have > expected that CPE was *already* deeply familiar with the importance of > dist-git / packager workflow integration. We are. > That CPE would *already* > understand that this - along with F/OSS principles - would be the meat > of Fedora's "unique" requirements in this process. The latter was not a requirement as per council directives on git hosting as mentioned in this thread already and the fact it did not end up as a formal requirement. > And that the rolling > of these into "user stories" would be a useful technique for > facilitating the process, but not the Holy Grail of the process outside > of which nothing would matter at all; I would have expected the > knowledge that (for instance) pingou already has about this stuff to > have just been included directly in the decision-making process. > Pingou was involved in the analysis of the requirements, as were other team members. > > However, as you've described it, that doesn't seem to have been what's > happened. Instead, this deep base of existing knowledge within your team > seems to have been sidelined in the decision process, and instead the > decision has been made based mostly or entirely on this apparently > somewhat wobbly process of bundling up and "summarizing" requirements > (through three levels of the telephone game, from Fedora lists to the > Council to this "summarized" list) as "User Stories". Or at least, that > is how you are presenting the process as having worked and the decision > as having been made, and the focus on generic forge-type requirements > and the repeated contention that Fedora had "very few" "unique" > requirements reinforces that impression. > > On a general note, I'm really having trouble understanding the overall > logic behind this "requirements" process at all. You've mentioned there > were over 300 requirements from the stakeholders and you summarized them > into this list. But when people have pressed you on individual items in > the summarized list, you've said stuff like this: > > [on private PR comments] "It is a valid requirement from a stakeholder > that we had to take at face value." > > OK, but why did *this* requirement make the "summarized" list when > others - as I've mentioned above - didn't? Because that is a unique requirement, with a specific need, that came from 2 stakeholder groups and was therefore merged in. I'm unsure what ones did not meet your summarised list, but like I said already, the entire list received was parsed and analysed and the ask in the requirement rolled into generic requirements to capture them at a bucket level. I cannot be any more clearer on this. > I am discounting the claim > that the list somehow summarizes all the given requirements, because it > very clearly and inarguably does not. You had to "take it at face > value", but there were 300 other requirements that had to be taken "at > face value" as well, and not all of them made the list. As that's true, > you can't duck Neal's question: why did *this specific request*, which > Neal gives various reasons for not considering a great priority, make > the summarized list? You can say "we think you're wrong that it's not a > very important request, we think it is an important request for reason > X", but you can't just duck the question by saying there was no choice > made that can be questioned. Clearly there *was*. > > [on the mobile app requirement] "It's a requirement that came from the > Fedora Community and one we could not satisfy as soon as Github was > ruled out. I do agree the experience is not great on it." > > Again, given the context that there clearly was some sort of filtering > process here, this reads as very bizarre. A bells-and-whistles feature > requirement that you acknowledge only Github barely fulfils, badly, > makes the summarized list, but "we want it to be F/OSS" and multiple > very specific fundamental dist-git integration requirements somehow > don't make the summarized list or are garbled beyond recognition? > The mobile one made it to the list as it was a unique standalone requirement, as stated previously. > > [on gists] "It's not up to me to gauge the merit of that as a use case > but it's a valid requirement which we considered." > > But the question is why *this* item from the list of 300+ made the > "summarized" list rather than being smooshed into something very vaguely > related, or dropped entirely. Did someone *else* gauge the merit of that > as a use case? Or did you make the decision on some other grounds? > > You have also implied there was some kind of "weighting" involved, when > asked if a requirement that "Github" be at the top of the UI would have > been accepted: > I asked and suggested that a MoSCoW style weighting be put in place by stakeholders, this did not happen for Fedora requirements. > > "Would it be weighed equally with a more core functional requirement? > The answer is of course no but we would > have respected that request." > > but you have not provided any further detail on exactly how this > "weighting" worked and how Fedora's requirements (those which were not, > apparently, entirely dropped) were "weighted". How was Fedora's "we want > it to be open source" "weighed" against requests that cannot currently > be satisfied by an open source choice, if it was considered at all? > That was not a requirement but we did not weigh anything that was not a critical use case of a git forge, so mobile apps is not going to make or break the usage of a git forge. > > I will also make a side note: it was claimed earlier in this thread that > the "mobile app" requirement "came from Fedora", but there is a rather > interesting discrepancy there. The Fedora requirement reads: > > "As a Fedora contributor, I can perform issue and pull request actions > on mobile devices via a native mobile app or a mobile-friendly webapp so > that I can contribute while away from my desk." > > The "summarized" version reads: > > "As a General User > I want a mobile native app > To allow me contribute while away from my desk" > I don't really find that an interesting discrepancy that we choose the generic wording for a mobile app requirement. If you like, I can update the wording to be verbatim, it doesn't change the actual ask of the requirement which is mobility use cases, which is what we evaluated on. > > somehow the "or a mobile-friendly webapp" bit got lost. (And the > specific actions). > -- > Adam Williamson > Fedora QA Community Monkey > IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net > http://www.happyassassin.net > > -- Leigh Griffin Engineering Manager Red Hat Waterford <https://www.redhat.com/> Communications House Cork Road, Waterford City lgrif...@redhat.com M: +353877545162 IM: lgriffin @redhatjobs <https://twitter.com/redhatjobs> redhatjobs <https://www.facebook.com/redhatjobs> @redhatjobs <https://instagram.com/redhatjobs> <https://red.ht/sig>
_______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
