Hi Frank,
I’m not sure I would equate the “rough metrics” implied in the current project lifecycle as providing significant guidance or establishing a meaningful measure of a projects maturity. I might argue that they are sorely lacking on community discussion and agreement. I see this dialog on community metrics as one vehicle for our community to chime in and help set these guidelines with the TSC. I would not propose we stop doing this because we feel the lifecycle metrics are sufficient, rather the opposite in that this work may help us find meaningful lifecycle metrics as a community. / Chris From: "Frank Brockners (fbrockne)" <fbroc...@cisco.com> Date: Thursday 8 September 2016 at 15:28 To: Raymond Paik <rp...@linuxfoundation.org> Cc: Daniel Smith <daniel.sm...@ericsson.com>, Christopher Price <chrispric...@gmail.com>, Dave Neary <dne...@redhat.com>, "SULLIVAN, BRYAN L" <bs3...@att.com>, TECH-DISCUSS OPNFV <opnfv-tech-discuss@lists.opnfv.org> Subject: RE: [opnfv-tech-discuss] Following up on Project Health metrics discussion Ray, thanks – in which case, let’s first try to get the overall project lifecycle concept adopted. Once done – and we have experiences with the project life cycle, we can look into whether we need to create additional metrics etc. We could do that as another iteration of the project lifecycle doc – but for now, it would be key to get the lifecycle really adopted by projects. Let’s do one step after the other. Thanks, Frank From: Raymond Paik [mailto:rp...@linuxfoundation.org] Sent: Donnerstag, 8. September 2016 15:23 To: Frank Brockners (fbrockne) <fbroc...@cisco.com> Cc: Daniel Smith <daniel.sm...@ericsson.com>; Christopher Price <chrispric...@gmail.com>; Dave Neary <dne...@redhat.com>; SULLIVAN, BRYAN L <bs3...@att.com>; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] Following up on Project Health metrics discussion Frank, It's in the same spirit as the project lifecycle you mentioned earlier. For example, I'd like to use it as one of the data points for guidance (as you noted) to highlight projects that are not doing as well.... Thanks, Ray On Thu, Sep 8, 2016 at 2:28 AM, Frank Brockners (fbrockne) <fbroc...@cisco.com> wrote: Hi Ray, question: What problem are we trying to solve with the metrics for project health? For the project lifecycle we have a defined process already – so nothing new is needed. Thanks, Frank From: Raymond Paik [mailto:rp...@linuxfoundation.org] Sent: Donnerstag, 8. September 2016 07:14 To: Daniel Smith <daniel.sm...@ericsson.com> Cc: Christopher Price <chrispric...@gmail.com>; Dave Neary <dne...@redhat.com>; Frank Brockners (fbrockne) <fbroc...@cisco.com>; SULLIVAN, BRYAN L <bs3...@att.com>; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] Following up on Project Health metrics discussion All, Glad to see the thread coming back to life (plus comments on the wiki) :-) Definitely understand concerns about the composite score. Maybe another option would be to start by looking at 4 areas (git, Jira, wiki, etc.) individually. I also want to suggest that if a project is active (and even if most of the work is being done upstream) I doubt the activities in OPNFV repo or other tools would be zero over a period of 3-6 months. So I think having a regular metrics (composite or otherwise) would be helpful in identifying projects that are either in need of help or are candidates for "archiving" My 2 cents.... Ray On Wed, Sep 7, 2016 at 9:42 PM, Daniel Smith <daniel.sm...@ericsson.com> wrote: Hello All. My take on this and sorry maybe a bit blunt, but I don’t see what the purpose is here? While guideline, guidance and such are good things, the discussion below seems awfully heavy and very boxed in. As Chris in the evaluation on how to improve the community, I am not sure how we could ever resolve what one person thinks is an adequate measure of something over another (i.e endless debate), since metrics, improvements - unless we are talking about something quantifiable - and that would mean taking this to code optimization level ultimately - are subjective based and implement/reaction based measured over time in the case of adjustment. As well +1 to doing our own assessments - since it should be us that outlines what we want to say about how we feel we have improved, and this only makes sense I would think as we are the only ones who have the context to see where we came from to where we are and what we are talking about doing next. I would say however, that I am really concerned at that level of discussion across all meetings on project handling and process and such - release and otherwise - while this shows we are active and "moving" sure - there is a lot of dissonance in the communication, mixed messages and again, lots of information overflow. I would like to see the "Operational" parts of OPNFV (Release Project, INFRA, TSC) focus more on how we support the projects in terms of day to day pain points - that is a solid, referable release process, a solid, regularly maintained method of publish, a solid Change process (for anything) that ensures that we don’t discuss and move from one week to another - not because an idea is good or bad, but simply cause we need to recognize that there are more than the 25 or so regular attendees in the groups / this mailing list. Would there be a market for such publishing outside our domain I would wonder? For Code stats and fun numbers those are nice to have, but a qualitative review of how we have improved our processes as a community, im not sure? I would vouchsafe that our work in the projects themselves are what OPNFV is pushing out, the operational aspects of "how we are doing things" - while nice to outline, against the backdrop of every growing laundry list of things to "Decide" is really not high on a list? I didn’t understand from below the "why" and "who" for this fully. Cheers and thank you D -----Original Message----- From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Christopher Price Sent: Wednesday, September 07, 2016 9:20 PM To: Dave Neary <dne...@redhat.com>; Frank Brockners (fbrockne) <fbroc...@cisco.com>; SULLIVAN, BRYAN L <bs3...@att.com>; Raymond Paik <rp...@linuxfoundation.org>; opnfv-tech-discuss@lists.opnfv.org Subject: Re: [opnfv-tech-discuss] Following up on Project Health metrics discussion Good comments Dave and everyone, I’d like to share my take on it is this. I don’t see any problem in looking at the metrics we already publish and using them to help us create a better understanding across our community on how we go about getting things done. (Maybe also helping find ways of improving ourselves.) As was mentioned we publish all these metrics today. I would prefer to see the OPNFV community drawing it’s own assessments of what that means rather than leaving it up to the imagination of people who are not engaged in our project to inform us of what they think. Reflection and introspection are useful activities; however, I also agree that a metric is not definitive of an activity. I suggest we approach this activity with a view to evaluate and improve our community. Also to maybe publish our activities in a way we want to be viewed, if we can arrive at such a thing. / Chris On 07/09/16 15:24, "Dave Neary" <opnfv-tech-discuss-boun...@lists.opnfv.org on behalf of dne...@redhat.com> wrote: Hi, On 09/07/2016 02:24 AM, Frank Brockners (fbrockne) wrote: > +1. > > Also note that when we defined the project lifecycle we used metrics > like the ones mentioned only as guidance rather than something to > compute a composite value – and even there, we did not constrain things > to metrics in OPNFV only. > > > > Frank > > > > *From:*SULLIVAN, BRYAN L [mailto:bs3...@att.com] > *Sent:* Dienstag, 6. September 2016 18:48 > *To:* Frank Brockners (fbrockne) <fbroc...@cisco.com>; Raymond Paik > <rp...@linuxfoundation.org>; opnfv-tech-discuss@lists.opnfv.org > *Subject:* RE: [opnfv-tech-discuss] Following up on Project Health > metrics discussion > > > > I’m unsure of the overall value of this exercise. Simply ask the PTLs > what the “health” of the project is. An honest PTL will tell you, and > that’s the only type we should elect. I dispute that this is a question of honesty. When I was starting out my software engineering career, I had an experienced manager who would ask me for estimates on how projects I was working on were going. "Fine," I would answer, "I should be finished that feature next week." Next week rolled around, and I'd get the question again. "Almost done, just a few bugs to work out. By next week it'll be done." I wasn't lying the first week, I just had no idea how to estimate software development. Similarly, if you ask a PTL if their project is "healthy", I would fully expect all projects to say yes - after all, what does an unhealthy project mean? This is where metrics come in... if we can flag certain sets of behaviours as indicating an issue, that allows adjustment. It's not enough to say "30 messages per month with that tag to tech-discuss - that seems pretty good" - by looking at behaviours, we can see who is not engaging effectively upstream, who is developing a lot of code in an OPNFV repo, which projects seem stuck in wiki/email discussions, which projects are not using Jira so well, etc. I don't know what those sets of behaviours/metrics might be, I figure that is the point of the project health metrics initiative. That said, I agree with both Frank and Bryan that unadorned/contextless composite metrics can mask, rather than reveal, some of these issues, and as such are not useful. With context, and with a human eye to evaluate things, some form of composite can be a useful diagnostic tool. Thanks, Dave. > Publish metrics if you want (we already do), but I would avoid trying to > draw conclusions from them. We do not have the luxury (if you can even > call it that!) of creating and maintaining a project-introspection > framework ala what you might see in corporate development shops. Even > considering what metrics are “useful” for specific purposes (e.g. what > “useful”/reliable implications can you draw from them) takes too much > time away from the real work. > > > > Thanks, > > Bryan Sullivan | AT&T > > > > *From:*opnfv-tech-discuss-boun...@lists.opnfv.org > <mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] *On Behalf Of *Frank > Brockners (fbrockne) > *Sent:* Tuesday, September 06, 2016 7:39 AM > *To:* Raymond Paik; opnfv-tech-discuss@lists.opnfv.org > <mailto:opnfv-tech-discuss@lists.opnfv.org> > *Subject:* Re: [opnfv-tech-discuss] Following up on Project Health > metrics discussion > > > > Hi Ray, > > > > thanks for posting the initial cut. IMHO a "composite score", as > proposed on the page, could be **very** misleading, especially for > projects which do most of the work upstream. So unless we track all > upstream repos and upstream Jiras (or similar), I would suggest to > **not** compute a composite score but evaluate things qualitatively only. > > > > Thanks, Frank > > > > *From:*opnfv-tech-discuss-boun...@lists.opnfv.org > <mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] *On Behalf Of > *Raymond Paik > *Sent:* Montag, 29. August 2016 19:33 > *To:* opnfv-tech-discuss@lists.opnfv.org > <mailto:opnfv-tech-discuss@lists.opnfv.org> > *Subject:* [opnfv-tech-discuss] Following up on Project Health metrics > discussion > > > > All, > > > > I had an action item from last week to start a wiki page for the > "project health metrics". You can find a proposal page > at https://wiki.opnfv.org/display/PROJ/Project+Health+Metrics. > > > > Please add your comments/feedback via email or directly on the wiki > page. I listed four activity areas that was discussed on the TSC call, > but feel free to add other activities that the community should consider. > > > > Thanks, > > > > Ray > > > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > -- Dave Neary - NFV/SDN Community Strategy Open Source and Standards, Red Hat - http://community.redhat.com Ph: +1-978-399-2182 / Cell: +1-978-799-3338 _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss _______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
