All, Glad to see the thread coming back to life (plus comments on the wiki) :-) Definitely understand concerns about the composite score. Maybe another option would be to start by looking at 4 areas (git, Jira, wiki, etc.) individually.
I also want to suggest that if a project is active (and even if most of the work is being done upstream) I doubt the activities in OPNFV repo or other tools would be zero over a period of 3-6 months. So I think having a regular metrics (composite or otherwise) would be helpful in identifying projects that are either in need of help or are candidates for "archiving" My 2 cents.... Ray On Wed, Sep 7, 2016 at 9:42 PM, Daniel Smith <daniel.sm...@ericsson.com> wrote: > Hello All. > > My take on this and sorry maybe a bit blunt, but I don’t see what the > purpose is here? > > While guideline, guidance and such are good things, the discussion below > seems awfully heavy and very boxed in. > > As Chris in the evaluation on how to improve the community, I am not sure > how we could ever resolve what one person thinks is an adequate measure of > something over another (i.e endless debate), since metrics, improvements - > unless we are talking about something quantifiable - and that would mean > taking this to code optimization level ultimately - are subjective based > and implement/reaction based measured over time in the case of adjustment. > > As well +1 to doing our own assessments - since it should be us that > outlines what we want to say about how we feel we have improved, and this > only makes sense I would think as we are the only ones who have the context > to see where we came from to where we are and what we are talking about > doing next. > > I would say however, that I am really concerned at that level of > discussion across all meetings on project handling and process and such - > release and otherwise - while this shows we are active and "moving" sure - > there is a lot of dissonance in the communication, mixed messages and > again, lots of information overflow. I would like to see the > "Operational" parts of OPNFV (Release Project, INFRA, TSC) focus more on > how we support the projects in terms of day to day pain points - that is a > solid, referable release process, a solid, regularly maintained method of > publish, a solid Change process (for anything) that ensures that we don’t > discuss and move from one week to another - not because an idea is good or > bad, but simply cause we need to recognize that there are more than the 25 > or so regular attendees in the groups / this mailing list. > > Would there be a market for such publishing outside our domain I would > wonder? For Code stats and fun numbers those are nice to have, but a > qualitative review of how we have improved our processes as a community, im > not sure? I would vouchsafe that our work in the projects themselves are > what OPNFV is pushing out, the operational aspects of "how we are doing > things" - while nice to outline, against the backdrop of every growing > laundry list of things to "Decide" is really not high on a list? I > didn’t understand from below the "why" and "who" for this fully. > > > Cheers and thank you > D > > > > > -----Original Message----- > From: opnfv-tech-discuss-boun...@lists.opnfv.org [mailto: > opnfv-tech-discuss-boun...@lists.opnfv.org] On Behalf Of Christopher Price > Sent: Wednesday, September 07, 2016 9:20 PM > To: Dave Neary <dne...@redhat.com>; Frank Brockners (fbrockne) < > fbroc...@cisco.com>; SULLIVAN, BRYAN L <bs3...@att.com>; Raymond Paik < > rp...@linuxfoundation.org>; opnfv-tech-discuss@lists.opnfv.org > Subject: Re: [opnfv-tech-discuss] Following up on Project Health metrics > discussion > > Good comments Dave and everyone, I’d like to share my take on it is this. > > I don’t see any problem in looking at the metrics we already publish and > using them to help us create a better understanding across our community on > how we go about getting things done. (Maybe also helping find ways of > improving ourselves.) > > As was mentioned we publish all these metrics today. I would prefer to > see the OPNFV community drawing it’s own assessments of what that means > rather than leaving it up to the imagination of people who are not engaged > in our project to inform us of what they think. > > Reflection and introspection are useful activities; however, I also agree > that a metric is not definitive of an activity. > I suggest we approach this activity with a view to evaluate and improve > our community. Also to maybe publish our activities in a way we want to be > viewed, if we can arrive at such a thing. > > / Chris > > On 07/09/16 15:24, "Dave Neary" <opnfv-tech-discuss-bounces@ > lists.opnfv.org on behalf of dne...@redhat.com> wrote: > > Hi, > > On 09/07/2016 02:24 AM, Frank Brockners (fbrockne) wrote: > > +1. > > > > Also note that when we defined the project lifecycle we used metrics > > like the ones mentioned only as guidance rather than something to > > compute a composite value – and even there, we did not constrain > things > > to metrics in OPNFV only. > > > > > > > > Frank > > > > > > > > *From:*SULLIVAN, BRYAN L [mailto:bs3...@att.com] > > *Sent:* Dienstag, 6. September 2016 18:48 > > *To:* Frank Brockners (fbrockne) <fbroc...@cisco.com>; Raymond Paik > > <rp...@linuxfoundation.org>; opnfv-tech-discuss@lists.opnfv.org > > *Subject:* RE: [opnfv-tech-discuss] Following up on Project Health > > metrics discussion > > > > > > > > I’m unsure of the overall value of this exercise. Simply ask the PTLs > > what the “health” of the project is. An honest PTL will tell you, and > > that’s the only type we should elect. > > I dispute that this is a question of honesty. > > When I was starting out my software engineering career, I had an > experienced manager who would ask me for estimates on how projects I > was > working on were going. "Fine," I would answer, "I should be finished > that feature next week." > > Next week rolled around, and I'd get the question again. "Almost done, > just a few bugs to work out. By next week it'll be done." > > I wasn't lying the first week, I just had no idea how to estimate > software development. > > Similarly, if you ask a PTL if their project is "healthy", I would > fully > expect all projects to say yes - after all, what does an unhealthy > project mean? > > This is where metrics come in... if we can flag certain sets of > behaviours as indicating an issue, that allows adjustment. It's not > enough to say "30 messages per month with that tag to tech-discuss - > that seems pretty good" - by looking at behaviours, we can see who is > not engaging effectively upstream, who is developing a lot of code in > an > OPNFV repo, which projects seem stuck in wiki/email discussions, which > projects are not using Jira so well, etc. I don't know what those sets > of behaviours/metrics might be, I figure that is the point of the > project health metrics initiative. > > That said, I agree with both Frank and Bryan that unadorned/contextless > composite metrics can mask, rather than reveal, some of these issues, > and as such are not useful. With context, and with a human eye to > evaluate things, some form of composite can be a useful diagnostic > tool. > > Thanks, > Dave. > > > > > > Publish metrics if you want (we already do), but I would avoid > trying to > > draw conclusions from them. We do not have the luxury (if you can > even > > call it that!) of creating and maintaining a project-introspection > > framework ala what you might see in corporate development shops. Even > > considering what metrics are “useful” for specific purposes (e.g. > what > > “useful”/reliable implications can you draw from them) takes too much > > time away from the real work. > > > > > > > > Thanks, > > > > Bryan Sullivan | AT&T > > > > > > > > *From:*opnfv-tech-discuss-boun...@lists.opnfv.org > > <mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> > > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] *On Behalf Of > *Frank > > Brockners (fbrockne) > > *Sent:* Tuesday, September 06, 2016 7:39 AM > > *To:* Raymond Paik; opnfv-tech-discuss@lists.opnfv.org > > <mailto:opnfv-tech-discuss@lists.opnfv.org> > > *Subject:* Re: [opnfv-tech-discuss] Following up on Project Health > > metrics discussion > > > > > > > > Hi Ray, > > > > > > > > thanks for posting the initial cut. IMHO a "composite score", as > > proposed on the page, could be **very** misleading, especially for > > projects which do most of the work upstream. So unless we track all > > upstream repos and upstream Jiras (or similar), I would suggest to > > **not** compute a composite score but evaluate things qualitatively > only. > > > > > > > > Thanks, Frank > > > > > > > > *From:*opnfv-tech-discuss-boun...@lists.opnfv.org > > <mailto:opnfv-tech-discuss-boun...@lists.opnfv.org> > > [mailto:opnfv-tech-discuss-boun...@lists.opnfv.org] *On Behalf Of > > *Raymond Paik > > *Sent:* Montag, 29. August 2016 19:33 > > *To:* opnfv-tech-discuss@lists.opnfv.org > > <mailto:opnfv-tech-discuss@lists.opnfv.org> > > *Subject:* [opnfv-tech-discuss] Following up on Project Health > metrics > > discussion > > > > > > > > All, > > > > > > > > I had an action item from last week to start a wiki page for the > > "project health metrics". You can find a proposal page > > at https://wiki.opnfv.org/display/PROJ/Project+Health+Metrics. > > > > > > > > Please add your comments/feedback via email or directly on the wiki > > page. I listed four activity areas that was discussed on the TSC > call, > > but feel free to add other activities that the community should > consider. > > > > > > > > Thanks, > > > > > > > > Ray > > > > > > > > _______________________________________________ > > opnfv-tech-discuss mailing list > > opnfv-tech-discuss@lists.opnfv.org > > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > > -- > Dave Neary - NFV/SDN Community Strategy > Open Source and Standards, Red Hat - http://community.redhat.com > Ph: +1-978-399-2182 / Cell: +1-978-799-3338 > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss > > > > > _______________________________________________ > opnfv-tech-discuss mailing list > opnfv-tech-discuss@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss >
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
