On 25/06/15 10:03, Lana Brindley wrote: > Hi core team, > > One of the things that came up as a pain point at Summit was the DocImpact > automation. I�ve been doing some thinking about this, and this is where I�m > at right now: > > Workflow: Devs create a patch, and at commit time they think "oh yeah, > probably there should be some docs about that" and whack in a 'DocImpact' > flag. In other words, there's not a lot of thought going in here, it's just > an easy thing to do, they get a warm fuzzy, and there�s an assumption that > the documentation fairy comes along and takes care of things. In reality, > what then happens is some docs triager spends an hour looking at the patch, > searching through the docs, then deciding it has nothing to do with > openstack-manuals. Often, the actual doc impact (if there even is one) is > against docs in the dev repo, not openstack-manuals. > > In short, devs recognise they need docs, and DocImpact is an easy way to feel > like they're doing something productive to make that happen. What really > happens is that those bugs are largely irrelevant for openstack-manuals, > which puts pressure on our core team to triage them effectively. To try and > get some perspective on the size of the problem, here are some numbers: > > Of the 508 current openstack-manuals bugs, 214 are the result of the > DocImpact script. 51 of these are yet to be triaged. > Right now, there are 170 patches in-flight with the DocImpact tag. To state > the obvious, if all them land, that�s 170 new bugs in our queue. > > So, solutions: > > 1: We have a crack team of docs people (potentially with automation > assistance) going through docimpact bugs, cc'ing the original patch authors, > and moving them into dev repos where necessarily, and marking the rest > invalid. We could team this with a an education campaign on the dev mailing > list. > > 2: We ditch docimpact, and force devs to create their own docs bugs (and > patches). This would mean fewer devs get on with the job of documentation, > but at least what we do get would be well-considered, rather than hastily > added to their commit message. > > 3: We adjust docimpact to raise a bug in their own dev repo. So, patch > against swift with docimpact raises a bug in the swift bug queue, not the > openstack-manuals doc queue. Maintenance overhead then remains with dev, and > I bet you any money they would self-police that better than we ever could. > > 4: Some other thing I haven�t thought of yet � ? > > I have a feeling 3 is where the money's at. I took the liberty of quickly > checking with an Infra core, and this change should be relatively easy to > implement, too, for the record. > > Thoughts?
Combo of modified #1 & modified #3, with a bit of #4 (communication back on their patch), with a bit of #5 (modified docimpact parameters). While I fully agree that DocImpact is in am imperfect state, and developers are falling into the "magic fairy" trap, I want to first go back to the reason DocImpact exists and understand the large benefit it does give us, despite misgivings. Prior to enlisting developer assistance to identify patches that potentially resulted in the need for some documentation, our team had to put in an enormous amount of effort to try and work out what was going on dev-wise. We kinda had an inkling that there were some new features going in, and there were probably some pretty important bugs being fixed. To work that out, it involved reading almost every single blueprint for features, and then trying to read a few hundred random patches that looked interesting enough that they might result in docs. What the introduction of DocImpact did was, for the most part, solve this tracking problem. We successfully leveraged the collective developer effort to let us know what the interesting things were. Our developers aren't perfect in doing this, but in general, we get bugs lodged so we can track major features, ensure our scripts for config changes are working, and find out rather important updates. In addition, we are able to fairly apply a priority to the patch that comes in. Vendor driver? Don't really care: low. Deprecation/Introduction of a major feature? Important: High. Bug Fix that seriously changes behaviour: High. See also https://wiki.openstack.org/wiki/Documentation/HowTo#Doc_Bug_Triaging_Guidelines Though flawed and in need of a serious kick in the pants, currently DocImpact provides the only place this tracking occurs. So, whatever solution we adopt, wherever/however it ends up, we need to make sure this tracking ability continues - leveraging the effort of many to filter the firehose a little. I now think this email is too long - I think I might address my first line response/actual solution ideas in a separate one instead, standby! Regards, Tom -- Mailing list: https://launchpad.net/~openstack-doc-core Post to : openstack-doc-core@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-doc-core More help : https://help.launchpad.net/ListHelp
