On Fri, Mar 11, 2016 at 3:46 AM, Michael Chapman <wop...@gmail.com> wrote: > > > On Sat, Mar 5, 2016 at 10:31 AM, Giulio Fidente <gfide...@redhat.com> wrote: >> >> On 03/04/2016 03:23 PM, Emilien Macchi wrote: >>> >>> That's not the name of any Summit's talk, it's just an e-mail I wanted >>> to write for a long time. >>> >>> It is an attempt to expose facts or things I've heard a lot; and bring >>> constructive thoughts about why it's challenging to contribute in >>> TripleO project. >> >> >> hi Emilien, >> >> thanks for bringing this up, it's not an easy topic and yet of most >> crucial. As a core contributors I feel, to some extent, responsible for the >> current status of things and I think it's time for us to reflect more about >> what we can, individually, do. >> >> I have some ideas but I want to start by commenting to your points. >> >>> 1/ "I don't review this patch, we don't have CI coverage." >>> >>> One thing I've noticed in TripleO is that a very few people are involved >>> in CI work. >>> In my opinion, CI system is more critical than any feature in a product. >>> Developing Software without tests is a bit like http://goo.gl/OlgFRc >>> All people - specially core - in the project should be involved in CI >>> work. If you are TripleO core and you don't contribute on CI, you might >>> ask yourself why. >> >> >> Agreed, we need more 'eyes' on out CI to cope with both the infra and the >> inavoidable failures due to changes/bugs in the puppet modules or openstack >> itself. >> >> But there is more hiding behind this problem ... we already have quite a >> number of optional and even pluggable features in TripleO and we're even >> designing an interface to make this easier; testing them all isn't going to >> happen. So we'll always hit something we don't have coverage for. >> >> Let's have a conversation on how we can improve coverage at the summit! >> Maybe we can make simply make our CI scenarios more variegated/complex in >> the attempt to touch more features? >> >>> 2/ "I don't review this patch, CI is broken." >>> >>> Another thing I've noticed in TripleO is that when CI is broken, again, >>> a very few people are actually working on fixing failures. >>> My experience over the last years taught me to stop my daily work when >>> CI is broken and fix it asap. >> >> >> Agreed. More eyes and more coverage to increase its dependability. >> >>> 3/ "I don't review it, because this feature / code is not my area". >>> >>> My first though is "Aren't we supposed to be engineers and learn new >>> areas?" >>> My second though is that I think we have a problem with TripleO Heat >>> Templates. >>> THT or TripleO Heat Templates's code is 80% of Puppet / Hiera. If >>> TripleO core say "I'm not familiar with Puppet", we have a problem here, >>> isn't? >>> Maybe should we split this repository? Or revisit the list of people who >>> can +2 patches on THT. >> >> >> Not sure here, I find that manifests and templates are pretty much "meant >> to go together" so I am worried that a split could solve some problems but >> also cause others. > > > This is pretty much what I proposed last week > (https://blueprints.launchpad.net/tripleo/+spec/refactor-puppet-manifests) > and I noticed Dan approved the blueprint yesterday (cheers). It's definitely > going to cause problems in that THT defines the data interface and > puppet-tripleo is going to have to keep up with that interface in lock-step > in some cases so be prepared to deal with that as a patch author. This isn't > really any different to non-tripleo puppet module situations where a change > to the repo holding hiera data will be tied to changes in modules. > > Ideally I'd like to incrementally decouple the puppet-tripleo profiles from > the data heat provides but for the first cut they'll be joined at the hip.
Michael, I've also been thinking at decoupling THT into puppet-tripleo manifests, please review: puppet-tripleo: glance api/registry: https://review.openstack.org/289459 THT: use puppet-tripleo to deploy Glance: https://review.openstack.org/289466 Any feedback is welcome, > So given a new home (puppet-tripleo) for a large portion of the code > (starting with overcloud controller and controller_pacemaker), hopefully > this paves the way for giving those who know puppet well the opportunity to > take on responsibility for the manifests without necessarily being > intimately familiar with the rest of the system, which I guess helps with > Emilien's original concern that there's a skill split across the tooling > lines. > >> >> >> This said, let's be honest, an effective patch for THT requires a good >> understanding of many different problems which can be TripleO specific (eg. >> implications on upgrades), tooling specific (eg. Heat/Puppet), OpenStack >> specific (eg. cooperation with other, optional, features) so I have myself >> skipped changes when I didn't feel comfortable with it. >> >> But one problem which I think is more recently slowing reviews and which >> is somewhat concause of 3) is that we're not dealing too well with code >> duplication in the yamls and with conditional logic in the manifests. >> >> Maybe we could stop and think a together about new HOT functionalities >> which could help us? Interesting for the summit as well? >> >>> 4/ Patches are stalled. Most of the time. >>> >>> Over the last 12 months, I've pushed a lot of patches in TripleO and one >>> thing I've noticed is that if I don't ping people, my patch got no >>> review. And I have to rebase it, every week, because the interface >>> changed. I got +2, cool ! Oh, merge conflict. Rebasing. Waiting for +2 >>> again... and so on.. >>> >>> I personally spent 20% of my time to review code, every day. >>> I wrote a blog post about how I'm doing review, with Gertty: >>> http://my1.fr/blog/reviewing-puppet-openstack-patches/ >>> I suggest TripleO folks to spend more time on reviews, for some reasons: >>> >>> * decreasing frustration from contributors >>> * accelerate development process >>> * teach new contributors to work on TripleO, and eventually scale-up the >>> core team. It's a time investment, but worth it. >> >> >> I'm inclined to think that this is a bit of a consequence of 1), 2) and 3) >> together. >> >>> In Puppet team, we have weekly triage sessions and it's pretty helpful. >> >> >> Right. I think we experimented with something like this before but it was >> probably perceived as an emergency measure so we put it on a side after a >> while. >> >> I remember we had a list of 'hot reviews' which we would review during the >> weekly meetings. But it isn't trivial to understand which type of review is >> considered hot. What is the purpose of the puppet team triaging? To find old >> reviews? Mergeable reviews? To dropping stale reviews? To speed up bug >> fixes? To get attention on features? >> >>> 5/ Most of the tests are run... manually. >>> >>> How many times I've heard "I've tested this patch locally, and it does >>> not work so -1". >>> >>> The only test we do in current CI is a ping to an instance. Seriously? >>> Most of OpenStack CIs (Fuel included), run Tempest, for testing APIs and >>> real scenarios. And we run a ping. >>> That's similar to 1/ but I wanted to raise it too. >> >> >> I'd say a consequence of 1) as well >> >>> If we don't change our way to work on TripleO, people will be more >>> frustrated and reduce contributions at some point. >>> I hope from here we can have a open and constructive discussion to try >>> to improve the TripleO project. >> >> >> me too so let me add my idea as the 6th point >> >> 5/ Documentation isn't great >> >> We did some good things, we have a repo with usable doc and a website to >> point people to, but the docs honestly are a bit confusing and even lack >> documentation about quite some features for end users. >> >> Maybe we can start some minor restructuring of the docs, splitting them >> into a 'quickstart' guide and a 'feature-complete' guide and ask people to >> submit together with a feature the matching documentation in the >> 'feature-complete' guide? >> -- >> Giulio Fidente >> GPG KEY: 08D733BA >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -- Emilien Macchi __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
