On Thu, Mar 02, 2017 at 10:13:07AM +0100, Marcin Juszkiewicz wrote: > I am working on some improvements for Kolla. Part of that work is > sending patches for review. > > Once patch is set for review.openstack.org there is a set of Jenkins > jobs started to make sure that patch does not break already working > code. And this is good thing. > > How it is done is not good ;( > > 1. Kolla output is nightmare to debug. > > There is --logs-dir option to provide separate logs for each image build > but it is not used. IMHO it should be as digging through such logs is > easier. > > Several images feels like they try to do install again and again to pass > through what distribution consider a bug - like "user XY already exists" > bugs while Debian/Ubuntu are used as base distro. Which adds several > error messages to check/ignore. > > > 2. Some jobs fail because "sometimes (not always) the gate can't access > some source on the internet". > > I spent most of my career on building software. Having builds fail for > such reasons was hardly acceptable when building could take hours. So we > mirrored all sources and used own mirror(s) as fallback. On other > systems I used 10-20GB w3cache to handle that for me (because there was > no way to provide local mirror of used sources). > > OpenStack infrastructure lacks any of it. Using "recheck" comment in > review to restart Jenkins jobs is not a solution - how many times it has > to fail to make sure that it is patch's fault not infrastructure one? > Lets not use absolutes please. In fact, the openstack-infra team does mirror a lot of things today[1] but it sounds like not everything you need for jobs.
For the most part, the openstack-infra team is pretty accommodation to mirroring requests. I admit, sometimes we are not the fastest to implement them, but we try. At the PTG, the openstack-ansible team reminded me about setting up a mirror for mariada, which I plan on working on today / tomorrow. > > As a contributor I started to ignore Jenkins tests. Instead I do builds > on several machines to check does everything works with my patches. If > something does not then I update my patchset. > Now, this is my personal opinion not openstack-infra, if you have to rely on local testing over testing in the gate, I think there is some serious issues with your project. Testing is hard, I get that. Nobody wants to recheck their patch multiple times to get a green check mark. At the same time, there are very few people in general that gravitate towards testing infrastructure (this is a general comment, not specific to kolla). As a suggestion, I would have a serious talk about stopping the addition of additional features to your code base and maybe work towards improving your testing coverage. Having a strong test base to me makes a world of difference for your developers and moral. I can tell you from personal experience, if a project (inside openstack or outside) doesn't have good test coverage, and I am always fighting with flapping tests, I find it difficult to keep contributing like you mentioned above. [1] http://mirror.dfw.rax.openstack.org/ __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
