Hi, Fatih Thanks for the reply. It aligned my understanding of releng with the current situation and future planning.
I agree that some problem may be caused by incorrect usage of releng repo and we have already made some effort to make it right from project's side[1]. In my opinion, there are something we could do in releng 1. clean up and refactor the wrappers and triggers, some work is already in progress[2] 2. provide a guideline for projects about best practise of using releng I would be very happy to contribute on these part if the releng team wants to push these things forward :-) Excuse me for being a bit off topic on "docker build resources"... [1]: https://gerrit.opnfv.org/gerrit/#/c/39395/ [2]: https://gerrit.opnfv.org/gerrit/#/c/39309/ On Wed, Nov 1, 2017 at 5:42 PM Fatih Degirmenci < fatih.degirme...@ericsson.com> wrote: > Hi Yujun, > > > > Thanks for the feedback. > > > > Releng aims to provide a similar service to the OPNFV projects like you > explained but some of the limitations you listed below are due to Jenkins > itself and how to manage CI in a consistent manner. > > We need to keep some kind of alignment and ensure that what we are doing > on Jenkins and CI in general fits into the overall framework. > > > > In general, only the Jenkins job configurations and simple wrapper scripts > should be located in releng repo. (excluding common utilities such as Test > Result API which will move into a separate repo) > > The scripts that do project specific stuff such as running unit tests, > builds and so on should be owned, developed, and maintained by > corresponding projects. > > > > The overall structure followed by releng is like this or from [1]. > > - preprocessing > > - actual build/test > > - postprocessing > > > > Preprocessing is simply doing general cleanup stuff, fixing permissions > and so on before the actual build starts and this is common for all > projects. > > Postprocessing deals with similar common stuff such as uploading artifacts > to artifact repo. > > > > The scripts that are consumed by releng from projects are the ones that > matter and I expect that these change more often than Jenkins jobs or > wrappes themselves. > > Normally you don't need to touch jobs/wrappers if you are not doing a > total structure change such as changing the job type from Freestyle to > Multijob. > > These scripts are generally located in <project_repo>/ci/build.sh, > <project_repo>/ci/test.sh which get executed by the wrapper scripts which > in turn run by Jenkins jobs. > > > > When it comes to docker; it is a bit different since the use of docker > containers for the test projects started in releng as common/community > initiative and the build scripts have been in releng repo > developed/maintained by the community at large, not just releng itself. > > But this is due wanting to have some kind of alignment and historical > reasons which can be changed if the projects require different way of doing > builds and so on. (I think Storperf is an example to this.) > > > > We have plans to look into Zuul v3 and start a PoC but we are waiting for > OpenStack to finalize their migration and fix the issues they are > experiencing. > > But we also need to keep in mind that we go further in our CI comparing to > OpenStack so it is not just about OpenStack fixing all the issues but also > fulfilling OPNFV needs. (one problem we will face is the number/types of > plugins we use on our Jenkins and the impacts of having long running tests.) > > > > If you can point to specific examples or list additional needs we can take > them into our backlog and improve what we do. > > > > [1] > https://wiki.opnfv.org/download/attachments/11699697/image2017-6-6%2022%3A56%3A30.png?version=1&modificationDate=1496783473000&api=v2 > > > > /Fatih > > > > *From: *<infra-wg-boun...@lists.opnfv.org> on behalf of "Yujun Zhang > (ZTE)" <zhangyujun+...@gmail.com> > *Date: *Wednesday, 1 November 2017 at 02:19 > *To: *David McBride <dmcbr...@linuxfoundation.org> > *Cc: *"infra...@lists.opnfv.org" <infra...@lists.opnfv.org>, test-wg < > test...@lists.opnfv.org>, "opnfv-tech-discuss@lists.opnfv.org" < > opnfv-tech-discuss@lists.opnfv.org>, Trevor Cooper < > trevor.coo...@intel.com> > *Subject: *Re: [infra-wg] [infra][releng] proposal for RELENG project to > take ownership of docker build resources > > > > Not sure I fully understand the word "ownership" and the scope of building > resources. For sure it is good to maintain the build scripts and *global* > config in releng. > > > > But for project specific configuration, it is not so convenient to manage > them in a central repository. Sometimes I have to wait for committers of > releng to merge my patch which just fixes a small bug of the the build job > for my project. Most time the validation of releng job is only possible > *after* the patch is merged. When I found it does not work, I have to go > over the steps again. It is not so productive since this workflow involves > two teams (releng and project). > > > > Ideally, I would expect releng to provide the building resource as a > service and the project specific configuration managed inside project > repository. This is much like the solution provided by some CI SaaS like > Travis-CI[1]. And also the trend in OpenStack infra brought in by zuul v3, > which is called in repo configuration. > > > > This is not just about docker building, but also applies for other > verification job. > > > > [1]: https://travis-ci.org/ > > [2]: https://docs.openstack.org/infra/manual/zuulv3.html#what-is-zuul-v3 > > > > On Wed, Nov 1, 2017 at 5:56 AM David McBride <dmcbr...@linuxfoundation.org> > wrote: > > please +1, 0, or -1 to indicate your position > > > > Team, > > > > This topic came up during our release meeting two weeks ago. I took an > action to get a discussion started in the infra meeting. > > > > The proposal is for the RELENG project to take ownership of docker build > resources (scripts,config files, etc.) > <https://gerrit.opnfv.org/gerrit/gitweb?p=releng.git;a=tree;f=jjb/releng> > for the purpose of providing a common docker build capability available to > all users of OPNFV CI. Active committers to these resources will remain > unchanged. > > > > We had a good discussion on Monday. There was general consensus that this > proposal makes sense. The minutes are here > <http://ircbot.wl.linuxfoundation.org/meetings/opnfv-meeting/2017/opnfv-meeting.2017-10-30-14.59.html>. > > > > > I took an action at the infra meeting to start a discussion on the mailing > list to seek additional feedback. Please respond to this email to indicate > whether you support the proposal. Also feel free to add your comments. > Thanks. > > > > David > > > > > -- > > *David McBride* > > Release Manager, OPNFV > > Mobile: +1.805.276.8018 > > Email/Google Talk: dmcbr...@linuxfoundation.org > > Skype: davidjmcbride1 > > IRC: dmcbride > > _______________________________________________ > infra-wg mailing list > infra...@lists.opnfv.org > https://lists.opnfv.org/mailman/listinfo/infra-wg > > -- > > Yujun Zhang > -- Yujun Zhang
_______________________________________________ opnfv-tech-discuss mailing list opnfv-tech-discuss@lists.opnfv.org https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
