Alex, Yes, spec is much better place to introduce this. BTW, "perestroika" builds new package for every commit and patch set and publishes them via HTTP. Please check here http://172.18.160.74/osci/review/CR-202763/mos/7.0/fuel/base/centos6/Packages/fuel-library7.0-7.0.0-1.mos6888.git.bac86fe.noarch.rpm if the package contains all necessary upstream modules.
Vladimir Kozhukalov On Sat, Jul 18, 2015 at 3:28 AM, Alex Schultz <aschu...@mirantis.com> wrote: > Not until we start using it then any ci that tests with that module will > validate the modules inclusion. You can check the output of the jobs as we > are printing what modules are managed by librarian. > > -Alex > On Jul 17, 2015 6:17 PM, "Andrew Woodward" <xar...@gmail.com> wrote: > >> Fantastic, do we have some way to validate that the module was pulled in >> properly as part of fuel-library CI? >> >> On Fri, Jul 17, 2015 at 2:48 PM Alex Schultz <aschu...@mirantis.com> >> wrote: >> >>> Hey All, >>> >>> I've figured it out without having to modify the fuel-main build code. >>> I've updated the fuel-library spec with a build action that invokes the >>> script to pull down external modules. Please take some time to review the >>> two reviews out there for this change to see if there are any issues with >>> the way it is implemented. >>> >>> https://review.openstack.org/#/c/202763/ >>> https://review.openstack.org/#/c/202767/ >>> >>> This is a first step towards being able to pull in unmodified external >>> puppet modules. >>> >>> Thanks, >>> -Alex >>> >>> On Fri, Jul 17, 2015 at 4:23 PM, Andrew Woodward <awoodw...@mirantis.com >>> > wrote: >>> >>>> >>>> >>>> On Fri, Jul 17, 2015 at 12:24 PM Vladimir Kozhukalov < >>>> vkozhuka...@mirantis.com> wrote: >>>> >>>>> Alex, >>>>> >>>>> Great that you did this. Now I think I can prepare fuel-main patch to >>>>> invoke this script right before building fuel-library package. I'll add >>>>> you >>>>> to review it. Is it ok if I do this monday morning? >>>>> >>>> >>>> Keep in minde we agreeded to a deadline to get this sorted and in shape >>>> to land by EOD monday or we will have to retain the old, and crappy fork >>>> method. If possible please work out how this needs to work as early as >>>> possible so Alex can continue. >>>> >>>> >>>>> Vladimir Kozhukalov >>>>> >>>>> On Fri, Jul 17, 2015 at 5:51 PM, Alex Schultz <aschu...@mirantis.com> >>>>> wrote: >>>>> >>>>>> Hey Vladimir, >>>>>> >>>>>> On Fri, Jul 17, 2015 at 7:33 AM, Vladimir Kozhukalov < >>>>>> vkozhuka...@mirantis.com> wrote: >>>>>> >>>>>>> Alex, >>>>>>> >>>>>>> Gathering upstream modules certainly should be implemented as a >>>>>>> separate script so as to make it possible to use it wherever we need >>>>>>> this >>>>>>> (tests, builds, etc.) According to builds there are two things >>>>>>> >>>>>>> 1) We have so called "perestroika" package build system (Dmitry >>>>>>> Burmistrov is a main contributor here). By the end of next week we are >>>>>>> going to switch building all the packages to "perestroika". And in >>>>>>> order to >>>>>>> gather upstream modules right before building fuel-library package, we >>>>>>> need >>>>>>> to change perestroika build scripts. >>>>>>> >>>>>>> 2) Currently we build packages using make system and you are right >>>>>>> about the place where you need to make changes. >>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>>>> If you create shell script, I'll help you to add it to make code. >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> I have updated my review[0] to extract the update logic to it's own >>>>>> bash script that can be invoked by the build scripts. Let me know what >>>>>> would be the best way to wedge this in there. I think for the >>>>>> "perestroika" this would also be needed for the fuel-library build, so if >>>>>> you point me at that I can see if I can help make that change as well. >>>>>> >>>>>> Thanks, >>>>>> -Alex >>>>>> >>>>>> [0] https://review.openstack.org/#/c/202763/ >>>>>> >>>>>> >>>>>>> >>>>>>> Vladimir Kozhukalov >>>>>>> >>>>>>> On Fri, Jul 17, 2015 at 2:56 PM, Aleksandr Didenko < >>>>>>> adide...@mirantis.com> wrote: >>>>>>> >>>>>>>> I believe build_repo function is the best way to do this [0]. So >>>>>>>> for fuel-library we'll need to run a shell script right from the repo >>>>>>>> before 'touch $$@'. We can make it either conditional ( test -f >>>>>>>> ./path/additional_build_script.sh && bash >>>>>>>> ./path/additional_build_script.sh >>>>>>>> ) or as additional parameter to function and add it in fuel-library >>>>>>>> call [1] >>>>>>>> >>>>>>>> Regards, >>>>>>>> Alex >>>>>>>> >>>>>>>> [0] >>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L16-L37 >>>>>>>> [1] >>>>>>>> https://github.com/stackforge/fuel-main/blob/master/repos.mk#L45 >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Jul 17, 2015 at 2:37 PM, Alex Schultz < >>>>>>>> aschu...@mirantis.com> wrote: >>>>>>>> >>>>>>>>> Hey Alex, >>>>>>>>> >>>>>>>>> On Jul 17, 2015 4:32 AM, "Aleksandr Didenko" < >>>>>>>>> adide...@mirantis.com> wrote: >>>>>>>>> > >>>>>>>>> > Hi, >>>>>>>>> > >>>>>>>>> > I think that we should provide a separate script that will fetch >>>>>>>>> the upstream modules into fuel-library/deployment/puppet/ directory. >>>>>>>>> It >>>>>>>>> will allow us to have everything in a single place and use this >>>>>>>>> script in >>>>>>>>> ISO build process and CI jobs. >>>>>>>>> > >>>>>>>>> >>>>>>>>> Right. That is what I'm going for. The issue I need help with is >>>>>>>>> the best way to execute this as part of the build process. From what >>>>>>>>> i >>>>>>>>> understand of the build process is that we are using git archive for >>>>>>>>> all >>>>>>>>> pieces so I'm not sure how to wedge in an extra script execution to >>>>>>>>> do the >>>>>>>>> module fetch. The creation of the script isn't the issue, the issue >>>>>>>>> is how >>>>>>>>> can I properly run it as part of the build process. >>>>>>>>> >>>>>>>>> >>>>>>>>> > Regards, >>>>>>>>> > Alex >>>>>>>>> > >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> -Alex >>>>>>>>> >>>>>>>>> > On Thu, Jul 16, 2015 at 11:17 PM, Alex Schultz < >>>>>>>>> aschu...@mirantis.com> wrote: >>>>>>>>> >> >>>>>>>>> >> Hello everyone, >>>>>>>>> >> >>>>>>>>> >> I have committed the initial configuration required to start >>>>>>>>> leveraging librarian-puppet as part of the way we pull in upstream >>>>>>>>> puppet >>>>>>>>> modules[0]. Additionally, I have also committed a change that would >>>>>>>>> pull in >>>>>>>>> the openstack-ironic module[1]. The one piece that is missing from >>>>>>>>> this >>>>>>>>> being a complete solution is the ability to run librarian-puppet as >>>>>>>>> part of >>>>>>>>> our build process for the fuel-library. I've looked into the >>>>>>>>> fuel-main >>>>>>>>> build scripts and I think it's over my head to figure this out just by >>>>>>>>> looking. Can anyone explain to me or assist me in how I could go about >>>>>>>>> modifying the existing build system to be able to run >>>>>>>>> librarian-puppet to >>>>>>>>> prepare the source for the package? In my initial investigation, it >>>>>>>>> looks >>>>>>>>> like it would be a modification of the fuel-main/packages/ >>>>>>>>> module.mk[3] file. I basically need to do the prepare_library[3] >>>>>>>>> function from the 202763 review[0] after we've pulled all the sources >>>>>>>>> together to fetch the upstream modules. >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> Thanks, >>>>>>>>> >> -Alex >>>>>>>>> >> >>>>>>>>> >> [0] https://review.openstack.org/202763 >>>>>>>>> >> [1] https://review.openstack.org/202767 >>>>>>>>> >> [2] >>>>>>>>> https://github.com/stackforge/fuel-main/blob/master/packages/module.mk#L63-L82 >>>>>>>>> >> [3] >>>>>>>>> https://review.openstack.org/#/c/202763/1/utils/jenkins/fuel_noop_tests.rb >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> __________________________________________________________________________ >>>>>>>>> >> OpenStack Development Mailing List (not for usage questions) >>>>>>>>> >> Unsubscribe: >>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>> >> >>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>> >> >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> __________________________________________________________________________ >>>>>>>>> > OpenStack Development Mailing List (not for usage questions) >>>>>>>>> > Unsubscribe: >>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>> > >>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>> > >>>>>>>>> >>>>>>>>> >>>>>>>>> __________________________________________________________________________ >>>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>>> Unsubscribe: >>>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> __________________________________________________________________________ >>>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>>> Unsubscribe: >>>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> __________________________________________________________________________ >>>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>>> Unsubscribe: >>>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> __________________________________________________________________________ >>>>>> OpenStack Development Mailing List (not for usage questions) >>>>>> Unsubscribe: >>>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>>> >>>>>> >>>>> >>>>> __________________________________________________________________________ >>>>> OpenStack Development Mailing List (not for usage questions) >>>>> Unsubscribe: >>>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>>> >>>> -- >>>> -- >>>> Andrew Woodward >>>> Mirantis >>>> Fuel Community Ambassador >>>> Ceph Community >>>> >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> -- >> >> -- >> >> Andrew Woodward >> >> Mirantis >> >> Fuel Community Ambassador >> >> Ceph Community >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
