Hi Saravanan, Ah, ok. I'd assumed ironic didn't touch the bootloader as the docs show a mkconfig & reboot to customize grub for localboot: http://docs.openstack.org/project-install-guide/baremetal/draft/advanced.html#appending-kernel-parameters-to-boot-instances.
Thanks, Ollie On 13 December 2016 at 11:03, Saravanan KR <skram...@redhat.com> wrote: > Hi Oliver, > > During the deployment, Ironic will start the node with IPA ramdisk, > which will copy the overcloud image to the node's disk, then configure > the grub cfg [1], set the node to local boot and reboot the node. > After which the node will be boot with the overcloud image. So no > reboot required in the overcloud image, as the "deploy steps" will be > run by the IPA itself. > > Regards, > Saravanan KR > > [1] > https://github.com/openstack/ironic-python-agent/blob/master/ironic_python_agent/extensions/image.py#L136 > > > On Tue, Dec 13, 2016 at 4:18 PM, Oliver Walsh <owa...@redhat.com> wrote: >> Hi Yolanda, >> >>> these changes will be created by ironic before the image is deployed >> >> The question I'm asking is how are the changed created without a reboot? >> >> Typically when setting this via a manual change or via tuned the >> process is to modify /etc/default/grub, run grub2-mkconfig, and then >> reboot. Are you suggesting we drop in a pre-build grub cfg before >> deployment? >> >> Thanks, >> Ollie >> >> On 13 December 2016 at 10:33, Yolanda Robla Mota <yrobl...@redhat.com> wrote: >>> It won't need a reboot, because these changes will be created by ironic >>> before the image is deployed. So it will boot with the right parameters. The >>> alternative of doing with puppet after the image was deployed, needed a >>> reboot, because the changes were done post-deploy. >>> So ironic build steps are pre-deploy without reboot, puppet changes are >>> post-deploy with a reboot. >>> >>> On Tue, Dec 13, 2016 at 11:24 AM, Oliver Walsh <owa...@redhat.com> wrote: >>>> >>>> Hi, >>>> >>>> Saravanan wrote: >>>> > If ironic "deploy steps" can configure this "tuned" setting and run the >>>> > command >>>> >>>> How does this avoid the reboot? >>>> >>>> Yolanda wrote: >>>> > The idea will be to define custom deployment steps for ironic, like >>>> > including the kernel boot parameters. >>>> >>>> Again, is this avoiding the reboot or just moving it? >>>> >>>> Thanks, >>>> Ollie >>>> >>>> On 13 December 2016 at 09:02, Saravanan KR <skram...@redhat.com> wrote: >>>> > Hi Yolanda, >>>> > >>>> > The flow for "tuned" will be like set the "tuned" configuration files, >>>> > and then activate the profile by running the command "tuned-adm >>>> > tuned-profile-nfv". This command will actually write the required >>>> > configuration files for tuning the host. If ironic "deploy steps" can >>>> > configure this "tuned" setting and run the command, then it is good >>>> > enough. >>>> > >>>> > Regards, >>>> > Saravanan KR >>>> > >>>> > On Tue, Dec 13, 2016 at 1:04 PM, Yolanda Robla Mota >>>> > <yrobl...@redhat.com> wrote: >>>> >> Hi Saravanan >>>> >> Thanks for your comments. With this new module, I guess a reboot is >>>> >> still >>>> >> needed after os-host-config ? >>>> >> Right now we have been guided by TripleO and Ironic people to start >>>> >> using >>>> >> what in Ironic is called "custom deployment steps". An initial spec is >>>> >> reflected here: >>>> >> https://review.openstack.org/#/c/382091 >>>> >> >>>> >> The idea will be to define custom deployment steps for ironic, like >>>> >> including the kernel boot parameters. Can that be a solution for your >>>> >> "tuned" needs as well? >>>> >> >>>> >> Best >>>> >> Yolanda >>>> >> >>>> >> On Tue, Dec 13, 2016 at 7:59 AM, Saravanan KR <skram...@redhat.com> >>>> >> wrote: >>>> >>> >>>> >>> Hello, >>>> >>> >>>> >>> Thanks Yolanda for starting the thread. The list of requirements in >>>> >>> the host configuration, related to boot parameters and reboot are: >>>> >>> >>>> >>> * DPDK - For vfio-pci driver binding, iommu support on kernel args is >>>> >>> mandatory, which has to be configured before os-net-config runs >>>> >>> * DPDK & RealTime - Enabling "tuned" profile for nfv or rt, will >>>> >>> update the boot parameters and a reboot is required >>>> >>> * Other items mentioned by Yolanda >>>> >>> >>>> >>> If it is configuring only, the boot parameters, then ironic's deploy >>>> >>> feature may help, but there are other requirement to enable the >>>> >>> "tuned" profile which tunes the host for the required configuration, >>>> >>> which also requires reboot, as it will alter the boot parameters. If >>>> >>> we can collate the all the configurations which requires reboot >>>> >>> together, we will improve the reboot time. And if we reboot before the >>>> >>> actual openstack services are started, then the reboot time _may_ >>>> >>> improve. >>>> >>> >>>> >>> Can I propose a *new* module for TripleO deployments, like > >>>> >>> os-host-config <, which will run after os-collect-config and before >>>> >>> os-net-config, then we can collate all the host specific configuration >>>> >>> inside this module. This module can be a set of ansible scripts, which >>>> >>> will only configure the host. Ofcource the parameter to this module >>>> >>> should be provided via os-collect-config. Separating the host >>>> >>> configuration will help in the containerized TripleO deployment also. >>>> >>> >>>> >>> Or any other better alternatives are welcome. >>>> >>> >>>> >>> Please pour in your views if you think for/against it. >>>> >>> >>>> >>> Regards, >>>> >>> Saravanan KR >>>> >>> >>>> >>> >>>> >>> On Fri, Dec 2, 2016 at 9:31 PM, Yolanda Robla Mota >>>> >>> <yrobl...@redhat.com> >>>> >>> wrote: >>>> >>> > Hi , Dmitry >>>> >>> > That's what i didn't get very clear. If all the deployment steps are >>>> >>> > pre-imaging as that statement says, or every deploy step could be >>>> >>> > isolated >>>> >>> > and configured somehow. >>>> >>> > I'm also a bit confused with that spec, because it mixes the concept >>>> >>> > of >>>> >>> > "deployment steps", will all the changes needed for runtime RAID. >>>> >>> > Could it >>>> >>> > be possible to separate into two separate ones? >>>> >>> > >>>> >>> > ----- Original Message ----- >>>> >>> > From: "Dmitry Tantsur" <dtant...@redhat.com> >>>> >>> > To: openstack-dev@lists.openstack.org >>>> >>> > Sent: Friday, December 2, 2016 3:51:30 PM >>>> >>> > Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update >>>> >>> > kernel >>>> >>> > parameters on local boot >>>> >>> > >>>> >>> > On 12/02/2016 01:28 PM, Yolanda Robla Mota wrote: >>>> >>> >> Hi Dmitry >>>> >>> >> >>>> >>> >> So we've been looking at that spec you suggested, but we are >>>> >>> >> wondering >>>> >>> >> if that will be useful for our use case. As the text says: >>>> >>> >> >>>> >>> >> The ``ironic-python-agent`` project and ``agent`` driver will be >>>> >>> >> adjusted to >>>> >>> >> support ``get_deploy_steps``. That way, ``ironic-python-agent`` >>>> >>> >> will be >>>> >>> >> able >>>> >>> >> to declare deploy steps to run prior to disk imaging, and operators >>>> >>> >> will be >>>> >>> >> able to extend ``ironic-python-agent`` to add any custom step. >>>> >>> >> >>>> >>> >> Our needs are different, actually we need to create a deployment >>>> >>> >> step >>>> >>> >> after imaging. We'd need an step that drops config on >>>> >>> >> /etc/default/grub , >>>> >>> >> and updates it. This is a post-imaging deploy step, that modifies >>>> >>> >> the base >>>> >>> >> image. Could ironic support these kind of steps, if there is a base >>>> >>> >> system >>>> >>> >> to just define per-user steps? >>>> >>> > >>>> >>> > I thought that all deployment operations are converted to steps, >>>> >>> > with >>>> >>> > partitioning, writing the image, writing the configdrive and >>>> >>> > installing >>>> >>> > the boot >>>> >>> > loader being four default ones (as you see, two steps actually >>>> >>> > happen >>>> >>> > after the >>>> >>> > image is written). >>>> >>> > >>>> >>> >> >>>> >>> >> The idea we had on mind is: >>>> >>> >> - from tripleo, add a property to each flavor, that defines the >>>> >>> >> boot >>>> >>> >> parameters: openstack flavor set compute --property >>>> >>> >> os:kernel_boot_params='abc' >>>> >>> >> - define a "ironic post-imaging deploy step", that will grab this >>>> >>> >> property from the flavor, drop it on /etc/default/grub and >>>> >>> >> regenerate it >>>> >>> >> - then on local boot, the proper kernel parameters will be applied >>>> >>> >> >>>> >>> >> What is your feedback there? >>>> >>> >> >>>> >>> >> ----- Original Message ----- >>>> >>> >> From: "Dmitry Tantsur" <dtant...@redhat.com> >>>> >>> >> To: openstack-dev@lists.openstack.org >>>> >>> >> Sent: Friday, December 2, 2016 12:44:29 PM >>>> >>> >> Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update >>>> >>> >> kernel >>>> >>> >> parameters on local boot >>>> >>> >> >>>> >>> >> On 11/28/2016 04:46 PM, Jay Faulkner wrote: >>>> >>> >>> >>>> >>> >>>> On Nov 28, 2016, at 7:36 AM, Yolanda Robla Mota >>>> >>> >>>> <yrobl...@redhat.com> >>>> >>> >>>> wrote: >>>> >>> >>>> >>>> >>> >>>> Hi, good afternoon >>>> >>> >>>> >>>> >>> >>>> I wanted to start an email thread about how to properly setup >>>> >>> >>>> kernel >>>> >>> >>>> parameters on local boot, for our overcloud images on TripleO. >>>> >>> >>>> These parameters may vary depending on the needs of our end >>>> >>> >>>> users, >>>> >>> >>>> and even can be different ( for different roles ) per deployment. >>>> >>> >>>> As an >>>> >>> >>>> example, we need it for: >>>> >>> >>>> - enable FIPS kernel in terms of security >>>> >>> >>>> (https://bugs.launchpad.net/tripleo/+bug/1640235) >>>> >>> >>>> - enable functionality for DPDK/SR-IOV >>>> >>> >>>> (https://review.openstack.org/#/c/331564/) >>>> >>> >>>> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image) >>>> >>> >>>> - etc.. >>>> >>> >>>> >>>> >>> >>>> So far, the solutions we got were on several directions: >>>> >>> >>>> >>>> >>> >>>> 1. Update the golden overcloud-full image with virt-customize, >>>> >>> >>>> modifying /etc/default/grub settings according to our needs: this >>>> >>> >>>> is a >>>> >>> >>>> manual process, not really driven by TripleO. End users will want >>>> >>> >>>> to avoid >>>> >>> >>>> manual steps as much as possible. Also if we announce that >>>> >>> >>>> OpenStack ships >>>> >>> >>>> features in TripleO like DPDK, SR-IOV... doesn't make sense to >>>> >>> >>>> tell end >>>> >>> >>>> users that if they want to consume that feature, they need to do >>>> >>> >>>> manual >>>> >>> >>>> updates on the image. It shall be natively supported, or >>>> >>> >>>> configurable per >>>> >>> >>>> TripleO environments. >>>> >>> >>>> >>>> >>> >>>> 2. Create our own images using diskimage-builder and custom >>>> >>> >>>> elements: >>>> >>> >>>> in this case, we have the problem that the partners will loose >>>> >>> >>>> support, as >>>> >>> >>>> building their own images is good for upstream, but not accepted >>>> >>> >>>> into the >>>> >>> >>>> OSP environment. Also the combination of images needed can be >>>> >>> >>>> huge, that can >>>> >>> >>>> be a blocker for QA. >>>> >>> >>>> >>>> >>> >>>> 3. Add Ironic support for it. Images can be uploaded to glance, >>>> >>> >>>> and >>>> >>> >>>> some properties can be set on metadata, like a json with kernel >>>> >>> >>>> parameters. >>>> >>> >>>> Ironic will modify these kernel parameters when deploying the >>>> >>> >>>> image (in a >>>> >>> >>>> similar way that when it installs bootloader, or generates >>>> >>> >>>> partitions). >>>> >>> >>>> >>>> >>> >>> >>>> >>> >>> This has been proposed before in ironic-specs >>>> >>> >>> (https://review.openstack.org/#/c/331564/) and was rejected, as it >>>> >>> >>> would >>>> >>> >>> require Ironic to reach out and modify image contents, which >>>> >>> >>> traditionally >>>> >>> >>> has been considered out of scope for Ironic. I would personally >>>> >>> >>> recommend >>>> >>> >>> #4, as post-boot automation is the safest way to configure >>>> >>> >>> node-specific >>>> >>> >>> options inside an image. >>>> >>> >> >>>> >>> >> I'm still a bit divided about our decision back then.. On one hand, >>>> >>> >> this does >>>> >>> >> seem somewhat out of scope. On the other, I quite understand why >>>> >>> >> reboot >>>> >>> >> is >>>> >>> >> suboptimal. I wonder if the ongoing deploy steps work will actually >>>> >>> >> solve it by >>>> >>> >> allowing hardware managers to provide additional deploy steps. >>>> >>> >> >>>> >>> >> Yolanda, you may want to check the spec >>>> >>> >> https://review.openstack.org/#/c/382091/ >>>> >>> >> as it lays the foundation for the deploy steps idea. >>>> >>> >> >>>> >>> >>> >>>> >>> >>> Thanks, >>>> >>> >>> Jay Faulkner >>>> >>> >>> OSIC >>>> >>> >>> >>>> >>> >>> >>>> >>> >>>> 4. Configure it post-deployment: there can be some puppet element >>>> >>> >>>> that updates kernel parameters. But it will need a node reboot to >>>> >>> >>>> be >>>> >>> >>>> applied, and it's very far from being optimal and acceptable for >>>> >>> >>>> the end >>>> >>> >>>> users. Reboots are slow, they can be a problem depending on the >>>> >>> >>>> number of >>>> >>> >>>> nodes/hardware, and also the timing of reboot shall be totally >>>> >>> >>>> controlled >>>> >>> >>>> (after all puppet has been applied properly). >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> In the first three cases, we also hit the problem that TripleO >>>> >>> >>>> only >>>> >>> >>>> accepts one single overcloud image for all deployments - there is >>>> >>> >>>> no way to >>>> >>> >>>> instruct TripleO to upload and use several images, depending on >>>> >>> >>>> the node >>>> >>> >>>> type (although Ironic supports it). Also, we are worried about >>>> >>> >>>> upgrade paths >>>> >>> >>>> if we do image customizations. We need a clear way to move >>>> >>> >>>> forward on it. >>>> >>> >>>> >>>> >>> >>>> So, we'd like to discuss the possible options there and the >>>> >>> >>>> action >>>> >>> >>>> items to take (raise bugs, create some blueprints...). To >>>> >>> >>>> summarize, our end >>>> >>> >>>> goal is the following: >>>> >>> >>>> >>>> >>> >>>> - need to map overcloud-full images to roles >>>> >>> >>>> - need to be done in an automated way, no manual steps enforced, >>>> >>> >>>> and >>>> >>> >>>> in a way that can pass properly quality controls >>>> >>> >>>> - reboots are sub-optimal >>>> >>> >>>> >>>> >>> >>>> What are your thoughts there? >>>> >>> >>>> >>>> >>> >>>> Best, >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> Yolanda Robla >>>> >>> >>>> yrobl...@redhat.com >>>> >>> >>>> Principal Software Engineer - NFV Partner Engineer >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> >>>> >>> >>>> __________________________________________________________________________ >>>> >>> >>>> OpenStack Development Mailing List (not for usage questions) >>>> >>> >>>> Unsubscribe: >>>> >>> >>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> >>>> >>> >>> __________________________________________________________________________ >>>> >>> >>> OpenStack Development Mailing List (not for usage questions) >>>> >>> >>> Unsubscribe: >>>> >>> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>> >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> __________________________________________________________________________ >>>> >>> >> OpenStack Development Mailing List (not for usage questions) >>>> >>> >> Unsubscribe: >>>> >>> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> __________________________________________________________________________ >>>> >>> >> OpenStack Development Mailing List (not for usage questions) >>>> >>> >> Unsubscribe: >>>> >>> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >> >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > __________________________________________________________________________ >>>> >>> > OpenStack Development Mailing List (not for usage questions) >>>> >>> > Unsubscribe: >>>> >>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > __________________________________________________________________________ >>>> >>> > OpenStack Development Mailing List (not for usage questions) >>>> >>> > Unsubscribe: >>>> >>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>> >>>> >>> >>>> >>> __________________________________________________________________________ >>>> >>> OpenStack Development Mailing List (not for usage questions) >>>> >>> Unsubscribe: >>>> >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >> >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Yolanda Robla Mota >>>> >> NFV Partner Engineer >>>> >> yrobl...@redhat.com >>>> >> +34 605641639 >>>> >> >>>> >> >>>> >> __________________________________________________________________________ >>>> >> OpenStack Development Mailing List (not for usage questions) >>>> >> Unsubscribe: >>>> >> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >> >>>> > >>>> > >>>> > __________________________________________________________________________ >>>> > OpenStack Development Mailing List (not for usage questions) >>>> > Unsubscribe: >>>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>>> >>>> __________________________________________________________________________ >>>> OpenStack Development Mailing List (not for usage questions) >>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >>> >>> >>> -- >>> Yolanda Robla Mota >>> NFV Partner Engineer >>> yrobl...@redhat.com >>> +34 605641639 >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
