Yes, we are talking about physical hardware and large clusters. Also in a very demanding environment. So reboots are really suboptimal in that case.
----- Original Message ----- From: "Alex Schultz" <aschu...@redhat.com> To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org> Sent: Friday, December 2, 2016 3:38:53 PM Subject: Re: [openstack-dev] [tripleo] [ironic] Need to update kernel parameters on local boot On Fri, Dec 2, 2016 at 7:29 AM, Oliver Walsh <owa...@redhat.com> wrote: > Hi Yolanda, > > I've the same requirements for a real-time compute role. > > I'm current using #4. Puppet sets the kernel cmdline in > /etc/defaults/grub, then touches a file that triggers a reboot in an > os-refresh-config post-configure script. > > How much of an issue is the reboot? In my dev env it doesn't make a > significant difference to the overall deployment time of a node. > If this is physical hardware, the reboot can add 5-10+ minutes depending on the vendor. VM's aren't a good representation of the actual cost of a reboot. Thanks, -Alex > I'm thinking we could use #4 as the general solution. In cases where a > reboot is too expensive then #1/#2/#3 could be used to prime the image > with same /etc/default/grub that puppet would create. As puppet > doesn't change it doesn't trigger a reboot. > > > Re custom images. Support in python-tripleoclient could be improved > but for now this is what I'm doing: > > Uploading a custom image: > OS_IMAGE=custom-overcloud-full.qcow2 openstack overcloud image upload > > Set the Image for the custom role in param_defaults: > parameter_defaults: > CustomRoleImage: custom-overcloud-full > > Thanks, > Ollie > > On 28 November 2016 at 15:36, Yolanda Robla Mota <yrobl...@redhat.com> wrote: >> Hi, good afternoon >> >> I wanted to start an email thread about how to properly setup kernel >> parameters on local boot, for our overcloud images on TripleO. >> These parameters may vary depending on the needs of our end users, and even >> can be different ( for different roles ) per deployment. As an example, we >> need it for: >> - enable FIPS kernel in terms of security >> (https://bugs.launchpad.net/tripleo/+bug/1640235) >> - enable functionality for DPDK/SR-IOV >> (https://review.openstack.org/#/c/331564/) >> - enable rd.iscsi.firmware=1 flag (this for the ramdisk image) >> - etc.. >> >> So far, the solutions we got were on several directions: >> >> 1. Update the golden overcloud-full image with virt-customize, modifying >> /etc/default/grub settings according to our needs: this is a manual process, >> not really driven by TripleO. End users will want to avoid manual steps as >> much as possible. Also if we announce that OpenStack ships features in >> TripleO like DPDK, SR-IOV... doesn't make sense to tell end users that if >> they want to consume that feature, they need to do manual updates on the >> image. It shall be natively supported, or configurable per TripleO >> environments. >> >> 2. Create our own images using diskimage-builder and custom elements: in >> this case, we have the problem that the partners will loose support, as >> building their own images is good for upstream, but not accepted into the >> OSP environment. Also the combination of images needed can be huge, that can >> be a blocker for QA. >> >> 3. Add Ironic support for it. Images can be uploaded to glance, and some >> properties can be set on metadata, like a json with kernel parameters. >> Ironic will modify these kernel parameters when deploying the image (in a >> similar way that when it installs bootloader, or generates partitions). >> >> 4. Configure it post-deployment: there can be some puppet element that >> updates kernel parameters. But it will need a node reboot to be applied, and >> it's very far from being optimal and acceptable for the end users. Reboots >> are slow, they can be a problem depending on the number of nodes/hardware, >> and also the timing of reboot shall be totally controlled (after all puppet >> has been applied properly). >> >> >> In the first three cases, we also hit the problem that TripleO only accepts >> one single overcloud image for all deployments - there is no way to instruct >> TripleO to upload and use several images, depending on the node type >> (although Ironic supports it). Also, we are worried about upgrade paths if >> we do image customizations. We need a clear way to move forward on it. >> >> So, we'd like to discuss the possible options there and the action items to >> take (raise bugs, create some blueprints...). To summarize, our end goal is >> the following: >> >> - need to map overcloud-full images to roles >> - need to be done in an automated way, no manual steps enforced, and in a >> way that can pass properly quality controls >> - reboots are sub-optimal >> >> What are your thoughts there? >> >> Best, >> >> >> Yolanda Robla >> yrobl...@redhat.com >> Principal Software Engineer - NFV Partner Engineer >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
