On 07/19/2018 06:55 PM, Paul Belanger wrote: > On Thu, Jul 19, 2018 at 05:30:27PM +0200, Cédric Jeanneret wrote: >> Hello, >> >> While trying to get a new validation¹ in the undercloud preflight >> checks, I hit an (not so) unexpected issue with the CI: >> it doesn't provide flavors with the minimal requirements, at least >> regarding the disk space. >> >> A quick-fix is to disable the validations in the CI - Wes has already >> pushed a patch for that in the upstream CI: >> https://review.openstack.org/#/c/583275/ >> We can consider this as a quick'n'temporary fix². >> >> The issue is on the RDO CI: apparently, they provide instances with >> "only" 55G of free space, making the checks fail: >> https://logs.rdoproject.org/17/582917/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/c9cf398/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-07-17_10_23_46 >> >> So, the question is: would it be possible to lower the requirment to, >> let's say, 50G? Where does that 60G³ come from? >> >> Thanks for your help/feedback. >> >> Cheers, >> >> C. >> >> >> >> ¹ https://review.openstack.org/#/c/582917/ >> >> ² as you might know, there's a BP for a unified validation framework, >> and it will allow to get injected configuration in CI env in order to >> lower the requirements if necessary: >> https://blueprints.launchpad.net/tripleo/+spec/validation-framework >> >> ³ >> http://tripleo.org/install/environments/baremetal.html#minimum-system-requirements >> > Keep in mind, upstream we don't really have control over partitions of nodes, > in > some case it is a single, other multiple. I'd suggest looking more at:
After some checks on y locally deployed containerized undercloud (hence, Rocky) without real activity, here's what I could get: - most data are located in /var - this explains the current check. If we go a bit deeper, here are the "actually used" directory in /var/lib: 20K alternatives 36K certmonger 4.0K chrony 1.2G config-data 4.0K dhclient 6.0G docker 28K docker-config-scripts 92K docker-container-startup-configs.json 44K docker-puppet 592K heat-config 832K ironic 4.0K ironic-inspector 236K kolla 4.0K logrotate 286M mysql 48K neutron 4.0K ntp 4.0K postfix 872K puppet 3.8M rabbitmq 59M rpm 4.0K rsyslog 64K systemd 20K tripleo 236K tripleo-config 9.8M yum 7.5G total Most of the "default installer" partition schema don't go further than putting /var, /tmp, /home and /usr in dedicated volumes - of course, end-user can chose to ignore that and provide a custom schema. That said, we can get the "used" paths. In addition to /var/lib, there's obviously /usr. We might want to: - loop on known locations - check if they are on dedicated mount points - check the available disk space on those mount points. An interesting thing in bash: df /var/lib/docker Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 104846316 10188828 94657488 10% / This allows to get: - actual volume - free space on the volume. More than that, we might also try to figure out some pattern. For instance, "docker" seems to be a pretty good candidate for space, as it will get the images and container data. This is probably even the biggest eater, at least on the undercloud - as well as the logs (/var/logs). We might do a check ensuring we can, at least, DEPLOY the app. This would require far less than the required 60G, and with a proper doc announcing that, we can get a functional test, aiming on its purpose: ensure we can deploy (so asking, let's say, 10G in /var/lib/docker, 5G in /var/lib/config-data, 5G in /usr, 1G in /var/log) and, later, upgrade (requiring the same amount of *free* space). That would require some changes in the validation check of course. But at least, we might get a pretty nice covering, while allowing it to run smoothly in the CI. But, as said: proper documentation should be set, and the "60G minimum required" should be rephrased in order to point the locations needing space (with the appropriate warning about "none exhaustiveness" and the like). Would that suit better the actual needs, and allow to get a proper disk space check/validation? Cheers, C. > > https://docs.openstack.org/infra/manual/testing.html > > As for downstream RDO, the same is going to apply once we start adding more > cloud providers. I would look to see if you actually need that much space for > deployments, and make try to mock the testing of that logic. > > - Paul > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Cédric Jeanneret Software Engineer DFG:DF
signature.asc
Description: OpenPGP digital signature
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
