Hi Paul, This is great news, good job, naming apart. :-)
Is ESXi a hard requirement on running Trillian? -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro ----- Original Message ----- > From: "Paul Angus" <paul.an...@shapeblue.com> > To: dev@cloudstack.apache.org > Cc: us...@cloudstack.apache.org > Sent: Tuesday, 2 February, 2016 10:49:46 > Subject: [Discuss] Flexible CI/CD - Trillian > Project: Trillian > We have been working on PoC of a CI environment design which will: > > · Provide fast build or rebuild of environments for testing. > > · Enable multiple independent concurrent builds > > · Be available on-demand through automation or individual request. > > · Be capable of fully utilising all available hardware > > · Flexible enough to be used to build super-realistic development > environments. > We intend to contribute and maintain our work within the Apache repos. > However, > we are currently building the POC, figuring out the requirements (and quirks) > of the individual pieces, before pushing something concrete for to the > community to review. > > We envision that Trillian would cater for a number of use cases: > > 1. CloudStack community integration testing of master against multiple > deployment scenarios (using ASF infra) > > 2. CloudStack community integration testing of PRs against multiple > deployment scenarios (using ASF infra) > > 3. Organisations/individuals running the full suites of tests available > in > Marvin against any physical environment they have. > > 4. Organisations/individuals deploying and running the full suites of > tests available in Marvin against virtualised infrastructures which can be > deployed by Marvin. > As we intend Trillian to test multiple environments concurrently, we use > nested > virtualization on ESXi hosts (our testing has shown that this is the only > hypervisor which can support the nested virtualisation of all other > hypervisors > with reasonable performance). We use Ansible to deploy and configure all > aspects of the build as this will greatly lower the barrier to entry for > independent testers. > We use CloudStack to provision the management server and virtualised (nested) > hosts on the physical hosts. We are creating Ansible playbooks and roles which > can: > > 1. Create guest instances using Rene’s Ansible 2.0 CloudStack modules - > a > Marvin VM, a Mgmt Server (CentOS or Ubuntu), any number of compute hosts (KVM, > vSphere or XenServer. Hyper-V later) > > 2. Configure hosts (inc. installing the relevant CloudStack agent where > required) > > 3. Install required ACS packages on management server > > 4. Configure a zone (including adding the compute hosts) via Marvin. > > 5. Run the required Marvin tests. > > 6. Return the results > We may need to propose enhancements to Marvin in order to sync the > configuration > of hosts with the configuration used by Marvin. > > Using virtualised test environments, we can have multiple test scenarios > running > concurrently. To do this we have found that it is necessary to create pools or > ranges of VLANs and IP addresses and allocate them to environments. So for > any > given physical environment which will be used for testing in, we take the > total > range(s) of IPs and VLANs available and carve them into non-overlapping chunks > suitable for concurrent use as mgmt, public and guest networks. These are > stored in a MariaDB database. When a range is being used in a testing > environment, that range is marked as ‘inuse’ in the database. When creating a > test environment, Trillian looks in the database for the next available VLAN > range, the next available public IP range and so on. The returned values are > used to populate a Marvin cfg file which in turn will be used to both build > the > environment and when running the Marvin testing. When the virtualised infra > is > cleaned up, the database will be updated to reflect that the used ranges are > available again. > This initiative has only recently been started, and as stated earlier we are > currently figuring out the requirements (and quirks) of the individual pieces > and looking for the most suitable wrapper to glue it all together. > Also I have found that Marvin requires a little work to make the output more > meaningful/readable (especially in the case of errors and exceptions) and to > make it a little more intelligent about the tests it can/can’t run based on > the > chosen infrastructure components. I have also found unreachable or very slow > ISO and template paths hardcoded into Marvin or individual tests. > We plan to enhance tests to address these issues and also reduce runtimes > where > possible. > > NEXT STEPS > > We will continue to our work on the PoC and share the results. > > We’d like to get some community agreement around the goals and use cases which > we are planning to satisfy before we get too far down the road of our PoC. So > please share your thoughts. > > > > [ShapeBlue]<http://www.shapeblue.com> > Paul Angus > VP Technology , ShapeBlue > > > d: +44 203 617 0528 | s: +44 203 603 > 0540<tel:+44%20203%20617%200528%20|%20s:%20+44%20203%20603%200540> | > m: +44 7711 418784<tel:+44%207711%20418784> > > e: paul.an...@shapeblue.com | t: > @cloudyangus<mailto:paul.an...@shapeblue.com%20|%20t:%20@cloudyangus> | > w: www.shapeblue.com<http://www.shapeblue.com> > > a: 53 Chandos Place, Covent Garden London WC2N 4HS UK > > > [cid:imagea932fa.png@ce3f6520.47a350c4] > > > Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue > Services > India LLP is a company incorporated in India and is operated under license > from > Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated > in > Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd > is a company registered by The Republic of South Africa and is traded under > license from Shape Blue Ltd. ShapeBlue is a registered trademark. > This email and any attachments to it may be confidential and are intended > solely > for the use of the individual to whom it is addressed. Any views or opinions > expressed are solely those of the author and do not necessarily represent > those > of Shape Blue Ltd or related companies. If you are not the intended recipient > of this email, you must neither take any action based upon its contents, nor > copy or show it to anyone. Please contact the sender if you believe you have > received this email in error. > > > > > Find out more about ShapeBlue and our range of CloudStack related services: > IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> > | > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | > CloudStack > Software Engineering<http://shapeblue.com/cloudstack-software-engineering/> > CloudStack Infrastructure > Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack > Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
