Absolutely Remi, we're looking to learn all that we can from those who already use Marvin. It's tougher for us non-developers to get started, but once we get the understanding in terms which we are used to working with, then we can better explain it to users and expand the number of people who can carry out testing.
Paul Angus VP Technology , ShapeBlue t: @cloudyangus<tel:@cloudyangus> e: paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> | w: www.shapeblue.com<http://www.shapeblue.com> -----Original Message----- From: Remi Bergsma [mailto:rberg...@schubergphilis.com] Sent: 20 January 2016 10:48 To: dev@cloudstack.apache.org Subject: Re: [PROPOSAL] LTS Release Cycle Hi Paul, I just hope you won’t reinvent the wheel ;-) Feel free to use what was build to test the 500+ PRs that got merged over the last couple of months to build 4.6, 4.7 and 4.8. For results see the comments in all these PRs. More info: https://github.com/schubergphilis/MCT-shared This is already being used by PCExtreme, Leaseweb and Schuberg Philis. Regards, Remi On 19/01/16 22:01, "Paul Angus" <paul.an...@shapeblue.com> wrote: >Hi Ilya (and all others), > >We (ShapeBlue) agree with you regarding the importance of automated >integration testing. The consulting team are currently working to understand >Marvin fully and what pieces we need to be able deploy properly representative >(virtualised) infrastructures to test against. We intend to open source our >resultant framework, with a view to community members being able to use it >themselves in their own labs but also so that as a community we can build it >out somewhere to be used as part of the projects' testing regime. > >We'll be reaching out to the community soon to work on the failures we're >seeing currently and understand where they come from - ie Marvin build, the >environment, the tests or CloudStack (watch out Mike T - I'm about to watch >your presentation video). > >We see this an important initiative for CloudStack as a whole, agile or >otherwise. But ultimately this is 'only' regression testing and the >organisations which need/want LTS, require it for a number of reasons, >including the fact new features bring new bugs which, if we knew we had to >test for, probably wouldn't have been there in the first place. > > > >Paul Angus >VP Technology , ShapeBlue > > >t: @cloudyangus<tel:@cloudyangus> > >e: paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com> | w: >www.shapeblue.com<http://www.shapeblue.com> > > > > > >-----Original Message----- >From: ilya [mailto:ilya.mailing.li...@gmail.com] >Sent: Tuesday, January 19, 2016 6:39 PM >To: dev@cloudstack.apache.org >Subject: Re: [PROPOSAL] LTS Release Cycle > >> Therefore, the process should strive to make as a few releases as >necessary to achieve this goal. > >I guess part two to this question would be - we need the automated testing >environments. This can ensure rapid release testing and acutal release, and we >dont have to restrains ourselves to limited number of releases LTS release due >to QA pains. > >We need a separate initiative with CloudStack testing framework. > >On 1/18/16 6:54 PM, John Burwell wrote: >> Ilya, >> >> Unless we have a bug fix that addresses a significant, widespread system >> stability problem or a high priority/impact security issue, an LTS will roll >> up a number of fixes. Each release would receive the full system test to >> verify that the patch set does not introduce regression defects. I believe >> that most LTS users want a few releases as necessary to keep their systems >> up-to-date and stable because each upgrade carries operational risk and >> downtime. Therefore, the process should strive to make as a few releases as >> necessary to achieve this goal. >> >> Thanks, >> -John >> >>> On Jan 15, 2016, at 3:22 PM, ilya <ilya.mailing.li...@gmail.com> wrote: >>> >>> John >>> >>> Thank you for taking time writing out the LTS proposal. >>> >>>> Broad community support is vital to guarantee the twenty (20) month >>>> support period for each LTS branch. Given the ebbs and flows of >>>> contribution and committer priorities, ShapeBlue will provide a >>>> release manager, as well as, engineering support to fill any >>>> contribution gaps to ensure that the community fulfills LTS commitments. >>> >>> You guys rock!! >>> >>> I'm +1 on this, >>> >>> Can you please expand on the QA side of LTS. Since this is more >>> around long term bug/security fix - i'd think - the testing will be >>> minimal, to the scope that fix applies - which will speed up the >>> release process in general. What are your thoughts on this? >>> >>> >>> Thanks >>> ilya >>> >>> >>> >>> >>> >>> On 1/15/16 10:48 AM, John Burwell wrote: >>>> Motivation >>>> ======== >>>> >>>> The current monthly release cycle addresses the needs of users >>>> focused on deploying new functionality as quickly as possible. It >>>> does not address the needs of users oriented towards stability >>>> rather than new functionality. These users typically employ QA >>>> processes to comply with corporate policy and/or regulatory >>>> requirements. To maintain a growing, thriving community, we must address >>>> the needs of both user types. >>>> Therefore, I propose that we overlay a LTS release cycle onto the >>>> monthly release cycle to address the needs of stability-oriented >>>> users with minimal to no impact on the monthly release cycle. This >>>> proposed LTS release cycle has the following goals: >>>> >>>> * Prefer Stability to New Functionality: Deliver releases that only >>>> address defects and CVEs. This narrowly focused change scope >>>> greatly reduces the upgrade risk/operational impact and shorter internal >>>> QA cycles. >>>> * Reliable Release Lifetimes: Embracing a time-based release >>>> strategy, the LTS release cycle will provide users with a reliable >>>> support time frames. Users can use these time frames provide users >>>> with an 20 month window in which to plan upgrades. >>>> * Support Sustainability: With a defined end of support for LTS >>>> releases and a maximum of two (2) LTS releases under active >>>> maintenance at any given time, community members can better plan >>>> their commitments to release support activities. We also have a >>>> agreed upon policy for release end-of-life (EOL) to debate about >>>> continuing work on old releases. >>>> >>>> Proposed Process >>>> ============== >>>> >>>> LTS release branches will be cut twice year on 1 Jan and 1 July >>>> from the tag of the most recent monthly release. The branch will be >>>> named <base >>>> version>-LTS and each LTS release will be versioned in the form of >>>> version><base -<LTS revision number>. For example, if we cut an LTS >>>> version>branch >>>> based on 4.7.0, the branch would be named 4.7.0-LTS and the version >>>> of the first LTS release would be 4.7.0-0, the second would be >>>> 4.7.0-1, etc. This release naming convention differentiates LTS and >>>> monthly releases, communicates the version on which the LTS release >>>> is based, and allows the maintenance releases for monthly releases >>>> without version number contention/conflict. Finally, like master, >>>> an LTS branch would be always deployable following its initial release. >>>> While it is unlikely that LTS users would deploy from the branch, >>>> the quality discipline of this requirement will benefit the long term >>>> stability of LTS releases. >>>> All PRs targeting an LTS would require two LGTMs in order to be merged. >>>> >>>> The following are the types of changes that would permitted and >>>> guarantees provided to users: >>>> >>>> * No features or enhancements would be backported to LTS release branches. >>>> * Database changes would be limited to those required to address >>>> the backported defect fixes. >>>> * Support for the release/version of the following components from >>>> the release on which the LTS is based throughout the entire release cycle: >>>> * MySQL/MariaDB >>>> * JDK/JRE >>>> * Linux distributions >>>> * API compatibility for between all LTS revisions. API changes >>>> would be limited to those required to fix defects or address security >>>> issues. >>>> >>>> An LTS release would have a twenty (20) month lifetime from the >>>> date the release branch is cut. This support period allows up to >>>> two (2) months of branch stabilization before initial release with >>>> a minimum of eighteen (18) months of availability for deployment. >>>> LTS releases would have the following support periods: >>>> >>>> * 0-14 months: backport all defect fixes in the scope of the LTS >>>> release functionality; fix all blocker and critical priority >>>> defects identified in the branch >>>> * 14-20 months: backport only CVE fixes in the scope of the LTS >>>> release functionality; fix all blocker priority defects in the >>>> branch >>>> >>>> Defect fixes that originate in an LTS branch will be pulled forward >>>> to master only. Finally, an LTS branch should be release the fewest >>>> times necessary to deliver fixes in a relatively timely manner. >>>> Therefore, LTS releases will be triggered based on the number of >>>> pending of fixes and their severity with no defect fix awaiting >>>> release more than forty-five >>>> (45) days. CVE fixes would trigger an immediate release of an LTS branch. >>>> >>>> Resourcing and Proposed Timeline >>>> =========================== >>>> >>>> Broad community support is vital to guarantee the twenty (20) month >>>> support period for each LTS branch. Given the ebbs and flows of >>>> contribution and committer priorities, ShapeBlue will provide a >>>> release manager, as well as, engineering support to fill any >>>> contribution gaps to ensure that the community fulfills LTS commitments. >>>> >>>> In order to prepare for supporting LTS releases, we would need to >>>> complete the following items: >>>> >>>> 1. All tools that do version number comparisons would be made aware >>>> of the LTS versioning scheme 2. Officially support running the >>>> management server on Java 8 since Java >>>> 7 has been EOL since last April [1] (i.e. compile to 1.7 with the >>>> Java >>>> 1.8 compiler and run on Java 1.8). Providing a 20-month support >>>> window on an EOL JVM is an unacceptable security risk. >>>> 3. Update the wiki and website to explain the new release cycle and >>>> help users decide which release type suits their needs 4. Define an >>>> initial test plan for LTS stabilization 5. Agree on the definitions >>>> of ticket severities >>>> >>>> In order to address these items and start on a regular rhythm, I >>>> propose that first LTS cycle begin on 1 July 2015. In the interim, >>>> we would continue to ship critical bug fixes from the 4.5 release >>>> as this release seems to be the most commonly deployed in the community. >>>> >>>> Together, the monthly and LTS release cycles address the needs of >>>> the entire Apache CloudStack user community. I believe that the >>>> process described in the proposal will yield releases that meet the >>>> needs of users requiring release stability without adversely >>>> affecting the velocity of the monthly release cycle. >>>> >>>> Thanks, >>>> -John >>>> >>>> [1]: http://www.oracle.com/technetwork/java/eol-135779.html >>>> >>>> >>>> ShapeBlue <http://www.shapeblue.com> John Burwell ShapeBlue >>>> >>>> d: *+44 (20) 3603 0542 | s: +1 (571) 403-2411 * >>>> <tel:+44%20(20)%203603%200542%20|%20s:%20+1%20(571)%20403-2411> >>>> >>>> e: *john.burw...@shapeblue.com | t: * >>>> <mailto:john.burw...@shapeblue.com%20|%20t:> | w: >>>> *www.shapeblue.com* <http://www.shapeblue.com> >>>> >>>> a: 53 Chandos Place, Covent Garden London WC2N 4HS UK >>>> >>>> Shape Blue Ltd is a company incorporated in England & Wales. >>>> ShapeBlue Services India LLP is a company incorporated in India and >>>> is operated under license from Shape Blue Ltd. Shape Blue Brasil >>>> Consultoria Ltda is a company incorporated in Brasil and is >>>> operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is >>>> a company registered by The Republic of South Africa and is traded >>>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>>> This email and any attachments to it may be confidential and are >>>> intended solely for the use of the individual to whom it is addressed. >>>> Any views or opinions expressed are solely those of the author and >>>> do not necessarily represent those of Shape Blue Ltd or related companies. >>>> If you are not the intended recipient of this email, you must >>>> neither take any action based upon its contents, nor copy or show it to >>>> anyone. >>>> Please contact the sender if you believe you have received this >>>> email in error. >>>> >>>> >>>> Find out more about ShapeBlue and our range of CloudStack related services: >>>> IaaS Cloud Design & Build >>>> <http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – >>>> rapid IaaS deployment framework <http://shapeblue.com/csforge/> >>>> CloudStack Consulting >>>> <http://shapeblue.com/cloudstack-consultancy/> >>>> | CloudStack Software Engineering >>>> <http://shapeblue.com/cloudstack-software-engineering/> >>>> CloudStack Infrastructure Support >>>> <http://shapeblue.com/cloudstack-infrastructure-support/> | >>>> CloudStack Bootcamp Training Courses >>>> <http://shapeblue.com/cloudstack-training/> >> >> Find out more about ShapeBlue and our range of CloudStack related services: >> IaaS Cloud Design & >> Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – >> rapid IaaS deployment framework<http://shapeblue.com/csforge/> >> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | >> CloudStack Software >> Engineering<http://shapeblue.com/cloudstack-software-engineering/> >> CloudStack Infrastructure >> Support<http://shapeblue.com/cloudstack-infrastructure-support/> | >> CloudStack Bootcamp Training >> Courses<http://shapeblue.com/cloudstack-training/> >> >Find out more about ShapeBlue and our range of CloudStack related services: >IaaS Cloud Design & >Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – >rapid IaaS deployment framework<http://shapeblue.com/csforge/> >CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | >CloudStack Software >Engineering<http://shapeblue.com/cloudstack-software-engineering/> >CloudStack Infrastructure >Support<http://shapeblue.com/cloudstack-infrastructure-support/> | >CloudStack Bootcamp Training >Courses<http://shapeblue.com/cloudstack-training/> Find out more about ShapeBlue and our range of CloudStack related services: IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> | CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> | CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/> | CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
