All,
We don't have any outstanding blocker PRs now. I could n't reproduce any regression wrt only VM-HA in RC1. By killing a HV host (and keep it that way, i.e. not rebooting it), HA enabled VMs did migrate to other hosts in a test I did. The only potential (blocker) issue seems to be a ldap related regression reported on users@ that Daan is looking into, which may or may not be a blocker but a configuration issue. Given, we're approaching the end of the week, it's likely we cut RC2 on Monday. Please continue testing RC1 in the meanwhile. Regards, Rohit Yadav ________________________________ From: Nux! <n...@li.nux.ro> Sent: Wednesday, January 24, 2018 9:29:21 AM To: users Cc: dev Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) Paul, Ok, let me play some more with those tunings params, see if I can find a satisfactory combination and I'll withhold a -1 on this on the basis that I've just misunderstood the whole thing. :) Thanks -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro rohit.ya...@shapeblue.com www.shapeblue.com 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue ----- Original Message ----- > From: "Paul Angus" <paul.an...@shapeblue.com> > To: "users" <us...@cloudstack.apache.org> > Cc: "dev" <dev@cloudstack.apache.org> > Sent: Tuesday, 23 January, 2018 19:55:01 > Subject: RE: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) > Hey Nux, > > There is quite a bit of tuning you can do, to speed or slow CloudStack's > decision making, but we need to be sure that when we lose contact with a host > agent, that the VMs themselves really are dead. By default host-ha is set to > be super sure. > > There are various timeouts which can be configured to decide how long to wait > for a host to restart before deciding that it is not going to start as well as > how many times we should check for disk activity from the resident VMs of a > suspect host. > > The parameters are detailed here. > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Host+HA > > Honestly, the aim of Host HA was to fix the particular issue that you are > describing as we can't remember a time when it did work reliably. > > > > paul.an...@shapeblue.com > www.shapeblue.com<http://www.shapeblue.com> > 53 Chandos Place, Covent Garden, London WC2N 4HSUK > @shapeblue > > > > > -----Original Message----- > From: Nux! [mailto:n...@li.nux.ro] > Sent: 23 January 2018 19:08 > To: users <us...@cloudstack.apache.org> > Cc: dev <dev@cloudstack.apache.org> > Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) > > Hi Paul, > > To be honest I do not remember when I last saw this, as I have not been > testing > ACS in 2017. > You'd kill a HV, the VMs would pop up on another after a few minutes. > > Even with Host HA, the VMs remain down until the hypervisor is back up, > restarted by OOBM - however if that HV has suffered a HW fault and needs to be > removed, then those VM will be down for a long time ... > > Unless I got things quite wrong, (VM) HA - one of the big selling points of > ACS > - is essentially broken? > > -- > Sent from the Delta quadrant using Borg technology! > > Nux! > www.nux.ro > > ----- Original Message ----- >> From: "Paul Angus" <paul.an...@shapeblue.com> >> To: "users" <us...@cloudstack.apache.org>, "dev" >> <dev@cloudstack.apache.org> >> Sent: Tuesday, 23 January, 2018 16:02:54 >> Subject: RE: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) > >> Hi Nux, >> >> When have you seen the VMs on KVM behaving in the manner which you are >> expecting? I recall it didn’t work that way in the mid 4.5 versions >> (we found out the hard way in front of a customer) and it doesn't >> behave the way you are expecting 4.9 - I've just tested it. >> >> You need host-ha enabled to get reliable HA in the event of a host >> crash, that is why we developed the host ha feature. >> >> Kind regards, >> >> Paul Angus >> >> paul.an...@shapeblue.com >> www.shapeblue.com<http://www.shapeblue.com> >> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >> >> >> >> >> -----Original Message----- >> From: Nux! [mailto:n...@li.nux.ro] >> Sent: 23 January 2018 15:06 >> To: dev <dev@cloudstack.apache.org> >> Cc: users <us...@cloudstack.apache.org> >> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >> >> Rohit, >> >> I'll also have to insist with the VM HA issue. >> https://issues.apache.org/jira/browse/CLOUDSTACK-10246 >> >> Lucian >> >> -- >> Sent from the Delta quadrant using Borg technology! >> >> Nux! >> www.nux.ro >> >> ----- Original Message ----- >>> From: "Rohit Yadav" <rohit.ya...@shapeblue.com> >>> To: "dev" <dev@cloudstack.apache.org>, "users" >>> <us...@cloudstack.apache.org> >>> Sent: Tuesday, 23 January, 2018 14:28:34 >>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >> >>> All, >>> >>> >>> Given we've outstanding blockers and PRs in review/testing, I'll cut >>> RC2 only after we manage to get them reviewed, tested and merged. >>> >>> >>> The outstanding PRs considered for RC2 are: >>> >>> https://github.com/apache/cloudstack/pull/2418 (Properly parse rules >>> for security groups) >>> >>> https://github.com/apache/cloudstack/pull/2419 (Password server >>> issue) >>> >>> >>> In addition we've following issues to receive fixes: >>> >>> - VR - DHCP/dnsmasq leases issue (reported by Ozhan) >>> >>> - Dynamic roles upgrade fixes: >>> https://issues.apache.org/jira/browse/CLOUDSTACK-10249 >>> >>> >>> Please share any other issues you've found, or I've missed. Thanks, >>> and continue testing RC1. >>> >>> >>> - Rohit >>> >>> <https://cloudstack.apache.org> >>> >>> >>> >>> ________________________________ >>> From: Rohit Yadav <rohit.ya...@shapeblue.com> >>> Sent: Monday, January 22, 2018 11:18:27 AM >>> To: Paul Angus; us...@cloudstack.apache.org; >>> dev@cloudstack.apache.org >>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>> >>> The same issue applies to any 4.9, 4.10 release. In case of 4.9, we >>> had discussed this as a doc bug and so it must be documented part of >>> the 4.11 release notes as well. >>> >>> >>> There are two ways admin can migrate to dynamic roles post-upgrade: >>> >>> >>> 1. Enable dynamic.apichecker.enabled to true which will use the >>> default api mapping of rules from 4.8 commands.properties and >>> automatic annotation based and (db-backed) dynamic rules from 4.9+. >>> Or, >>> >>> 2. The migration script is only useful where admins were not using >>> the default api rule mappings and they strictly want to >>> check/migrate each API. This approach requires admins to go through >>> new APIs and fix commands.properties before running the migration >>> scriopt (we've been sharing the new/change API list in release notes, for >>> example: >>> >>> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.9.3.0/api-changes.html#new-api-commands). >>> (for reference, doc: >>> >>> http://docs.cloudstack.apache.org/projects/cloudstack-administration/ >>> e >>> n/latest/accounts.html#using-dynamic-roles) >>> >>> >>> Unlike the dynamic API checker, the static checker does not even >>> allow the root API to access all the APIs which is why post upgrade, >>> if the UI calls any API that is not allowed for the root admin (in >>> this case the quotaIsEnabled API) the UI will logout the user on API >>> unauthorized failure which is what happened. >>> >>> >>> So, we can discuss two fixes: >>> >>> - Like dynamic checker, let the static checker allow all APIs only to >>> the root admin (id=1) (I would not prefer to change the legacy >>> behaviour though) >>> >>> - During upgrade, if commands.properties is missing we set the global >>> setting to true, i.e. switch to dynamic roles (which would happen if >>> someone tries to upgrade from 4.5->4.11 using a new mgmt server if >>> they fail to copy the commands.properties file from /usr/share or /etc >>> paths). >>> >>> >>> Thoughts? >>> >>> >>> - Rohit >>> >>> <https://cloudstack.apache.org> >>> >>> >>> >>> ________________________________ >>> >>> rohit.ya...@shapeblue.com >>> www.shapeblue.com<http://www.shapeblue.com> >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>> >>> >>> >>> >>> rohit.ya...@shapeblue.com >>> www.shapeblue.com<http://www.shapeblue.com> >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>> >>> >>> >>> From: Paul Angus >>> Sent: Monday, January 22, 2018 9:24:25 AM >>> To: us...@cloudstack.apache.org >>> Cc: Rohit Yadav; dev@cloudstack.apache.org; Daan Hoogland >>> Subject: RE: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>> >>> If I've understood the issue correctly, "not being able to log in if >>> upgrading >>> from 4.5" is a blocker in my book. I don't think that it should be the >>> duty >>> of the Admin, to fix our oversights. Migration to the use of dynamic >>> roles is also broken as the command will be missing from >>> commands.properties in the first place, so the 'migrated' commands >>> will not be complete. >>> >>> As there will need to be an RC2, IMO this upgrade issue should be >>> fixed as part of it. >>> >>> >>> >>> Kind regards, >>> >>> Paul Angus >>> >>> >>> VP Technology >>> paul.an...@shapeblue.com >>> www.shapeblue.com<http://www.shapeblue.com> >>> >>> >>> >>> >>> -----Original Message----- >>> From: Boris Stoyanov [mailto:boris.stoya...@shapeblue.com] >>> Sent: 22 January 2018 07:31 >>> To: us...@cloudstack.apache.org >>> Cc: Rohit Yadav <rohit.ya...@shapeblue.com>; >>> dev@cloudstack.apache.org; Daan Hoogland >>> <daan.hoogl...@shapeblue.com> >>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>> >>> Hi Paul, >>> Migration script considers only what’s in the command.properties >>> file, so if the ‘missing’ quotaIsEnabled=15 is not there it will not >>> create a rule for it. As Rohit mentioned it’s a duty of the admin to >>> take care of aligning this up. I’m also not big fan of having this >>> described in release notes, but would like to be included >>> automatically during upgrade. Main argument against it, its not a blocker. >>> >>> Bobby. >>> >>> >>> boris.stoya...@shapeblue.com >>> www.shapeblue.com<http://www.shapeblue.com> >>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>> >>> >>> >>>> On 19 Jan 2018, at 19:04, Paul Angus <paul.an...@shapeblue.com> wrote: >>>> >>>> OK, just to confirm ‘we’ the community have basically deprecated the >>>> use of commands.properties? >>>> >>>> But for people upgrading from a version before dynamic roles, does >>>> the migration script take into account (or need to take into >>>> account) the ‘missing’ >>>> quotaIsEnabled=15 parameter? >>>> >>>> >>>> >>>> >>>> paul.an...@shapeblue.com >>>> www.shapeblue.com<http://www.shapeblue.com> >>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>>> >>>> >>>> >>>> From: Rohit Yadav >>>> Sent: 19 January 2018 09:27 >>>> To: users <us...@cloudstack.apache.org>; dev@cloudstack.apache.org; >>>> Paul Angus <paul.an...@shapeblue.com> >>>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>>> >>>> >>>> Hi Bobby, >>>> >>>> >>>> >>>> Agree, it's not user-friendly which is why admins should migrate to >>>> the dynamic roles feature. But I'm not sure if this is a blocker and >>>> if an admin wants to stick to the old static (commands.properties) >>>> way, they need to manage changes themselves. We may add something to >>>> the release notes /cc @Paul Angus<mailto:paul.an...@shapeblue.com>. >>>> >>>> >>>> >>>> - Rohit >>>> >>>> >>>> >>>> Software Architect >>>> rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com> >>>> www.shapeblue.com<http://www.shapeblue.com> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> ________________________________ >>>> From: Boris Stoyanov >>>> <boris.stoya...@shapeblue.com<mailto:boris.stoya...@shapeblue.com>> >>>> Sent: Friday, January 19, 2018 2:51:32 PM >>>> To: users >>>> Cc: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> >>>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>>> >>>> Hi Rohit, >>>> >>>> That doesn’t sound much user friendly what do you think? Can we look >>>> for a way to automate this dependency in the upgrade process? >>>> >>>> Bobby. >>>> >>>> >>>> boris.stoya...@shapeblue.com<mailto:boris.stoya...@shapeblue.com> >>>> www.shapeblue.com<http://www.shapeblue.com> >>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>>> >>>> >>>> >>>>> On 19 Jan 2018, at 10:50, Rohit Yadav >>>>> <rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com>> wrote: >>>>> >>>>> Hi Bobby, >>>>> >>>>> >>>>> I checked the 4.5-4.11 upgrade environment, due to the nature of >>>>> how static checker with commands.properties work, admins will be >>>>> required to add/update new API/ACLs in the commands.properties file. >>>>> >>>>> Adding the following to commands.properties file and restarting >>>>> mgmt server fixes the issue: >>>>> >>>>> quotaIsEnabled=15 >>>>> >>>>> >>>>> Please continue testing, thanks. >>>>> >>>>> >>>>> - Rohit >>>>> >>>>> <https://cloudstack.apache.org> >>>>> >>>>> >>>>> >>>>> ________________________________ >>>>> From: Boris Stoyanov >>>>> <boris.stoya...@shapeblue.com<mailto:boris.stoya...@shapeblue.com>> >>>>> Sent: Wednesday, January 17, 2018 6:54:28 PM >>>>> To: us...@cloudstack.apache.org<mailto:us...@cloudstack.apache.org> >>>>> Cc: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> >>>>> Subject: Re: [VOTE] Apache Cloudstack 4.11.0.0 (LTS) >>>>> >>>>> I think I’ve hit a blocker when upgrading to 4.11 >>>>> >>>>> Here’s the jira id: >>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-10236 >>>>> >>>>> I’ve upgraded from 4.5 to 4.11, then I’ve logged in with admin and >>>>> got session expired immediately. >>>>> >>>>> Regards, >>>>> Boris Stoyanov >>>>> >>>>> >>>>> boris.stoya...@shapeblue.com<mailto:boris.stoya...@shapeblue.com> >>>>> www.shapeblue.com<http://www.shapeblue.com> >>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >>>>> >>>>> >>>>> >>>>> >>>>> rohit.ya...@shapeblue.com<mailto:rohit.ya...@shapeblue.com> >>>>> www.shapeblue.com<http://www.shapeblue.com> >>>>> 53 Chandos Place, Covent Garden, London WC2N 4HSUK >>>>> @shapeblue >>>>> >>>>> >>>>> >>>>> On 17 Jan 2018, at 8:42, Tutkowski, Mike >>>>> <mike.tutkow...@netapp.com<mailto:mike.tutkow...@netapp.com<mailto:mike.tutkow...@netapp.com%3cmailto:mike.tutkow...@netapp.com>>> >>>>> wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> For the past couple days, I have been running the KVM managed-storage >>>>> regression-test suite against RC1. >>>>> >>>>> With the exception of one issue (more on this below), all of these tests >>>>> have >>>>> passed. >>>>> >>>>> Tomorrow I plan to start in on the VMware-related managed-storage tests. >>>>> >>>>> Once I’ve completed running those, I expect to move on to the >>>>> XenServer-related >>>>> managed-storage tests. >>>>> >>>>> I ran these XenServer and VMware tests just prior to RC1 being created, >>>>> so I >>>>> suspect all of those tests will come back successful. >>>>> >>>>> Now, with regards to the one issue I found on KVM with managed storage: >>>>> >>>>> It relates to a new feature whereby you can online migrate the storage of >>>>> a VM >>>>> from NFS or Ceph to managed storage. >>>>> >>>>> During the code-review process, I made a change per a suggestion and it >>>>> introduced an issue with this feature. The solution is just a couple >>>>> lines of >>>>> code and only impacts this one use case. If you are testing this release >>>>> candidate and don’t really care about this particular feature, it should >>>>> not at >>>>> all impact your ability to test RC1. >>>>> >>>>> Thanks! >>>>> Mike >>>>> >>>>> On Jan 15, 2018, at 4:33 AM, Rohit Yadav >>>>> <ro...@apache.org<mailto:ro...@apache.org<mailto:ro...@apache.org%3cmailto:ro...@apache.org>>> >>>>> wrote: >>>>> >>>>> Hi All, >>>>> >>>>> I've created a 4.11.0.0 release, with the following artifacts up for >>>>> testing and a vote: >>>>> >>>>> Git Branch and Commit SH: >>>>> https://gitbox.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.11.0.0-RC20180115T1603 >>>>> Commit: 1b8a532ba52127f388847690df70e65c6b46f4d4 >>>>> >>>>> Source release (checksums and signatures are available at the same >>>>> location): >>>>> https://dist.apache.org/repos/dist/dev/cloudstack/4.11.0.0/ >>>>> >>>>> PGP release keys (signed using 5ED1E1122DC5E8A4A45112C2484248210EE3D884): >>>>> https://dist.apache.org/repos/dist/release/cloudstack/KEYS >>>>> >>>>> The vote will be open for 72 hours. >>>>> >>>>> For sanity in tallying the vote, can PMC members please be sure to >>>>> indicate >>>>> "(binding)" with their vote? >>>>> >>>>> [ ] +1 approve >>>>> [ ] +0 no opinion >>>>> [ ] -1 disapprove (and reason why) >>>>> >>>>> Additional information: >>>>> >>>>> For users' convenience, I've built packages from >>>>> 1b8a532ba52127f388847690df70e65c6b46f4d4 and published RC1 repository >>>>> here: >>>>> http://cloudstack.apt-get.eu/testing/4.11-rc1 >>>>> >>>>> The release notes are still work-in-progress, but the systemvmtemplate >>>>> upgrade section has been updated. You may refer the following for >>>>> systemvmtemplate upgrade testing: >>>>> http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/latest/index.html >>>>> >>>>> 4.11 systemvmtemplates are available from here: >>>>> https://download.cloudstack.org/systemvm/4.11/ >>>>> >>>>> Regards, > > >>> Rohit Yadav
