date:20140902

I've resolved my ticket as a duplicate of 
https://issues.apache.org/jira/browse/CLOUDSTACK-7464

Please note that this has nothing to do with RHEL7 - I saw it on RHEL6.3

John

From: Koushik Das [koushik@citrix.com]
Sent: 02 September 2014 08:02
To: dev@cloudstack.apache.org
Subject: RE: Unable to connect to management server on recent builds

This is the erring commit

commit c730bc3491f8b684c5ae51e0bff54cf7577cc246
Author: Rohit Yadav 
Date:   Mon Sep 1 21:14:13 2014 +0200

server: Add event bus bean as commented xml in META-INF core

This adds a spring bean xml to have EventBus for ACS, but the bean is 
commented
so the event bus service won't start by default. I'm adding this for any 
developer
who may want to hack on events and may use it just by uncommenting it and 
fixing
options.

Signed-off-by: Rohit Yadav 

-Original Message-
From: John Dilley [mailto:john.dil...@citrix.com]
Sent: Tuesday, 2 September 2014 12:23 PM
To: dev@cloudstack.apache.org
Subject: Unable to connect to management server on recent builds

Hi all,

The latest build seems to have broken starting the management server - I've 
raised https://issues.apache.org/jira/browse/CLOUDSTACK-7465

http://jenkins.buildacloud.org/job/simulator-singlerun/259/ has failed, and I'm 
assuming it's the same cause, but I can't see a lot of information on that run 
as to why the tests haven't run. If someone familiar with the code would mind 
looking at this and seeing if they could reproduce/fix, that would be good!

Thanks,

John

Review Request 25240: Management Server setup doesn't proceed to completion

2014-09-02 Thread Damodar Reddy Talakanti


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25240/
---

Review request for cloudstack and Kishan Kavala.


Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7464

https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7464


Repository: cloudstack-git


Description
---

The commit "c730bc3491f8b684c5ae51e0bff54cf7577cc246" caused the issue to not 
start the management server as it is having an empty context file to load.


Diffs
-

  server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml 
3e1a8ed 

Diff: https://reviews.apache.org/r/25240/diff/


Testing
---

Tested by starting management server after this fix whether it is starting up 
or not.


Thanks,

Damodar Reddy Talakanti

Re: [2/2] git commit: updated refs/heads/master to 23c7c73

Hi Hugo,

On Tue, Sep 2, 2014 at 10:09 AM,  wrote:

> Revert "server: Add event bus bean as commented xml in META-INF core"
>
> Breaks CloudStack startup. You're better off putting this on the wiki
>

Alright will do, but this is just a file in which the bean is commented so
it does not break CloudStack startup.

Regards.


>
> This reverts commit c730bc3491f8b684c5ae51e0bff54cf7577cc246.
>
>
> Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
> Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/f636611c
> Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/f636611c
> Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/f636611c
>
> Branch: refs/heads/master
> Commit: f636611caca7dfe1f6b7a2da6d0da530ffb6e502
> Parents: 3d04b52
> Author: Hugo Trippaers 
> Authored: Tue Sep 2 09:58:25 2014 +0200
> Committer: Hugo Trippaers 
> Committed: Tue Sep 2 10:09:04 2014 +0200
>
> --
>  .../core/spring-event-bus-context.xml   | 38 
>  1 file changed, 38 deletions(-)
> --
>
>
>
> http://git-wip-us.apache.org/repos/asf/cloudstack/blob/f636611c/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> --
> diff --git
> a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> b/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> deleted file mode 100644
> index 3e1a8ed..000
> ---
> a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> +++ /dev/null
> @@ -1,38 +0,0 @@
> -
> -
>
>

Re: [2/2] git commit: updated refs/heads/master to 23c7c73

2014-09-02 Thread Damoder Reddy

Right it is a commented out file but I think spring does not like Empty Context 
Files so it is failing to start.

On 02-Sep-2014, at 1:49 pm, Rohit Yadav  wrote:

> Hi Hugo,
> 
> On Tue, Sep 2, 2014 at 10:09 AM,  wrote:
> 
>> Revert "server: Add event bus bean as commented xml in META-INF core"
>> 
>> Breaks CloudStack startup. You're better off putting this on the wiki
>> 
> 
> Alright will do, but this is just a file in which the bean is commented so
> it does not break CloudStack startup.
> 
> Regards.
> 
> 
>> 
>> This reverts commit c730bc3491f8b684c5ae51e0bff54cf7577cc246.
>> 
>> 
>> Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
>> Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/f636611c
>> Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/f636611c
>> Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/f636611c
>> 
>> Branch: refs/heads/master
>> Commit: f636611caca7dfe1f6b7a2da6d0da530ffb6e502
>> Parents: 3d04b52
>> Author: Hugo Trippaers 
>> Authored: Tue Sep 2 09:58:25 2014 +0200
>> Committer: Hugo Trippaers 
>> Committed: Tue Sep 2 10:09:04 2014 +0200
>> 
>> --
>> .../core/spring-event-bus-context.xml   | 38 
>> 1 file changed, 38 deletions(-)
>> --
>> 
>> 
>> 
>> http://git-wip-us.apache.org/repos/asf/cloudstack/blob/f636611c/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
>> --
>> diff --git
>> a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
>> b/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
>> deleted file mode 100644
>> index 3e1a8ed..000
>> ---
>> a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
>> +++ /dev/null
>> @@ -1,38 +0,0 @@
>> -
>> -
>> 
>>

Re: [2/2] git commit: updated refs/heads/master to 23c7c73

2014-09-02 Thread Hugo Trippaers

Hey Rohit,

It does break CS startup, otherwise i would not have reverted it. Spring does 
not tolerate empty xml files.

Cheers,

Hugo


On 2 sep. 2014, at 10:19, Rohit Yadav  wrote:

> Hi Hugo,
> 
> On Tue, Sep 2, 2014 at 10:09 AM,  wrote:
> Revert "server: Add event bus bean as commented xml in META-INF core"
> 
> Breaks CloudStack startup. You're better off putting this on the wiki
> 
> Alright will do, but this is just a file in which the bean is commented so it 
> does not break CloudStack startup.
> 
> Regards.
>  
> 
> This reverts commit c730bc3491f8b684c5ae51e0bff54cf7577cc246.
> 
> 
> Project: http://git-wip-us.apache.org/repos/asf/cloudstack/repo
> Commit: http://git-wip-us.apache.org/repos/asf/cloudstack/commit/f636611c
> Tree: http://git-wip-us.apache.org/repos/asf/cloudstack/tree/f636611c
> Diff: http://git-wip-us.apache.org/repos/asf/cloudstack/diff/f636611c
> 
> Branch: refs/heads/master
> Commit: f636611caca7dfe1f6b7a2da6d0da530ffb6e502
> Parents: 3d04b52
> Author: Hugo Trippaers 
> Authored: Tue Sep 2 09:58:25 2014 +0200
> Committer: Hugo Trippaers 
> Committed: Tue Sep 2 10:09:04 2014 +0200
> 
> --
>  .../core/spring-event-bus-context.xml   | 38 
>  1 file changed, 38 deletions(-)
> --
> 
> 
> http://git-wip-us.apache.org/repos/asf/cloudstack/blob/f636611c/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> --
> diff --git 
> a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml 
> b/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> deleted file mode 100644
> index 3e1a8ed..000
> --- a/server/resources/META-INF/cloudstack/core/spring-event-bus-context.xml
> +++ /dev/null
> @@ -1,38 +0,0 @@
> -
> -
> 
>

Jenkins build is unstable: simulator-singlerun #261

See

Review Request 25243: CLOUDSTACK-7463: UI: Domain Admin UI shows 'Add LDAP Users'

2014-09-02 Thread Gabor Apati-Nagy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25243/
---

Review request for cloudstack, Brian Federle and Jessica Wang.


Bugs: CLOUDSTACK-7463
https://issues.apache.org/jira/browse/CLOUDSTACK-7463


Repository: cloudstack-git


Description
---

After this patch the button will be shown only for root admins if LDAP is 
enabled.


Diffs
-

  ui/scripts/accounts.js 2ebfe82 

Diff: https://reviews.apache.org/r/25243/diff/


Testing
---


Thanks,

Gabor Apati-Nagy

[GitHub] cloudstack pull request: IGNORE: Testing that travis-ci runs on pu...

2014-09-02 Thread imduffy15

Github user imduffy15 closed the pull request at:

https://github.com/apache/cloudstack/pull/13


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: IGNORE: Testing that travis-ci runs on pu...

2014-09-02 Thread imduffy15

Github user imduffy15 commented on the pull request:

https://github.com/apache/cloudstack/pull/13#issuecomment-54129007
  
Status successfully shown. Test completed. Closing.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 25228: hitting java.lang.reflect.InvocationTargetException while starting usage server

2014-09-02 Thread Kishan Kavala


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25228/#review52018
---

Ship it!


commit 51e0488e5c07595219142fb2f954726f72e0adf2

- Kishan Kavala


On Sept. 2, 2014, 9:38 a.m., Damodar Reddy Talakanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25228/
> ---
> 
> (Updated Sept. 2, 2014, 9:38 a.m.)
> 
> 
> Review request for cloudstack, Kishan Kavala and Rayees Namathponnan.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7316
> 
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7316
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Usage server is failing to start when we install Usage server on the 
> management server with DB encryption enabled. Usage is server is unable to 
> get "key" file from the classpath. 
> 
> 
> Diffs
> -
> 
>   packaging/centos63/cloud.spec 790f57b 
> 
> Diff: https://reviews.apache.org/r/25228/diff/
> 
> 
> Testing
> ---
> 
> Tested on centos 6.3 and rhel 7
> 
> 
> Thanks,
> 
> Damodar Reddy Talakanti
> 
>

Jenkins build is still unstable: simulator-singlerun #262

See

Build failed in Jenkins: build-master-noredist #3438

See 

Changes:

[kishan] CLOUDSTACK-7316: Usage Server is not getting started when we install 
it on management server. This is happening when encryption is enabled. For 
usage server it is not able to get key file in the classpath.

--
[...truncated 10619 lines...]
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-md5/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-md5-4.5.0-SNAPSHOT.jar
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-md5/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-md5-4.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Plugin - User Authenticator Plain Text 
4.5.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Deleting 

 (includes = [target, dist], excludes = [])
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Surefire report directory: 


---
 T E S T S
---

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-plaintext/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-plaintext-4.5.0-SNAPSHOT.jar
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-plaintext/4.5.0-SNAPSHOT

Re: Review Request 25243: CLOUDSTACK-7463: UI: Domain Admin UI shows 'Add LDAP Users'


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25243/#review52020
---

Ship it!


Ship It!

- Rajani Karuturi


On Sept. 2, 2014, 9:37 a.m., Gabor Apati-Nagy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25243/
> ---
> 
> (Updated Sept. 2, 2014, 9:37 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Jessica Wang.
> 
> 
> Bugs: CLOUDSTACK-7463
> https://issues.apache.org/jira/browse/CLOUDSTACK-7463
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After this patch the button will be shown only for root admins if LDAP is 
> enabled.
> 
> 
> Diffs
> -
> 
>   ui/scripts/accounts.js 2ebfe82 
> 
> Diff: https://reviews.apache.org/r/25243/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gabor Apati-Nagy
> 
>

Re: Review Request 25243: CLOUDSTACK-7463: UI: Domain Admin UI shows 'Add LDAP Users'



> On Sept. 2, 2014, 10:37 a.m., Rajani Karuturi wrote:
> > Ship It!

commit c200ada863dd4a7d5659538e4378c77a3e89d597


- Rajani


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25243/#review52020
---


On Sept. 2, 2014, 9:37 a.m., Gabor Apati-Nagy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25243/
> ---
> 
> (Updated Sept. 2, 2014, 9:37 a.m.)
> 
> 
> Review request for cloudstack, Brian Federle and Jessica Wang.
> 
> 
> Bugs: CLOUDSTACK-7463
> https://issues.apache.org/jira/browse/CLOUDSTACK-7463
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> After this patch the button will be shown only for root admins if LDAP is 
> enabled.
> 
> 
> Diffs
> -
> 
>   ui/scripts/accounts.js 2ebfe82 
> 
> Diff: https://reviews.apache.org/r/25243/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gabor Apati-Nagy
> 
>

Issue with accessing DNS from VPN IP

Hi,

When I connect to CloudStack's VPN on a network (L2TP over IPSEC), I’m assigned 
an IP like 10.1.2.2 and the DNS assigned is 10.1.2.1, but the virtual router is 
not listening on this IP (VPN) for DNS queries but on guest network so I cannot 
access resources by using internal dns domain names.

Is this normal behaviour? If it’s not a bug should we fix it? A workaround was 
to set the necessary DNS IP in the client before connecting (in my case the 
router’s IP, 10.1.1.1).

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Build failed in Jenkins: build-master #1521

See 

Changes:

[rajanikaruturi] CLOUDSTACK-7463: UI: Domain Admin UI shows 'Add LDAP Users' 
button (should not be shown)

--
[...truncated 3885 lines...]
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] Surefire report directory: 


---
 T E S T S
---
Running com.cloud.server.auth.MD5UserAuthenticatorTest
log4j:WARN No appenders could be found for logger 
(com.cloud.server.auth.MD5UserAuthenticator).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
info.
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.578 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Plugin - User Authenticator Plain Text 
4.5.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Deleting 

 (includes = [**/*], excludes = [])
[INFO] Deleting 

 (includes = [target, dist], excludes = [])
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Surefire report directory: 


---
 T E S T S
---

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Plugin - User Authenticator SAML2 
4.5.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
cloud-plugin-user-authenticator-saml2 ---
[INFO] Deleting 

 (includes = [**/*], excludes = [])
[INFO] Deleting 

 (includes = [target, dist], excludes = [])
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-plugin-user-authenticator-saml2 ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- maven-remote-resour

Jenkins build is still unstable: simulator-singlerun #263

See

Re: Review Request 25235: vGPU service offering upgrade test automation

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25235/#review52023
---

1. It should be possible to refactor these tests to put code into functions - 
at present there is a lot of duplicated code between each testcase
2. Additional validation should be performed either in the guest or on the 
XenServer host to validate that the correct vGPU has been attached
3. Validation should be performed after the first VM has been deployed, mostly 
so that if the testcase fails, it's clear whether it also failed on the first 
service offering.

- John Dilley

On Sept. 1, 2014, 5:34 p.m., sailaja mada wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25235/
> ---
> 
> (Updated Sept. 1, 2014, 5:34 p.m.)
> 
> 
> Review request for cloudstack, Doug Clark, John Dilley, and Sanjay Tripathi.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> vGPU service offering upgrade test automation
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_deploy_vgpu_vm.py 534a7e3 
> 
> Diff: https://reviews.apache.org/r/25235/diff/
> 
> 
> Testing
> ---
> 
> Yes. All the testcases are passed with XenServer.
> 
> 
> Thanks,
> 
> sailaja mada
> 
>

Re: [ACS44][DISCUSS]release 4.4.1

On Mon, Sep 1, 2014 at 10:20 AM, Daan Hoogland  wrote:
> SO May 1st and 1st manday in september are both due to may 4th...
>
> Nice read Stephen.
>
> Now back to 4.4.x
>
> I would like to create a new release if people feel we have something to
> release. How is the general population of this list thinking about that? Is
> there still focus on the 4.4 branch? Are all major issues from the 4.4.0
> release solved?
>
> thanks for any (encouraging) responds,
> Daan
>

I've explicitly held off on branching 4.5 so you can maintain focus on
4.4.1 as I'd like to see it, and see it promoted before we get
distracted with another release.

--David

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

GitHub user wilderrodrigues opened a pull request:

https://github.com/apache/cloudstack/pull/14

Pull request of changes in the "cloud-server" module

In the last 10 weeks we have been working in the cloud-server, focusing our 
time in the refactor of the [Vpc]VirtualNetworkApplianceManagerImpl. We had a 
mains goals increase of Maintainability, Extensibility, Readability and test 
coverage. That was just a first step towards the development, still in 
progress, of the Redundant Virtual Routers for VPC.

== What has been done so far:

â¢ The VirtualNetworkApplianceManagerImpl class line numbers dropped from  
4440 to 2558
â¢ The VpcVirtualNetworkApplianceImpl class line numbers dropped from 1484 
to 749
â¢ We created 35 new classes in order to split the code/responsibility
â¢ We added 97.8% unit test coverage for com.cloud.network.element/router 
and org.cloud.network.router.deployment packages
o   The most complex classes we changed are in those packages
o   About 1700 lines of unit tests
â¢ We started doing some integration tests
o   Deployment of Basic and Advanced Zones
o   Test Create Account and user for that account
o   Test Sub domain allowed to launch VM  when a Domain level zone 
is created
o   Test delete domain without force option
o   Test delete domain with force option
o   Test update admin details
o   Test update domain admin details
o   Test user update API
o   Test login API with domain
o   Test if Login API does not return UUID's
â¢ VPC related tests were still done manually.
o   Will have it automated as well.

We started the changes in the network area, trying to identify the 
differences in the 2 types of network we have. For that we created Basic and 
Advanced Network Topology classes. The network topology classes are responsible 
by invoking the Apply/Setup/Create/Save rules that were previously done by the 
[Vpc]VirtualNetworkAppliance. A topology instance is retrieved via a context 
object that is injected in the [Vpc]VirtualElement. The context object will 
return the most appropriate topology instance based on the Network Type, which 
is defined in the Data Centre. That was the first step towards the refactor.

From the topology class we reach the Rule Applier implementation that will 
be used to do all the rule setup preparation (i.e. invoke DAOs and prepare the 
command object). The RuleApplier interface was extracted from the 
VirtualNetworkApplianceManagerImpl, where it use to be an inner interface. For 
each anonymous implementation of the RuleApplier we created a concrete class. 
The rules are used as elements of a Visitor class, which will perform some 
extra logic, depending on the rule it's visiting, and call the send commands to 
router method. The latter has also been extracted from the 
VirtualNetworkApplianceManagerImpl and is now in a new helper class: 
NetworkHelperImpl.

The visitor has been used because we were aiming to split the 
responsibility and also because the way the RuleApplier was implemented before, 
it was clear that every command sent to the router was following a 2-steps 
approach: gather information to create the commands, apply some logic to send 
to the router. For those reason we implemented the visitor pattern. Since we 
already had the Basic/Advanced Network Topology classes, we created 2 concrete 
classes to visit the rules: Basic/Advanced Network Visitors. Both classes 
extend the abstract class NetworkTopologyVisitor, which defines all the visit 
methods per type of rule. By doing so, we can use the same rule and separate 
the logic based on the type of visitor that we have - Basic or Advanced.

Continuing on the refactor, we also added some helper classes for the 
"getSomething" related methods. Following this approach we ended up having the 
following classes:

â¢ NetworkHelper (interface)
â¢ NetworkHelperImpl
â¢ VpcNetworkHelperImpl
â¢ CommandSetupHelper
â¢ NicProfileHelper
â¢ RouterControlHelper

Last, but not least - and actually the most crucial part of the code - 
there was also a huge refactor in terms of how the routers are deployed. The 
previous deployeRouter and deployVpcInrouter methods do not exist any more. 
Instead of having the logics spread, or sometime tangled, in the 
[Vpc]VirtualNetworkApplianceManagerImpl, we have created a Router Deployment 
Definition mechanism, with classes that follow the same naming convention. The 
deployment definition has 2 implementations, Router and Vpc router, which are 
created with the aid of a Builder class. Most of the work, which is common to 
both implementation, is being done by the RouterDeploymentDefinition class. The 
specific bits are done by their implementation. for example, when 
findOrDeployVirtu

Jenkins build is still unstable: simulator-singlerun #264

See

Re: [ACS44][DISCUSS]release 4.4.1




On 09/01/2014 04:30 PM, Wido den Hollander wrote:



On 09/01/2014 04:20 PM, Daan Hoogland wrote:

SO May 1st and 1st manday in september are both due to may 4th...

Nice read Stephen.

Now back to 4.4.x

I would like to create a new release if people feel we have something to
release. How is the general population of this list thinking about
that? Is
there still focus on the 4.4 branch? Are all major issues from the 4.4.0
release solved?



Would I say all issues? I don't know. I do know however that all issues
that I encountered have been fixed in 4.4.1

Wido



Let me correct myself there.

I had a fresh 4.4.0 system running and just installed 4.4.1 packages 
(deb) from the 4.4 branch.


After restarting the management server I got:

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: 
Unknown column 'snapshot_policy.display' in 'field list'


In the 430 to 440 upgrade SQL file there is this statement however:

ALTER TABLE `cloud`.`snapshot_policy` ADD COLUMN `display` tinyint(1) 
NOT NULL DEFAULT '1' COMMENT 'True if the policy can be displayed to the 
end user';


After running it manually on my database it was resolved. Really weird 
why that didn't get executed during the initial installation.


Any ideas before I create a issue?

Wido


thanks for any (encouraging) responds,
Daan


On Mon, Sep 1, 2014 at 10:33 AM, Stephen Turner

wrote:


Interestingly, the reason Labour Day is on May 1 in the rest of the
world
is the same reason it's not in May in the US and Canada, to do with
communists and anarchists commemorating a riot in Chicago. See
http://en.wikipedia.org/wiki/Labor_Day
http://en.wikipedia.org/wiki/Haymarket_affair

--
Stephen Turner


-Original Message-
From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
Sent: 01 September 2014 09:05
To: dev; Mike Tutkowski
Cc: Francois Gaudreault
Subject: Re: [ACS44][DISCUSS]release 4.4.1

Hey Mike, You guys are not supposed to add random red holidays to your
calendars. You're the commy bashers. May 1st not enough for ya'll?


On Mon, Sep 1, 2014 at 12:00 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:


Just an FYI that it's Labor Day Weekend in the U.S., so you might not
get many, if any, replies until Tuesday.


On Sat, Aug 30, 2014 at 2:06 PM, Francois Gaudreault <
fgaudrea...@cloudops.com> wrote:


On 2014-08-30, 4:01 PM, Daan Hoogland wrote:


H,

How is the felling on list about 4.4.1? Do we have a stable enough

branch

4.4 at this moment? Especially the answer on this of KVM saffy
people is appreciated.

  I am running on 4.4.1 since a week now. I did hit couple small
bugs,

but

no real blocker. We also try to fix the SSL offload code to work
with projects. We would really really like to get this feature in

4.4.1.


FG

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform &
Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_





--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*





--
Daan

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

2014-09-02 Thread bhaisaab

Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54144949
  
Thanks for your work Wilder, can you please rebase this branch against 
master as I see there are a lot of commits it is showing on the PR which would 
result in fast forward or merge/conflicts.

I would then like to review this branch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54145868
  
Hi Rohit,

I did first a rebase from master onto the visitor-rebase branch, in order 
to make sure everything was fine. I also built, executed and tested just to be 
100% sure.

After that, I rebased from visitor-rebase onto master (switched to master 
branch and did a git rebase visitor-rebase). All changed were applied without 
conflicts.

Could you just me to the problems you saw?

Thanks for your willingness to review our changes.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

2014-09-02 Thread bhaisaab

Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54146863
  
Hey, the issue is the PR includes a lot of other changes which already 
exists on master branch. So, this PR includes these extra commits and if I 
merge your PR on master it may end up in a lot of conflicts.

The idea is to have clean PRs or patches; the feature branch rebases from 
master; i.e. you checkout visitor-rebase and do a git pull --rebase origin 
master and push this branch to us for review after fixing any conflicts. If you 
send us a rebased branch against master, it's much easier for us to review and 
merge since you've handled conflicts at your end.

More; 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git#Git-Createcleancommits


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54147257
  
Okay, I'm already on that.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Cherry-picking fix that may change 4.3.1 schema

Hi,

I’m going through list of issues that affect 4.3.0 and 4.3.1 and cherry-picking 
those which are already fixed elsewhere. This is the issue which would involve 
some modifications to the schema: 
https://issues.apache.org/jira/browse/CLOUDSTACK-6756

@Sebatien: can you please take a look at this bug and cherry-pick if we want it 
in 4.3.1?

It is doable either in the schema upgrade path 4.3.0 to 4.3.1 (sql file) or in 
the upgrade class.

Since, we’re going to have 4.4.1 soon as well, it makes it tricky how to to 
deal with upgrade paths. So, at present if 4.4.1 is out already, people on 
4.3.1 (in near future) won’t be able to upgrade to 4.4.1 etc. Should in such a 
case try to release 4.3.1 out first and then 4.4.1?

What about if in future we have more such situations, we perhaps need to fix 
release management process and policies, one way is to refactor out the db 
upgrade logic as a separate tool that has a more quick release cycle like 
cloudmonkey?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54150383
  
Hi Rohit,

Did as you suggested and got no conflicts detected.

[INFO] 

[INFO] BUILD SUCCESS
[INFO] 

[INFO] Total time: 13:13.154s
[INFO] Finished at: Tue Sep 02 15:13:50 CEST 2014
[INFO] Final Memory: 83M/211M
[INFO] 


I will now push the branch to you guys.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cherry-picking fix that may change 4.3.1 schema

2014-09-02 Thread sebgoa


On Sep 2, 2014, at 3:20 PM, Rohit Yadav  wrote:

> Hi,
> 
> I’m going through list of issues that affect 4.3.0 and 4.3.1 and 
> cherry-picking those which are already fixed elsewhere. This is the issue 
> which would involve some modifications to the schema: 
> https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> 
> @Sebatien: can you please take a look at this bug and cherry-pick if we want 
> it in 4.3.1?
> 

I have always been of the impression that we should not have any schema changes 
in minor releases even if it fixes bugs ? 

But I am happy to commit this, if folks agree that we should do it.

> It is doable either in the schema upgrade path 4.3.0 to 4.3.1 (sql file) or 
> in the upgrade class.
> 
> Since, we’re going to have 4.4.1 soon as well, it makes it tricky how to to 
> deal with upgrade paths. So, at present if 4.4.1 is out already, people on 
> 4.3.1 (in near future) won’t be able to upgrade to 4.4.1 etc. Should in such 
> a case try to release 4.3.1 out first and then 4.4.1?
> 

I soon as I see the smoke tests green on Travis , I will launch a vote for 
4.3.1 

> What about if in future we have more such situations, we perhaps need to fix 
> release management process and policies, one way is to refactor out the db 
> upgrade logic as a separate tool that has a more quick release cycle like 
> cloudmonkey?
> 
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
> 
> 
> 
> Find out more about ShapeBlue and our range of CloudStack related services
> 
> IaaS Cloud Design & Build
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure 
> Support
> CloudStack Bootcamp Training 
> Courses
> 
> This email and any attachments to it may be confidential and are intended 
> solely for the use of the individual to whom it is addressed. Any views or 
> opinions expressed are solely those of the author and do not necessarily 
> represent those of Shape Blue Ltd or related companies. If you are not the 
> intended recipient of this email, you must neither take any action based upon 
> its contents, nor copy or show it to anyone. Please contact the sender if you 
> believe you have received this email in error. Shape Blue Ltd is a company 
> incorporated in England & Wales. ShapeBlue Services India LLP is a company 
> incorporated in India and is operated under license from Shape Blue Ltd. 
> Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
> operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
> registered by The Republic of South Africa and is traded under license from 
> Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Cherry-picking fix that may change 4.3.1 schema




On 09/02/2014 03:30 PM, sebgoa wrote:


On Sep 2, 2014, at 3:20 PM, Rohit Yadav  wrote:


Hi,

I’m going through list of issues that affect 4.3.0 and 4.3.1 and cherry-picking 
those which are already fixed elsewhere. This is the issue which would involve 
some modifications to the schema: 
https://issues.apache.org/jira/browse/CLOUDSTACK-6756

@Sebatien: can you please take a look at this bug and cherry-pick if we want it 
in 4.3.1?



I have always been of the impression that we should not have any schema changes 
in minor releases even if it fixes bugs ?



I thought so as well. Since how are people going to upgrade to 4.4.1 
from 4.3.1?


Wido


But I am happy to commit this, if folks agree that we should do it.


It is doable either in the schema upgrade path 4.3.0 to 4.3.1 (sql file) or in 
the upgrade class.

Since, we’re going to have 4.4.1 soon as well, it makes it tricky how to to 
deal with upgrade paths. So, at present if 4.4.1 is out already, people on 
4.3.1 (in near future) won’t be able to upgrade to 4.4.1 etc. Should in such a 
case try to release 4.3.1 out first and then 4.4.1?



I soon as I see the smoke tests green on Travis , I will launch a vote for 4.3.1


What about if in future we have more such situations, we perhaps need to fix 
release management process and policies, one way is to refactor out the db 
upgrade logic as a separate tool that has a more quick release cycle like 
cloudmonkey?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England & Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Review Request 24740: CLOUDSTACK-7354: Set isdynamicallyscalable before attempting to scale the VM


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24740/
---

(Updated Sept. 2, 2014, 1:38 p.m.)


Review request for cloudstack, Santhosh Edukulla and SrikanteswaraRao Talluri.


Bugs: CLOUDSTACK-7354
https://issues.apache.org/jira/browse/CLOUDSTACK-7354


Repository: cloudstack-git


Description
---

The test looks like it was supposed to set the isdynamicallyscalable property 
on the VM. Unfortunately it did it after attempting to scale the VM, so it had 
no effect.


Diffs
-

  test/integration/smoke/test_scale_vm.py b3804dc 

Diff: https://reviews.apache.org/r/24740/diff/


Testing
---

Tested on VMWare


Thanks,

John Dilley

Re: Review Request 24696: Only attempt to shrink volume in test if CLVM is in use


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24696/
---

(Updated Sept. 2, 2014, 1:38 p.m.)


Review request for cloudstack, Santhosh Edukulla and SrikanteswaraRao Talluri.


Bugs: CLOUDSTACK-7228
https://issues.apache.org/jira/browse/CLOUDSTACK-7228


Repository: cloudstack-git


Description
---

Currently only CLVM supports shrinking volumes. This patch will only attempt to 
shrink a volume if CLVM is in use.


Diffs
-

  test/integration/smoke/test_volumes.py e2e0d76 

Diff: https://reviews.apache.org/r/24696/diff/


Testing
---

Tested on XenServer to check that the test now passes.


Thanks,

John Dilley

Review Request 25248: Fix NPE in case VM is started and its template does not exist anymore


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25248/
---

Review request for cloudstack, Alena Prokharchyk, edison su, Darren Shepherd, 
Sebastien Goasguen, and Hugo Trippaers.


Bugs: CLOUDSTACK-6945
https://issues.apache.org/jira/browse/CLOUDSTACK-6945


Repository: cloudstack-git


Description
---

Fixes https://issues.apache.org/jira/browse/CLOUDSTACK-6945


Diffs
-

  engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 2fd7a52 

Diff: https://reviews.apache.org/r/25248/diff/


Testing
---

Builds cleanly, will throw resource not available exception when template does 
not exist.


Thanks,

Rohit Yadav

Re: Cherry-picking fix that may change 4.3.1 schema

On Tue, Sep 2, 2014 at 9:34 AM, Wido den Hollander  wrote:
>
>
> On 09/02/2014 03:30 PM, sebgoa wrote:
>>
>>
>> On Sep 2, 2014, at 3:20 PM, Rohit Yadav  wrote:
>>
>>> Hi,
>>>
>>> I’m going through list of issues that affect 4.3.0 and 4.3.1 and
>>> cherry-picking those which are already fixed elsewhere. This is the issue
>>> which would involve some modifications to the schema:
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-6756
>>>
>>> @Sebatien: can you please take a look at this bug and cherry-pick if we
>>> want it in 4.3.1?
>>>
>>
>> I have always been of the impression that we should not have any schema
>> changes in minor releases even if it fixes bugs ?
>>
>
> I thought so as well. Since how are people going to upgrade to 4.4.1 from
> 4.3.1?
>
> Wido
>

I agree - how will users upgrade from 4.3.1 to 4.4.x or later - schema
changes aren't necessarily going to screw up the upgrade process, but
certainly have a high likelihood of doing so. What happens when
4.3.0-to-4.4.0.sql runs? The table already exists - and upgrade will
puke. 4.4.0 is already released; so not like you can roll that change
back.

--David

Re: Cherry-picking fix that may change 4.3.1 schema

On 02-Sep-2014, at 4:38 pm, David Nalley  wrote:
> I agree - how will users upgrade from 4.3.1 to 4.4.x or later - schema
> changes aren't necessarily going to screw up the upgrade process, but
> certainly have a high likelihood of doing so. What happens when
> 4.3.0-to-4.4.0.sql runs? The table already exists - and upgrade will
> puke. 4.4.0 is already released; so not like you can roll that change
> back.

Since 4.4.0 is already released, people who will install 4.3.1 won’t be able to 
upgrade to 4.4.0 as the 4.4.0 artifacts won’t have an upgrade path from 4.3.1 
to 4.4.0; but this can be only fixed (i.e. have an upgrade path) in 4.4.1.

For 4.3.1, we can still have the changes but 4.3.1 users would only be able to 
upgrade to 4.4.1 or later releases and not 4.4.0 because there is no path from 
4.3.1 to 4.4.0 in 4.4.0 release.

This is true for all ACS releases, only the latest releases know and have 
upgrade paths. So, what I also proposed here was to refactor the whole upgrade 
logic as a tool and maintain it separately and this tool can perhaps has its 
own release process that is faster than the ACS releases and this tool knows 
about all the releases and has the necessary upgrade paths.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Cherry-picking fix that may change 4.3.1 schema

2014-09-02 Thread Francois Gaudreault


Do I read the ticket wrong or that fix is already in 4.4?

FG

On 2014-09-02, 10:48 AM, Rohit Yadav wrote:

On 02-Sep-2014, at 4:38 pm, David Nalley  wrote:

I agree - how will users upgrade from 4.3.1 to 4.4.x or later - schema
changes aren't necessarily going to screw up the upgrade process, but
certainly have a high likelihood of doing so. What happens when
4.3.0-to-4.4.0.sql runs? The table already exists - and upgrade will
puke. 4.4.0 is already released; so not like you can roll that change
back.

Since 4.4.0 is already released, people who will install 4.3.1 won’t be able to 
upgrade to 4.4.0 as the 4.4.0 artifacts won’t have an upgrade path from 4.3.1 
to 4.4.0; but this can be only fixed (i.e. have an upgrade path) in 4.4.1.

For 4.3.1, we can still have the changes but 4.3.1 users would only be able to 
upgrade to 4.4.1 or later releases and not 4.4.0 because there is no path from 
4.3.1 to 4.4.0 in 4.4.0 release.

This is true for all ACS releases, only the latest releases know and have 
upgrade paths. So, what I also proposed here was to refactor the whole upgrade 
logic as a tool and maintain it separately and this tool can perhaps has its 
own release process that is faster than the ACS releases and this tool knows 
about all the releases and has the necessary upgrade paths.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England & Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.





--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: Cherry-picking fix that may change 4.3.1 schema

On 02-Sep-2014, at 4:54 pm, Francois Gaudreault  
wrote:
> Do I read the ticket wrong or that fix is already in 4.4?
>
> FG

Yes, this already is in 4.4 but the issue is — do we want it for 4.3 branch as 
it affects 4.3.0 release as well and it’s a major issue for people using ACS 
"usage".

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Cherry-picking fix that may change 4.3.1 schema

2014-09-02 Thread Francois Gaudreault

On 2014-09-02, 11:00 AM, Rohit Yadav wrote:

On 02-Sep-2014, at 4:54 pm, Francois Gaudreault
wrote:

Do I read the ticket wrong or that fix is already in 4.4?

Yes, this already is in 4.4 but the issue is — do we want it for 4.3 branch as it affects
4.3.0 release as well and it’s a major issue for people using ACS "usage".

I see. Well, I think we were impacted by that too, and we made the decision to
move on 4.4.1-snap (even if it's technically less stable?) and then upgrade to
4.4.1 GA (next week?)

I personally don't think pulling back DB changes in lower releases is a good
idea :S

But that's only my opinion :)

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions
expressed are solely those of the author and do not necessarily represent those of
Shape Blue Ltd or related companies. If you are not the intended recipient of this
email, you must neither take any action based upon its contents, nor copy or show
it to anyone. Please contact the sender if you believe you have received this email
in error. Shape Blue Ltd is a company incorporated in England & Wales.
ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue
SA Pty Ltd is a company registered by The Republic of South Africa and is traded
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

2014-09-02 Thread lsimons

Github user lsimons commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54167891
  
Thanks Travis! So cool that this is working...the failure is due to the 
change here:


https://github.com/schubergphilis/cloudstack/commit/680dd7ece887829645afee99c45d32af6fe57134#diff-11

Wilder will update the pull request to filter it out.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cherry-picking fix that may change 4.3.1 schema

On 02-Sep-2014, at 5:07 pm, Francois Gaudreault  
wrote:
> I see. Well, I think we were impacted by that too, and we made the decision 
> to move on 4.4.1-snap (even if it's technically less stable?) and then 
> upgrade to 4.4.1 GA (next week?)
>
> I personally don't think pulling back DB changes in lower releases is a good 
> idea :S
>
> But that's only my opinion :)

I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like to 
stick to that only. I understand there are too many numbers, versions and 
branches to follow, so please if you can try to understand the issue;

This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756 requires 
that there are some extra columns in the database to do book keeping when 
delete ips so you don’t actually delete db/table rows.

The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750

But, we already have 4.4.0 release and db upgrade paths are always in the next 
release versions.

So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does not 
care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0 to 
4.3.1. So, the limitation is that people won’t be able to upgrade from 4.3.1 to 
4.4.0, because 4.4.0 is already released.

If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can put the 
fix from the JIRA issue on 4.3 branch so the issue is fixed for 4.3.1. The fix 
will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release 4.4.1 after 
4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4 branch such that

The abstract issue is — we’ll have such issue in future, how do we fix it. I 
suggested — we make a separate tool that does (rolling) upgrades.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

2014-09-02 Thread bhaisaab

Github user bhaisaab commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54169234
  
Thanks to @imduffy15 we have TravisCI fire up on every commit.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Cherry-picking fix that may change 4.3.1 schema

The right way to do this is to do commit to commit migrations and use some
db versioning tools like liquibase or flywaydb which will execute the new
change sets. We discussed this before and IIRC, someone started working on
it.

On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav
wrote:

On 02-Sep-2014, at 5:07 pm, Francois Gaudreault > wrote:
> I see. Well, I think we were impacted by that too, and we made the
decision to move on 4.4.1-snap (even if it's technically less stable?) and
then upgrade to 4.4.1 GA (next week?)
>
> I personally don't think pulling back DB changes in lower releases is a
good idea :S
>
> But that's only my opinion :)

I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like to
stick to that only. I understand there are too many numbers, versions and
branches to follow, so please if you can try to understand the issue;

This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756 requires
that there are some extra columns in the database to do book keeping when
delete ips so you don’t actually delete db/table rows.

The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750

But, we already have 4.4.0 release and db upgrade paths are always in the
next release versions.

So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does not
care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
to 4.3.1. So, the limitation is that people won’t be able to upgrade from
4.3.1 to 4.4.0, because 4.4.0 is already released.

If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can put
the fix from the JIRA issue on 4.3 branch so the issue is fixed for 4.3.1.
The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
branch such that

The abstract issue is — we’ll have such issue in future, how do we fix it.
I suggested — we make a separate tool that does (rolling) upgrades.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure Support<
http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<
http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
a company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.

--
~Rajani
Sent from Windows Phone

Re: Cherry-picking fix that may change 4.3.1 schema

@Rohit
I think we are playing with fire here a bit.  DB changes should be made on
major releases and we should try to avoid DB changes on minor releases
because of this type of problem.  We have to keep the upgrade paths as
clean as possible because we don't want people running into issues when
they upgrade their prod boxes.

I think we are going to have a problem with this one because I think the
4.4.1 branch is going to be released before the 4.3.1 branch, so the
upgrade path will be broken.  My understanding is that the 4.4.1 branch is
planned to be launched on Monday (Sept 8th), so I expect there will be a
problem with the upgrade path if we implement this change in 4.3.1.  Thats
my two cents anyway...

@Rajani
I am not entirely sure which technology is being used for the db upgrade
paths, but I think it is probably something like liquibase already.  I
would have to check on the technology, but this is already in place.  The
issue is more about maintaining an upgradeable path for each version that
is released.  This is a bit complicated by the fact that we currently have
4.3, 4.4 and 4.5 version all having changes made to them at the same time...


*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_


On Tue, Sep 2, 2014 at 11:28 AM, Rohit Yadav 
wrote:

>
> On 02-Sep-2014, at 5:07 pm, Francois Gaudreault 
> wrote:
> > I see. Well, I think we were impacted by that too, and we made the
> decision to move on 4.4.1-snap (even if it's technically less stable?) and
> then upgrade to 4.4.1 GA (next week?)
> >
> > I personally don't think pulling back DB changes in lower releases is a
> good idea :S
> >
> > But that's only my opinion :)
>
> I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
> to stick to that only. I understand there are too many numbers, versions
> and branches to follow, so please if you can try to understand the issue;
>
> This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> requires that there are some extra columns in the database to do book
> keeping when delete ips so you don’t actually delete db/table rows.
>
> The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
>
> But, we already have 4.4.0 release and db upgrade paths are always in the
> next release versions.
>
> So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
> not care about 4.4.0 schema changes. There is only an upgrade path from
> 4.3.0 to 4.3.1. So, the limitation is that people won’t be able to upgrade
> from 4.3.1 to 4.4.0, because 4.4.0 is already released.
>
> If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
> put the fix from the JIRA issue on 4.3 branch so the issue is fixed for
> 4.3.1. The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we
> release 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1
> on 4.4 branch such that
>
> The abstract issue is — we’ll have such issue in future, how do we fix it.
> I suggested — we make a separate tool that does (rolling) upgrades.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registe

Re: Cherry-picking fix that may change 4.3.1 schema


On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
> The right way to do this is to do commit to commit migrations and use some
> db versioning tools like liquibase or flywaydb which will execute the new
> change sets. We discussed this before and IIRC, someone started working on
> it.

This is not a proposal thread to introduce new tools and I would certainly 
avoid the “right way to do things” as I’m a fan of iterative development.

I’m interested in a concrete solution to the concrete issue I raised. Here’s 
how I will do it:

- Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0 upgrade 
path
- Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
- Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for the 
above change
- On 4.4 branch, in the upgrade (db upgrader) map there won’t be an upgrade 
path from 4.3.1 for any class except for the map entry where version is 4.3.1
- Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a 
choice) and the upgrade path will exist only in 4.4.1 and later releases

I think if I do the above, no one will have any issue. I’m sharing them to run 
by you all in case you find any issue?

>
>
> On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
> wrote:
>
>
> On 02-Sep-2014, at 5:07 pm, Francois Gaudreault  > wrote:
>> I see. Well, I think we were impacted by that too, and we made the
> decision to move on 4.4.1-snap (even if it's technically less stable?) and
> then upgrade to 4.4.1 GA (next week?)
>>
>> I personally don't think pulling back DB changes in lower releases is a
> good idea :S
>>
>> But that's only my opinion :)
>
> I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like to
> stick to that only. I understand there are too many numbers, versions and
> branches to follow, so please if you can try to understand the issue;
>
> This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756 requires
> that there are some extra columns in the database to do book keeping when
> delete ips so you don’t actually delete db/table rows.
>
> The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
>
> But, we already have 4.4.0 release and db upgrade paths are always in the
> next release versions.
>
> So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does not
> care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
> to 4.3.1. So, the limitation is that people won’t be able to upgrade from
> 4.3.1 to 4.4.0, because 4.4.0 is already released.
>
> If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can put
> the fix from the JIRA issue on 4.3 branch so the issue is fixed for 4.3.1.
> The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
> 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
> branch such that
>
> The abstract issue is — we’ll have such issue in future, how do we fix it.
> I suggested — we make a separate tool that does (rolling) upgrades.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com 
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build>
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
>
>
> --
> ~Rajani
> Sent from Windows Phone

Regards,
Rohit Yadav
Software Architect, ShapeBl

Re: Cherry-picking fix that may change 4.3.1 schema

If you’re not going to read anything because the thread has gone too long 
already, read this:

4.4.0 is already out there, if we release 4.3.1 there is no way people can 
upgrade to 4.4.0 as the upgrade paths exist in the 4.4.0 release itself which 
does not know about 4.3.1

On 02-Sep-2014, at 5:48 pm, Will Stevens  wrote:
> @Rohit
> I think we are playing with fire here a bit.  DB changes should be made on
> major releases and we should try to avoid DB changes on minor releases
> because of this type of problem.  We have to keep the upgrade paths as
> clean as possible because we don't want people running into issues when
> they upgrade their prod boxes.
>
> I think we are going to have a problem with this one because I think the
> 4.4.1 branch is going to be released before the 4.3.1 branch, so the
> upgrade path will be broken.  My understanding is that the 4.4.1 branch is
> planned to be launched on Monday (Sept 8th), so I expect there will be a
> problem with the upgrade path if we implement this change in 4.3.1.  Thats
> my two cents anyway…

Sorry to break you this, the issue I’m raising here has always existed before. 
Just that it was not widely shared and was silently fixed by developers.

This is also the first time that we’re planning to release a minor version from 
previous major release. That is we already released 4.4.0 and now we’re going 
to release 4.3.1.

Database upgrade paths from previous releases are only added in next releases. 
And, you may go look at the code to see that all minor upgrades do introduce 
some db changes and not just read emails. Sometime they don’t have to put an 
explicit schema-AtoB.sql and their cleanup files.

In any case, due to the way we release ACS. People won’t be able to upgrade 
from 4.3.1 to 4.4.0 anyway.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Cherry-picking fix that may change 4.3.1 schema


On Sep 2, 2014, at 11:56 AM, Rohit Yadav  wrote:

> 
> On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
>> The right way to do this is to do commit to commit migrations and use some
>> db versioning tools like liquibase or flywaydb which will execute the new
>> change sets. We discussed this before and IIRC, someone started working on
>> it.
> 
> This is not a proposal thread to introduce new tools and I would certainly 
> avoid the “right way to do things” as I’m a fan of iterative development.
> 
> I’m interested in a concrete solution to the concrete issue I raised. Here’s 
> how I will do it:
> 
> - Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0 
> upgrade path
> - Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
> - Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for the 
> above change
> - On 4.4 branch, in the upgrade (db upgrader) map there won’t be an upgrade 
> path from 4.3.1 for any class except for the map entry where version is 4.3.1
> - Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a 
> choice) and the upgrade path will exist only in 4.4.1 and later releases
> 
> I think if I do the above, no one will have any issue. I’m sharing them to 
> run by you all in case you find any issue?
> 

Or:

#1: we don't release 4.3.1

The main reason that I wanted to do 4.3.1 is to start providing some sort of 
LTS…but maybe our LTS strategy is to move towards rolling upgrades.


>> 
>> 
>> On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
>> wrote:
>> 
>> 
>> On 02-Sep-2014, at 5:07 pm, Francois Gaudreault > > wrote:
>>> I see. Well, I think we were impacted by that too, and we made the
>> decision to move on 4.4.1-snap (even if it's technically less stable?) and
>> then upgrade to 4.4.1 GA (next week?)
>>> 
>>> I personally don't think pulling back DB changes in lower releases is a
>> good idea :S
>>> 
>>> But that's only my opinion :)
>> 
>> I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like to
>> stick to that only. I understand there are too many numbers, versions and
>> branches to follow, so please if you can try to understand the issue;
>> 
>> This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756 requires
>> that there are some extra columns in the database to do book keeping when
>> delete ips so you don’t actually delete db/table rows.
>> 
>> The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
>> 
>> But, we already have 4.4.0 release and db upgrade paths are always in the
>> next release versions.
>> 
>> So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does not
>> care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
>> to 4.3.1. So, the limitation is that people won’t be able to upgrade from
>> 4.3.1 to 4.4.0, because 4.4.0 is already released.
>> 
>> If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can put
>> the fix from the JIRA issue on 4.3 branch so the issue is fixed for 4.3.1.
>> The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
>> 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
>> branch such that
>> 
>> The abstract issue is — we’ll have such issue in future, how do we fix it.
>> I suggested — we make a separate tool that does (rolling) upgrades.
>> 
>> Regards,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +41 779015219 | rohit.ya...@shapeblue.com 
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>> 
>> 
>> 
>> Find out more about ShapeBlue and our range of CloudStack related services
>> 
>> IaaS Cloud Design & Build>> 
>> CSForge – rapid IaaS deployment framework
>> CloudStack Consulting
>> CloudStack Infrastructure Support<
>> http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<
>> http://shapeblue.com/cloudstack-training/>
>> 
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue B

Re: Cherry-picking fix that may change 4.3.1 schema

2014-09-02 Thread Marcus

I think we're discussing the matter of policy, rather than this case. Can
we make this one work, I'm confident you can, Rohit.  But as a policy, do
we want to make changes to DB schemas in minor (bugfix) releases, and I
think the answer being given is no. Perhaps that means the concrete
solution is that we don't fix a known bug. Given our track record and that
we're juggling 4.4.1 and the upcoming 4.5, I'm not entirely sure 4.3.1 will
even get the time to be tested and released.


On Tue, Sep 2, 2014 at 9:56 AM, Rohit Yadav 
wrote:

>
> On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
> > The right way to do this is to do commit to commit migrations and use
> some
> > db versioning tools like liquibase or flywaydb which will execute the new
> > change sets. We discussed this before and IIRC, someone started working
> on
> > it.
>
> This is not a proposal thread to introduce new tools and I would certainly
> avoid the “right way to do things” as I’m a fan of iterative development.
>
> I’m interested in a concrete solution to the concrete issue I raised.
> Here’s how I will do it:
>
> - Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0
> upgrade path
> - Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
> - Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for
> the above change
> - On 4.4 branch, in the upgrade (db upgrader) map there won’t be an
> upgrade path from 4.3.1 for any class except for the map entry where
> version is 4.3.1
> - Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a
> choice) and the upgrade path will exist only in 4.4.1 and later releases
>
> I think if I do the above, no one will have any issue. I’m sharing them to
> run by you all in case you find any issue?
>
> >
> >
> > On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
> > wrote:
> >
> >
> > On 02-Sep-2014, at 5:07 pm, Francois Gaudreault <
> fgaudrea...@cloudops.com
> > > wrote:
> >> I see. Well, I think we were impacted by that too, and we made the
> > decision to move on 4.4.1-snap (even if it's technically less stable?)
> and
> > then upgrade to 4.4.1 GA (next week?)
> >>
> >> I personally don't think pulling back DB changes in lower releases is a
> > good idea :S
> >>
> >> But that's only my opinion :)
> >
> > I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
> to
> > stick to that only. I understand there are too many numbers, versions and
> > branches to follow, so please if you can try to understand the issue;
> >
> > This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> requires
> > that there are some extra columns in the database to do book keeping when
> > delete ips so you don’t actually delete db/table rows.
> >
> > The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
> >
> > But, we already have 4.4.0 release and db upgrade paths are always in the
> > next release versions.
> >
> > So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
> not
> > care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
> > to 4.3.1. So, the limitation is that people won’t be able to upgrade from
> > 4.3.1 to 4.4.0, because 4.4.0 is already released.
> >
> > If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
> put
> > the fix from the JIRA issue on 4.3 branch so the issue is fixed for
> 4.3.1.
> > The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
> > 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
> > branch such that
> >
> > The abstract issue is — we’ll have such issue in future, how do we fix
> it.
> > I suggested — we make a separate tool that does (rolling) upgrades.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.ya...@shapeblue.com 
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//
> >>
> > CSForge – rapid IaaS deployment framework
> > CloudStack Consulting
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> >

test failure on 4.3 branch

Hi, with simulator now running on travis we are seeing test failures a bit more 
clearly.

test_04_extrat_Iso is failing on 4.3 branch

Could someone fix the test or investigate what's happening.

https://travis-ci.org/apache/cloudstack/jobs/34183172

thanks

-sebastien

Clarification on github or and TravisCI

As some of you may have noticed github PR has been turned on for our main repo 
(it was already the case for the docs repo).

In addition, TravisCI config files have been added to master and 4.3 (not yet 
for 4.4).

https://travis-ci.org/apache/cloudstack/builds

The Travis jobs, compile cloudstack and deploy the simulator to run the smoke 
tests.

This is an *experimentation* to see how we can change our commit mechanism (pr 
vs. RB) as well as add CI for every commit.
We moved forward with this without a proposal or former vote to get a feel for 
it and see if it could help.

Basically, every pr from a personal fork of cloudstack will trigger a Travis 
job and the PR will show the Travis job status. 
The main idea of course being that if the tests pass they we can merge.

I personally see it as an addition to Jenkins jobs not a replacement and a 
quick way to get free CI while we get our act together with a real infra.

Comments and help (with tests and travis config) welcome, of course feel free 
to start sending pr that way knowing that this is still an experiment.

ps: thanks to Ian and Rohit for getting it working.

-sebastien

Re: Cherry-picking fix that may change 4.3.1 schema

I think this covers the main use case we have to be concerned about, so I
think this is fine.  I don't think we should have to worry about upgrading
from 4.3.1 to 4.4.0.  I agree that if we maintain an upgrade path from
4.3.1 to 4.4.1, we should be fine.


*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_


On Tue, Sep 2, 2014 at 11:56 AM, Rohit Yadav 
wrote:

>
> On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
> > The right way to do this is to do commit to commit migrations and use
> some
> > db versioning tools like liquibase or flywaydb which will execute the new
> > change sets. We discussed this before and IIRC, someone started working
> on
> > it.
>
> This is not a proposal thread to introduce new tools and I would certainly
> avoid the “right way to do things” as I’m a fan of iterative development.
>
> I’m interested in a concrete solution to the concrete issue I raised.
> Here’s how I will do it:
>
> - Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0
> upgrade path
> - Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
> - Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for
> the above change
> - On 4.4 branch, in the upgrade (db upgrader) map there won’t be an
> upgrade path from 4.3.1 for any class except for the map entry where
> version is 4.3.1
> - Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a
> choice) and the upgrade path will exist only in 4.4.1 and later releases
>
> I think if I do the above, no one will have any issue. I’m sharing them to
> run by you all in case you find any issue?
>
> >
> >
> > On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
> > wrote:
> >
> >
> > On 02-Sep-2014, at 5:07 pm, Francois Gaudreault <
> fgaudrea...@cloudops.com
> > > wrote:
> >> I see. Well, I think we were impacted by that too, and we made the
> > decision to move on 4.4.1-snap (even if it's technically less stable?)
> and
> > then upgrade to 4.4.1 GA (next week?)
> >>
> >> I personally don't think pulling back DB changes in lower releases is a
> > good idea :S
> >>
> >> But that's only my opinion :)
> >
> > I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
> to
> > stick to that only. I understand there are too many numbers, versions and
> > branches to follow, so please if you can try to understand the issue;
> >
> > This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> requires
> > that there are some extra columns in the database to do book keeping when
> > delete ips so you don’t actually delete db/table rows.
> >
> > The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
> >
> > But, we already have 4.4.0 release and db upgrade paths are always in the
> > next release versions.
> >
> > So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
> not
> > care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
> > to 4.3.1. So, the limitation is that people won’t be able to upgrade from
> > 4.3.1 to 4.4.0, because 4.4.0 is already released.
> >
> > If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
> put
> > the fix from the JIRA issue on 4.3 branch so the issue is fixed for
> 4.3.1.
> > The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
> > 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
> > branch such that
> >
> > The abstract issue is — we’ll have such issue in future, how do we fix
> it.
> > I suggested — we make a separate tool that does (rolling) upgrades.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.ya...@shapeblue.com 
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//
> >>
> > CSForge – rapid IaaS deployment framework
> > CloudStack Consulting
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> > intended recipient of this email, you must neither take any action based
>

Re: Cherry-picking fix that may change 4.3.1 schema

On 02-Sep-2014, at 6:05 pm, Marcus  wrote:
> I think we're discussing the matter of policy, rather than this case. Can
> we make this one work, I'm confident you can, Rohit.  But as a policy, do
> we want to make changes to DB schemas in minor (bugfix) releases, and I
> think the answer being given is no. Perhaps that means the concrete
> solution is that we don't fix a known bug. Given our track record and that
> we're juggling 4.4.1 and the upcoming 4.5, I'm not entirely sure 4.3.1 will
> even get the time to be tested and released.

As a policy I think we should *not* be doing this, but we also need to support 
major releases with minor ones as we have to understand that we’ve users using 
ACS in production.

FYI — we’ve been modifying databases in minor releases as well, checkout 
Upgrade421to430 and Upgrade420to421, we’re certainly changing databases. Just 
that we’re not adding new tables, doing something drastic but we’re certainly 
doing db migrations.

4.3.1 will be unique as it will be released after 4.4.0, and from my $dayjob I 
know there are some users interested in 4.3.1 and I don’t want to fork ACS for 
them or create a new distribution.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

RE: Clarification on github or and TravisCI

2014-09-02 Thread Stephen Turner

So just to be clear, Sebastien, does that mean it's acceptable for 
non-committers to submit their code for review via a Github pull request, 
instead of using ReviewBoard? (I hope so: I think that will make the process 
easier and so encourage contributions).

-- 
Stephen Turner


-Original Message-
From: Sebastien Goasguen [mailto:run...@gmail.com] 
Sent: 02 September 2014 17:19
To: dev@cloudstack.apache.org
Subject: Clarification on github or and TravisCI

As some of you may have noticed github PR has been turned on for our main repo 
(it was already the case for the docs repo).

In addition, TravisCI config files have been added to master and 4.3 (not yet 
for 4.4).

https://travis-ci.org/apache/cloudstack/builds

The Travis jobs, compile cloudstack and deploy the simulator to run the smoke 
tests.

This is an *experimentation* to see how we can change our commit mechanism (pr 
vs. RB) as well as add CI for every commit.
We moved forward with this without a proposal or former vote to get a feel for 
it and see if it could help.

Basically, every pr from a personal fork of cloudstack will trigger a Travis 
job and the PR will show the Travis job status. 
The main idea of course being that if the tests pass they we can merge.

I personally see it as an addition to Jenkins jobs not a replacement and a 
quick way to get free CI while we get our act together with a real infra.

Comments and help (with tests and travis config) welcome, of course feel free 
to start sending pr that way knowing that this is still an experiment.

ps: thanks to Ian and Rohit for getting it working.

-sebastien

Shell commands vs Native Java file handling

2014-09-02 Thread David Bierce

While investigating an issue with secondary storage templates, I noticed all 
the file handling I came across shelled and execute things like mkdir, rm, etc, 
etc.  With the migration to Java 7 is there still a reason to continue that 
method of file handling or in the future could/should file operations be 
handled by Java?


Thanks,
David Bierce

Build failed in Jenkins: simulator-hotfix-trigger #28

See 

--
[...truncated 1125 lines...]
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-framework-cluster ---
[INFO] Surefire report directory: 


---
 T E S T S
---

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ cloud-framework-cluster ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ 
cloud-framework-cluster ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-framework-cluster ---
[INFO] Installing 

 to 
/var/lib/jenkins/.m2/repository/org/apache/cloudstack/cloud-framework-cluster/4.3.1-SNAPSHOT/cloud-framework-cluster-4.3.1-SNAPSHOT.jar
[INFO] Installing 

 to 
/var/lib/jenkins/.m2/repository/org/apache/cloudstack/cloud-framework-cluster/4.3.1-SNAPSHOT/cloud-framework-cluster-4.3.1-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Framework - Jobs 4.3.1-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-framework-jobs 
---
[INFO] Deleting 

 (includes = [**/*], excludes = [])
[INFO] Deleting 
 
(includes = [target, dist], excludes = [])
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-framework-jobs ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
cloud-framework-jobs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
cloud-framework-jobs ---
[INFO] Compiling 33 source files to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
cloud-framework-jobs ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
cloud-framework-jobs ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-framework-jobs ---
[INFO] Surefire report directory: 


---
 T E S T S
---

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ cloud-framework-jobs ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.1:attach-descriptor (attach-descriptor) @ 
cloud-framework-jobs ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-framework-jobs ---
[INFO] Installing 

 to 
/var/lib/jenkins/.m2/repository/org/apache/cloudstack/cloud-framework-jobs/4.3.1-SNAPSHOT/cloud-framework-jobs-4.3.1-SNAPSHOT.jar
[INFO] Installing 

 to 
/var/lib/jenkins/.m2/repository/org/apache/cloudstack/cloud-framework-jobs/4.3.1-SNAPSHOT/cloud-framework-jobs-4.3.1-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Cloud Engine Schema Component 4.3.1-SNAPSHOT
[INFO] --

Re: Shell commands vs Native Java file handling

If we could keep code like that all in Java, that would be my personal
preference.


On Tue, Sep 2, 2014 at 10:42 AM, David Bierce 
wrote:

> While investigating an issue with secondary storage templates, I noticed
> all the file handling I came across shelled and execute things like mkdir,
> rm, etc, etc.  With the migration to Java 7 is there still a reason to
> continue that method of file handling or in the future could/should file
> operations be handled by Java?
>
>
> Thanks,
> David Bierce
>
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: Cherry-picking fix that may change 4.3.1 schema


On 02-Sep-2014, at 6:19 pm, Will Stevens  wrote:
> I think this covers the main use case we have to be concerned about, so I
> think this is fine.  I don't think we should have to worry about upgrading
> from 4.3.1 to 4.4.0.  I agree that if we maintain an upgrade path from
> 4.3.1 to 4.4.1, we should be fine.

Alright so I’m dropping this issue, we’re not going to cherry pick it on 4.3. I 
would still go ahead and implement an upgrade path from 4.3.1 to 4.4.1. I’ll 
add an upgrade path and have a hotfix branch for Daan to review.


> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
>
> On Tue, Sep 2, 2014 at 11:56 AM, Rohit Yadav 
> wrote:
>
>>
>> On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
>>> The right way to do this is to do commit to commit migrations and use
>> some
>>> db versioning tools like liquibase or flywaydb which will execute the new
>>> change sets. We discussed this before and IIRC, someone started working
>> on
>>> it.
>>
>> This is not a proposal thread to introduce new tools and I would certainly
>> avoid the “right way to do things” as I’m a fan of iterative development.
>>
>> I’m interested in a concrete solution to the concrete issue I raised.
>> Here’s how I will do it:
>>
>> - Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0
>> upgrade path
>> - Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
>> - Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for
>> the above change
>> - On 4.4 branch, in the upgrade (db upgrader) map there won’t be an
>> upgrade path from 4.3.1 for any class except for the map entry where
>> version is 4.3.1
>> - Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a
>> choice) and the upgrade path will exist only in 4.4.1 and later releases
>>
>> I think if I do the above, no one will have any issue. I’m sharing them to
>> run by you all in case you find any issue?
>>
>>>
>>>
>>> On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
>>> wrote:
>>>
>>>
>>> On 02-Sep-2014, at 5:07 pm, Francois Gaudreault <
>> fgaudrea...@cloudops.com
>>> > wrote:
 I see. Well, I think we were impacted by that too, and we made the
>>> decision to move on 4.4.1-snap (even if it's technically less stable?)
>> and
>>> then upgrade to 4.4.1 GA (next week?)

 I personally don't think pulling back DB changes in lower releases is a
>>> good idea :S

 But that's only my opinion :)
>>>
>>> I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
>> to
>>> stick to that only. I understand there are too many numbers, versions and
>>> branches to follow, so please if you can try to understand the issue;
>>>
>>> This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
>> requires
>>> that there are some extra columns in the database to do book keeping when
>>> delete ips so you don’t actually delete db/table rows.
>>>
>>> The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
>>>
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
>>>
>>> But, we already have 4.4.0 release and db upgrade paths are always in the
>>> next release versions.
>>>
>>> So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
>> not
>>> care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
>>> to 4.3.1. So, the limitation is that people won’t be able to upgrade from
>>> 4.3.1 to 4.4.0, because 4.4.0 is already released.
>>>
>>> If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
>> put
>>> the fix from the JIRA issue on 4.3 branch so the issue is fixed for
>> 4.3.1.
>>> The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
>>> 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
>>> branch such that
>>>
>>> The abstract issue is — we’ll have such issue in future, how do we fix
>> it.
>>> I suggested — we make a separate tool that does (rolling) upgrades.
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +41 779015219 | rohit.ya...@shapeblue.com 
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related
>> services
>>>
>>> IaaS Cloud Design & Build<
>> http://shapeblue.com/iaas-cloud-design-and-build//

>>> CSForge – rapid IaaS deployment framework
>>> CloudStack Consulting
>>> CloudStack Infrastructure Support<
>>> http://shapeblue.com/cloudstack-infrastructure-support/>
>>> CloudStack Bootcamp Training Courses<
>>> http://shapeblue.com/cloudstack-training/>
>>>
>>> This email and any attachments to it may be confidenti

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54187941
  
Fixing it right now.

Sorry for the incovenience.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Review Request 25258: Fix TestVolumes.test_07_resize_fail

2014-09-02 Thread Alex Brett


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25258/
---

Review request for cloudstack and Mike Tutkowski.


Bugs: CLOUDSTACK-7467
https://issues.apache.org/jira/browse/CLOUDSTACK-7467


Repository: cloudstack-git


Description
---

Previously if you had a volume using a non customisable disk offering, and
attempted to resize it passing in the same disk offering id, the command would
be accepted, but it would actually be resized to its current size (i.e. the
provided size parameter was ignored). This is what the test used to check.

Commit de6a3112b6b80952d1598acaa112ac50a3ef9d32 modified the logic to check if
the provided diskofferingid was the same as the current one, and if so treat it
as if it hadn't been provided - this means the resize command now fails, which
is probably the more sensible thing to do (rather than giving the impression it
will be resized but actually not doing so).

This change therefore modifies the test logic to match.


Diffs
-

  test/integration/smoke/test_volumes.py e2e0d76 

Diff: https://reviews.apache.org/r/25258/diff/


Testing
---

Test passed running against current build


Thanks,

Alex Brett

Re: Shell commands vs Native Java file handling




On 02-09-14 19:06, Mike Tutkowski wrote:

If we could keep code like that all in Java, that would be my personal
preference.



+1

Makes error handling (using Exception) a lot easier and cleaner.

Stay away from Shell if not required.

Wido



On Tue, Sep 2, 2014 at 10:42 AM, David Bierce 
wrote:


While investigating an issue with secondary storage templates, I noticed
all the file handling I came across shelled and execute things like mkdir,
rm, etc, etc.  With the migration to Java 7 is there still a reason to
continue that method of file handling or in the future could/should file
operations be handled by Java?


Thanks,
David Bierce

Re: Shell commands vs Native Java file handling

2014-09-02 Thread David Bierce

Me too, but I’m new to this code base and to Java so I didn’t know if there was 
a rational behind it or not.  :)

There were a few places in SSVM where it is handling symlinks, for which 
support wasn’t added till Java 7 (according to the Docs), that was the only 
reason I could think of.

Thanks,
David Bierce
On Sep 2, 2014, at 12:06 PM, Mike Tutkowski  
wrote:

> If we could keep code like that all in Java, that would be my personal
> preference.
> 
> 
> On Tue, Sep 2, 2014 at 10:42 AM, David Bierce 
> wrote:
> 
>> While investigating an issue with secondary storage templates, I noticed
>> all the file handling I came across shelled and execute things like mkdir,
>> rm, etc, etc.  With the migration to Java 7 is there still a reason to
>> continue that method of file handling or in the future could/should file
>> operations be handled by Java?
>> 
>> 
>> Thanks,
>> David Bierce
>> 
>> 
>> 
> 
> 
> -- 
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> *™*

Re: Shell commands vs Native Java file handling




On 02-09-14 20:11, David Bierce wrote:

Me too, but I’m new to this code base and to Java so I didn’t know if there was 
a rational behind it or not.  :)

There were a few places in SSVM where it is handling symlinks, for which 
support wasn’t added till Java 7 (according to the Docs), that was the only 
reason I could think of.



Ah, ok. I'm not aware of specific cases, but there are multiple 
occasions where I replaced Shell by Java code.


If you spot any of those, please, do send patches!

Shell execution is not reliable and also doesn't make code portable.

Wido


Thanks,
David Bierce
On Sep 2, 2014, at 12:06 PM, Mike Tutkowski  
wrote:


If we could keep code like that all in Java, that would be my personal
preference.


On Tue, Sep 2, 2014 at 10:42 AM, David Bierce 
wrote:


While investigating an issue with secondary storage templates, I noticed
all the file handling I came across shelled and execute things like mkdir,
rm, etc, etc.  With the migration to Java 7 is there still a reason to
continue that method of file handling or in the future could/should file
operations be handled by Java?


Thanks,
David Bierce






--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: Review Request 25248: Fix NPE in case VM is started and its template does not exist anymore

2014-09-02 Thread Nitin Mehta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25248/#review52059
---



engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java


Checking for template==null masks the whole problem. 
1. Such validations should have happenned in the deployvm api layer if it 
comes from that api.
2. If its coming from a startvm api its perfectly fine to have the template 
removed since the volume already exists. 
3. If you see how template is used below...if it has to 'create' a new 
volume the template shouldnt be removed but again the validations should be in 
api layer.


- Nitin Mehta


On Sept. 2, 2014, 1:53 p.m., Rohit Yadav wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25248/
> ---
> 
> (Updated Sept. 2, 2014, 1:53 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk, edison su, Darren Shepherd, 
> Sebastien Goasguen, and Hugo Trippaers.
> 
> 
> Bugs: CLOUDSTACK-6945
> https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixes https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Diffs
> -
> 
>   engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 
> 2fd7a52 
> 
> Diff: https://reviews.apache.org/r/25248/diff/
> 
> 
> Testing
> ---
> 
> Builds cleanly, will throw resource not available exception when template 
> does not exist.
> 
> 
> Thanks,
> 
> Rohit Yadav
> 
>

Jenkins build is back to normal : build-master #1522

See

Review Request 25266: Simulator build support need to extends for RPM build

2014-09-02 Thread Rayees Namathponnan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25266/
---

Review request for cloudstack, Frank Zhang and Hugo Trippaers.


Bugs: CLOUDSTACK-7469
https://issues.apache.org/jira/browse/CLOUDSTACK-7469


Repository: cloudstack-git


Description
---

Currently there is no option to build rpm with simulator,  as part this patch 
modified package.sh file to accept simulator

also updated cloud.spec file to build both oss and nooss simulator buids


To build noredist and simulator, you need to use below package command
./package.sh --pack noredist --s simulator

default package with simulato, need to call  ./package.sh


Diffs
-

  packaging/centos63/cloud.spec 7306d1f 
  packaging/centos63/package.sh 6de66e6 

Diff: https://reviews.apache.org/r/25266/diff/


Testing
---

Yes


Thanks,

Rayees Namathponnan

Re: Review Request 25258: Fix TestVolumes.test_07_resize_fail


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25258/#review52061
---

Ship it!


Committed with SHA 24dd6cee78ec779498b532d5dbb5015490642129

Thanks for changing the test! :)

- Mike Tutkowski


On Sept. 2, 2014, 11:48 a.m., Alex Brett wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25258/
> ---
> 
> (Updated Sept. 2, 2014, 11:48 a.m.)
> 
> 
> Review request for cloudstack and Mike Tutkowski.
> 
> 
> Bugs: CLOUDSTACK-7467
> https://issues.apache.org/jira/browse/CLOUDSTACK-7467
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Previously if you had a volume using a non customisable disk offering, and
> attempted to resize it passing in the same disk offering id, the command would
> be accepted, but it would actually be resized to its current size (i.e. the
> provided size parameter was ignored). This is what the test used to check.
> 
> Commit de6a3112b6b80952d1598acaa112ac50a3ef9d32 modified the logic to check if
> the provided diskofferingid was the same as the current one, and if so treat 
> it
> as if it hadn't been provided - this means the resize command now fails, which
> is probably the more sensible thing to do (rather than giving the impression 
> it
> will be resized but actually not doing so).
> 
> This change therefore modifies the test logic to match.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_volumes.py e2e0d76 
> 
> Diff: https://reviews.apache.org/r/25258/diff/
> 
> 
> Testing
> ---
> 
> Test passed running against current build
> 
> 
> Thanks,
> 
> Alex Brett
> 
>

Jenkins build is back to normal : build-master-noredist #3439

See

Re: Review Request 25248: Fix NPE in case VM is started and its template does not exist anymore



> On Sept. 2, 2014, 7:01 p.m., Nitin Mehta wrote:
> > engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java, line 
> > 814
> > 
> >
> > Checking for template==null masks the whole problem. 
> > 1. Such validations should have happenned in the deployvm api layer if 
> > it comes from that api.
> > 2. If its coming from a startvm api its perfectly fine to have the 
> > template removed since the volume already exists. 
> > 3. If you see how template is used below...if it has to 'create' a new 
> > volume the template shouldnt be removed but again the validations should be 
> > in api layer.

So, I can read code too, upper layers are not passing the template so what do 
you propose? How may I fix this then?


- Rohit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25248/#review52059
---


On Sept. 2, 2014, 1:53 p.m., Rohit Yadav wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25248/
> ---
> 
> (Updated Sept. 2, 2014, 1:53 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk, edison su, Darren Shepherd, 
> Sebastien Goasguen, and Hugo Trippaers.
> 
> 
> Bugs: CLOUDSTACK-6945
> https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixes https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Diffs
> -
> 
>   engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 
> 2fd7a52 
> 
> Diff: https://reviews.apache.org/r/25248/diff/
> 
> 
> Testing
> ---
> 
> Builds cleanly, will throw resource not available exception when template 
> does not exist.
> 
> 
> Thanks,
> 
> Rohit Yadav
> 
>

Storing passwords in the DB

Hi,

I was wondering what our current "best practices" are around storing
passwords in the DB?

For example, if you want to store the username and password of a resource
that CloudStack manages, how do we recommend storing the password?

Thanks!

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: Storing passwords in the DB




On 02-09-14 21:22, Mike Tutkowski wrote:

Hi,

I was wondering what our current "best practices" are around storing
passwords in the DB?

For example, if you want to store the username and password of a resource
that CloudStack manages, how do we recommend storing the password?



Using the build-in encryption mechanism? CloudStack also saves the VNC 
passwords for KVM that way for example.


Wido


Thanks!

Re: Cherry-picking fix that may change 4.3.1 schema

On Tue, Sep 2, 2014 at 10:48 AM, Rohit Yadav  wrote:
>
> On 02-Sep-2014, at 4:38 pm, David Nalley  wrote:
>> I agree - how will users upgrade from 4.3.1 to 4.4.x or later - schema
>> changes aren't necessarily going to screw up the upgrade process, but
>> certainly have a high likelihood of doing so. What happens when
>> 4.3.0-to-4.4.0.sql runs? The table already exists - and upgrade will
>> puke. 4.4.0 is already released; so not like you can roll that change
>> back.
>
> Since 4.4.0 is already released, people who will install 4.3.1 won’t be able 
> to upgrade to 4.4.0 as the 4.4.0 artifacts won’t have an upgrade path from 
> 4.3.1 to 4.4.0; but this can be only fixed (i.e. have an upgrade path) in 
> 4.4.1.
>

Yes, I understand that, but think about all of the changes in 4.4.0 -
those changes (including the duplicated ones you're proposing) still
have to run against the database - e.g. - the database has to go
through the cycles to make it to 4.4.0, even if it is but an interim
step. Unless you are talking about obviating the need to run the 4.4.0
schema changes entirely - and that also scares me. That's new upgrade
path from 4.4.1, while we think we have a decent upgrade path at
present.

> For 4.3.1, we can still have the changes but 4.3.1 users would only be able 
> to upgrade to 4.4.1 or later releases and not 4.4.0 because there is no path 
> from 4.3.1 to 4.4.0 in 4.4.0 release.
>

Think in database releases rather than code release - you can't get to
4.4.1 without traversing 4.4.0 - there are other changes; unless you
are talking rebuilding the database upgrade portion entirelyand
skipping 4.4.0.

> This is true for all ACS releases, only the latest releases know and have 
> upgrade paths. So, what I also proposed here was to refactor the whole 
> upgrade logic as a tool and maintain it separately and this tool can perhaps 
> has its own release process that is faster than the ACS releases and this 
> tool knows about all the releases and has the necessary upgrade paths.
>

Honestly the db upgrade process does concern me - others have
mentioned tools like flyway and I like the idea - esp if we do
database design in such a way that we can downgrade as well as
upgrade. I would love to see a proposal on doing this better, though I
don't think it would solve this particular problem.

Re: Storing passwords in the DB

Thanks, Wido

Do you happen to know a relevant class off the top of your head?


On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander  wrote:

>
>
> On 02-09-14 21:22, Mike Tutkowski wrote:
>
>> Hi,
>>
>> I was wondering what our current "best practices" are around storing
>> passwords in the DB?
>>
>> For example, if you want to store the username and password of a resource
>> that CloudStack manages, how do we recommend storing the password?
>>
>>
> Using the build-in encryption mechanism? CloudStack also saves the VNC
> passwords for KVM that way for example.
>
> Wido
>
>  Thanks!
>>
>>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: Storing passwords in the DB




On 02-09-14 21:29, Mike Tutkowski wrote:

Thanks, Wido

Do you happen to know a relevant class off the top of your head?



No sorry, but if you search for where it fetches the VNC password for 
KVM VMs you should find it.


It's probably the DB layer which does the encryption and decryption.

Wido



On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander  wrote:




On 02-09-14 21:22, Mike Tutkowski wrote:


Hi,

I was wondering what our current "best practices" are around storing
passwords in the DB?

For example, if you want to store the username and password of a resource
that CloudStack manages, how do we recommend storing the password?



Using the build-in encryption mechanism? CloudStack also saves the VNC
passwords for KVM that way for example.

Wido

  Thanks!

Re: Storing passwords in the DB

OK - thanks!


On Tue, Sep 2, 2014 at 1:33 PM, Wido den Hollander  wrote:

>
>
> On 02-09-14 21:29, Mike Tutkowski wrote:
>
>> Thanks, Wido
>>
>> Do you happen to know a relevant class off the top of your head?
>>
>>
> No sorry, but if you search for where it fetches the VNC password for KVM
> VMs you should find it.
>
> It's probably the DB layer which does the encryption and decryption.
>
> Wido
>
>
>
>> On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander 
>> wrote:
>>
>>
>>>
>>> On 02-09-14 21:22, Mike Tutkowski wrote:
>>>
>>>  Hi,

 I was wondering what our current "best practices" are around storing
 passwords in the DB?

 For example, if you want to store the username and password of a
 resource
 that CloudStack manages, how do we recommend storing the password?


  Using the build-in encryption mechanism? CloudStack also saves the VNC
>>> passwords for KVM that way for example.
>>>
>>> Wido
>>>
>>>   Thanks!
>>>



>>
>>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Jenkins build is still unstable: simulator-singlerun #265

See

Pushing hotfix branches

Hey All,
I am a committer and when I tried to push a hotfix branch to '
https://github.com/apache/cloudstack.git' today, I got a "Permission
Denied" error.  Is this a problem with my account or is there an issue with
the github repo right now?

Thx,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

Re: Pushing hotfix branches


On Sep 2, 2014, at 3:57 PM, Will Stevens  wrote:

> Hey All,
> I am a committer and when I tried to push a hotfix branch to '
> https://github.com/apache/cloudstack.git' today, I got a "Permission
> Denied" error.  Is this a problem with my account or is there an issue with
> the github repo right now?
> 

you can't, it's a mirror…you need to create a branch on the asf repo first.

> Thx,
> 
> *Will STEVENS*
> Lead Developer
> 
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_

Re: Pushing hotfix branches

You can't push to github. It's a RO mirror.
You push commits to git-wip-us.a.o.
Very few people have write access to anything on github.com/apache



On Tue, Sep 2, 2014 at 3:57 PM, Will Stevens  wrote:
> Hey All,
> I am a committer and when I tried to push a hotfix branch to '
> https://github.com/apache/cloudstack.git' today, I got a "Permission
> Denied" error.  Is this a problem with my account or is there an issue with
> the github repo right now?
>
> Thx,
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_

Re: Pushing hotfix branches

Oh.  I thought for sure I had done that before, but maybe I am just losing
my mind (totally possible).  :)

Thanks guys,

Will


*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_


On Tue, Sep 2, 2014 at 4:03 PM, David Nalley  wrote:

> You can't push to github. It's a RO mirror.
> You push commits to git-wip-us.a.o.
> Very few people have write access to anything on github.com/apache
>
>
>
> On Tue, Sep 2, 2014 at 3:57 PM, Will Stevens 
> wrote:
> > Hey All,
> > I am a committer and when I tried to push a hotfix branch to '
> > https://github.com/apache/cloudstack.git' today, I got a "Permission
> > Denied" error.  Is this a problem with my account or is there an issue
> with
> > the github repo right now?
> >
> > Thx,
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
>

Re: Storing passwords in the DB

2014-09-02 Thread Amogh Vasekar

Hi,
You can add @Encrypt tag to the field and it would be stored in encrypted
form in the DB, and decrypted automatically when reading. It uses the key
file provided in db.properties for encryption.
You can check many of the VOs, for example UserVO, as a reference.

HTH
Amogh  


On 9/2/14 12:38 PM, "Mike Tutkowski"  wrote:

>OK - thanks!
>
>
>On Tue, Sep 2, 2014 at 1:33 PM, Wido den Hollander  wrote:
>
>>
>>
>> On 02-09-14 21:29, Mike Tutkowski wrote:
>>
>>> Thanks, Wido
>>>
>>> Do you happen to know a relevant class off the top of your head?
>>>
>>>
>> No sorry, but if you search for where it fetches the VNC password for
>>KVM
>> VMs you should find it.
>>
>> It's probably the DB layer which does the encryption and decryption.
>>
>> Wido
>>
>>
>>
>>> On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander 
>>> wrote:
>>>
>>>

 On 02-09-14 21:22, Mike Tutkowski wrote:

  Hi,
>
> I was wondering what our current "best practices" are around storing
> passwords in the DB?
>
> For example, if you want to store the username and password of a
> resource
> that CloudStack manages, how do we recommend storing the password?
>
>
>  Using the build-in encryption mechanism? CloudStack also saves the
>VNC
 passwords for KVM that way for example.

 Wido

   Thanks!

>
>
>
>>>
>>>
>
>
>-- 
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkow...@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the cloud
>**

Re: Review Request 25248: Fix NPE in case VM is started and its template does not exist anymore

2014-09-02 Thread Nitin Mehta



> On Sept. 2, 2014, 7:01 p.m., Nitin Mehta wrote:
> > engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java, line 
> > 814
> > 
> >
> > Checking for template==null masks the whole problem. 
> > 1. Such validations should have happenned in the deployvm api layer if 
> > it comes from that api.
> > 2. If its coming from a startvm api its perfectly fine to have the 
> > template removed since the volume already exists. 
> > 3. If you see how template is used below...if it has to 'create' a new 
> > volume the template shouldnt be removed but again the validations should be 
> > in api layer.
> 
> Rohit Yadav wrote:
> So, I can read code too, upper layers are not passing the template so 
> what do you propose? How may I fix this then?

Firstly, check whether the issue is reproducible. Just realized that from 4.3 
onwards templates have 'Inactive' state to mark it removed. Removed attribute 
should never be set so this exception shouldnt be hit . Check when is the 
removed flag set as it should be a bug(check CLOUDSTACK-5997 and backporting it 
already fixes that). 
Secondly, even if this bug doesnt exist, do a sanity check and see this kind of 
check is in the api which will ultimately call this method. Check if apis are 
missing them. But some apis might not need it like StartVm, Rebootvm where new 
volume is not created. Do write a util method which could be shared by all 
apis. I guess that should be good enough.


- Nitin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25248/#review52059
---


On Sept. 2, 2014, 1:53 p.m., Rohit Yadav wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25248/
> ---
> 
> (Updated Sept. 2, 2014, 1:53 p.m.)
> 
> 
> Review request for cloudstack, Alena Prokharchyk, edison su, Darren Shepherd, 
> Sebastien Goasguen, and Hugo Trippaers.
> 
> 
> Bugs: CLOUDSTACK-6945
> https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixes https://issues.apache.org/jira/browse/CLOUDSTACK-6945
> 
> 
> Diffs
> -
> 
>   engine/orchestration/src/com/cloud/vm/VirtualMachineManagerImpl.java 
> 2fd7a52 
> 
> Diff: https://reviews.apache.org/r/25248/diff/
> 
> 
> Testing
> ---
> 
> Builds cleanly, will throw resource not available exception when template 
> does not exist.
> 
> 
> Thanks,
> 
> Rohit Yadav
> 
>

Re: Storing passwords in the DB

Thanks, Amogh

In my case, I'm storing the password in the storage_pool_details table's
value field. Not all cells in this column will need to be encrypted,
though. What do you suggest there?


On Tue, Sep 2, 2014 at 2:28 PM, Amogh Vasekar 
wrote:

> Hi,
> You can add @Encrypt tag to the field and it would be stored in encrypted
> form in the DB, and decrypted automatically when reading. It uses the key
> file provided in db.properties for encryption.
> You can check many of the VOs, for example UserVO, as a reference.
>
> HTH
> Amogh
>
>
> On 9/2/14 12:38 PM, "Mike Tutkowski"  wrote:
>
> >OK - thanks!
> >
> >
> >On Tue, Sep 2, 2014 at 1:33 PM, Wido den Hollander 
> wrote:
> >
> >>
> >>
> >> On 02-09-14 21:29, Mike Tutkowski wrote:
> >>
> >>> Thanks, Wido
> >>>
> >>> Do you happen to know a relevant class off the top of your head?
> >>>
> >>>
> >> No sorry, but if you search for where it fetches the VNC password for
> >>KVM
> >> VMs you should find it.
> >>
> >> It's probably the DB layer which does the encryption and decryption.
> >>
> >> Wido
> >>
> >>
> >>
> >>> On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander 
> >>> wrote:
> >>>
> >>>
> 
>  On 02-09-14 21:22, Mike Tutkowski wrote:
> 
>   Hi,
> >
> > I was wondering what our current "best practices" are around storing
> > passwords in the DB?
> >
> > For example, if you want to store the username and password of a
> > resource
> > that CloudStack manages, how do we recommend storing the password?
> >
> >
> >  Using the build-in encryption mechanism? CloudStack also saves the
> >VNC
>  passwords for KVM that way for example.
> 
>  Wido
> 
>    Thanks!
> 
> >
> >
> >
> >>>
> >>>
> >
> >
> >--
> >*Mike Tutkowski*
> >*Senior CloudStack Developer, SolidFire Inc.*
> >e: mike.tutkow...@solidfire.com
> >o: 303.746.7302
> >Advancing the way the world uses the cloud
> >* *
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Jenkins build is still unstable: simulator-singlerun #266

See

Re: Storing passwords in the DB

2014-09-02 Thread Amogh Vasekar

Hi,

You can check DBEncryptionUtil, it provides utility methods for encryption
/ decryption. May be add some custom logic for only your cell?
HostEntityDaoImpl might be useful reference.

Thanks,
Amogh

On 9/2/14 1:47 PM, "Mike Tutkowski"  wrote:

>Thanks, Amogh
>
>In my case, I'm storing the password in the storage_pool_details table's
>value field. Not all cells in this column will need to be encrypted,
>though. What do you suggest there?
>
>
>On Tue, Sep 2, 2014 at 2:28 PM, Amogh Vasekar 
>wrote:
>
>> Hi,
>> You can add @Encrypt tag to the field and it would be stored in
>>encrypted
>> form in the DB, and decrypted automatically when reading. It uses the
>>key
>> file provided in db.properties for encryption.
>> You can check many of the VOs, for example UserVO, as a reference.
>>
>> HTH
>> Amogh
>>
>>
>> On 9/2/14 12:38 PM, "Mike Tutkowski" 
>>wrote:
>>
>> >OK - thanks!
>> >
>> >
>> >On Tue, Sep 2, 2014 at 1:33 PM, Wido den Hollander 
>> wrote:
>> >
>> >>
>> >>
>> >> On 02-09-14 21:29, Mike Tutkowski wrote:
>> >>
>> >>> Thanks, Wido
>> >>>
>> >>> Do you happen to know a relevant class off the top of your head?
>> >>>
>> >>>
>> >> No sorry, but if you search for where it fetches the VNC password for
>> >>KVM
>> >> VMs you should find it.
>> >>
>> >> It's probably the DB layer which does the encryption and decryption.
>> >>
>> >> Wido
>> >>
>> >>
>> >>
>> >>> On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander 
>> >>> wrote:
>> >>>
>> >>>
>> 
>>  On 02-09-14 21:22, Mike Tutkowski wrote:
>> 
>>   Hi,
>> >
>> > I was wondering what our current "best practices" are around
>>storing
>> > passwords in the DB?
>> >
>> > For example, if you want to store the username and password of a
>> > resource
>> > that CloudStack manages, how do we recommend storing the password?
>> >
>> >
>> >  Using the build-in encryption mechanism? CloudStack also saves
>>the
>> >VNC
>>  passwords for KVM that way for example.
>> 
>>  Wido
>> 
>>    Thanks!
>> 
>> >
>> >
>> >
>> >>>
>> >>>
>> >
>> >
>> >--
>> >*Mike Tutkowski*
>> >*Senior CloudStack Developer, SolidFire Inc.*
>> >e: mike.tutkow...@solidfire.com
>> >o: 303.746.7302
>> >Advancing the way the world uses the cloud
>> >* *
>>
>>
>
>
>-- 
>*Mike Tutkowski*
>*Senior CloudStack Developer, SolidFire Inc.*
>e: mike.tutkow...@solidfire.com
>o: 303.746.7302
>Advancing the way the world uses the cloud
>**

Re: Realhostip service extended till Sep 30th

2014-09-02 Thread Nitin Mehta

Created another wiki [1] for troubleshooting and information on the
implementation details.
[1] 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Troubleshooting+-+up
loading+custom+domain+certificate+instead+of+using+realhostip.com



Thanks,
-Nitin

On 05/06/14 7:03 PM, "Nitin Mehta"  wrote:

>Please find wiki [1] containing the procedure to replace realhostip.com
>with Your own custom domain name.
>
>[1] 
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replac
>e
>+realhostip.com+with+Your+Own+Domain+Name
>
>
>Thanks,
>-Nitin
>
>On 05/06/14 4:23 PM, "Animesh Chaturvedi" 
>wrote:
>
>>
>>Folks 
>>
>>I wanted to provide an update on shutting down of RealhostIp service.
>>Citrix has decided to move the date by one quarter and the new date will
>>be Sep 30th.  The new dates allow users of CloudStack additional time for
>>updating their infrastructure. While testing Realhostip fixes additional
>>product issues were found which are tracked in following three defects
>>CLOUDSTACK-6499 [1], CLOUDSTACK-6599 [2], CLOUDSTACK-6824 [3] . These
>>defects are fixed in 4.4 and master branch. Please verify in your setups.
>>If you are on older version of CloudStack do plan on upgrading to 4.4 .
>>Additional details on fixes are available in JIRA.  Updated wiki with
>>steps will be published shortly.
>>
>>[1] https://issues.apache.org/jira/browse/CLOUDSTACK-6499 
>>[2] https://issues.apache.org/jira/browse/CLOUDSTACK-6599
>>[3] https://issues.apache.org/jira/browse/CLOUDSTACK-6824
>>
>>
>>Thanks
>>Animesh
>>
>>
>>
>>
>>> -Original Message-
>>> From: John Kinsella [mailto:j...@stratosec.co]
>>> Sent: Thursday, April 17, 2014 8:35 AM
>>> To: ; users...@cloudstack.apache.org;
>>> 
>>> Subject: REMINDER RealhostIP going away
>>> 
>>> Reminder, folks - please migrate off RealhostIP.com or you¹re going to
>>>get a
>>> nasty surprise this summer. More info at link below.
>>> 
>>> 
>>>https://blogs.apache.org/cloudstack/entry/RealhostIP_service_is_being_re
>>>t
>>> ired
>

Build failed in Jenkins: simulator-hotfix-trigger #29

See 

--
[...truncated 7107 lines...]
[INFO] 
[INFO] Building Apache CloudStack Developer Mode 4.4.1-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-developer ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties 
(default) @ cloud-developer ---
[WARNING] Ignoring missing properties file: 

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-developer ---
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (default) @ cloud-developer ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] >>> exec-maven-plugin:1.2.1:java (create-schema-simulator) @ 
cloud-developer >>>
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-developer ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] <<< exec-maven-plugin:1.2.1:java (create-schema-simulator) @ 
cloud-developer <<<
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:java (create-schema-simulator) @ 
cloud-developer ---
log4j:WARN No appenders could be found for logger 
(org.springframework.core.env.StandardEnvironment).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
info.
> WARNING: Provided file does not exist: 

> Initializing database=simulator with host=localhost port=3306 
username=cloud password=cloud
> Running query: drop database if exists `simulator`
> Running query: create database `simulator`
> Running query: GRANT ALL ON simulator.* to 'cloud'@`localhost` 
identified by 'cloud'
> Running query: GRANT ALL ON simulator.* to 'cloud'@`%` identified 
by 'cloud'
> Processing SQL file at 

> Processing SQL file at 

> Processing SQL file at 

> Processing upgrade: com.cloud.upgrade.DatabaseUpgradeChecker
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
cloud-developer ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-developer ---
[INFO] Installing 

 to 
/var/lib/jenkins/.m2/repository/org/apache/cloudstack/cloud-developer/4.4.1-SNAPSHOT/cloud-developer-4.4.1-SNAPSHOT.pom
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 16.840s
[INFO] Finished at: Tue Sep 02 17:12:53 EDT 2014
[INFO] Final Memory: 40M/181M
[INFO] 
[simulator-hotfix-trigger] $ /bin/bash -x /tmp/hudson4887887998073909737.sh
+ jps -l
+ grep -q Launcher
+ rm -f xunit.xml
+ rm -rf /tmp/MarvinLogs
+ echo Check for initialization of the management server
Check for initialization of the management server
+ COUNTER=0
+ SERVER_PID=16405
+ '[' 0 -lt 44 ']'
+ mvn -P systemvm,simulator -pl :cloud-client-ui jetty:run
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=1
+ '[' 1 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=2
+ '[' 2 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=3
+ '[' 3 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=4
+ '[' 4 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=5
+ '[' 5 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=6
+ '[' 6 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=7
+ '[' 7 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=8
+ '[' 8 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=9
+ '[' 9 -lt 44 ']'
+ grep -q 'Management server node 127.0.0.1 is up' jetty-output.out
+ sleep 5
+ COUNTER=10
+

Jenkins build is still unstable: simulator-singlerun #267

See

Re: Storing passwords in the DB

Thanks, Amogh

It looks like I can simply call the static encrypt method before I store
the password in the DB, then pull out the encrypted value and call the
static decrypt method before I send the password to the resource in
question.


On Tue, Sep 2, 2014 at 3:05 PM, Amogh Vasekar 
wrote:

> Hi,
>
> You can check DBEncryptionUtil, it provides utility methods for encryption
> / decryption. May be add some custom logic for only your cell?
> HostEntityDaoImpl might be useful reference.
>
> Thanks,
> Amogh
>
> On 9/2/14 1:47 PM, "Mike Tutkowski"  wrote:
>
> >Thanks, Amogh
> >
> >In my case, I'm storing the password in the storage_pool_details table's
> >value field. Not all cells in this column will need to be encrypted,
> >though. What do you suggest there?
> >
> >
> >On Tue, Sep 2, 2014 at 2:28 PM, Amogh Vasekar 
> >wrote:
> >
> >> Hi,
> >> You can add @Encrypt tag to the field and it would be stored in
> >>encrypted
> >> form in the DB, and decrypted automatically when reading. It uses the
> >>key
> >> file provided in db.properties for encryption.
> >> You can check many of the VOs, for example UserVO, as a reference.
> >>
> >> HTH
> >> Amogh
> >>
> >>
> >> On 9/2/14 12:38 PM, "Mike Tutkowski" 
> >>wrote:
> >>
> >> >OK - thanks!
> >> >
> >> >
> >> >On Tue, Sep 2, 2014 at 1:33 PM, Wido den Hollander 
> >> wrote:
> >> >
> >> >>
> >> >>
> >> >> On 02-09-14 21:29, Mike Tutkowski wrote:
> >> >>
> >> >>> Thanks, Wido
> >> >>>
> >> >>> Do you happen to know a relevant class off the top of your head?
> >> >>>
> >> >>>
> >> >> No sorry, but if you search for where it fetches the VNC password for
> >> >>KVM
> >> >> VMs you should find it.
> >> >>
> >> >> It's probably the DB layer which does the encryption and decryption.
> >> >>
> >> >> Wido
> >> >>
> >> >>
> >> >>
> >> >>> On Tue, Sep 2, 2014 at 1:25 PM, Wido den Hollander 
> >> >>> wrote:
> >> >>>
> >> >>>
> >> 
> >>  On 02-09-14 21:22, Mike Tutkowski wrote:
> >> 
> >>   Hi,
> >> >
> >> > I was wondering what our current "best practices" are around
> >>storing
> >> > passwords in the DB?
> >> >
> >> > For example, if you want to store the username and password of a
> >> > resource
> >> > that CloudStack manages, how do we recommend storing the password?
> >> >
> >> >
> >> >  Using the build-in encryption mechanism? CloudStack also saves
> >>the
> >> >VNC
> >>  passwords for KVM that way for example.
> >> 
> >>  Wido
> >> 
> >>    Thanks!
> >> 
> >> >
> >> >
> >> >
> >> >>>
> >> >>>
> >> >
> >> >
> >> >--
> >> >*Mike Tutkowski*
> >> >*Senior CloudStack Developer, SolidFire Inc.*
> >> >e: mike.tutkow...@solidfire.com
> >> >o: 303.746.7302
> >> >Advancing the way the world uses the cloud
> >> >* *
> >>
> >>
> >
> >
> >--
> >*Mike Tutkowski*
> >*Senior CloudStack Developer, SolidFire Inc.*
> >e: mike.tutkow...@solidfire.com
> >o: 303.746.7302
> >Advancing the way the world uses the cloud
> >* *
>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Build failed in Jenkins: build-master-noredist #3442

See 

Changes:

[min.chen] CLOUDSTACK-7471:Regular user is allowed to 
deleteNetwork/RestartNetwork

[min.chen] Fix incorrectly written unit tests.

--
[...truncated 5058 lines...]
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-plugin-user-authenticator-md5 ---
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-md5/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-md5-4.5.0-SNAPSHOT.jar
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-md5/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-md5-4.5.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Apache CloudStack Plugin - User Authenticator Plain Text 
4.5.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Deleting 

 (includes = [**/*], excludes = [])
[INFO] Deleting 

 (includes = [target, dist], excludes = [])
[INFO] 
[INFO] --- maven-checkstyle-plugin:2.11:check (cloudstack-checkstyle) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Starting audit...
Audit done.

[INFO] 
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] 
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Compiling 1 source file to 

[INFO] 
[INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ 
cloud-plugin-user-authenticator-plaintext ---
[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] No sources to compile
[INFO] 
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Surefire report directory: 


---
 T E S T S
---

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

[INFO] 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Building jar: 

[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] 
[INFO] --- maven-install-plugin:2.3.1:install (default-install) @ 
cloud-plugin-user-authenticator-plaintext ---
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-plaintext/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-plaintext-4.5.0-SNAPSHOT.jar
[INFO] Installing 

 to 
/jenkins/.m2/repository/org/apache/cloudstack/cloud-plugin-user-authenticator-plaintext/4.5.0-SNAPSHOT/cloud-plugin-user-authenticator-plaintext-4.5.0-SNAPSHOT.pom
[INFO]

Jenkins build is still unstable: simulator-singlerun #268

See

Jenkins build is still unstable: simulator-singlerun #269

See

Re: Cherry-picking fix that may change 4.3.1 schema

On Tue, Sep 2, 2014 at 9:18 PM, Will Stevens  wrote:

>
> @Rajani
> I am not entirely sure which technology is being used for the db upgrade
> paths, but I think it is probably something like liquibase already.  I
> would have to check on the technology, but this is already in place.  The
> issue is more about maintaining an upgradeable path for each version that
> is released.  This is a bit complicated by the fact that we currently have
> 4.3, 4.4 and 4.5 version all having changes made to them at the same
> time...
>
>
we do not use any external library. It is home grown and has the
limitations like not handling commit to commit migration. it only does
version to version which is why we see lot of db issues on master as
well(current development branch)



~Rajani



>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
>
> On Tue, Sep 2, 2014 at 11:28 AM, Rohit Yadav 
> wrote:
>
> >
> > On 02-Sep-2014, at 5:07 pm, Francois Gaudreault <
> fgaudrea...@cloudops.com>
> > wrote:
> > > I see. Well, I think we were impacted by that too, and we made the
> > decision to move on 4.4.1-snap (even if it's technically less stable?)
> and
> > then upgrade to 4.4.1 GA (next week?)
> > >
> > > I personally don't think pulling back DB changes in lower releases is a
> > good idea :S
> > >
> > > But that's only my opinion :)
> >
> > I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
> > to stick to that only. I understand there are too many numbers, versions
> > and branches to follow, so please if you can try to understand the issue;
> >
> > This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> > requires that there are some extra columns in the database to do book
> > keeping when delete ips so you don’t actually delete db/table rows.
> >
> > The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
> >
> > But, we already have 4.4.0 release and db upgrade paths are always in the
> > next release versions.
> >
> > So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
> > not care about 4.4.0 schema changes. There is only an upgrade path from
> > 4.3.0 to 4.3.1. So, the limitation is that people won’t be able to
> upgrade
> > from 4.3.1 to 4.4.0, because 4.4.0 is already released.
> >
> > If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
> > put the fix from the JIRA issue on 4.3 branch so the issue is fixed for
> > 4.3.1. The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we
> > release 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to
> 4.4.1
> > on 4.4 branch such that
> >
> > The abstract issue is — we’ll have such issue in future, how do we fix
> it.
> > I suggested — we make a separate tool that does (rolling) upgrades.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.ya...@shapeblue.com
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> > http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge – rapid IaaS deployment framework
> > CloudStack Consulting
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> > intended recipient of this email, you must neither take any action based
> > upon its contents, nor copy or show it to anyone. Please contact the
> sender
> > if you believe you have received this email in error. Shape Blue Ltd is a
> > company incorporated in England & Wales. ShapeBlue Services India LLP is
> a
> > company incorporated in India and is operated under license from Shape
> Blue
> > Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil
> > and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd
> is
> > a company registered by The Republic of South Africa and is traded under
> > license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> >
>

Re: Cherry-picking fix that may change 4.3.1 schema

this was to your suggestion of "a separate tool that does (rolling)
upgrades" which i think is a new proposal. I should have made it clear in
my mail about it and should have started it with "right way according to
me".

"right way to do things" is in no way against iterative development. May be
at a stage one would realise it was not the right way.

Anyways, thanks for the gyan. I would stay away from any such discussions
in future.

~Rajani


On Tue, Sep 2, 2014 at 9:26 PM, Rohit Yadav 
wrote:

>
> On 02-Sep-2014, at 5:40 pm, Rajani Karuturi  wrote:
> > The right way to do this is to do commit to commit migrations and use
> some
> > db versioning tools like liquibase or flywaydb which will execute the new
> > change sets. We discussed this before and IIRC, someone started working
> on
> > it.
>
> This is not a proposal thread to introduce new tools and I would certainly
> avoid the “right way to do things” as I’m a fan of iterative development.
>
> I’m interested in a concrete solution to the concrete issue I raised.
> Here’s how I will do it:
>
> - Fix the issue on 4.3.1, cherry pick and get rid of the 4.3.0 to 4.4.0
> upgrade path
> - Put the db changes in upgrade path from 4.3.0 to 4.3.1, on 4.3 branch
> - Add an upgrade path from 4.3.1 to 4.4.1 on 4.4 branch which counts for
> the above change
> - On 4.4 branch, in the upgrade (db upgrader) map there won’t be an
> upgrade path from 4.3.1 for any class except for the map entry where
> version is 4.3.1
> - Users who upgrade to 4.3.1 can upgrade only to 4.4.1 (they don’t have a
> choice) and the upgrade path will exist only in 4.4.1 and later releases
>
> I think if I do the above, no one will have any issue. I’m sharing them to
> run by you all in case you find any issue?
>
> >
> >
> > On Tue, Sep 2, 2014 at 20:58 PM, Rohit Yadav 
> > wrote:
> >
> >
> > On 02-Sep-2014, at 5:07 pm, Francois Gaudreault <
> fgaudrea...@cloudops.com
> > > wrote:
> >> I see. Well, I think we were impacted by that too, and we made the
> > decision to move on 4.4.1-snap (even if it's technically less stable?)
> and
> > then upgrade to 4.4.1 GA (next week?)
> >>
> >> I personally don't think pulling back DB changes in lower releases is a
> > good idea :S
> >>
> >> But that's only my opinion :)
> >
> > I raised a concrete issue regarding 4.3.1/4.4.1 release and I would like
> to
> > stick to that only. I understand there are too many numbers, versions and
> > branches to follow, so please if you can try to understand the issue;
> >
> > This issue — https://issues.apache.org/jira/browse/CLOUDSTACK-6756
> requires
> > that there are some extra columns in the database to do book keeping when
> > delete ips so you don’t actually delete db/table rows.
> >
> > The issue above is fixed in an upgrade path from 4.3.0 to 4.4.0:
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=setup/db/db/schema-430to440.sql;h=226260804523c79e3ce3cfa3c407b5ac698d749c;hp=3b525c41a1befd94c5ffc324c357b566606a97d0;hb=ce6a53e;hpb=d0f806b3a486c58b033083fc57f39dd686e31750
> >
> > But, we already have 4.4.0 release and db upgrade paths are always in the
> > next release versions.
> >
> > So, there is no upgrade path from 4.3.1 to 4.4.0; as 4.3.1 version does
> not
> > care about 4.4.0 schema changes. There is only an upgrade path from 4.3.0
> > to 4.3.1. So, the limitation is that people won’t be able to upgrade from
> > 4.3.1 to 4.4.0, because 4.4.0 is already released.
> >
> > If we release 4.4.1 before 4.3.1, we’ll have the same issue. So, we can
> put
> > the fix from the JIRA issue on 4.3 branch so the issue is fixed for
> 4.3.1.
> > The fix will be in the 4.3.0 to 4.3.1 upgrade path. And, if we release
> > 4.4.1 after 4.3.1, we can fix the upgrade path from 4.3.1 to 4.4.1 on 4.4
> > branch such that
> >
> > The abstract issue is — we’ll have such issue in future, how do we fix
> it.
> > I suggested — we make a separate tool that does (rolling) upgrades.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.ya...@shapeblue.com 
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//
> >>
> > CSForge – rapid IaaS deployment framework
> > CloudStack Consulting
> > CloudStack Infrastructure Support<
> > http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> > http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are intended
> > solely for the use of the individual to whom it is addressed. Any views
> or
> > opinions expressed are solely those of the author and do not necessarily
> > represent those of Shape Blue Ltd or related companies. If you are not
> the
> > intended recipient of this email, yo

Re: Unable to start a VM due to insufficient capacity

2014-09-02 Thread Giri Prasad



CS on Ubuntu 12.04 LTS, 4.1 is working correctly on the same hardware 
(partition1).

This problem is with CS 4.3 on Ubuntu 14.04 LTS (partition2):


2014-09-01 17:21:23,267 INFO  [o.a.c.f.j.i.AsyncJobMonitor] 
(Job-Executor-6:ctx-d1e72afb) Add job-27 into job monitoring
2014-09-01 17:21:23,267 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] 
(Job-Executor-6:ctx-d1e72afb) Executing AsyncJobVO {id:27, userId: 2, 
accountId: 2, instanceType: VirtualMachine, instanceId: 6, cmd: 
org.apache.cloudstack.api.command.user.vm.DeployVMCmd, cmdInfo: 
{"sessionkey":"aa/b/","serviceofferingid":"","cmdEventType":"VM.CREATE","ctxUserId":"2","zoneid":"","httpmethod":"GET","networkids":"","response":"json","id":"6","hypervisor":"KVM","name":"ubuntu-12-04-64bit-admin-Instance","_":"1409572282666","ctxAccountId":"2","diskofferingid":"99218ad6-b601-4dc1-a266-9b0616e7474a","ctxStartEventId":"105","displayname":"ubuntu-12-04-64bit-admin-Instance"},
 cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: 
null, initMsid: 279278805451282, completeMsid: null, lastUpdated: null, 
lastPolled: null, created: null}
2014-09-01 17:21:23,272 DEBUG [c.c.a.ApiDispatcher] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) InfrastructureEntity name 
is:com.cloud.offering.ServiceOffering
2014-09-01 17:21:23,273 DEBUG [c.c.a.ApiDispatcher] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) ControlledEntity name 
is:com.cloud.template.VirtualMachineTemplate
2014-09-01 17:21:23,275 DEBUG [c.c.a.ApiDispatcher] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) ControlledEntity name 
is:com.cloud.network.Network
2014-09-01 17:21:23,277 DEBUG [c.c.a.ApiDispatcher] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) InfrastructureEntity name 
is:com.cloud.offering.DiskOffering
2014-09-01 17:21:23,443 DEBUG [c.c.n.NetworkModelImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Service SecurityGroup is not 
supported in the network id=204
2014-09-01 17:21:23,445 DEBUG [c.c.n.NetworkModelImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Service SecurityGroup is not 
supported in the network id=204
2014-09-01 17:21:23,455 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Deploy avoids pods: [], clusters: 
[], hosts: []
2014-09-01 17:21:23,456 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) DeploymentPlanner allocation 
algorithm: com.cloud.deploy.FirstFitPlanner@7c4a754a
2014-09-01 17:21:23,456 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Trying to allocate a host and 
storage pools from dc:1, pod:null,cluster:null, requested cpu: 1000, requested 
ram: 1073741824
2014-09-01 17:21:23,456 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Is ROOT volume READY (pool already 
allocated)?: No
2014-09-01 17:21:23,456 DEBUG [c.c.d.FirstFitPlanner] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Searching all possible resources 
under this Zone: 1
2014-09-01 17:21:23,457 DEBUG [c.c.d.FirstFitPlanner] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Listing clusters in order of 
aggregate capacity, that have (atleast one host with) enough CPU and RAM 
capacity under this Zone: 1
2014-09-01 17:21:23,459 DEBUG [c.c.d.FirstFitPlanner] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Removing from the clusterId list 
these clusters from avoid set: []
2014-09-01 17:21:23,463 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336) Checking resources in Cluster: 1 
under Pod: 1
2014-09-01 17:21:23,463 DEBUG [c.c.a.m.a.i.FirstFitAllocator] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) Looking for 
hosts in dc: 1  pod:1  cluster:1
2014-09-01 17:21:23,465 DEBUG [c.c.a.m.a.i.FirstFitAllocator] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) 
FirstFitAllocator has 1 hosts to check for allocation: [Host[-1-Routing]]
2014-09-01 17:21:23,466 DEBUG [c.c.a.m.a.i.FirstFitAllocator] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) Found 1 
hosts for allocation after prioritization: [Host[-1-Routing]]
2014-09-01 17:21:23,466 DEBUG [c.c.a.m.a.i.FirstFitAllocator] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) Looking for 
speed=1000Mhz, Ram=1024
2014-09-01 17:21:23,469 DEBUG [c.c.c.CapacityManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) Host: 1 has 
cpu capability (cpu:4, speed:2801) to support requested CPU: 1 and requested 
speed: 1000
2014-09-01 17:21:23,469 DEBUG [c.c.c.CapacityManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitRoutingAllocator) Checking if 
host: 1 has enough capacity for requested CPU: 1000 and requested RAM: 
1073741824 , cpuOverprovisioningFactor: 1.0
2014-09-01 17:21:23,471 DEBUG [c.c.c.CapacityManagerImpl] 
(Job-Executor-6:ctx-d1e72afb ctx-21bde336 FirstFitR

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54255040
  
Hi Rohit,

I changed the db.properties last night and from the 3 builds 1 was 
successful (job 30.1). Concerning the other 2, jobs 30.2 and 30.3, I saw 
problems related to some Marvin tests. .

I had a look and the logs and identify the following:

30.2
=== TestName: test_privategw_acl | Status : EXCEPTION ===
   test/integration/smoke/test_privategw_acl.py
=== TestName: None | Status : EXCEPTION ===
   test/integration/smoke/test_reset_vm_on_reboot.py
=== TestName: None | Status : EXCEPTION ===
   test/integration/smoke/test_routers.py

30.3
=== TestName: test_01_create_service_offering | Status : EXCEPTION ===
   test/integration/smoke/test_service_offerings.py
=== TestName: None | Status : EXCEPTION ===
   test/integration/smoke/test_templates.py
=== TestName: None | Status : EXCEPTION ===
   test/integration/smoke/test_vm_life_cycle.py 
=== TestName: None | Status : EXCEPTION ===
   test/integration/smoke/test_volumes.py
=== TestName: test_vpc_remote_access_vpn | Status : EXCEPTION ===
   test/integration/smoke/test_vpc_vpn.py
=== TestName: test_vpc_site2site_vpn | Status : EXCEPTION ===
   test/integration/smoke/test_vpc_vpn.py



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

GitHub user wilderrodrigues reopened a pull request:

https://github.com/apache/cloudstack/pull/14

Pull request of changes in the "cloud-server" module

In the last 10 weeks we have been working in the cloud-server, focusing our 
time in the refactor of the [Vpc]VirtualNetworkApplianceManagerImpl. We had a 
mains goals increase of Maintainability, Extensibility, Readability and test 
coverage. That was just a first step towards the development, still in 
progress, of the Redundant Virtual Routers for VPC.

== What has been done so far:

â¢ The VirtualNetworkApplianceManagerImpl class line numbers dropped from  
4440 to 2558
â¢ The VpcVirtualNetworkApplianceImpl class line numbers dropped from 1484 
to 749
â¢ We created 35 new classes in order to split the code/responsibility
â¢ We added 97.8% unit test coverage for com.cloud.network.element/router 
and org.cloud.network.router.deployment packages
o   The most complex classes we changed are in those packages
o   About 1700 lines of unit tests
â¢ We started doing some integration tests
o   Deployment of Basic and Advanced Zones
o   Test Create Account and user for that account
o   Test Sub domain allowed to launch VM  when a Domain level zone 
is created
o   Test delete domain without force option
o   Test delete domain with force option
o   Test update admin details
o   Test update domain admin details
o   Test user update API
o   Test login API with domain
o   Test if Login API does not return UUID's
â¢ VPC related tests were still done manually.
o   Will have it automated as well.

We started the changes in the network area, trying to identify the 
differences in the 2 types of network we have. For that we created Basic and 
Advanced Network Topology classes. The network topology classes are responsible 
by invoking the Apply/Setup/Create/Save rules that were previously done by the 
[Vpc]VirtualNetworkAppliance. A topology instance is retrieved via a context 
object that is injected in the [Vpc]VirtualElement. The context object will 
return the most appropriate topology instance based on the Network Type, which 
is defined in the Data Centre. That was the first step towards the refactor.

From the topology class we reach the Rule Applier implementation that will 
be used to do all the rule setup preparation (i.e. invoke DAOs and prepare the 
command object). The RuleApplier interface was extracted from the 
VirtualNetworkApplianceManagerImpl, where it use to be an inner interface. For 
each anonymous implementation of the RuleApplier we created a concrete class. 
The rules are used as elements of a Visitor class, which will perform some 
extra logic, depending on the rule it's visiting, and call the send commands to 
router method. The latter has also been extracted from the 
VirtualNetworkApplianceManagerImpl and is now in a new helper class: 
NetworkHelperImpl.

The visitor has been used because we were aiming to split the 
responsibility and also because the way the RuleApplier was implemented before, 
it was clear that every command sent to the router was following a 2-steps 
approach: gather information to create the commands, apply some logic to send 
to the router. For those reason we implemented the visitor pattern. Since we 
already had the Basic/Advanced Network Topology classes, we created 2 concrete 
classes to visit the rules: Basic/Advanced Network Visitors. Both classes 
extend the abstract class NetworkTopologyVisitor, which defines all the visit 
methods per type of rule. By doing so, we can use the same rule and separate 
the logic based on the type of visitor that we have - Basic or Advanced.

Continuing on the refactor, we also added some helper classes for the 
"getSomething" related methods. Following this approach we ended up having the 
following classes:

â¢ NetworkHelper (interface)
â¢ NetworkHelperImpl
â¢ VpcNetworkHelperImpl
â¢ CommandSetupHelper
â¢ NicProfileHelper
â¢ RouterControlHelper

Last, but not least - and actually the most crucial part of the code - 
there was also a huge refactor in terms of how the routers are deployed. The 
previous deployeRouter and deployVpcInrouter methods do not exist any more. 
Instead of having the logics spread, or sometime tangled, in the 
[Vpc]VirtualNetworkApplianceManagerImpl, we have created a Router Deployment 
Definition mechanism, with classes that follow the same naming convention. The 
deployment definition has 2 implementations, Router and Vpc router, which are 
created with the aid of a Builder class. Most of the work, which is common to 
both implementation, is being done by the RouterDeploymentDefinition class. The 
specific bits are done by their implementation. for example, when 
findOrDeployVir

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues closed the pull request at:

https://github.com/apache/cloudstack/pull/14


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] cloudstack pull request: Pull request of changes in the "cloud-ser...

Github user wilderrodrigues commented on the pull request:

https://github.com/apache/cloudstack/pull/14#issuecomment-54255074
  
Sorry mouse went over the wrong button my mistake and I closed the PR.

Cheers,
Wilder


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: CloudStack Snapshot Question