[IP clearance] merge IP cleared features into master

[Murali Reddy]

I revoked the code for the features 'optional public IP assignment for EIP' [1] 
and 'network services in shared network' [2] features from master due to IP 
clearance issues. Both the features have passed IP clearance window [3][4]. Due 
to multiple refactors that happened at 4.1 code freeze time, patches are 
invalid now. I did not get time to work on new patch and test them out. I would 
like work (its almost rework now) on these features now and merge them back 
into master.

Let me know if there are any concerns.

[1] https://issues.apache.org/jira/browse/CLOUDSTACK-265
[2] https://issues.apache.org/jira/browse/CLOUDSTACK-312
[3] 
http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.mbox/%3ccakprhvzyidegdeu3ptb_dctzjc5xyutsof_p+gexkzhnt_+...@mail.gmail.com%3E
[4] 
http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.mbox/%3ccakprhvyp0xozlwuj7yomboi12a02c+woe0f9omke0k2uuxs...@mail.gmail.com%3E

Thanks,
Murali

Re: Review Request: (CLOUDSTACK-1638) Network plugins won't be notified VM migration.

[Hiroaki Kawai]

> On March 29, 2013, 8 p.m., Chiradeep Vittal wrote:
> > I do think an explicit migration interface on NetworkElement is the right 
> > way to do it. This way, network elements can decide explicitly when and how 
> > to handle this state. 
> > Sprinkling
> >if(!nic.getReservationId().equals(context.getReservationId())){
> > // migration operation
> > return; 
> > }
> > everywhere is error prone:
> > - Implementors of new NetworkElements are not aware of this indirectly 
> > expressed dependency
> > - This snippet of code (except for the comment) does not in any way 
> > indicate the operation.

I agree that introducing a new interface is a good solution. But the kind of 
interface changes seems to happen in the next cloudstack refactoring, so I 
implemented as shown not to change the interface as possible as I can. If we 
add a new interface, we must spread that implementation for that interface to 
every plugins anyway.

If you do want to add a new interface right now, I'll create another patch.


- Hiroaki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9871/#review18531
---


On March 29, 2013, 1:49 a.m., Hiroaki Kawai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9871/
> ---
> 
> (Updated March 29, 2013, 1:49 a.m.)
> 
> 
> Review request for cloudstack, Hugo Trippaers and Chiradeep Vittal.
> 
> 
> Description
> ---
> 
> The location of the virtual machine is provided by DeployDestination, which 
> will be passed in NetworkGuru#reserve and NetworkElement#prepare. 
> 
> During the virtual machine migration, it actually changes DeployDestination 
> and it looks like that it will tell that event to network components as it 
> has NetworkManager#prepareNicForMigration. The problem is that althogh the 
> interface has that method, NetworkManagerImpl does not tell the 
> DeployDestination changes to network components. 
> 
> So IMHO, we need to add calls of NetworkGuru#reserve and 
> NetworkElement#prepare in NetworkManagerImpl#prepareNicForMigration . And 
> then, we also need to add calls NetworkGuru#release and 
> NetworkElement#release after the migration, otherwise the network resources 
> that plugin reserved will be kept even when the vm leaves off.
> 
> Created a first minimum patch to show the concept.
> 
> 
> This addresses bug CLOUDSTACK-1638.
> 
> 
> Diffs
> -
> 
>   docs/en-US/plugin-niciranvp-tables.xml 4f81655 
>   
> plugins/network-elements/nicira-nvp/src/com/cloud/network/NiciraNvpNicMappingVO.java
>  0779e69 
>   
> plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraNvpElement.java
>  1fcccdb 
>   server/src/com/cloud/network/NetworkManager.java 4124b19 
>   server/src/com/cloud/network/NetworkManagerImpl.java a98bdd4 
>   server/src/com/cloud/network/guru/ControlNetworkGuru.java 934cd70 
>   server/src/com/cloud/network/guru/DirectNetworkGuru.java ee824af 
>   server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java 354d7ed 
>   server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java 24d24f8 
>   server/src/com/cloud/network/guru/GuestNetworkGuru.java cebfb08 
>   server/src/com/cloud/network/guru/PodBasedNetworkGuru.java b513325 
>   server/src/com/cloud/network/guru/StorageNetworkGuru.java 879d0cd 
>   server/src/com/cloud/vm/VirtualMachineManagerImpl.java 9230f4a 
>   setup/db/create-schema.sql 5b6dc04 
> 
> Diff: https://reviews.apache.org/r/9871/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Hiroaki Kawai
> 
>

Re: Review Request: CLOUDSTACK-1453: support restore for VM created from ISO

[Koushik Das]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9779/#review18578
---



api/src/org/apache/cloudstack/api/command/user/vm/RestoreVMCmd.java


What it means to restore from new iso? Do we need a new iso parameter?



server/src/com/cloud/vm/UserVmManagerImpl.java


Handling of iso case is not clear. Are you expecting the iso id in the 
newTemplateId parameter?


- Koushik Das


On March 6, 2013, 12:23 p.m., Harikrishna Patnala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9779/
> ---
> 
> (Updated March 6, 2013, 12:23 p.m.)
> 
> 
> Review request for cloudstack, Nitin Mehta and Koushik Das.
> 
> 
> Description
> ---
> 
> CLOUDSTACK-1453: support restore for VM created from ISO
> Previously this was not there. Fixed with small changes
> 
> 
> This addresses bug CLOUDSTACK-1453.
> 
> 
> Diffs
> -
> 
>   api/src/org/apache/cloudstack/api/command/user/vm/RestoreVMCmd.java 9c33f97 
>   server/src/com/cloud/vm/UserVmManagerImpl.java ca9c13f 
> 
> Diff: https://reviews.apache.org/r/9779/diff/
> 
> 
> Testing
> ---
> 
> tested locally
> 
> 
> Thanks,
> 
> Harikrishna Patnala
> 
>

Review Request: debian: adding ipset as part of agent install.

[Pradeep Soundararajan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10216/
---

Review request for cloudstack, Chip Childers, Hugo Trippaers, Wido den 
Hollander, and Noa Resare.


Description
---

debian: adding ipset as part of agent install as per Chip.

Signed-off-by: Pradeep 


Diffs
-

  debian/control 8f82fc3ab2faefa1bc41e45a79abf376c0f2bc2a 

Diff: https://reviews.apache.org/r/10216/diff/


Testing
---

Debian packaging went through.


Thanks,

Pradeep Soundararajan

Re: Review Request: BUG-ID: CLOUDSTACK-1689

[Pradeep Soundararajan]

> On March 29, 2013, 12:46 p.m., ASF Subversion and Git Services wrote:
> > Commit 5a66ec35ad08ff3313e72b54eba7e1754868eee4 in branch refs/heads/bvt 
> > from Prasanna Santhanam 
> > [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=5a66ec3 ]
> > 
> > CLOUDSTACK-1689: KVM agent install.sh should install ipset command
> > 
> > Added ipset in cloud.spec as part of agent install.
> > 
> > Signed-off-by: Pradeep 
> > Signed-off-by: Prasanna Santhanam 
> >

I hope I have added as part of debian agent install. 

Chip, please review this https://reviews.apache.org/r/10216 


- Pradeep


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9946/#review18508
---


On March 29, 2013, 10:41 a.m., Pradeep Soundararajan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9946/
> ---
> 
> (Updated March 29, 2013, 10:41 a.m.)
> 
> 
> Review request for cloudstack, David Nalley, Chip Childers, Hugo Trippaers, 
> and Wido den Hollander.
> 
> 
> Description
> ---
> 
> BUG-ID: CLOUDSTACK-1689
> 
> Summary: KVM agent install.sh should install ipset command
> 
> Added ipset in cloud.spec as part of agent install.
> 
> Signed-off-by: Pradeep 
> 
> 
> Diffs
> -
> 
>   packaging/centos63/cloud.spec e10ae1e 
> 
> Diff: https://reviews.apache.org/r/9946/diff/
> 
> 
> Testing
> ---
> 
> Packaging ipset properly after this change. 
> 
> 
> Thanks,
> 
> Pradeep Soundararajan
> 
>

Re: [ACS41][QA] Unassigned Critical Upgrade defects

[Rohit Yadav]

On Sun, Mar 31, 2013 at 2:47 PM, Sudha Ponnaganti <
sudha.ponnaga...@citrix.com> wrote:

> Hi,
>
> Wanted to check if anyone is interested to pick up the following upgrade
> defects. Once upgrade testing is done, we would get a little closer to 4.1
> RC.
>
> CLOUDSTACK-1839  Upgrade 4.0 -> 4.1 - Upgraded DB has lot more
> keys and indexes for many tables compare to the fresh installed 4.1 DB.
>

Check what happened to create-index-fk.sql, why was it removed and fix
upgrade path from 40 to 410.


> CLOUDSTACK-1856  Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3
> parameters missing in db. Properties compared to an upgraded 4.0 set up.
>

Properties files should be backed up or ask user to replace/keep the file,
this is a packaging issue.

Cheers.


>
> Thanks
> /Sudha
>

Re: cloudmonkey printing enhancements proposal

[Rohit Yadav]

Hi Justin, thanks for your first patch.

As per our workflows, we use git for our version control and have a
reviewboard for reviewing patches and an issue tracker for tracking
bugs/features.

Git:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git

Review board:
https://reviews.apache.org

For filtering only passed key names, we can cut down the whole if-else tree
using filter (which you may see I'd used at several places) and reduce list.

Something like; filter(lamba key: , dictionary.keys()), using this filtered failsafe
list of keys, we can generate dictionary, using a map; map(lambda x:
myjson[x] = jsondictionary[x], failsafekeys) etc.

I can help with this, thanks for your patch. Since it's small I can help
apply that for future please use review board or share git formatted patch
which would apply cleanly on master.

Regards.

On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien  wrote:

> Greetings,
>
> I am posting my first patch. I am not sure exactly how to submit a patch
> but I figure I will paste it here and someone will tell me the right
> method. I am keeping track of the work that I am doing and as Rohit
> suggested I will update the mailing list as I progress. This first patch
> adds basic JSON output with filtering. I have tested it on cloudmonkey
> 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are appreciated.
>
> --- README ---
> Added
> 1. display = [default|json|tabularize] has been added in the config to
> replace
> tabularize = [true|false]
> 2. tabularize is deprecated but we will still set it as "false" once the
> user
> removes it out of their config to avoid throwing an error. This will be
> removed in the next major version.
> 3. display = "default" is added to the [ui] section of the config if it is
> not
> present.
> 4. You can now output JSON formatted text by setting the config display =
> json
> 5. You can now filter text in JSON output mode. (i.e. list users
> account=grudzien filter=account,id,email). Filtered output returns a
> properly formatted JSON document.
>
> Removed
> 1. Removed the printing of attr keys in read_config().
>
> Deprecated
> 1. tabularize = [true|false] is now messaged as deprecated.
>
> ToDo
> 1. Need to format empty return messages with proper JSON messaging.
> 2. Need to standardize JSON output messaging.
> A. Add return code [Error|Success] to all calls.
> B. Add called API verb.
> 3. Count is not decreased in results when filtering completely eliminates a
> result.
> 4. JSON print needs to be implemented manually to support colors. Right now
> json.dumps() pretty prints the output.
> 5. Color tagging needs to be added to JSON printing.
> 6. Error messages need to have proper JSON formatting.
> 7. Various help text has grammar or consistency errors that need fixing.
> 8. Make color a passable option to a command instead of a config parameter.
> (i.e. list users account=grudzien color=true)
> A. This will require reworking the result_filter passable argument to
> the
> various print methods since they only expect filter= input.
> 9. The API in CloudStack 4.0.1 does not all return proper JSON formatted
> text.
> As a result some of the JSON printing looks funny. I will get the later
> versions into a lab and test them to make sure the formatting bugs are
> in
> newer revisions and submit them as bugs.
> --- README ---
>
> --- PATCH ---
> diff -rupN cloudstack/tools/cli/cloudmonkey/cloudmonkey.py
> cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py
> --- cloudstack/tools/cli/cloudmonkey/cloudmonkey.py 2013-03-31
> 16:58:26.0 -0500
> +++ cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py 2013-03-31
> 22:03:38.0 -0500
> @@ -27,6 +27,7 @@ try:
>  import shlex
>  import sys
>  import types
> +import copy
>
>  from cachemaker import loadcache, savecache, monkeycache,
> splitverbsubject
>  from config import __version__, __description__, __projecturl__
> @@ -162,6 +163,44 @@ class CloudMonkeyShell(cmd.Cmd, object):
>  self.monkeyprint(printer)
>  return PrettyTable(toprow)
>
> +# method: print_result_json( result, result_filter )
> +# parameters: result - raw results from the API call
> +# result_filter - filterset
> +# description: prints result as a json object
> +def print_result_json(result, result_filter=None):
> +tfilter = {} # temp var to hold a dict of the filters
> +tresult = copy.deepcopy(result) # dupe the result to filter
> +if result_filter != None:
> +for res in result_filter:
> +tfilter[ res ] = 1
> +myresults = {}
> +for okey, oval in result.iteritems():
> +if isinstance( oval, dict ):
> +for tkey in x:
> +if tkey not in tfilter:
> +  

RE: [ACS41][QA] Unassigned Critical Upgrade defects

[Ram Ganesh]

> -Original Message-
> From: rohityada...@gmail.com [mailto:rohityada...@gmail.com] On Behalf
> Of Rohit Yadav
> Sent: 01 April 2013 13:31
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS41][QA] Unassigned Critical Upgrade defects
> 
> On Sun, Mar 31, 2013 at 2:47 PM, Sudha Ponnaganti <
> sudha.ponnaga...@citrix.com> wrote:
> 
> > Hi,
> >
> > Wanted to check if anyone is interested to pick up the following
> > upgrade defects. Once upgrade testing is done, we would get a little
> > closer to 4.1 RC.
> >
> > CLOUDSTACK-1839  Upgrade 4.0 -> 4.1 - Upgraded DB has lot more
> > keys and indexes for many tables compare to the fresh installed 4.1 DB.
> >
> 
> Check what happened to create-index-fk.sql, why was it removed and fix
> upgrade path from 40 to 410.
> 
> 
> > CLOUDSTACK-1856  Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3
> > parameters missing in db. Properties compared to an upgraded 4.0 set up.
> >
> 
> Properties files should be backed up or ask user to replace/keep the file, 
> this
> is a packaging issue.

I see following commit removing these parameters - 
http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-commits/201212.mbox/%3c20121219220428.9e565324...@tyr.zones.apache.org%3E.
 Are they obsolete now?

> 
> Cheers.
> 
> 
> >
> > Thanks
> > /Sudha
> >

Re: Review Request: Mulitiple ip address per nic support for advacned shared networks

[Jayapal Reddy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10040/
---

(Updated April 1, 2013, 8:32 a.m.)


Review request for cloudstack, Abhinandan Prateek, Murali Reddy, and anthony xu.


Changes
---

Updated for review comments.


Description
---

Changes added for supporting multiple ip address per nic for shared networks


Diffs (updated)
-

  server/src/com/cloud/network/NetworkServiceImpl.java d503459 

Diff: https://reviews.apache.org/r/10040/diff/


Testing
---

1. Tested adding and removing the ip for nic using the APIs


Thanks,

Jayapal Reddy

Re: [ACS41][QA] Unassigned Critical Upgrade defects

[prasanna]

>> > CLOUDSTACK-1856  Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3
>> > parameters missing in db. Properties compared to an upgraded 4.0 set up.
>> >
>>
>> Properties files should be backed up or ask user to replace/keep the file, 
>> this
>> is a packaging issue.
>
> I see following commit removing these parameters - 
> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-commits/201212.mbox/%3c20121219220428.9e565324...@tyr.zones.apache.org%3E.
>  Are they obsolete now?

That's right. Those properties never come into use anywhere within our code.



--
Prasanna.,

RE: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1

[Sailaja Mada]

Hi, 

I tried to upgrade 4.0 Ubuntu Management server to 4.1 with below instructions :

Procedure 1:

1.  Add/Modify the line "deb http://cloudstack.apt-get.eu/ubuntu precise 4.1 " 
in /etc/apt/sources.list.d/cloudstack.list 
2.  apt-get update

Observation: It failed with GPG Error.  

Reading package lists... Done
W: GPG error: http://cloudstack.apt-get.eu precise Release: The following 
signatures couldn't be verified because the public key is not available: 
NO_PUBKEY BBFCFE5386C278E3

I tried to key KEYS from 
http://www.apache.org/dist/dev/incubator/cloudstack/KEYS   as mentioned @ 
http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/Installation_Guide/sect-source-verify.html
  .  
But I am getting 404 error as KEYS are not available at this location. 

Procedure 2:

1. Tried to create APT Repo with packages that are built @ 
http://jenkins.cloudstack.org/job/package-deb-4.1/ and tried apt-get update .  
But there are no packages got upgraded.  Filed the JIRA ticket for this issue 
couple of days back @  https://issues.apache.org/jira/browse/CLOUDSTACK-1848 


Thanks,
Sailaja.M


-Original Message-
From: Wido den Hollander [mailto:w...@widodh.nl] 
Sent: Sunday, March 31, 2013 12:38 AM
To: dev@cloudstack.apache.org
Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1



On 03/30/2013 04:02 PM, Marcus Sorensen wrote:
> One of the things that helped me quite a bit With the CentOS stuff was 
> that I created a VM with agent/management server running in it. I set 
> up a 4.0 system with an advanced zone, pulled down the source code 
> into it, and then took a copy of the VM. Then I ran through the 
> upgrade process 10 or 15 times, testing, making note of what was 
> missing, rolling back and then trying again, slowly building a patch 
> for the cloud.spec. With the database upgrade and everything else that 
> needs to be done, it's really hard to roll back a real environment and 
> try the upgrade again from scratch.
>

I did the same with the Deb packaging, I have a couple of VMs where I test all 
of this.

> At this point upgrading CentOS is just installing the new RPMs along 
> with a few new dependencies, restarting the services, and restarting 
> the system vms so they get upgraded software. Your e-mail has prompted 
> me to want to create a wiki though, in case anything goes wrong, 
> unless upgrade notes will be released somewhere in particular. Just a 
> place to put upgrade tips like "Old configs will be moved from 
> /etc/cloud to /etc/cloud.rpmsave" in case something does need to be 
> moved manually by the admin.
>

The problem is that we are moving files from location to location and we don't 
simply want to do all of that without telling the admin what is going on.

I'll try to do as much as I can in the packages and pre- and post-installation 
files, but I'll leave the "scary work" to the admins.

The big warning should say: BACKUP YOUR DATABASE before continuing.

So yes, the packages should do all the heavy lifting, but some guidance would 
be nice for a lot of users I think.

Wido

> On Sat, Mar 30, 2013 at 6:17 AM, Wido den Hollander  wrote:
>> Hi,
>>
>> I've started a Wiki[0] to describe the upgrade process going from 4.0 
>> to 4.1 with Ubuntu packages.
>>
>> Since the packages got renamed from cloud-* to cloudstack-* a lot has 
>> changed and some manual intervention is required.
>>
>> The Wiki is work-in-progress while I'm upgrading one of my 4.0 
>> clusters to
>> 4.1 and see how it all works out.
>>
>> Feel free to change the Wiki if you find something that needs to be 
>> adressed!
>>
>> Wido
>>
>> [0]:
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ubuntu+upgrade
>> +process
>>

Re: cloudmonkey printing enhancements proposal

[Justin Grudzien]

Thanks for the information. If you haven't already put the patch for review
I would like to do so and follow the directions so I can get used to the
process. Let me know. Also, I saw the lambda functions and chose not to use
them because I think they make the code harder to read for the more novice
python programmer. I can go back and add it if the goal is to have shorter
code but if it were up to me I would have the code more readable. :) I hope
people find this work to be useful, I appreciate the guidance through the
submission process.

Justin


On Mon, Apr 1, 2013 at 3:07 AM, Rohit Yadav  wrote:

> Hi Justin, thanks for your first patch.
>
> As per our workflows, we use git for our version control and have a
> reviewboard for reviewing patches and an issue tracker for tracking
> bugs/features.
>
> Git:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git
>
> Review board:
> https://reviews.apache.org
>
> For filtering only passed key names, we can cut down the whole if-else tree
> using filter (which you may see I'd used at several places) and reduce
> list.
>
> Something like; filter(lamba key:  key in my set of filters>, dictionary.keys()), using this filtered failsafe
> list of keys, we can generate dictionary, using a map; map(lambda x:
> myjson[x] = jsondictionary[x], failsafekeys) etc.
>
> I can help with this, thanks for your patch. Since it's small I can help
> apply that for future please use review board or share git formatted patch
> which would apply cleanly on master.
>
> Regards.
>
> On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien 
> wrote:
>
> > Greetings,
> >
> > I am posting my first patch. I am not sure exactly how to submit a patch
> > but I figure I will paste it here and someone will tell me the right
> > method. I am keeping track of the work that I am doing and as Rohit
> > suggested I will update the mailing list as I progress. This first patch
> > adds basic JSON output with filtering. I have tested it on cloudmonkey
> > 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are appreciated.
> >
> > --- README ---
> > Added
> > 1. display = [default|json|tabularize] has been added in the config to
> > replace
> > tabularize = [true|false]
> > 2. tabularize is deprecated but we will still set it as "false" once the
> > user
> > removes it out of their config to avoid throwing an error. This will
> be
> > removed in the next major version.
> > 3. display = "default" is added to the [ui] section of the config if it
> is
> > not
> > present.
> > 4. You can now output JSON formatted text by setting the config display =
> > json
> > 5. You can now filter text in JSON output mode. (i.e. list users
> > account=grudzien filter=account,id,email). Filtered output returns a
> > properly formatted JSON document.
> >
> > Removed
> > 1. Removed the printing of attr keys in read_config().
> >
> > Deprecated
> > 1. tabularize = [true|false] is now messaged as deprecated.
> >
> > ToDo
> > 1. Need to format empty return messages with proper JSON messaging.
> > 2. Need to standardize JSON output messaging.
> > A. Add return code [Error|Success] to all calls.
> > B. Add called API verb.
> > 3. Count is not decreased in results when filtering completely
> eliminates a
> > result.
> > 4. JSON print needs to be implemented manually to support colors. Right
> now
> > json.dumps() pretty prints the output.
> > 5. Color tagging needs to be added to JSON printing.
> > 6. Error messages need to have proper JSON formatting.
> > 7. Various help text has grammar or consistency errors that need fixing.
> > 8. Make color a passable option to a command instead of a config
> parameter.
> > (i.e. list users account=grudzien color=true)
> > A. This will require reworking the result_filter passable argument to
> > the
> > various print methods since they only expect filter= input.
> > 9. The API in CloudStack 4.0.1 does not all return proper JSON formatted
> > text.
> > As a result some of the JSON printing looks funny. I will get the
> later
> > versions into a lab and test them to make sure the formatting bugs
> are
> > in
> > newer revisions and submit them as bugs.
> > --- README ---
> >
> > --- PATCH ---
> > diff -rupN cloudstack/tools/cli/cloudmonkey/cloudmonkey.py
> > cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py
> > --- cloudstack/tools/cli/cloudmonkey/cloudmonkey.py 2013-03-31
> > 16:58:26.0 -0500
> > +++ cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py 2013-03-31
> > 22:03:38.0 -0500
> > @@ -27,6 +27,7 @@ try:
> >  import shlex
> >  import sys
> >  import types
> > +import copy
> >
> >  from cachemaker import loadcache, savecache, monkeycache,
> > splitverbsubject
> >  from config import __version__, __description__, __projecturl__
> > @@ -162,6 +163,44 @@ class CloudMonkeyShell(cmd.Cmd, object):
> >  self.monkeyprint(printer)
> >  return Pretty

Re: Review Request: Mulitiple ip address per nic support for advacned shared networks

[Murali Reddy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10040/#review18583
---

Ship it!


submitted with 8dd0b7747b2da1aa71110e871b0bd4ffdbcdfcb4

- Murali Reddy


On April 1, 2013, 8:32 a.m., Jayapal Reddy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10040/
> ---
> 
> (Updated April 1, 2013, 8:32 a.m.)
> 
> 
> Review request for cloudstack, Abhinandan Prateek, Murali Reddy, and anthony 
> xu.
> 
> 
> Description
> ---
> 
> Changes added for supporting multiple ip address per nic for shared networks
> 
> 
> Diffs
> -
> 
>   server/src/com/cloud/network/NetworkServiceImpl.java d503459 
> 
> Diff: https://reviews.apache.org/r/10040/diff/
> 
> 
> Testing
> ---
> 
> 1. Tested adding and removing the ip for nic using the APIs
> 
> 
> Thanks,
> 
> Jayapal Reddy
> 
>

Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1

[prasanna]

The keys are found here : http://www.apache.org/dist/incubator/cloudstack/KEYS

On 1 April 2013 16:13, Sailaja Mada  wrote:
> Hi,
>
> I tried to upgrade 4.0 Ubuntu Management server to 4.1 with below 
> instructions :
>
> Procedure 1:
>
> 1.  Add/Modify the line "deb http://cloudstack.apt-get.eu/ubuntu precise 4.1 
> " in /etc/apt/sources.list.d/cloudstack.list
> 2.  apt-get update
>
> Observation: It failed with GPG Error.
>
> Reading package lists... Done
> W: GPG error: http://cloudstack.apt-get.eu precise Release: The following 
> signatures couldn't be verified because the public key is not available: 
> NO_PUBKEY BBFCFE5386C278E3
>
> I tried to key KEYS from 
> http://www.apache.org/dist/dev/incubator/cloudstack/KEYS   as mentioned @ 
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/Installation_Guide/sect-source-verify.html
>   .
> But I am getting 404 error as KEYS are not available at this location.
>
> Procedure 2:
>
> 1. Tried to create APT Repo with packages that are built @ 
> http://jenkins.cloudstack.org/job/package-deb-4.1/ and tried apt-get update . 
>  But there are no packages got upgraded.  Filed the JIRA ticket for this 
> issue couple of days back @  
> https://issues.apache.org/jira/browse/CLOUDSTACK-1848
>
>
> Thanks,
> Sailaja.M
>
>
> -Original Message-
> From: Wido den Hollander [mailto:w...@widodh.nl]
> Sent: Sunday, March 31, 2013 12:38 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1
>
>
>
> On 03/30/2013 04:02 PM, Marcus Sorensen wrote:
>> One of the things that helped me quite a bit With the CentOS stuff was
>> that I created a VM with agent/management server running in it. I set
>> up a 4.0 system with an advanced zone, pulled down the source code
>> into it, and then took a copy of the VM. Then I ran through the
>> upgrade process 10 or 15 times, testing, making note of what was
>> missing, rolling back and then trying again, slowly building a patch
>> for the cloud.spec. With the database upgrade and everything else that
>> needs to be done, it's really hard to roll back a real environment and
>> try the upgrade again from scratch.
>>
>
> I did the same with the Deb packaging, I have a couple of VMs where I test 
> all of this.
>
>> At this point upgrading CentOS is just installing the new RPMs along
>> with a few new dependencies, restarting the services, and restarting
>> the system vms so they get upgraded software. Your e-mail has prompted
>> me to want to create a wiki though, in case anything goes wrong,
>> unless upgrade notes will be released somewhere in particular. Just a
>> place to put upgrade tips like "Old configs will be moved from
>> /etc/cloud to /etc/cloud.rpmsave" in case something does need to be
>> moved manually by the admin.
>>
>
> The problem is that we are moving files from location to location and we 
> don't simply want to do all of that without telling the admin what is going 
> on.
>
> I'll try to do as much as I can in the packages and pre- and 
> post-installation files, but I'll leave the "scary work" to the admins.
>
> The big warning should say: BACKUP YOUR DATABASE before continuing.
>
> So yes, the packages should do all the heavy lifting, but some guidance would 
> be nice for a lot of users I think.
>
> Wido
>
>> On Sat, Mar 30, 2013 at 6:17 AM, Wido den Hollander  wrote:
>>> Hi,
>>>
>>> I've started a Wiki[0] to describe the upgrade process going from 4.0
>>> to 4.1 with Ubuntu packages.
>>>
>>> Since the packages got renamed from cloud-* to cloudstack-* a lot has
>>> changed and some manual intervention is required.
>>>
>>> The Wiki is work-in-progress while I'm upgrading one of my 4.0
>>> clusters to
>>> 4.1 and see how it all works out.
>>>
>>> Feel free to change the Wiki if you find something that needs to be
>>> adressed!
>>>
>>> Wido
>>>
>>> [0]:
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ubuntu+upgrade
>>> +process
>>>



-- 
Prasanna.,

Re: [ACS41][QA] Unassigned Critical Upgrade defects

[Wido den Hollander]

On 04/01/2013 10:00 AM, Rohit Yadav wrote:

On Sun, Mar 31, 2013 at 2:47 PM, Sudha Ponnaganti <
sudha.ponnaga...@citrix.com> wrote:


Hi,

Wanted to check if anyone is interested to pick up the following upgrade
defects. Once upgrade testing is done, we would get a little closer to 4.1
RC.

CLOUDSTACK-1839  Upgrade 4.0 -> 4.1 - Upgraded DB has lot more
keys and indexes for many tables compare to the fresh installed 4.1 DB.



Check what happened to create-index-fk.sql, why was it removed and fix
upgrade path from 40 to 410.



CLOUDSTACK-1856  Upgrade 4.0 -> 4.1 - Fresh install of 4.1 has 3
parameters missing in db. Properties compared to an upgraded 4.0 set up.



Properties files should be backed up or ask user to replace/keep the file,
this is a packaging issue.



I fixed those things yesterday (I hope/think) with this commit: 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=11892d34d7ec0e37e3b5cf568a2a30d4be23590f


See the changes in cloudstack-agent.postinst and 
cloudstack-management.postinst


Wido


Cheers.




Thanks
/Sudha

Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1

[Wido den Hollander]

On 04/01/2013 12:43 PM, Sailaja Mada wrote:

Hi,

I tried to upgrade 4.0 Ubuntu Management server to 4.1 with below instructions :

Procedure 1:

1.  Add/Modify the line "deb http://cloudstack.apt-get.eu/ubuntu precise 4.1 " 
in /etc/apt/sources.list.d/cloudstack.list
2.  apt-get update

Observation: It failed with GPG Error.

Reading package lists... Done
W: GPG error: http://cloudstack.apt-get.eu precise Release: The following 
signatures couldn't be verified because the public key is not available: 
NO_PUBKEY BBFCFE5386C278E3

I tried to key KEYS from 
http://www.apache.org/dist/dev/incubator/cloudstack/KEYS   as mentioned @ 
http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/Installation_Guide/sect-source-verify.html
  .
But I am getting 404 error as KEYS are not available at this location.



The key is available here: http://cloudstack.apt-get.eu/release.asc

wget -O - http://cloudstack.apt-get.eu/release.asc|sudo apt-key add -

It's not an official Apache mirror, so it's not using any Apache keys.


Procedure 2:

1. Tried to create APT Repo with packages that are built @ 
http://jenkins.cloudstack.org/job/package-deb-4.1/ and tried apt-get update .  
But there are no packages got upgraded.  Filed the JIRA ticket for this issue 
couple of days back @  https://issues.apache.org/jira/browse/CLOUDSTACK-1848



Please read the Wiki again, since the packages have been renamed you 
need to "apt-get install" the new packages.


apt-get update only retrieves the new package lists from the server.

Wido



Thanks,
Sailaja.M


-Original Message-
From: Wido den Hollander [mailto:w...@widodh.nl]
Sent: Sunday, March 31, 2013 12:38 AM
To: dev@cloudstack.apache.org
Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1



On 03/30/2013 04:02 PM, Marcus Sorensen wrote:

One of the things that helped me quite a bit With the CentOS stuff was
that I created a VM with agent/management server running in it. I set
up a 4.0 system with an advanced zone, pulled down the source code
into it, and then took a copy of the VM. Then I ran through the
upgrade process 10 or 15 times, testing, making note of what was
missing, rolling back and then trying again, slowly building a patch
for the cloud.spec. With the database upgrade and everything else that
needs to be done, it's really hard to roll back a real environment and
try the upgrade again from scratch.



I did the same with the Deb packaging, I have a couple of VMs where I test all 
of this.


At this point upgrading CentOS is just installing the new RPMs along
with a few new dependencies, restarting the services, and restarting
the system vms so they get upgraded software. Your e-mail has prompted
me to want to create a wiki though, in case anything goes wrong,
unless upgrade notes will be released somewhere in particular. Just a
place to put upgrade tips like "Old configs will be moved from
/etc/cloud to /etc/cloud.rpmsave" in case something does need to be
moved manually by the admin.



The problem is that we are moving files from location to location and we don't 
simply want to do all of that without telling the admin what is going on.

I'll try to do as much as I can in the packages and pre- and post-installation files, but 
I'll leave the "scary work" to the admins.

The big warning should say: BACKUP YOUR DATABASE before continuing.

So yes, the packages should do all the heavy lifting, but some guidance would 
be nice for a lot of users I think.

Wido


On Sat, Mar 30, 2013 at 6:17 AM, Wido den Hollander  wrote:

Hi,

I've started a Wiki[0] to describe the upgrade process going from 4.0
to 4.1 with Ubuntu packages.

Since the packages got renamed from cloud-* to cloudstack-* a lot has
changed and some manual intervention is required.

The Wiki is work-in-progress while I'm upgrading one of my 4.0
clusters to
4.1 and see how it all works out.

Feel free to change the Wiki if you find something that needs to be
adressed!

Wido

[0]:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ubuntu+upgrade
+process

RE: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1

[Sailaja Mada]

Hi,

Retried with latest packages and with change in steps . Now 4.1 deb packages 
got installed.

1 wget http://www.apache.org/dist/incubator/cloudstack/KEYS
2) gpg --import KEYS
3)  apt-get install cloudstack*

But it  failed  while connecting to DB and upgrade is not initiated . 

Thanks,
Sailaja.M


-Original Message-
From: Wido den Hollander [mailto:w...@widodh.nl] 
Sent: Monday, April 01, 2013 4:38 PM
To: dev@cloudstack.apache.org
Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1



On 04/01/2013 12:43 PM, Sailaja Mada wrote:
> Hi,
>
> I tried to upgrade 4.0 Ubuntu Management server to 4.1 with below 
> instructions :
>
> Procedure 1:
>
> 1.  Add/Modify the line "deb http://cloudstack.apt-get.eu/ubuntu 
> precise 4.1 " in /etc/apt/sources.list.d/cloudstack.list
> 2.  apt-get update
>
> Observation: It failed with GPG Error.
>
> Reading package lists... Done
> W: GPG error: http://cloudstack.apt-get.eu precise Release: The 
> following signatures couldn't be verified because the public key is 
> not available: NO_PUBKEY BBFCFE5386C278E3
>
> I tried to key KEYS from 
> http://www.apache.org/dist/dev/incubator/cloudstack/KEYS   as mentioned @ 
> http://cloudstack.apache.org/docs/en-US/Apache_CloudStack/4.0.1-incubating/html/Installation_Guide/sect-source-verify.html
>   .
> But I am getting 404 error as KEYS are not available at this location.
>

The key is available here: http://cloudstack.apt-get.eu/release.asc

wget -O - http://cloudstack.apt-get.eu/release.asc|sudo apt-key add -

It's not an official Apache mirror, so it's not using any Apache keys.

> Procedure 2:
>
> 1. Tried to create APT Repo with packages that are built @ 
> http://jenkins.cloudstack.org/job/package-deb-4.1/ and tried apt-get 
> update .  But there are no packages got upgraded.  Filed the JIRA 
> ticket for this issue couple of days back @  
> https://issues.apache.org/jira/browse/CLOUDSTACK-1848
>

Please read the Wiki again, since the packages have been renamed you need to 
"apt-get install" the new packages.

apt-get update only retrieves the new package lists from the server.

Wido

>
> Thanks,
> Sailaja.M
>
>
> -Original Message-
> From: Wido den Hollander [mailto:w...@widodh.nl]
> Sent: Sunday, March 31, 2013 12:38 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [ACS41] Ubuntu upgrade process from 4.0.X to 4.1
>
>
>
> On 03/30/2013 04:02 PM, Marcus Sorensen wrote:
>> One of the things that helped me quite a bit With the CentOS stuff 
>> was that I created a VM with agent/management server running in it. I 
>> set up a 4.0 system with an advanced zone, pulled down the source 
>> code into it, and then took a copy of the VM. Then I ran through the 
>> upgrade process 10 or 15 times, testing, making note of what was 
>> missing, rolling back and then trying again, slowly building a patch 
>> for the cloud.spec. With the database upgrade and everything else 
>> that needs to be done, it's really hard to roll back a real 
>> environment and try the upgrade again from scratch.
>>
>
> I did the same with the Deb packaging, I have a couple of VMs where I test 
> all of this.
>
>> At this point upgrading CentOS is just installing the new RPMs along 
>> with a few new dependencies, restarting the services, and restarting 
>> the system vms so they get upgraded software. Your e-mail has 
>> prompted me to want to create a wiki though, in case anything goes 
>> wrong, unless upgrade notes will be released somewhere in particular. 
>> Just a place to put upgrade tips like "Old configs will be moved from 
>> /etc/cloud to /etc/cloud.rpmsave" in case something does need to be 
>> moved manually by the admin.
>>
>
> The problem is that we are moving files from location to location and we 
> don't simply want to do all of that without telling the admin what is going 
> on.
>
> I'll try to do as much as I can in the packages and pre- and 
> post-installation files, but I'll leave the "scary work" to the admins.
>
> The big warning should say: BACKUP YOUR DATABASE before continuing.
>
> So yes, the packages should do all the heavy lifting, but some guidance would 
> be nice for a lot of users I think.
>
> Wido
>
>> On Sat, Mar 30, 2013 at 6:17 AM, Wido den Hollander  wrote:
>>> Hi,
>>>
>>> I've started a Wiki[0] to describe the upgrade process going from 
>>> 4.0 to 4.1 with Ubuntu packages.
>>>
>>> Since the packages got renamed from cloud-* to cloudstack-* a lot 
>>> has changed and some manual intervention is required.
>>>
>>> The Wiki is work-in-progress while I'm upgrading one of my 4.0 
>>> clusters to
>>> 4.1 and see how it all works out.
>>>
>>> Feel free to change the Wiki if you find something that needs to be 
>>> adressed!
>>>
>>> Wido
>>>
>>> [0]:
>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Ubuntu+upgrad
>>> e
>>> +process
>>>

Re: [IP clearance] merge IP cleared features into master

[Chip Childers]

On Mon, Apr 01, 2013 at 07:00:07AM +, Murali Reddy wrote:
> I revoked the code for the features 'optional public IP assignment for EIP' 
> [1] and 'network services in shared network' [2] features from master due to 
> IP clearance issues. Both the features have passed IP clearance window 
> [3][4]. Due to multiple refactors that happened at 4.1 code freeze time, 
> patches are invalid now. I did not get time to work on new patch and test 
> them out. I would like work (its almost rework now) on these features now and 
> merge them back into master.
> 
> Let me know if there are any concerns.

+1

I'd suggest creating a feature branch, and then requesting a merge once
you have things sorted out (WRT the refactoring that forces changes to
the previously approved patches).

> 
> [1] https://issues.apache.org/jira/browse/CLOUDSTACK-265
> [2] https://issues.apache.org/jira/browse/CLOUDSTACK-312
> [3] 
> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.mbox/%3ccakprhvzyidegdeu3ptb_dctzjc5xyutsof_p+gexkzhnt_+...@mail.gmail.com%3E
> [4] 
> http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.mbox/%3ccakprhvyp0xozlwuj7yomboi12a02c+woe0f9omke0k2uuxs...@mail.gmail.com%3E
> 
> Thanks,
> Murali

Re: cloudmonkey printing enhancements proposal

[Chip Childers]

On Mon, Apr 01, 2013 at 05:56:11AM -0500, Justin Grudzien wrote:
> Thanks for the information. If you haven't already put the patch for review
> I would like to do so and follow the directions so I can get used to the
> process. Let me know. 

Go for it Justin.

Also, give me your Jira ID once you have one, so that I can give you
karma to assign the "improvement" bug to yourself (and resolve it once
the patch is committed).

> Also, I saw the lambda functions and chose not to use
> them because I think they make the code harder to read for the more novice
> python programmer. I can go back and add it if the goal is to have shorter
> code but if it were up to me I would have the code more readable. :) I hope
> people find this work to be useful, I appreciate the guidance through the
> submission process.
> 
> Justin
> 
> 
> On Mon, Apr 1, 2013 at 3:07 AM, Rohit Yadav  wrote:
> 
> > Hi Justin, thanks for your first patch.
> >
> > As per our workflows, we use git for our version control and have a
> > reviewboard for reviewing patches and an issue tracker for tracking
> > bugs/features.
> >
> > Git:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git
> >
> > Review board:
> > https://reviews.apache.org
> >
> > For filtering only passed key names, we can cut down the whole if-else tree
> > using filter (which you may see I'd used at several places) and reduce
> > list.
> >
> > Something like; filter(lamba key:  > key in my set of filters>, dictionary.keys()), using this filtered failsafe
> > list of keys, we can generate dictionary, using a map; map(lambda x:
> > myjson[x] = jsondictionary[x], failsafekeys) etc.
> >
> > I can help with this, thanks for your patch. Since it's small I can help
> > apply that for future please use review board or share git formatted patch
> > which would apply cleanly on master.
> >
> > Regards.
> >
> > On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien 
> > wrote:
> >
> > > Greetings,
> > >
> > > I am posting my first patch. I am not sure exactly how to submit a patch
> > > but I figure I will paste it here and someone will tell me the right
> > > method. I am keeping track of the work that I am doing and as Rohit
> > > suggested I will update the mailing list as I progress. This first patch
> > > adds basic JSON output with filtering. I have tested it on cloudmonkey
> > > 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are appreciated.
> > >
> > > --- README ---
> > > Added
> > > 1. display = [default|json|tabularize] has been added in the config to
> > > replace
> > > tabularize = [true|false]
> > > 2. tabularize is deprecated but we will still set it as "false" once the
> > > user
> > > removes it out of their config to avoid throwing an error. This will
> > be
> > > removed in the next major version.
> > > 3. display = "default" is added to the [ui] section of the config if it
> > is
> > > not
> > > present.
> > > 4. You can now output JSON formatted text by setting the config display =
> > > json
> > > 5. You can now filter text in JSON output mode. (i.e. list users
> > > account=grudzien filter=account,id,email). Filtered output returns a
> > > properly formatted JSON document.
> > >
> > > Removed
> > > 1. Removed the printing of attr keys in read_config().
> > >
> > > Deprecated
> > > 1. tabularize = [true|false] is now messaged as deprecated.
> > >
> > > ToDo
> > > 1. Need to format empty return messages with proper JSON messaging.
> > > 2. Need to standardize JSON output messaging.
> > > A. Add return code [Error|Success] to all calls.
> > > B. Add called API verb.
> > > 3. Count is not decreased in results when filtering completely
> > eliminates a
> > > result.
> > > 4. JSON print needs to be implemented manually to support colors. Right
> > now
> > > json.dumps() pretty prints the output.
> > > 5. Color tagging needs to be added to JSON printing.
> > > 6. Error messages need to have proper JSON formatting.
> > > 7. Various help text has grammar or consistency errors that need fixing.
> > > 8. Make color a passable option to a command instead of a config
> > parameter.
> > > (i.e. list users account=grudzien color=true)
> > > A. This will require reworking the result_filter passable argument to
> > > the
> > > various print methods since they only expect filter= input.
> > > 9. The API in CloudStack 4.0.1 does not all return proper JSON formatted
> > > text.
> > > As a result some of the JSON printing looks funny. I will get the
> > later
> > > versions into a lab and test them to make sure the formatting bugs
> > are
> > > in
> > > newer revisions and submit them as bugs.
> > > --- README ---
> > >
> > > --- PATCH ---
> > > diff -rupN cloudstack/tools/cli/cloudmonkey/cloudmonkey.py
> > > cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py
> > > --- cloudstack/tools/cli/cloudmonkey/cloudmonkey.py 2013-03-31
> > > 16:58:26.0 -0500
> > > +++ cloudstackmodified/tools/cli/clo

[IP Reservation] Is VM creation using the reserved IPs on cloudstack managed hosts supported?

[Abhinav Roy]

Hi,

Is creation of VMs using the reserved IP (after doing IP reservation) on 
cloudstack managed hosts supported? Because when I tried to do that the VM got 
destroyed as soon as it was created.
While on hosts which are not managed by cloudstack VM creation using reserved 
IPs went through.



Thanks and regards,
Abhinav

Re: VM not obtaing DHCP address when migrating to another host.

[Chip Childers]

On Fri, Mar 29, 2013 at 12:44:54PM -0700, Chiradeep Vittal wrote:
> 
> 
> On 3/29/13 7:04 AM, "Sean Truman"  wrote:
> 
> >All,
> >
> >When I use the migrate to another host option for a running VM (Basic
> >networking), the VM refuses to get a DHCP address?  Is this a known issue?
> >or is something wrong with my hosts? is there something I can check? When
> >I
> >migrate it back to the original host it obtains an IP address.
> >
> >v/r
> >Sean
> 
> Which hypervisor? DHCP is only triggered at VM boot time, so I don't see
> what it has to do with migration (which can happen well after boot time).
> Are you migrating during boot?
> 
>

I believe that Sean solved this (based on IRC conversation)...
something to do with a network cable being unplugged. ;-P

-chip

Re: [ACS41][PACKAGING] Fixes from master into 4.1

[Chip Childers]

On Sun, Mar 31, 2013 at 05:25:09PM +0200, Wido den Hollander wrote:
> Hello,
> 
> I've been working on the Deb packaging today and I've committed some
> stuff into the 4.1 branch as well.
> 
> All fixes first went into master and I did one big commit
> (11892d34d7ec0e37e3b5cf568a2a30d4be23590f) into 4.1 and
> cherry-picked some other.
> 
> Since this doesn't touch any code of CloudStack I felt safe to do so
> without going through the review process first.
> 
> Wido
>

ACK - thanks Wido

Re: Review Request: BUG-ID: CLOUDSTACK-1689

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9946/#review18585
---


Commit 9eed6586092d4ecd268d403f1fec83ad196569b4 in branch refs/heads/4.1 from 
Chip Childers 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=9eed658 ]

CLOUDSTACK-1689: Adding ipset as part of agent install for debian
Signed-off-by: Chip Childers 


- ASF Subversion and Git Services


On March 29, 2013, 10:41 a.m., Pradeep Soundararajan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9946/
> ---
> 
> (Updated March 29, 2013, 10:41 a.m.)
> 
> 
> Review request for cloudstack, David Nalley, Chip Childers, Hugo Trippaers, 
> and Wido den Hollander.
> 
> 
> Description
> ---
> 
> BUG-ID: CLOUDSTACK-1689
> 
> Summary: KVM agent install.sh should install ipset command
> 
> Added ipset in cloud.spec as part of agent install.
> 
> Signed-off-by: Pradeep 
> 
> 
> Diffs
> -
> 
>   packaging/centos63/cloud.spec e10ae1e 
> 
> Diff: https://reviews.apache.org/r/9946/diff/
> 
> 
> Testing
> ---
> 
> Packaging ipset properly after this change. 
> 
> 
> Thanks,
> 
> Pradeep Soundararajan
> 
>

Re: Review Request: BUG-ID: CLOUDSTACK-1689

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9946/#review18586
---


Commit 6eac4229434b46ee74b5952d3abd26b646166b39 in branch refs/heads/master 
from Chip Childers 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=6eac422 ]

CLOUDSTACK-1689: Adding ipset as part of agent install for debian
Signed-off-by: Chip Childers 


- ASF Subversion and Git Services


On March 29, 2013, 10:41 a.m., Pradeep Soundararajan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9946/
> ---
> 
> (Updated March 29, 2013, 10:41 a.m.)
> 
> 
> Review request for cloudstack, David Nalley, Chip Childers, Hugo Trippaers, 
> and Wido den Hollander.
> 
> 
> Description
> ---
> 
> BUG-ID: CLOUDSTACK-1689
> 
> Summary: KVM agent install.sh should install ipset command
> 
> Added ipset in cloud.spec as part of agent install.
> 
> Signed-off-by: Pradeep 
> 
> 
> Diffs
> -
> 
>   packaging/centos63/cloud.spec e10ae1e 
> 
> Diff: https://reviews.apache.org/r/9946/diff/
> 
> 
> Testing
> ---
> 
> Packaging ipset properly after this change. 
> 
> 
> Thanks,
> 
> Pradeep Soundararajan
> 
>

Re: Review Request: debian: adding ipset as part of agent install.

[Chip Childers]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10216/#review18587
---

Ship it!


applied to 4.1 and master

- Chip Childers


On April 1, 2013, 7:49 a.m., Pradeep Soundararajan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10216/
> ---
> 
> (Updated April 1, 2013, 7:49 a.m.)
> 
> 
> Review request for cloudstack, Chip Childers, Hugo Trippaers, Wido den 
> Hollander, and Noa Resare.
> 
> 
> Description
> ---
> 
> debian: adding ipset as part of agent install as per Chip.
> 
> Signed-off-by: Pradeep 
> 
> 
> Diffs
> -
> 
>   debian/control 8f82fc3ab2faefa1bc41e45a79abf376c0f2bc2a 
> 
> Diff: https://reviews.apache.org/r/10216/diff/
> 
> 
> Testing
> ---
> 
> Debian packaging went through.
> 
> 
> Thanks,
> 
> Pradeep Soundararajan
> 
>

Re: Review Request: BUG-ID: CLOUDSTACK-1689

[Chip Childers]

On Mon, Apr 01, 2013 at 07:51:20AM +, Pradeep Soundararajan wrote:
> 
> 
> > On March 29, 2013, 12:46 p.m., ASF Subversion and Git Services wrote:
> > > Commit 5a66ec35ad08ff3313e72b54eba7e1754868eee4 in branch refs/heads/bvt 
> > > from Prasanna Santhanam 
> > > [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=5a66ec3 ]
> > > 
> > > CLOUDSTACK-1689: KVM agent install.sh should install ipset command
> > > 
> > > Added ipset in cloud.spec as part of agent install.
> > > 
> > > Signed-off-by: Pradeep 
> > > Signed-off-by: Prasanna Santhanam 
> > >
> 
> I hope I have added as part of debian agent install. 
> 
> Chip, please review this https://reviews.apache.org/r/10216 
> 
> 
> - Pradeep


ACK - and done.

Re: [PROPOSAL] EIP across zones

[David Nalley]

On Mon, Apr 1, 2013 at 2:11 AM, Murali Reddy  wrote:
> On 01/04/13 1:47 AM, "David Nalley"  wrote:
>
>>On Fri, Mar 29, 2013 at 2:04 AM, Murali Reddy 
>>wrote:
>>> On 26/03/13 8:10 PM, "Adam Grochowski" 
>>>wrote:
>>>
Hi,
So I'm curious - what is the proposed method to move a single IP
across zones (presuming these zones span regions).  As Chiradeep
mentioned earlier, /24 is the largest block that will be advertised,
so it would necessitate moving (or advertising in two locations, and
back hauling back to one)  an entire class C to accomplish this (or
some black magic like LISP).
>>>
>>> Adam, there is no proposed method to move IP across the zones. Having a
>>> native capability in CloudStack to move IP across the zones does not
>>>make
>>> sense. We will be just assuming CloudStack will have control to many
>>> infrastructure components (orchestrate route advertisements etc).
>>>Instead
>>> what is being proposed is, let the CloudStack raise an event when user
>>> moves IP across the zones, then let admin act up on the event and
>>>initiate
>>> the IP availability from new zone. There is no assumption or proposed
>>> method when and how admin to achieve this. Operator can opt for any
>>> solution that works in distributed data centre setup.
>>
>>
>>You don't have a proposed method for moving IPs between zones?
>>Is this all a theoretical solution?
>
> No, its not theoretical solution. Point I am trying to make is, CloudStack
> need not have a native capability to move IP across zone. From the
> CloudStack core perspective, all we need is abstraction of moving IP
> (presented as NAT) across the zones. Then we can have specific
> intelligence in the plug-ins which are providing EIP service. For
> e.g.'Route Health Injection' is commonly used solution in distributed data
> centres for disaster recovery supported by multiple vendors [1][2][3][4].
> My initial plan was to enhance NetScaler plug-in to integrate with
> NetScaler's RHI capability. But even to have such capability in plug-in
> means CloudStack will have to make several assumptions on operational
> aspects. So its better to off-load responsibility of IP advertisement to
> the operators. Let me know if you still have any confusion. May be I will
> add a reference deployment model/s once I progress with feature and test
> it.
>

Ahh - very helpful - thank you.
A reference deployment model would be nice as well.

--David

Re: [DISCUSS] Palo Alto Integration

[Will Stevens]

Thank you for all your help Murali...

So my Provider has been setup with isExternal = true this whole time.
public static final Provider PaloAlto = new Provider("PaloAlto", true);

If I run a debugger and then create a guest network, I see it enter the
'design' function of the ExternalGuestNetworkGuru, but it does not do
anything in there because the config is not null, but the config.getId() =
-1, so it just returns the config (Network object) and doesn't really do
anything.

Apparently the 'implement' method doesn't get called until a VM is
attempted to be launched on the network.

I must be missing something because, every Isolated guest network I create
on my provider is defaulting to the cidr of 10.1.1.0/24.  Even if I have
multiple Isolated networks associated with the same account, they all by
default have that cidr.

If the default behaviour of the ExternalGuestNetworkGuru is to create
non-overlapping guest cidrs, why does it always default to the
10.1.1.0/24cidr when I create a new network?  I can not specify a
gateway or netmask
because it is an external network (as you can see from the included
screenshot).
[image: Inline image 1]

What am I missing here?  Why am I unable to create non-overlapping cidrs
with the ExternalGuestNetworkGuru?

Thanks,

Will


On Fri, Mar 29, 2013 at 1:23 AM, Murali Reddy wrote:

> On 28/03/13 10:59 PM, "Will Stevens"  wrote:
>
> >I am trying to implement the non-overlapping cidrs right now and I have
> >some questions.  Does the ExternalGuestNetworkGuru create networks with
> >non-overlapping cidrs by default?  Or do I need to override it's 'design'
> >and 'implement' methods to implement non overlapping cidrs?
>
> Will, yes, it does by default. You can just use
> 'ExternalGuestNetworkGuru'. Just so that you know, there is check
> 'networkIsConfiguredForExternalNetworking' in ExternalGuestNetworkGuru.
> Which basically checks if provider is configured as service provider using
> external physical appliances. So when you declare provider, mark
> 'isExternal' as true in the provider constructor.
>
> >
> >If I have to write my own methods, I think I understand how to
> >override ExternalGuestNetworkGuru and then get it to run by adding it to
> >the components.xml (or nonoss-components.xml) as well as the
> >componentContext.xml.in.
> >
> >If I do not have to actually write the logic for the non-overlapping cidrs
> >(which i am hoping is the case), and the ExternalGuestNetworkGuru actually
> >implements that logic, how would I get the ExternalGuestNetworkGuru into
> >my
> >flow without actually overriding the class?  I understand that the
> >components are loaded through the components.xml stuff, but its not clear
> >how you specify which NetworkGuru should be used in my specific flow.
> >
> >I am basically working from this
> >document<
> https://cwiki.apache.org/CLOUDSTACK/extending-cloudstack-networki
> >ng.html>and
> >the code.  Is there any other resources I should be aware of for
> >extending the CloudStack networking functionality?
> >
> >I have a good start on a Resource, ExternalFirewallElement and an
> >ExternalFirewallService.  I can currently set the Palo Alto as the
> >provider
> >of Firewall, SourceNat, StaticNat and Port Forwarding services.  I can
> >currently Add, List, Configure and Delete my Palo Alto provider.
> >
> >I am getting there, but I still feel like there are gaps in my knowledge
> >when using the CS networking plugin functionality.
>
> Good the hear the progress. Feel free to ask any question.
>
> Thanks,
> Murali
>
> >
> >Thanks,
> >
> >Will
>
>

Incubator PMC/Board report for Apr 2013 ([ppmc])

[Marvin]

Dear podling,

This email was sent by an automated system on behalf of the Apache Incubator 
PMC.
It is an initial reminder to give you plenty of time to prepare your quarterly
board report.

The board meeting is scheduled for Wed, 17 April 2013, 10:30:00:00 PST. The 
report 
for your podling will form a part of the Incubator PMC report. The Incubator 
PMC 
requires your report to be submitted 2 weeks before the board meeting, to allow 
sufficient time for review and submission (Wed, Apr 3rd).

Please submit your report with sufficient time to allow the incubator PMC, and 
subsequently board members to review and digest. Again, the very latest you 
should submit your report is 2 weeks prior to the board meeting.

Thanks,

The Apache Incubator PMC

Submitting your Report
--

Your report should contain the following:

 * Your project name
 * A brief description of your project, which assumes no knowledge of the 
project
   or necessarily of its field
 * A list of the three most important issues to address in the move towards 
   graduation.
 * Any issues that the Incubator PMC or ASF Board might wish/need to be aware of
 * How has the community developed since the last report
 * How has the project developed since the last report.
 
This should be appended to the Incubator Wiki page at:

  http://wiki.apache.org/incubator/April2013

Note: This manually populated. You may need to wait a little before this page is
  created from a template.

Mentors
---
Mentors should review reports for their project(s) and sign them off on the 
Incubator wiki page. Signing off reports shows that you are following the 
project - projects that are not signed may raise alarms for the Incubator PMC.

Incubator PMC

Re: Incubator PMC/Board report for Apr 2013 ([ppmc])

[Chip Childers]

We actually owe this to the board itself, so I'll go and update our
IPMC status to "graduated" now.

I'll post a board report in the next couple of days for everyone to
review.

On Mon, Apr 01, 2013 at 03:27:22PM +, Marvin wrote:
> 
> 
> Dear podling,
> 
> This email was sent by an automated system on behalf of the Apache Incubator 
> PMC.
> It is an initial reminder to give you plenty of time to prepare your quarterly
> board report.
> 
> The board meeting is scheduled for Wed, 17 April 2013, 10:30:00:00 PST. The 
> report 
> for your podling will form a part of the Incubator PMC report. The Incubator 
> PMC 
> requires your report to be submitted 2 weeks before the board meeting, to 
> allow 
> sufficient time for review and submission (Wed, Apr 3rd).
> 
> Please submit your report with sufficient time to allow the incubator PMC, 
> and 
> subsequently board members to review and digest. Again, the very latest you 
> should submit your report is 2 weeks prior to the board meeting.
> 
> Thanks,
> 
> The Apache Incubator PMC
> 
> Submitting your Report
> --
> 
> Your report should contain the following:
> 
>  * Your project name
>  * A brief description of your project, which assumes no knowledge of the 
> project
>or necessarily of its field
>  * A list of the three most important issues to address in the move towards 
>graduation.
>  * Any issues that the Incubator PMC or ASF Board might wish/need to be aware 
> of
>  * How has the community developed since the last report
>  * How has the project developed since the last report.
>  
> This should be appended to the Incubator Wiki page at:
> 
>   http://wiki.apache.org/incubator/April2013
> 
> Note: This manually populated. You may need to wait a little before this page 
> is
>   created from a template.
> 
> Mentors
> ---
> Mentors should review reports for their project(s) and sign them off on the 
> Incubator wiki page. Signing off reports shows that you are following the 
> project - projects that are not signed may raise alarms for the Incubator PMC.
> 
> Incubator PMC
> 
> 

[DISCUSS] VMWare Snapshot Issue

[Francois Gaudreault]

Hi,

I think you should consider to "upgrade" this bug to critical for 4.1:
https://issues.apache.org/jira/browse/CLOUDSTACK-1224

We just replicated the exact same thing on our setup today.  We use 
up-to-date 4.1 with couple commit behind maybe.  This bug kills a key 
feature (snapshots!).


Thanks!

--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC  H3J 1S6
www.cloudops.com
@CloudOps_

RE: Unable to start domR(VR) in devcloud using laster 4.1/master.

[Rajesh Battala]

I had debugged and found that from devcloud unable to ssh to the router vm. 
When it tries to execute the GetDomRVersionCmd its getting failing and router 
vm is getting destroyed.

I had tried manually  ssh to router vm am getting Permission denied (publickey)

I had tried changing the permission of my ssh folder and keys, still the same 
issue. 
-rw--- 1 root root 1674 Apr  1 15:38 id_rsa.cloud
-rw--- 1 root root  394 Apr  1 15:38 id_rsa.cloud.pub
-rw--- 1 root root  884 Apr  1 15:51 known_hosts

How to resolve this issue?

Here is the output of ssh with verbose mode. Am using devCloud with xenserver 

ssh -i .ssh/id_rsa.cloud -p 3922 169.254.0.105
Permission denied (publickey).
root@devcloud:~# ssh -vvv -i .ssh/id_rsa.cloud -p 3922 169.254.0.105
OpenSSH_6.0p1 Debian-4, OpenSSL 1.0.1c 10 May 2012
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to 169.254.0.105 [169.254.0.105] port 3922.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug3: Incorrect RSA1 identifier
debug3: Could not load ".ssh/id_rsa.cloud" as a RSA1 public key
debug1: identity file .ssh/id_rsa.cloud type 1
debug1: Checking blacklist file /usr/share/ssh/blacklist.RSA-2048
debug1: Checking blacklist file /etc/ssh/blacklist.RSA-2048
debug1: identity file .ssh/id_rsa.cloud-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.5p1 
Debian-6+squeeze1
debug1: match: OpenSSH_5.5p1 Debian-6+squeeze1 pat OpenSSH_5*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0p1 Debian-4
debug2: fd 3 setting O_NONBLOCK
debug3: put_host_port: [169.254.0.105]:3922
debug3: load_hostkeys: loading entries for host "[169.254.0.105]:3922" from 
file "/root/.ssh/known_hosts "
debug3: load_hostkeys: found key type RSA in file /root/.ssh/known_hosts:1
debug3: load_hostkeys: loaded 1 keys
debug3: order_hostkeyalgs: prefer hostkeyalgs: 
ssh-rsa-cert-...@openssh.com,ssh-rsa-cert-...@openssh.com,ssh-rsa
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: 
ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sh

  a1
debug2: kex_parse_kexinit: 
ssh-rsa-cert-...@openssh.com,ssh-rsa-cert-...@openssh.com,ssh-rsa,ecdsa-sha2-nistp256-cert-...@openssh.com,ecdsa-sha2-nistp384-cert-...@openssh.com,ecdsa-sha2-nistp521-cert-v01@open

  
ssh.com,ssh-dss-cert-...@openssh.com,ssh-dss-cert-...@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-dss
debug2: kex_parse_kexinit: 
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-...@lysator.liu.se
debug2: kex_parse_kexinit: 
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-...@lysator.liu.se
debug2: kex_parse_kexinit: 
hmac-md5,hmac-sha1,umac...@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd...@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: 
hmac-md5,hmac-sha1,umac...@openssh.com,hmac-sha2-256,hmac-sha2-256-96,hmac-sha2-512,hmac-sha2-512-96,hmac-ripemd160,hmac-ripemd...@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,z...@openssh.com,zlib
debug2: kex_parse_kexinit: none,z...@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: 
diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: 
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-...@lysator.liu.se
debug2: kex_parse_kexinit: 
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-...@lysator.liu.se
debug2: kex_parse_kexinit: 
hmac-md5,hmac-sha1,umac...@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh. 
com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: 
hmac-md5,hmac-sha1,umac...@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh. 
com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,z...@openssh.com
debug2: kex_parse_kexinit: none,z...@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: fo

Re: [DISCUSS] git rebase vs git merge in your feature branch?

[Joe Brockmeier]

On Fri, Mar 29, 2013, at 06:36 PM, Edison Su wrote:
>Should we advocate "git rebase" in
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git?

Would you mind documenting how you think we should do it, step-by-step
on this page on the wiki? 

Right now the committer section is pretty bare. 

Best,

jzb
-- 
Joe Brockmeier
j...@zonker.net
Twitter: @jzb
http://www.dissociatedpress.net/

Re: [DISCUSS] git rebase vs git merge in your feature branch?

[Chip Childers]

On Fri, Mar 29, 2013 at 04:36:06PM -0700, Edison Su wrote:
> Hi all,
>  I am trying to review some feature branches, when I see merge requests 
> coming from mailing list, one thing that makes code review almost unrealistic 
> is that, developers tend to use "git merge" to master branch whenever rebase 
> is needed. I don't know other people really do review feature branch or not, 
> if so, how to review a feature branch, with several "merge branch "master""  
> on the feature branch. I really don't find a better way to do that.
> If, all we use "git rebase" to master branch, then the code review will 
> be much easier, at least, what kind of commits you did on the feature branch 
> can be easily identified. For example, I worked on storage_refactor branch 
> for a few months, with a lot of changes, before sending out merge request, 
> there are only less than 10 commits on the branch, reviewer can use "git diff 
> since..HEAD" to get all the changes.
>Should we advocate "git rebase" in 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git?

This is noted (although not as strongly as it could be) in the Branch
Merge Expectations page [1]:

  "The branch has been rebased against the latest version of master, so
  that the differences are limited to what's expected to merge (grab the
  master commit-sh, so that you can point to it in the merge email)."

If we add it to the git page, then please update the branch merge page
to point to the specific expectations over on the git page.

BTW - I'm +1 on this.  You can always do a diff between HEAD of master
and the feature branch, but that's always going to have some confusing
changes that would be sorted out via a rebase.

-chip

[1]
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Branch+Merge+Expectations

maven build failed for nonoss

[Jayapal Reddy Uradi]

Hi, 
master  non-oss build  got failed in netscaler.
 deps folder has cloud-netscaler-sdx.jar and cloud-netscaler.jar and ran the 
install-non-oss.sh

Can any one suggest me how  to build non-oss .

Please find below fail build logs.


[INFO] Apache CloudStack Plugin - F5 . SUCCESS [2.641s]
[INFO] Apache CloudStack Plugin - Network Netscaler .. FAILURE [2.473s]
[INFO] Apache CloudStack Plugin - Juniper SRX  SKIPPED
[INFO] Apache CloudStack VMware Base . SKIPPED
[INFO] Apache CloudStack Plugin - Hypervisor VMware .. SKIPPED
[INFO] Apache CloudStack SystemVM Patches  SKIPPED
[INFO] Apache CloudStack Cloud Services .. SKIPPED
[INFO] Apache CloudStack Console Proxy Service ... SKIPPED
[INFO] Apache CloudStack Console Proxy Plugin  SKIPPED
[INFO] Apache CloudStack Console Proxy ... SKIPPED
[INFO] Apache CloudStack Test  SKIPPED
[INFO] Apache CloudStack Client UI ... SKIPPED
[INFO] Apache CloudStack Developer Tools . SKIPPED
[INFO] Apache CloudStack Developer Tools . SKIPPED
[INFO] Apache CloudStack apidocs . SKIPPED
[INFO] Apache CloudStack marvin .. SKIPPED
[INFO] Apache CloudStack cloudmonkey cli . SKIPPED
[INFO] Apache CloudStack DevCloud  SKIPPED
[INFO] Apache CloudStack DevCloud-KVM  SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 3:30.412s
[INFO] Finished at: Mon Apr 01 22:27:36 GMT+05:30 2013
[INFO] Final Memory: 36M/99M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile (default-compile) 
on project cloud-plugin-network-netscaler: Compilation failure: Compilation 
failure:
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[21,59]
 package com.citrix.netscaler.nitro.resource.config.autoscale does not exist
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[22,59]
 package com.citrix.netscaler.nitro.resource.config.autoscale does not exist
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2230,30]
 cannot find symbol
[ERROR] symbol  : method set_monstate(java.lang.String)
[ERROR] location: class 
com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2232,90]
 cannot find symbol
[ERROR] symbol  : method 
add(com.citrix.netscaler.nitro.service.nitro_service,com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding)
[ERROR] location: class 
com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2257,41]
 cannot find symbol
[ERROR] symbol  : method 
delete(com.citrix.netscaler.nitro.service.nitro_service,com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding)
[ERROR] location: class 
com.citrix.netscaler.nitro.resource.config.basic.service_lbmonitor_binding
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2339,28]
 cannot find symbol
[ERROR] symbol  : method set_autoscale(java.lang.String)
[ERROR] location: class 
com.citrix.netscaler.nitro.resource.config.basic.servicegroup
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2340,28]
 cannot find symbol
[ERROR] symbol  : method set_memberport(int)
[ERROR] location: class 
com.citrix.netscaler.nitro.resource.config.basic.servicegroup
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2457,23]
 cannot find symbol
[ERROR] symbol  : method set_minautoscalemembers(int)
[ERROR] location: class com.citrix.netscaler.nitro.resource.config.lb.lbvserver
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java:[2458,23]
 cannot find symbol
[ERROR] symbol  : method set_maxautoscalemembers(int)
[ERROR] location: class com.citrix.netscaler.nitro.resource.config.lb.lbvserver
[ERROR] 
/Users/jayapalreddy/dev/cloudstack/plugins/network-elements/netscaler/src/com/cloud/network/resource/

Re: [DISCUSS] git rebase vs git merge in your feature branch?

[Chiradeep Vittal]

On 3/30/13 3:00 AM, "Wido den Hollander"  wrote:

>Hi,
>
>On 03/30/2013 12:36 AM, Edison Su wrote:
>> Hi all,
>>   I am trying to review some feature branches, when I see merge
>>requests coming from mailing list, one thing that makes code review
>>almost unrealistic is that, developers tend to use "git merge" to master
>>branch whenever rebase is needed. I don't know other people really do
>>review feature branch or not, if so, how to review a feature branch,
>>with several "merge branch "master""  on the feature branch. I really
>>don't find a better way to do that.
>>  If, all we use "git rebase" to master branch, then the code review
>>will be much easier, at least, what kind of commits you did on the
>>feature branch can be easily identified. For example, I worked on
>>storage_refactor branch for a few months, with a lot of changes, before
>>sending out merge request, there are only less than 10 commits on the
>>branch, reviewer can use "git diff since..HEAD" to get all the changes.
>> Should we advocate "git rebase" in
>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git?
>
>Yes, I'd vote for rebase, a couple of reasons:
>- You can track changes more easy
>- The feature can be committed to master in one single patch
>
>When stuff gets into master after a rebase in the feature branch (and
>even a squash) you can see which feature came in with which commit.
>
>A feature can be easily reverted, but also one patch can be reviewed to
>verify if it's OK.
>
>I just don't like merges, I think they make a mess of the repos.
>
>Wido

+1 on rebase 
-0 on squash
Sometimes features need to modify core components. Squashing loses the
history /reasoning behind that modification. I would leave the decision of
selective squashing to the committer.

Re: [DISCUSS] git rebase vs git merge in your feature branch?

[Chip Childers]

On Mon, Apr 01, 2013 at 10:08:52AM -0700, Chiradeep Vittal wrote:
> -0 on squash
> Sometimes features need to modify core components. Squashing loses the
> history /reasoning behind that modification. I would leave the decision of
> selective squashing to the committer.

Agreed.  It totally depends on the specifics of the situation.
Sometimes a squash of everything will make sense.  Other times, a squash
of some commits to tidy things up will make sense.  And finally,
sometimes it doesn't matter at all, and everything can be squashed.

Re: maven build failed for nonoss

[Chip Childers]

On Mon, Apr 01, 2013 at 05:06:06PM +, Jayapal Reddy Uradi wrote:
> Hi, 
> master  non-oss build  got failed in netscaler.
>  deps folder has cloud-netscaler-sdx.jar and cloud-netscaler.jar and ran the 
> install-non-oss.sh
> 
> Can any one suggest me how  to build non-oss .
> 
> Please find below fail build logs.

Search the list archive, because I believe you are seeing something that
has already been answered.  AFAIK, the netscaler SDK jar needs to be a
newer version than what we used during 4.0.0.

RE: cloudmonkey printing enhancements proposal

[Edison Su]

> -Original Message-
> From: prasanna [mailto:srivatsav.prasa...@gmail.com]
> Sent: Friday, March 29, 2013 10:16 PM
> To: dev@cloudstack.apache.org
> Subject: Re: cloudmonkey printing enhancements proposal
> 
> On 30 March 2013 09:20, Justin Grudzien  wrote:
> >
> > My company is building a private cloud and we are moving to
> > cloudstack. As we begun investigating the cloudmonkey CLI we found
> > that the output was slightly hard to read. I have begun working on
> > some optimizations that I think will benefit the community and I
> > reached out to Rohit, who recommended that I join this list and
> > present my ideas. Here is what I am
> > proposing:
> >
> > 1. Add json output to cloudmonkey
> > I have accomplished this by adding a config parameter called display,
> > which can be set to json, tabularize, or default. I have removed the
> > tabularize parameter.
> >
> > Justins-MacBook-Pro:cloudmonkey grudzien$ cloudmonkey list users
> > account=grudzien {
> >   "count": 1,
> >   "user": [
> > {
> >   "account": "grudzien",
> >   "accountid": "b799783d-e5bb-460a-be0e-3966bd69edda",
> >   "accounttype": 1,
> >   "apikey": "*nokey*",
> >   "created": "2013-03-27T16:09:17-0500",
> >   "domain": "ROOT",
> >   "domainid": "7e61c32f-9873-4944-947a-dcc00d3bebdc",
> >   "email": "grudz...@gmail.com",
> >   "firstname": "Justin",
> >   "id": "265930bc-62ef-41f8-849c-e58593ca4b1f",
> >   "lastname": "Grudzien",
> >   "secretkey": "*nokey*",
> >   "state": "enabled",
> >   "username": "grudzien"
> > }
> >   ]
> > }
> >
> > 2. Add filtering as a standard parameter for all output types.
> > The only thing that has filtering now is the tabular output and grep
> > breaks the json.
> >
> > Justins-MacBook-Pro:cloudmonkey grudzien$ cloudmonkey list users
> > account=grudzien filter=account,email,username,state
> > {
> >   "count": 1,
> >   "user": [
> > {
> >   "account": "grudzien",
> >   "email": "grudz...@gmail.com",
> >   "state": "enabled",
> >   "username": "grudzien"
> > }
> >   ]
> > }
> >
> 
> +1 - JSON output would make it very good to consume into other api
> response processing. Right now cloudmonkey actually isn't aware of the
> response object model. For certain API responses that return embedded
> objects (eg: listNetworks [1]) it would be good to return the JSON
> appropriately formatted as well.
> 
> [1] https://gist.github.com/vogxn/5255883

If cloudmonkey cares about json response now, it makes sense that both 
cloudmonkey and marvin share the same code to send api command and parse json 
response. Isn't it?

> 
> 
> > 3. Add color to the json output
> > I was thinking of colorizing the keys in the key/value pairs to
> > increase readability.
> >
> > 4. Move the color option from the config file to the command line.
> > There are two reasons for this. First, I want to be able to wrap a
> > script around cloudmonkey and not have to worry about colorization
> > that will impede me processing the output and second I think it would
> > be more useful to use the highlighting on demand rather than having to
> > back out of the shell to edit a config file.
> >
> > 5. Standardize messaging for the output types.
> > Right now certain kinds of messaging is presented differently for an
> > output type. For example, if I issue an api command that doesn't exist
> > it displays a generic error message, regardless of the output type
> > selected. Ideally, all output would be in the specified format.
> >
> > I have the first two working and am planning on implementing the
> > others as I flesh them out. I will submit a patch when I feel it is
> > ready. Any early feedback on whether these changes will be useful to
> others is appreciated.
> 
> Patches most welcome!
> 
> 
> 
> 
> --
> Prasanna.,

RE: cloudmonkey printing enhancements proposal

[Edison Su]

> -Original Message-
> From: Justin Grudzien [mailto:grudz...@gmail.com]
> Sent: Monday, April 01, 2013 3:56 AM
> To: dev@cloudstack.apache.org
> Subject: Re: cloudmonkey printing enhancements proposal
>
> Thanks for the information. If you haven't already put the patch for review I
> would like to do so and follow the directions so I can get used to the 
> process.
> Let me know. Also, I saw the lambda functions and chose not to use them
> because I think they make the code harder to read for the more novice
> python programmer. I can go back and add it if the goal is to have shorter

+2, I am amateur python programmer, hate to read obscure code, especially if 
the language is no type...

> code but if it were up to me I would have the code more readable. :) I hope
> people find this work to be useful, I appreciate the guidance through the
> submission process.
>
> Justin
>
>
> On Mon, Apr 1, 2013 at 3:07 AM, Rohit Yadav  wrote:
>
> > Hi Justin, thanks for your first patch.
> >
> > As per our workflows, we use git for our version control and have a
> > reviewboard for reviewing patches and an issue tracker for tracking
> > bugs/features.
> >
> > Git:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git
> >
> > Review board:
> > https://reviews.apache.org
> >
> > For filtering only passed key names, we can cut down the whole if-else
> > tree using filter (which you may see I'd used at several places) and
> > reduce list.
> >
> > Something like; filter(lamba key:  > like key in my set of filters>, dictionary.keys()), using this
> > filtered failsafe list of keys, we can generate dictionary, using a map;
> map(lambda x:
> > myjson[x] = jsondictionary[x], failsafekeys) etc.
> >
> > I can help with this, thanks for your patch. Since it's small I can
> > help apply that for future please use review board or share git
> > formatted patch which would apply cleanly on master.
> >
> > Regards.
> >
> > On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien 
> > wrote:
> >
> > > Greetings,
> > >
> > > I am posting my first patch. I am not sure exactly how to submit a
> > > patch but I figure I will paste it here and someone will tell me the
> > > right method. I am keeping track of the work that I am doing and as
> > > Rohit suggested I will update the mailing list as I progress. This
> > > first patch adds basic JSON output with filtering. I have tested it
> > > on cloudmonkey
> > > 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are
> appreciated.
> > >
> > > --- README ---
> > > Added
> > > 1. display = [default|json|tabularize] has been added in the config
> > > to replace
> > > tabularize = [true|false]
> > > 2. tabularize is deprecated but we will still set it as "false" once
> > > the user
> > > removes it out of their config to avoid throwing an error. This
> > > will
> > be
> > > removed in the next major version.
> > > 3. display = "default" is added to the [ui] section of the config if
> > > it
> > is
> > > not
> > > present.
> > > 4. You can now output JSON formatted text by setting the config
> > > display = json 5. You can now filter text in JSON output mode. (i.e.
> > > list users
> > > account=grudzien filter=account,id,email). Filtered output returns a
> > > properly formatted JSON document.
> > >
> > > Removed
> > > 1. Removed the printing of attr keys in read_config().
> > >
> > > Deprecated
> > > 1. tabularize = [true|false] is now messaged as deprecated.
> > >
> > > ToDo
> > > 1. Need to format empty return messages with proper JSON messaging.
> > > 2. Need to standardize JSON output messaging.
> > > A. Add return code [Error|Success] to all calls.
> > > B. Add called API verb.
> > > 3. Count is not decreased in results when filtering completely
> > eliminates a
> > > result.
> > > 4. JSON print needs to be implemented manually to support colors.
> > > Right
> > now
> > > json.dumps() pretty prints the output.
> > > 5. Color tagging needs to be added to JSON printing.
> > > 6. Error messages need to have proper JSON formatting.
> > > 7. Various help text has grammar or consistency errors that need fixing.
> > > 8. Make color a passable option to a command instead of a config
> > parameter.
> > > (i.e. list users account=grudzien color=true)
> > > A. This will require reworking the result_filter passable
> > > argument to the
> > > various print methods since they only expect filter= input.
> > > 9. The API in CloudStack 4.0.1 does not all return proper JSON
> > > formatted text.
> > > As a result some of the JSON printing looks funny. I will get
> > > the
> > later
> > > versions into a lab and test them to make sure the formatting
> > > bugs
> > are
> > > in
> > > newer revisions and submit them as bugs.
> > > --- README ---
> > >
> > > --- PATCH ---
> > > diff -rupN cloudstack/tools/cli/cloudmonkey/cloudmonkey.py
> > > cloudstackmodified/tools/cli/cloudmonkey/cloudmonkey.py
> > > --- cloudstack/tools/cli/

Re: Review Request: Make SHA256Salt the default password encoding and authentication mechanism for cloudstack

[Venkata Siva Vijayendra Bhamidipati]

> On March 29, 2013, 5:22 p.m., Min Chen wrote:
> > api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.java,
> >  line 66
> > 
> >
> > If password is default hashed to SHA256SALT, then it should not be 
> > clear text password. This description is contradictory to me.

Hi Min, this is the password that needs to be sent in by the client - it will 
need to be the clear text password itself -  even if plaintext authentication 
is being used and the password passed in is itself is a hash, technically it 
would still be a clear text password that matches the password stored in the 
db. That is what is reflected in the description above.. The sha encoding kicks 
in only on the server side. If the cleartext is to be protected, SSL via HTTPS 
would serve that purpose.


> On March 29, 2013, 5:22 p.m., Min Chen wrote:
> > developer/developer-prefill.sql, line 39
> > 
> >
> > Should this password be encoded using SHA if that is the default way? 
> > The pre-filled one is still MD5.

If the admin user is created as enabled, the mgmt server will leave the 
password as is and not change it. However, if created as disabled, the mgmt 
server code, when creating the admin user, will replace the password for the 
admin user as hash("password") where the hash function is the default function 
configured.


- Venkata Siva Vijayendra


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10039/#review18519
---


On March 28, 2013, 8:26 p.m., Venkata Siva Vijayendra Bhamidipati wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10039/
> ---
> 
> (Updated March 28, 2013, 8:26 p.m.)
> 
> 
> Review request for cloudstack, Hugo Trippaers, Kelven Yang, and Min Chen.
> 
> 
> Description
> ---
> 
> Changing default password encoding mechanism from MD5 to SHA256Salted.
> 
> 
> This addresses bug CS-1734.
> 
> 
> Diffs
> -
> 
>   
> api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.java 
> 89673ea 
>   api/src/org/apache/cloudstack/api/command/admin/user/CreateUserCmd.java 
> fb29e1a 
>   api/src/org/apache/cloudstack/api/command/admin/user/UpdateUserCmd.java 
> 1f31662 
>   client/tomcatconf/componentContext.xml.in 016df0a 
>   client/tomcatconf/nonossComponentContext.xml.in 8f8dae5 
>   developer/developer-prefill.sql 6300d35 
>   
> plugins/user-authenticators/ldap/src/com/cloud/server/auth/LDAPUserAuthenticator.java
>  61eebe5 
>   
> plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
>  026125e 
>   
> plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
>  52e7cb3 
>   
> plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
>  1b29f69 
>   server/src/com/cloud/server/ManagementServerImpl.java b689f93 
>   server/src/com/cloud/user/AccountManagerImpl.java b69f314 
> 
> Diff: https://reviews.apache.org/r/10039/diff/
> 
> 
> Testing
> ---
> 
> Manual testing done for both oss and nonoss components. Both admin and users 
> added later are encoded according to the scheme configured, and authenticated 
> by the same scheme.
> 
> To change the order of the schemes, modify the following list properties in 
> client/tomcatconf/nonossComponentContext.xml.in or 
> client/tomcatconf/componentContext.xml.in as applicable, to the desired order:
> 
> 
>  
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>  
>  
> 
>  
> 
> 
> Thanks,
> 
> Venkata Siva Vijayendra Bhamidipati
> 
>

Re: Review Request: Make SHA256Salt the default password encoding and authentication mechanism for cloudstack

[Vijayendra Bhamidipati]

Putting in a new authenticator/encoder for SHA512 would be good - I'll log
a ticket to track it as a separate effort and work on it when time
permits. Until then, let's use SHA256 as the default since it's the best
we've got implemented as of now.

Regards,
Vijay

On 3/29/13 5:56 PM, "Justin Grudzien"  wrote:

>Understandable that SHA256 is good for today but SHA512 is good for
>tomorrow and has excellent support in all major languages, products, etc.
>Oracle is the only thing I can think of that doesn't support it out of
>the box.
>
>I agree on balance but I can certainly spare the 32 bytes of storage. :)
>
>Justin
>
>On Mar 29, 2013, at 7:23 PM, John Kinsella  wrote:
>
>> Never too late. :)
>> 
>> Looking at [1], it'll take about 2E64 days to search the SHA256
>>keyspace with a brute forcer written for a GPU accelerator. Even with
>>10,000 of them, it'll take way, way too long. Bitcoin coin uses SHA256,
>>so I think it's safe for a while. :)
>> 
>> Yes, SHA512 would be "more safe," but ya gotta ask what you're really
>>getting by going to the bigger keyŠsecurity is about a balance. If
>>you're using ACS to run a cloud cracking SHA256 passwords, then maybe
>>you should use SHA512 passwords in ACS. ;)
>> 
>> John
>> http://www.insidepro.com/eng/egb.shtml
>> 
>> On Mar 29, 2013, at 4:36 PM, Justin Grudzien
>>mailto:grudz...@gmail.com>>
>> wrote:
>> 
>> I apologize for jumping into this conversation late, but I am new to
>>the developer mailing list. Why would we choose SHA256+salt over
>>SHA512+salt? SHA512+salt's storage is insignificant when compared to
>>SHA256 and the chances of a birthday attack are significantly reduced.
>>As a security professional I would argue for the best possible hashing
>>algorithm available, especially when there is little to no cost.
>> 
>> Justin
>> 
>> On Mar 29, 2013, at 6:00 PM, "Min Chen"
>>mailto:min.c...@citrix.com>> wrote:
>> 
>> 
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/10039/#review18548
>> ---
>> 
>> 
>> 
>> client/tomcatconf/componentContext.xml.in
>> 
>> 
>>  This componentContext.xml.in and nonossComponentContext.xml.in should
>>be put into applicationContext.xml since they are the same for nonoss
>>and oss versions.
>> 
>> 
>> - Min Chen
>> 
>> 
>> On March 28, 2013, 8:26 p.m., Venkata Siva Vijayendra Bhamidipati wrote:
>> 
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/10039/
>> ---
>> 
>> (Updated March 28, 2013, 8:26 p.m.)
>> 
>> 
>> Review request for cloudstack, Hugo Trippaers, Kelven Yang, and Min
>>Chen.
>> 
>> 
>> Description
>> ---
>> 
>> Changing default password encoding mechanism from MD5 to SHA256Salted.
>> 
>> 
>> This addresses bug CS-1734.
>> 
>> 
>> Diffs
>> -
>> 
>> 
>>api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.
>>java 89673ea
>> api/src/org/apache/cloudstack/api/command/admin/user/CreateUserCmd.java
>>fb29e1a
>> api/src/org/apache/cloudstack/api/command/admin/user/UpdateUserCmd.java
>>1f31662
>> client/tomcatconf/componentContext.xml.in 016df0a
>> client/tomcatconf/nonossComponentContext.xml.in 8f8dae5
>> developer/developer-prefill.sql 6300d35
>> 
>>plugins/user-authenticators/ldap/src/com/cloud/server/auth/LDAPUserAuthen
>>ticator.java 61eebe5
>> 
>>plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenti
>>cator.java 026125e
>> 
>>plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTex
>>tUserAuthenticator.java 52e7cb3
>> 
>>plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256
>>SaltedUserAuthenticator.java 1b29f69
>> server/src/com/cloud/server/ManagementServerImpl.java b689f93
>> server/src/com/cloud/user/AccountManagerImpl.java b69f314
>> 
>> Diff: https://reviews.apache.org/r/10039/diff/
>> 
>> 
>> Testing
>> ---
>> 
>> Manual testing done for both oss and nonoss components. Both admin and
>>users added later are encoded according to the scheme configured, and
>>authenticated by the same scheme.
>> 
>> To change the order of the schemes, modify the following list
>>properties in client/tomcatconf/nonossComponentContext.xml.in or
>>client/tomcatconf/componentContext.xml.in as applicable, to the desired
>>order:
>> 
>>  
>>   
>>  
>>  
>>  
>>  
>>  
>>  
>> 
>>  
>>  
>>  
>>   
>>   
>>  
>>   
>> 
>> 
>> Thanks,
>> 
>> Venkata Siva Vijayendra Bhamidipati
>> 
>> 
>> Stratosec - Secure Infrastructure as a Service
>> o: 415.315.9385
>> @johnlkinsella
>> 

Re: cloudmonkey printing enhancements proposal

[Rohit Yadav]

On Mon, Apr 1, 2013 at 4:26 PM, Justin Grudzien  wrote:

> Thanks for the information. If you haven't already put the patch for review
> I would like to do so and follow the directions so I can get used to the
> process. Let me know. Also, I saw the lambda functions and chose not to use
> them because I think they make the code harder to read for the more novice
> python programmer. I can go back and add it if the goal is to have shorter
> code but if it were up to me I would have the code more readable. :) I hope
> people find this work to be useful, I appreciate the guidance through the
> submission process.
>

Okay, let me see them if I can get some time tomorrow.

About keep it simple, I think it's a matter of art. IMO such thinkings has
been exploited by people to write a lot of verbose code. I think we're
artists and our art evolves with time and so does our tools and techniques.

I think it's time for functional programming, for example we now have
anonymous functions or lambdas in mainstream langs (or their iterations)
such as C++11 and Java8. There is really a use of such a paradigms, we're
not throwing them just because lambdas are cool (even though lamba calculus
gives you your eureka moments :)

Regards.
PS. An iterative programmer may think an OOP programmer writes hard to read
code and an OOP programmer may think the same for a functional programmer,
a functional programmer thinks nobody in the world knows how to really
program :P Lastly, a user does not care who're coding it as long as it
works lol :P


> Justin
>
>
> On Mon, Apr 1, 2013 at 3:07 AM, Rohit Yadav  wrote:
>
> > Hi Justin, thanks for your first patch.
> >
> > As per our workflows, we use git for our version control and have a
> > reviewboard for reviewing patches and an issue tracker for tracking
> > bugs/features.
> >
> > Git:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git
> >
> > Review board:
> > https://reviews.apache.org
> >
> > For filtering only passed key names, we can cut down the whole if-else
> tree
> > using filter (which you may see I'd used at several places) and reduce
> > list.
> >
> > Something like; filter(lamba key:  like
> > key in my set of filters>, dictionary.keys()), using this filtered
> failsafe
> > list of keys, we can generate dictionary, using a map; map(lambda x:
> > myjson[x] = jsondictionary[x], failsafekeys) etc.
> >
> > I can help with this, thanks for your patch. Since it's small I can help
> > apply that for future please use review board or share git formatted
> patch
> > which would apply cleanly on master.
> >
> > Regards.
> >
> > On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien 
> > wrote:
> >
> > > Greetings,
> > >
> > > I am posting my first patch. I am not sure exactly how to submit a
> patch
> > > but I figure I will paste it here and someone will tell me the right
> > > method. I am keeping track of the work that I am doing and as Rohit
> > > suggested I will update the mailing list as I progress. This first
> patch
> > > adds basic JSON output with filtering. I have tested it on cloudmonkey
> > > 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are
> appreciated.
> > >
> > > --- README ---
> > > Added
> > > 1. display = [default|json|tabularize] has been added in the config to
> > > replace
> > > tabularize = [true|false]
> > > 2. tabularize is deprecated but we will still set it as "false" once
> the
> > > user
> > > removes it out of their config to avoid throwing an error. This
> will
> > be
> > > removed in the next major version.
> > > 3. display = "default" is added to the [ui] section of the config if it
> > is
> > > not
> > > present.
> > > 4. You can now output JSON formatted text by setting the config
> display =
> > > json
> > > 5. You can now filter text in JSON output mode. (i.e. list users
> > > account=grudzien filter=account,id,email). Filtered output returns
> a
> > > properly formatted JSON document.
> > >
> > > Removed
> > > 1. Removed the printing of attr keys in read_config().
> > >
> > > Deprecated
> > > 1. tabularize = [true|false] is now messaged as deprecated.
> > >
> > > ToDo
> > > 1. Need to format empty return messages with proper JSON messaging.
> > > 2. Need to standardize JSON output messaging.
> > > A. Add return code [Error|Success] to all calls.
> > > B. Add called API verb.
> > > 3. Count is not decreased in results when filtering completely
> > eliminates a
> > > result.
> > > 4. JSON print needs to be implemented manually to support colors. Right
> > now
> > > json.dumps() pretty prints the output.
> > > 5. Color tagging needs to be added to JSON printing.
> > > 6. Error messages need to have proper JSON formatting.
> > > 7. Various help text has grammar or consistency errors that need
> fixing.
> > > 8. Make color a passable option to a command instead of a config
> > parameter.
> > > (i.e. list users account=grudzien color=true)
> > > A. This will require reworking the resul

Re: [DISCUSS] Palo Alto Integration

[Will Stevens]

So I have been stepping through the code and I can confirm that the
'design' method of ExternalGuestNetworkGuru is being hit, but it doesn't do
anything, so it passes off work of creating the network to the 'design'
method of GuestNetworkGuru which assigns 10.1.1.0/24 to the network every
time I create a network.

Something I am finding strange is that 'config.getId()' gives -1, so the
new network that is being created while in the 'design' method of
ExternalGuestNetworkGuru does not hit the only logic in the function:

NetworkVO config = (NetworkVO) super.design(offering, plan, userSpecified,
owner);
if (config == null) {
return null;
} else if
(_networkModel.networkIsConfiguredForExternalNetworking(plan.getDataCenterId(),
config.getId())) {
/* In order to revert userSpecified network setup */
config.setState(State.Allocated);
}

So the config.setState(State.Allocated) is not getting hit.

There does seem to be some logic for updating the cidr in the 'implement'
function of ExternalGuestNetworkGuru, but that is not run until a VM is
added to the network (from what I understand), so that is a bit strange to
me.

Are the non-overlapping cidrs implemented only when a VM is added to the
network and the same placeholder cidr is used until then?

Thanks,

Will


On Mon, Apr 1, 2013 at 11:22 AM, Will Stevens  wrote:

> Thank you for all your help Murali...
>
> So my Provider has been setup with isExternal = true this whole time.
> public static final Provider PaloAlto = new Provider("PaloAlto", true);
>
> If I run a debugger and then create a guest network, I see it enter the
> 'design' function of the ExternalGuestNetworkGuru, but it does not do
> anything in there because the config is not null, but the config.getId() =
> -1, so it just returns the config (Network object) and doesn't really do
> anything.
>
> Apparently the 'implement' method doesn't get called until a VM is
> attempted to be launched on the network.
>
> I must be missing something because, every Isolated guest network I create
> on my provider is defaulting to the cidr of 10.1.1.0/24.  Even if I have
> multiple Isolated networks associated with the same account, they all by
> default have that cidr.
>
> If the default behaviour of the ExternalGuestNetworkGuru is to create
> non-overlapping guest cidrs, why does it always default to the 
> 10.1.1.0/24cidr when I create a new network?  I can not specify a gateway or 
> netmask
> because it is an external network (as you can see from the included
> screenshot).
> [image: Inline image 1]
>
> What am I missing here?  Why am I unable to create non-overlapping cidrs
> with the ExternalGuestNetworkGuru?
>
> Thanks,
>
> Will
>
>
> On Fri, Mar 29, 2013 at 1:23 AM, Murali Reddy wrote:
>
>> On 28/03/13 10:59 PM, "Will Stevens"  wrote:
>>
>> >I am trying to implement the non-overlapping cidrs right now and I have
>> >some questions.  Does the ExternalGuestNetworkGuru create networks with
>> >non-overlapping cidrs by default?  Or do I need to override it's 'design'
>> >and 'implement' methods to implement non overlapping cidrs?
>>
>> Will, yes, it does by default. You can just use
>> 'ExternalGuestNetworkGuru'. Just so that you know, there is check
>> 'networkIsConfiguredForExternalNetworking' in ExternalGuestNetworkGuru.
>> Which basically checks if provider is configured as service provider using
>> external physical appliances. So when you declare provider, mark
>> 'isExternal' as true in the provider constructor.
>>
>> >
>> >If I have to write my own methods, I think I understand how to
>> >override ExternalGuestNetworkGuru and then get it to run by adding it to
>> >the components.xml (or nonoss-components.xml) as well as the
>> >componentContext.xml.in.
>> >
>> >If I do not have to actually write the logic for the non-overlapping
>> cidrs
>> >(which i am hoping is the case), and the ExternalGuestNetworkGuru
>> actually
>> >implements that logic, how would I get the ExternalGuestNetworkGuru into
>> >my
>> >flow without actually overriding the class?  I understand that the
>> >components are loaded through the components.xml stuff, but its not clear
>> >how you specify which NetworkGuru should be used in my specific flow.
>> >
>> >I am basically working from this
>> >document<
>> https://cwiki.apache.org/CLOUDSTACK/extending-cloudstack-networki
>> >ng.html>and
>> >the code.  Is there any other resources I should be aware of for
>> >extending the CloudStack networking functionality?
>> >
>> >I have a good start on a Resource, ExternalFirewallElement and an
>> >ExternalFirewallService.  I can currently set the Palo Alto as the
>> >provider
>> >of Firewall, SourceNat, StaticNat and Port Forwarding services.  I can
>> >currently Add, List, Configure and Delete my Palo Alto provider.
>> >
>> >I am getting there, but I still feel like there are gaps in my knowledge
>> >when using the CS networking plugin functionality.
>>
>> Good the hear the progress. Feel free to ask any question.
>>
>> Thanks,

Review Request: CLOUDSTACK-1814: Add LXC documentation

[Phong Nguyen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10224/
---

Review request for cloudstack, Jessica Tomechak and Joe Brockmeier.


Description
---

Initial submission for LXC documentation. Note the usage of xpointer references 
to avoid duplicate element IDs when including other xml fragments.


Diffs
-

  docs/en-US/hypervisor-host-install-firewall.xml ae82fc4 
  docs/en-US/hypervisor-host-install-network.xml 3a6dfac 
  docs/en-US/hypervisor-installation.xml b0fc9f4 
  docs/en-US/lxc-install.xml PRE-CREATION 
  docs/en-US/lxc-topology-req.xml PRE-CREATION 
  docs/en-US/management-server-install-systemvm.xml 8dc73de 
  docs/en-US/minimum-system-requirements.xml 0e497dd 
  docs/en-US/prepare-system-vm-template.xml b53a509 
  docs/en-US/topology-req.xml 65c9c2e 

Diff: https://reviews.apache.org/r/10224/diff/


Testing
---

Built successfully with Publican 3.0.


Thanks,

Phong Nguyen

CLOUDSTACK-1839: Upgrade 4.0 -> 4.1 - Upgraded DB has lot more keys and indexes for many tables compare to the fresh installed 4.1 DB.

[Chip Childers]

Rohit,

I know you are quasi-offline right now, but do you think you'd be able
to take a look at CLOUDSTACK-1839 for me in the next day or so?

If not, just say so and we'll try to find someone else to review / fix
it.

Thanks!

-chip

CLOUDSTACK-1842: ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 Guest OS Types on the Upgraded Setup

[Chip Childers]

I went digging into CLOUDSTACK-1842, and I see the following:

In looking at the SQL scripts, I see that the guest_os and
guest_os_hypervisor tables are typically populated via the upgrade
scripts during an upgrade.

I also see that templates.sql has the data to populate these tables:

INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name) VALUES 
(163, UUID(), 10, 'Ubuntu 12.04 (32-bit)');
INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name) VALUES 
(164, UUID(), 10, 'Ubuntu 12.04 (64-bit)');

and for VMware:

INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type, guest_os_name, 
guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (32-bit)', 162);
INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type, guest_os_name, 
guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (64-bit)', 163);

Nothing is in templates.sql for Ubuntu 12.04 on KVM, and no Ubuntu 
(any version) for XenServer.  Is this right?  Also, should we 
populate Ubuntu 12.04 into guest_os and guest_os_hypervisor via the 
40to41 upgrade script?

-chip

CLOUDSTACK-1868: GetVmStatsCommand throws NullPointerException with VMWare

[Chip Childers]

Can I get a volunteer to look at CLOUSTACK-1868 ASAP?  It's one of a
*handful* of remaining bugs for 4.1.

-chip

Re: [DISCUSS] VMWare Snapshot Issue

[Chip Childers]

Edison,

I bumped CLOUDSTACK-1224 up to critical, as Francois is mentioning that
it breaks snapshotting.

It appears to be already assigned to you. Can you look at the bug as soon 
as possible (or reply to let us know if you aren't able to get to it)?

Thanks!

-chip

On Mon, Apr 01, 2013 at 11:42:32AM -0400, Francois Gaudreault wrote:
> Hi,
> 
> I think you should consider to "upgrade" this bug to critical for 4.1:
> https://issues.apache.org/jira/browse/CLOUDSTACK-1224
> 
> We just replicated the exact same thing on our setup today.  We use
> up-to-date 4.1 with couple commit behind maybe.  This bug kills a
> key feature (snapshots!).
> 
> Thanks!
> 
> -- 
> Francois Gaudreault
> Architecte de Solution Cloud | Cloud Solutions Architect
> fgaudrea...@cloudops.com
> 514-629-6775
> - - -
> CloudOps
> 420 rue Guy
> Montréal QC  H3J 1S6
> www.cloudops.com
> @CloudOps_
> 
> 

[DOCS] LXC documentation

[Phong Nguyen]

Hi,

I have been working on adding LXC documentation [1].

LXC is very similar to KVM in terms of setup, so I would like to reuse most
of the KVM sections to avoid duplicate documentation. With the unique IDs
in the KVM sections, Publican will error out during build because it will
find duplicate IDs since these sections are included more than once.

As a workaround, I was able to define the  tags with unique IDs
for LXC and used xpointer references to source all children of the
 tag in the included KVM xml files. This works as expected,
however, it's brittle and I'd like to find a better solution.

What approach should we take for reusable documentation (within the same
book)? Should we not use any IDs in xml fragments that are deemed reusable?

[1] https://reviews.apache.org/r/10224/

Thanks,
Phong

[ACS41] Final push to an RC?

[Chip Childers]

Hi all,

We're getting there...  Here's what I see as outstanding:

* Edison - CLOUDSTACK-1224 Volume snapshot creation failing
This bug was recently bumped from major to Critical.  Edison, please
shout if you disagree with the priority bump.

* Rohit - CLOUDSTACK-1839 Upgrade 4.0 -> 4.1 - Upgraded DB has lot more keys 
and indexes for many tables compare to the fresh installed 4.1 DB.
We may need someone to review and fix if Rohit isn't around to fix this
one in the next day or so.  Pending his response to a query about
availability.

* Abhinandan - CLOUDSTACK-1842 ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 
Guest OS Types on the Upgraded Setup   Abhinandan Prateek
This bug was assigned to Abhinandan when I went to look at it.  I sent
an email out to the group asking a couple of questions as well.  Let's
see if we can wrap this one up (as I believe the solution should be
fairly straight forward).

* Marcus - CLOUDSTACK-1845 KVM - storage migration often fails Marcus Sorensen
* Marcus - CLOUDSTACK-1846 KVM - storage pools can silently fail to be 
unregistered, leading to failure to register later  Marcus Sorensen
Marcus, I applied the fix for these to 4.1.  Can you solve the 4.0
branch version of the problem so that the bugs can be resolved?  If it's
going to take awhile to get back to 4.0, I'd actually say that it's OK
to not worry about it much.  Unless you were trying to get the 4.0 fix
into 4.0.2, we weren't planning to do another bug-fix release for 4.0.x
after 4.0.2.  Feel free to resolve the bugs if that makes sense please.

* Unassigned - CLOUDSTACK-1868 GetVmStatsCommand throws NullPointerException 
with VMWare
We need someone to pick this one up.  Anyone?

* Joe - Release Notes
I think that the upgrade portion of the docs probably needs to be
updated to reflect Wido's notes on the wiki for how to perform the
upgrade process itself.  Do you have this one under control, or do you
want help?  I'd love to have the upgrade instructions in the release
itself (and will probably hold off on an RC until we have them).

* QA Team - Upgrade testing finalization
I see that there was some traffic about upgrade testing over the
weekend.  Where do we stand?  Are we done now that Wido provided the
fixes he provided?

Any finally...  does anyone else have any outstanding work / items /
bugs that you believe need to be addressed before we start the voting
process for 4.1.0's first RC?

-chip

RE: [ACS41] Final push to an RC?

[Sudha Ponnaganti]

Upgrade resolved defects are being validated tonight and tomorrow morning. 
1839/1842 are  required but they can be validated in RC Candidate drop as it 
seem to be isolated fix and may not impact anything else. 

I need to cleanup some of the New and Improved feature list i.e mark them 
closed based on resolved items and testing done. Haven't done in last couple of 
weeks. Will do after I finish one more task for day job. 

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Monday, April 01, 2013 1:09 PM
To: dev@cloudstack.apache.org
Cc: Edison Su; bhais...@apache.org; Abhinandan Prateek; shadow...@gmail.com
Subject: [ACS41] Final push to an RC?

Hi all,

We're getting there...  Here's what I see as outstanding:

* Edison - CLOUDSTACK-1224 Volume snapshot creation failing This bug was 
recently bumped from major to Critical.  Edison, please shout if you disagree 
with the priority bump.

* Rohit - CLOUDSTACK-1839 Upgrade 4.0 -> 4.1 - Upgraded DB has lot more keys 
and indexes for many tables compare to the fresh installed 4.1 DB.
We may need someone to review and fix if Rohit isn't around to fix this one in 
the next day or so.  Pending his response to a query about availability.

* Abhinandan - CLOUDSTACK-1842 ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 
Guest OS Types on the Upgraded Setup   Abhinandan Prateek
This bug was assigned to Abhinandan when I went to look at it.  I sent an email 
out to the group asking a couple of questions as well.  Let's see if we can 
wrap this one up (as I believe the solution should be fairly straight forward).

* Marcus - CLOUDSTACK-1845 KVM - storage migration often fails Marcus Sorensen
* Marcus - CLOUDSTACK-1846 KVM - storage pools can silently fail to be 
unregistered, leading to failure to register later  Marcus Sorensen Marcus, I 
applied the fix for these to 4.1.  Can you solve the 4.0 branch version of the 
problem so that the bugs can be resolved?  If it's going to take awhile to get 
back to 4.0, I'd actually say that it's OK to not worry about it much.  Unless 
you were trying to get the 4.0 fix into 4.0.2, we weren't planning to do 
another bug-fix release for 4.0.x after 4.0.2.  Feel free to resolve the bugs 
if that makes sense please.

* Unassigned - CLOUDSTACK-1868 GetVmStatsCommand throws NullPointerException 
with VMWare We need someone to pick this one up.  Anyone?

* Joe - Release Notes
I think that the upgrade portion of the docs probably needs to be updated to 
reflect Wido's notes on the wiki for how to perform the upgrade process itself. 
 Do you have this one under control, or do you want help?  I'd love to have the 
upgrade instructions in the release itself (and will probably hold off on an RC 
until we have them).

* QA Team - Upgrade testing finalization I see that there was some traffic 
about upgrade testing over the weekend.  Where do we stand?  Are we done now 
that Wido provided the fixes he provided?

Any finally...  does anyone else have any outstanding work / items / bugs that 
you believe need to be addressed before we start the voting process for 4.1.0's 
first RC?

-chip

Re: Review Request: Make SHA256Salt the default password encoding and authentication mechanism for cloudstack

[Chip Childers]

On Mon, Apr 01, 2013 at 11:30:16AM -0700, Vijayendra Bhamidipati wrote:
> Putting in a new authenticator/encoder for SHA512 would be good - I'll log
> a ticket to track it as a separate effort and work on it when time
> permits. Until then, let's use SHA256 as the default since it's the best
> we've got implemented as of now.
> 
> Regards,
> Vijay

Vijay,

Mind if I take this one up (CLOUDSTACK-1870)?

Review Request: CLOUDSTACK-1846, CLOUDSTACK-1845 - KVM Storage, sometimes KVMHA will remount deleted NFS pools, causing failures when defining new storage pools.

[Marcus Sorensen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10227/
---

Review request for cloudstack and Joe Brockmeier.


Description
---

CLOUDSTACK-1846, CLOUDSTACK-1845 - KVM Storage, sometimes KVMHA will remount
deleted NFS pools, causing failures when defining new storage pools. 
Sometimes
a storage pool has never been used on a host, and getStoragePool fails when
copying templates or in storage migration. deleteStoragePool(pool) often 
fails
silently, leaving no pool defined in libvirt, but a mountpoint left behind.
This patch handles some of these exceptions and brings forward any issues 
via
logging.


This addresses bugs CLOUDSTACK-1845 and CLOUDSTACK-1846.


Diffs
-

  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
 131c37e 

Diff: https://reviews.apache.org/r/10227/diff/


Testing
---

Deployed new 4.0 zone in devcloud-kvm, tested volume downloads, template 
creation, storage migration


Thanks,

Marcus Sorensen

Re: [IP clearance] merge IP cleared features into master

[Chiradeep Vittal]

+1

On 4/1/13 12:00 AM, "Murali Reddy"  wrote:

>I revoked the code for the features 'optional public IP assignment for
>EIP' [1] and 'network services in shared network' [2] features from
>master due to IP clearance issues. Both the features have passed IP
>clearance window [3][4]. Due to multiple refactors that happened at 4.1
>code freeze time, patches are invalid now. I did not get time to work on
>new patch and test them out. I would like work (its almost rework now) on
>these features now and merge them back into master.
>
>Let me know if there are any concerns.
>
>[1] https://issues.apache.org/jira/browse/CLOUDSTACK-265
>[2] https://issues.apache.org/jira/browse/CLOUDSTACK-312
>[3] 
>http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.m
>box/%3ccakprhvzyidegdeu3ptb_dctzjc5xyutsof_p+gexkzhnt_+...@mail.gmail.com%
>3E
>[4] 
>http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.m
>box/%3ccakprhvyp0xozlwuj7yomboi12a02c+woe0f9omke0k2uuxs...@mail.gmail.com%
>3E
>
>Thanks,
>Murali

Re: [ACS41] Final push to an RC?

[Marcus Sorensen]

I've put a patch for jzb on review board for 4.0, re 1845/1846

https://reviews.apache.org/r/10227/


On Mon, Apr 1, 2013 at 2:14 PM, Sudha Ponnaganti <
sudha.ponnaga...@citrix.com> wrote:

> Upgrade resolved defects are being validated tonight and tomorrow morning.
> 1839/1842 are  required but they can be validated in RC Candidate drop as
> it seem to be isolated fix and may not impact anything else.
>
> I need to cleanup some of the New and Improved feature list i.e mark them
> closed based on resolved items and testing done. Haven't done in last
> couple of weeks. Will do after I finish one more task for day job.
>
> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Monday, April 01, 2013 1:09 PM
> To: dev@cloudstack.apache.org
> Cc: Edison Su; bhais...@apache.org; Abhinandan Prateek;
> shadow...@gmail.com
> Subject: [ACS41] Final push to an RC?
>
> Hi all,
>
> We're getting there...  Here's what I see as outstanding:
>
> * Edison - CLOUDSTACK-1224 Volume snapshot creation failing This bug was
> recently bumped from major to Critical.  Edison, please shout if you
> disagree with the priority bump.
>
> * Rohit - CLOUDSTACK-1839 Upgrade 4.0 -> 4.1 - Upgraded DB has lot more
> keys and indexes for many tables compare to the fresh installed 4.1 DB.
> We may need someone to review and fix if Rohit isn't around to fix this
> one in the next day or so.  Pending his response to a query about
> availability.
>
> * Abhinandan - CLOUDSTACK-1842 ASF 4.0 to 4.1 Upgrade: Missing Ubuntu
> 12.04 Guest OS Types on the Upgraded Setup   Abhinandan Prateek
> This bug was assigned to Abhinandan when I went to look at it.  I sent an
> email out to the group asking a couple of questions as well.  Let's see if
> we can wrap this one up (as I believe the solution should be fairly
> straight forward).
>
> * Marcus - CLOUDSTACK-1845 KVM - storage migration often fails Marcus
> Sorensen
> * Marcus - CLOUDSTACK-1846 KVM - storage pools can silently fail to be
> unregistered, leading to failure to register later  Marcus Sorensen Marcus,
> I applied the fix for these to 4.1.  Can you solve the 4.0 branch version
> of the problem so that the bugs can be resolved?  If it's going to take
> awhile to get back to 4.0, I'd actually say that it's OK to not worry about
> it much.  Unless you were trying to get the 4.0 fix into 4.0.2, we weren't
> planning to do another bug-fix release for 4.0.x after 4.0.2.  Feel free to
> resolve the bugs if that makes sense please.
>
> * Unassigned - CLOUDSTACK-1868 GetVmStatsCommand throws
> NullPointerException with VMWare We need someone to pick this one up.
>  Anyone?
>
> * Joe - Release Notes
> I think that the upgrade portion of the docs probably needs to be updated
> to reflect Wido's notes on the wiki for how to perform the upgrade process
> itself.  Do you have this one under control, or do you want help?  I'd love
> to have the upgrade instructions in the release itself (and will probably
> hold off on an RC until we have them).
>
> * QA Team - Upgrade testing finalization I see that there was some traffic
> about upgrade testing over the weekend.  Where do we stand?  Are we done
> now that Wido provided the fixes he provided?
>
> Any finally...  does anyone else have any outstanding work / items / bugs
> that you believe need to be addressed before we start the voting process
> for 4.1.0's first RC?
>
> -chip
>
>

Re: Review Request: (CLOUDSTACK-1638) Network plugins won't be notified VM migration.

[Chiradeep Vittal]

Yes, please. If you add a no-op implementation for all the existing
network elements then there is no impact.

On 4/1/13 12:28 AM, "Hiroaki Kawai"  wrote:

>
>
>> On March 29, 2013, 8 p.m., Chiradeep Vittal wrote:
>> > I do think an explicit migration interface on NetworkElement is the
>>right way to do it. This way, network elements can decide explicitly
>>when and how to handle this state.
>> > Sprinkling
>> >if(!nic.getReservationId().equals(context.getReservationId())){
>> > // migration operation
>> > return;
>> > }
>> > everywhere is error prone:
>> > - Implementors of new NetworkElements are not aware of this
>>indirectly expressed dependency
>> > - This snippet of code (except for the comment) does not in any way
>>indicate the operation.
>
>I agree that introducing a new interface is a good solution. But the kind
>of interface changes seems to happen in the next cloudstack refactoring,
>so I implemented as shown not to change the interface as possible as I
>can. If we add a new interface, we must spread that implementation for
>that interface to every plugins anyway.
>
>If you do want to add a new interface right now, I'll create another
>patch.
>
>
>- Hiroaki
>
>
>---
>This is an automatically generated e-mail. To reply, visit:
>https://reviews.apache.org/r/9871/#review18531
>---
>
>
>On March 29, 2013, 1:49 a.m., Hiroaki Kawai wrote:
>> 
>> ---
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/9871/
>> ---
>> 
>> (Updated March 29, 2013, 1:49 a.m.)
>> 
>> 
>> Review request for cloudstack, Hugo Trippaers and Chiradeep Vittal.
>> 
>> 
>> Description
>> ---
>> 
>> The location of the virtual machine is provided by DeployDestination,
>>which will be passed in NetworkGuru#reserve and NetworkElement#prepare.
>> 
>> During the virtual machine migration, it actually changes
>>DeployDestination and it looks like that it will tell that event to
>>network components as it has NetworkManager#prepareNicForMigration. The
>>problem is that althogh the interface has that method,
>>NetworkManagerImpl does not tell the DeployDestination changes to
>>network components.
>> 
>> So IMHO, we need to add calls of NetworkGuru#reserve and
>>NetworkElement#prepare in NetworkManagerImpl#prepareNicForMigration .
>>And then, we also need to add calls NetworkGuru#release and
>>NetworkElement#release after the migration, otherwise the network
>>resources that plugin reserved will be kept even when the vm leaves off.
>> 
>> Created a first minimum patch to show the concept.
>> 
>> 
>> This addresses bug CLOUDSTACK-1638.
>> 
>> 
>> Diffs
>> -
>> 
>>   docs/en-US/plugin-niciranvp-tables.xml 4f81655
>>   
>>plugins/network-elements/nicira-nvp/src/com/cloud/network/NiciraNvpNicMap
>>pingVO.java 0779e69
>>   
>>plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraN
>>vpElement.java 1fcccdb
>>   server/src/com/cloud/network/NetworkManager.java 4124b19
>>   server/src/com/cloud/network/NetworkManagerImpl.java a98bdd4
>>   server/src/com/cloud/network/guru/ControlNetworkGuru.java 934cd70
>>   server/src/com/cloud/network/guru/DirectNetworkGuru.java ee824af
>>   server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java
>>354d7ed 
>>   server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java
>>24d24f8 
>>   server/src/com/cloud/network/guru/GuestNetworkGuru.java cebfb08
>>   server/src/com/cloud/network/guru/PodBasedNetworkGuru.java b513325
>>   server/src/com/cloud/network/guru/StorageNetworkGuru.java 879d0cd
>>   server/src/com/cloud/vm/VirtualMachineManagerImpl.java 9230f4a
>>   setup/db/create-schema.sql 5b6dc04
>> 
>> Diff: https://reviews.apache.org/r/9871/diff/
>> 
>> 
>> Testing
>> ---
>> 
>> 
>> Thanks,
>> 
>> Hiroaki Kawai
>> 
>>
>

RE: CLOUDSTACK-1868: GetVmStatsCommand throws NullPointerException with VMWare

[Musayev, Ilya]

Chip,

It happened for me as well only once and cannot find a way to reproduce it.

Regards
ilya

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Monday, April 01, 2013 3:52 PM
To: dev@cloudstack.apache.org
Subject: CLOUDSTACK-1868: GetVmStatsCommand throws NullPointerException with 
VMWare

Can I get a volunteer to look at CLOUSTACK-1868 ASAP?  It's one of a
*handful* of remaining bugs for 4.1.

-chip

Re: CLOUDSTACK-1842: ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 Guest OS Types on the Upgraded Setup

[Chiradeep Vittal]

On 4/1/13 12:46 PM, "Chip Childers"  wrote:

>I went digging into CLOUDSTACK-1842, and I see the following:
>
>In looking at the SQL scripts, I see that the guest_os and
>guest_os_hypervisor tables are typically populated via the upgrade
>scripts during an upgrade.
>
>I also see that templates.sql has the data to populate these tables:
>
>INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name)
>VALUES (163, UUID(), 10, 'Ubuntu 12.04 (32-bit)');
>INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name)
>VALUES (164, UUID(), 10, 'Ubuntu 12.04 (64-bit)');
>
>and for VMware:
>
>INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type,
>guest_os_name, guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (32-bit)',
>162);
>INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type,
>guest_os_name, guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (64-bit)',
>163);
>
>Nothing is in templates.sql for Ubuntu 12.04 on KVM, and no Ubuntu
>(any version) for XenServer.  Is this right?  Also, should we
>populate Ubuntu 12.04 into guest_os and guest_os_hypervisor via the
>40to41 upgrade script?
>
>-chip

I know that XS 6.0.2 didn't support Ubuntu 12.04 (XS 6.1 does). But I'm
not sure if ACS 4.1 supports XS 6.1. I think that ACS 4.0 didn't support
XS 6.1

Re: CLOUDSTACK-1842: ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 Guest OS Types on the Upgraded Setup

[Chip Childers]

On Mon, Apr 01, 2013 at 01:44:39PM -0700, Chiradeep Vittal wrote:
> 
> 
> On 4/1/13 12:46 PM, "Chip Childers"  wrote:
> 
> >I went digging into CLOUDSTACK-1842, and I see the following:
> >
> >In looking at the SQL scripts, I see that the guest_os and
> >guest_os_hypervisor tables are typically populated via the upgrade
> >scripts during an upgrade.
> >
> >I also see that templates.sql has the data to populate these tables:
> >
> >INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name)
> >VALUES (163, UUID(), 10, 'Ubuntu 12.04 (32-bit)');
> >INSERT INTO `cloud`.`guest_os` (id, uuid, category_id, display_name)
> >VALUES (164, UUID(), 10, 'Ubuntu 12.04 (64-bit)');
> >
> >and for VMware:
> >
> >INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type,
> >guest_os_name, guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (32-bit)',
> >162);
> >INSERT INTO `cloud`.`guest_os_hypervisor` (hypervisor_type,
> >guest_os_name, guest_os_id) VALUES ("VmWare", 'Ubuntu 12.04 (64-bit)',
> >163);
> >
> >Nothing is in templates.sql for Ubuntu 12.04 on KVM, and no Ubuntu
> >(any version) for XenServer.  Is this right?  Also, should we
> >populate Ubuntu 12.04 into guest_os and guest_os_hypervisor via the
> >40to41 upgrade script?
> >
> >-chip
> 
> I know that XS 6.0.2 didn't support Ubuntu 12.04 (XS 6.1 does). But I'm
> not sure if ACS 4.1 supports XS 6.1. I think that ACS 4.0 didn't support
> XS 6.1
> 
>

We upgraded the java bindings in the repo to allow for XS 6.1:

https://issues.apache.org/jira/browse/CLOUDSTACK-594

Re: [ACS41] Final push to an RC?

[Joe Brockmeier]

On Mon, Apr 1, 2013, at 03:08 PM, Chip Childers wrote:
> * Joe - Release Notes
> I think that the upgrade portion of the docs probably needs to be
> updated to reflect Wido's notes on the wiki for how to perform the
> upgrade process itself.  Do you have this one under control, or do you
> want help?  I'd love to have the upgrade instructions in the release
> itself (and will probably hold off on an RC until we have them).

Yep, I saw Wido's emails on that and have everything in front of me to
update the release notes. I've already pushed some minor updates to the
release notes and will get them finished shortly. 
 
> Any finally...  does anyone else have any outstanding work / items /
> bugs that you believe need to be addressed before we start the voting
> process for 4.1.0's first RC?

Are we going directly to a vote or are we going to have an RC for
comments and *then* a vote? 

Best,

jzb
-- 
Joe Brockmeier
j...@zonker.net
Twitter: @jzb
http://www.dissociatedpress.net/

Re: Review Request: Make SHA256Salt the default password encoding and authentication mechanism for cloudstack

[Vijayendra Bhamidipati]

Sure Chip, I'll xfer the bug over to you.

Regards,
Vijay



On 4/1/13 1:27 PM, "Chip Childers"  wrote:

>On Mon, Apr 01, 2013 at 11:30:16AM -0700, Vijayendra Bhamidipati wrote:
>> Putting in a new authenticator/encoder for SHA512 would be good - I'll
>>log
>> a ticket to track it as a separate effort and work on it when time
>> permits. Until then, let's use SHA256 as the default since it's the best
>> we've got implemented as of now.
>> 
>> Regards,
>> Vijay
>
>Vijay,
>
>Mind if I take this one up (CLOUDSTACK-1870)?
>

Re: [ACS41] Final push to an RC?

[David Nalley]

On Mon, Apr 1, 2013 at 5:14 PM, Joe Brockmeier  wrote:
> On Mon, Apr 1, 2013, at 03:08 PM, Chip Childers wrote:
>> * Joe - Release Notes
>> I think that the upgrade portion of the docs probably needs to be
>> updated to reflect Wido's notes on the wiki for how to perform the
>> upgrade process itself.  Do you have this one under control, or do you
>> want help?  I'd love to have the upgrade instructions in the release
>> itself (and will probably hold off on an RC until we have them).
>
> Yep, I saw Wido's emails on that and have everything in front of me to
> update the release notes. I've already pushed some minor updates to the
> release notes and will get them finished shortly.
>
>> Any finally...  does anyone else have any outstanding work / items /
>> bugs that you believe need to be addressed before we start the voting
>> process for 4.1.0's first RC?
>
> Are we going directly to a vote or are we going to have an RC for
> comments and *then* a vote?
>

What's the difference?

--David

Re: [ACS41] Final push to an RC?

[Joe Brockmeier]

On Mon, Apr 1, 2013, at 04:25 PM, David Nalley wrote:
> On Mon, Apr 1, 2013 at 5:14 PM, Joe Brockmeier  wrote:
> > Are we going directly to a vote or are we going to have an RC for
> > comments and *then* a vote?
> >
> 
> What's the difference?

Stopping and re-starting votes when more people start testing it in
earnest and find bugs. 

Best,

jzb
-- 
Joe Brockmeier
j...@zonker.net
Twitter: @jzb
http://www.dissociatedpress.net/

Re: Review Request: CLOUDSTACK-922: LXC Support in Cloudstack

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10122/#review18603
---


Commit aa79ccf985a172603a9ff15fbaa71dbfb3dad99d in branch refs/heads/master 
from Edison Su 
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=aa79ccf ]

CLOUDSTACK-922: LXC Support in Cloudstack.

Signed-off-by: Edison Su 


- ASF Subversion and Git Services


On March 26, 2013, 9:02 p.m., Phong Nguyen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10122/
> ---
> 
> (Updated March 26, 2013, 9:02 p.m.)
> 
> 
> Review request for cloudstack, akarasulu and edison su.
> 
> 
> Description
> ---
> 
> This is a patch for adding LXC support to Cloudstack.
> 
> Please see the following links for more details:
> 
> Design doc: 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+Cloudstack
> Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-922
> 
> 
> Diffs
> -
> 
>   agent/conf/agent.properties f7eac67 
>   api/src/com/cloud/agent/api/Command.java 9cd6749 
>   api/src/com/cloud/agent/api/RebootCommand.java 299e61b 
>   api/src/com/cloud/agent/api/StopCommand.java 9ee7ce3 
>   api/src/com/cloud/hypervisor/Hypervisor.java 2e0012d 
>   api/src/com/cloud/storage/Storage.java fba12b6 
>   client/tomcatconf/componentContext.xml.in a98a41f 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/DirectVifDriver.java
>  PRE-CREATION 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
>  8ee3ea4 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtConnection.java
>  981d343 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtDomainXMLParser.java
>  b622b6d 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStorageVolumeDef.java
>  d5cd91a 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java
>  c93aeeb 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMPhysicalDisk.java
>  08f51a4 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java
>  c2bfad9 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
>  d5e6ad6 
>   
> plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/LibvirtVMDefTest.java
>  b723875 
>   scripts/storage/secondary/cloud-install-sys-tmplt 2172b28 
>   server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java 544a803 
>   server/src/com/cloud/hypervisor/LXCGuru.java PRE-CREATION 
>   server/src/com/cloud/hypervisor/kvm/discoverer/KvmServerDiscoverer.java 
> 75e00ad 
>   server/src/com/cloud/hypervisor/kvm/discoverer/LibvirtServerDiscoverer.java 
> PRE-CREATION 
>   server/src/com/cloud/hypervisor/kvm/discoverer/LxcServerDiscoverer.java 
> PRE-CREATION 
>   server/src/com/cloud/network/SshKeysDistriMonitor.java fc7c882 
>   server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java 
> 6742f7b 
>   server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java 
> 46ac7af 
>   server/src/com/cloud/template/HypervisorTemplateAdapter.java 1426421 
>   server/src/com/cloud/vm/VirtualMachineManagerImpl.java 0aeef0e 
>   setup/db/db/schema-410to420.sql eb650cc 
>   ui/scripts/system.js c0a5d14 
>   ui/scripts/templates.js 040ce4a 
>   ui/scripts/zoneWizard.js 5108c5c 
> 
> Diff: https://reviews.apache.org/r/10122/diff/
> 
> 
> Testing
> ---
> 
> Performed end-to-end testing for starting an LXC instance:
> 
> - Add an LXC cluster
> - Add an LXC host
> - Register an LXC template
> - Spin up an LXC container
> 
> Additional testing:
> 
> - Verified connectivity with direct network settings
> - Used KVM systemvm with LXC cluster
> 
> 
> Thanks,
> 
> Phong Nguyen
> 
>

Re: Review Request: CLOUDSTACK-922: LXC Support in Cloudstack

[edison su]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10122/#review18604
---

Ship it!


Ship It!

- edison su


On March 26, 2013, 9:02 p.m., Phong Nguyen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10122/
> ---
> 
> (Updated March 26, 2013, 9:02 p.m.)
> 
> 
> Review request for cloudstack, akarasulu and edison su.
> 
> 
> Description
> ---
> 
> This is a patch for adding LXC support to Cloudstack.
> 
> Please see the following links for more details:
> 
> Design doc: 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+Cloudstack
> Jira ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-922
> 
> 
> Diffs
> -
> 
>   agent/conf/agent.properties f7eac67 
>   api/src/com/cloud/agent/api/Command.java 9cd6749 
>   api/src/com/cloud/agent/api/RebootCommand.java 299e61b 
>   api/src/com/cloud/agent/api/StopCommand.java 9ee7ce3 
>   api/src/com/cloud/hypervisor/Hypervisor.java 2e0012d 
>   api/src/com/cloud/storage/Storage.java fba12b6 
>   client/tomcatconf/componentContext.xml.in a98a41f 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/DirectVifDriver.java
>  PRE-CREATION 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
>  8ee3ea4 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtConnection.java
>  981d343 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtDomainXMLParser.java
>  b622b6d 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtStorageVolumeDef.java
>  d5cd91a 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtVMDef.java
>  c93aeeb 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMPhysicalDisk.java
>  08f51a4 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/KVMStoragePoolManager.java
>  c2bfad9 
>   
> plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/storage/LibvirtStorageAdaptor.java
>  d5e6ad6 
>   
> plugins/hypervisors/kvm/test/com/cloud/hypervisor/kvm/resource/LibvirtVMDefTest.java
>  b723875 
>   scripts/storage/secondary/cloud-install-sys-tmplt 2172b28 
>   server/src/com/cloud/consoleproxy/ConsoleProxyManagerImpl.java 544a803 
>   server/src/com/cloud/hypervisor/LXCGuru.java PRE-CREATION 
>   server/src/com/cloud/hypervisor/kvm/discoverer/KvmServerDiscoverer.java 
> 75e00ad 
>   server/src/com/cloud/hypervisor/kvm/discoverer/LibvirtServerDiscoverer.java 
> PRE-CREATION 
>   server/src/com/cloud/hypervisor/kvm/discoverer/LxcServerDiscoverer.java 
> PRE-CREATION 
>   server/src/com/cloud/network/SshKeysDistriMonitor.java fc7c882 
>   server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java 
> 6742f7b 
>   server/src/com/cloud/storage/secondary/SecondaryStorageManagerImpl.java 
> 46ac7af 
>   server/src/com/cloud/template/HypervisorTemplateAdapter.java 1426421 
>   server/src/com/cloud/vm/VirtualMachineManagerImpl.java 0aeef0e 
>   setup/db/db/schema-410to420.sql eb650cc 
>   ui/scripts/system.js c0a5d14 
>   ui/scripts/templates.js 040ce4a 
>   ui/scripts/zoneWizard.js 5108c5c 
> 
> Diff: https://reviews.apache.org/r/10122/diff/
> 
> 
> Testing
> ---
> 
> Performed end-to-end testing for starting an LXC instance:
> 
> - Add an LXC cluster
> - Add an LXC host
> - Register an LXC template
> - Spin up an LXC container
> 
> Additional testing:
> 
> - Verified connectivity with direct network settings
> - Used KVM systemvm with LXC cluster
> 
> 
> Thanks,
> 
> Phong Nguyen
> 
>

RE: Call for 4.2 and 4.1.1 Release Managers!

[Animesh Chaturvedi]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Wednesday, March 27, 2013 7:20 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Call for 4.2 and 4.1.1 Release Managers!
> 
> On Wed, Mar 27, 2013 at 01:00:52PM -0400, David Nalley wrote:
> > On Thu, Mar 21, 2013 at 4:09 PM, Musayev, Ilya 
> wrote:
> > > Chip
> > >
> > > Would you please collaborate as to what release manager does. An
> examples would be nice.
> > >
> > > Thanks
> > > ilya
> >
> >
> > Hi Ilya:
> >
> > So the short description is 'cat herder'
> >
> > The real tasks/duties - assuming a feature release:
> >
> > Act as the release schedule reminder - effectively driving and
> > enforcing the dates we agreed to earlier in the release cycle.
> > Triage/manage the bugs/new features coming into a feature release,
> > ensuring the severity is set appropriately and drawing attention to
> > things that get ignored/dropped.
> > Calling for votes
> > Creating releases (and signing, and getting them uploaded and
> > mirrored) Acting as change control when we start locking down a branch
> > for release - essentially ensuring that changes after a certain period
> > get some minimum level of review and testing, and that we aren't
> > deviating from that. Chip has called himself the human gerrit because of 
> > this.
> >
> >
> > The point releases are a bit different - there are no new features,
> > and you really are trying to focus on bugfixes, so point releases
> > should be a bit less work. Experience has shown that most folks aren't
> > as happy to fix bugs as to develop new features. So you become much
> > more of a 'attention seeker' - or perhaps 'attention driver' is a
> > better word. - driving attention to things that need to be fixed. It
> > also means spending copious amounts of time in Jira (as you would in a
> > feature release, but this is even more pervasive) You essentially have
> > to look at bugs reported against newer releases and see if they apply
> > - if patches for them are applicable to your release (e.g. if I fix a
> > bug for 4.2 - does that bug apply to 4.1? Should the fix be in 4.1.1?)
> > etc.
> >
> > --David
> >
> 
> So with David's description (thanks BTW, it's been a lng $dayjob week for
> me already), we have one volunteer in Animesh (and Animesh, does your offer
> still stand?).  Anyone else want to take a crack at it?
> 
[Animesh>] Chip my offer is still on. Like Ilya I also have some ambiguity or 
unknowns on release management, I am thinking of putting together a wiki page 
on release management tasks and responsibilities, and we can collaborate on 
refining it. By the way I feel given that CloudStack is a big project there can 
be more than one person help coordinate release activities. 

> -chip

RE: [DISCUSS] VMWare Snapshot Issue

[Edison Su]

> -Original Message-
> From: Chip Childers [mailto:chip.child...@sungard.com]
> Sent: Monday, April 01, 2013 12:55 PM
> To: dev@cloudstack.apache.org; fgaudrea...@cloudops.com
> Cc: Edison Su
> Subject: Re: [DISCUSS] VMWare Snapshot Issue
> 
> Edison,
> 
> I bumped CLOUDSTACK-1224 up to critical, as Francois is mentioning that it
> breaks snapshotting.
> 
> It appears to be already assigned to you. Can you look at the bug as soon as
> possible (or reply to let us know if you aren't able to get to it)?

Fang told me the snapshot feature works on her environment(which is on 4.1), so 
I need more mgt server logs to identify the issue.

> 
> Thanks!
> 
> -chip
> 
> On Mon, Apr 01, 2013 at 11:42:32AM -0400, Francois Gaudreault wrote:
> > Hi,
> >
> > I think you should consider to "upgrade" this bug to critical for 4.1:
> > https://issues.apache.org/jira/browse/CLOUDSTACK-1224
> >
> > We just replicated the exact same thing on our setup today.  We use
> > up-to-date 4.1 with couple commit behind maybe.  This bug kills a key
> > feature (snapshots!).
> >
> > Thanks!
> >
> > --
> > Francois Gaudreault
> > Architecte de Solution Cloud | Cloud Solutions Architect
> > fgaudrea...@cloudops.com
> > 514-629-6775
> > - - -
> > CloudOps
> > 420 rue Guy
> > Montréal QC  H3J 1S6
> > www.cloudops.com
> > @CloudOps_
> >
> >

Re: Can same Xen Server Host be used for different Management Servers?

[Chiradeep Vittal]

This can be made to work (but YMMV, not for production)
The configuration table has an entry with name = 'instance.value'.
This is set to 'VM' by default (and hence the vm names on the hypervisors
have 'VM') on them.
Change this to something else (e.g., KM) on the 4.0 installation.

On 3/28/13 10:50 PM, "prasanna"  wrote:

>On 29 March 2013 11:11, Kausal Malladi  wrote:
>> Hi,
>> As part of development, I have configured a XenServer host with
>> Management Server built from CloudStack 3.0 source code and have
>> changed few things (resource allocation algorithm) in CloudStack 4.0
>> source code (for my experiment).
>> Can I use the same host for testing my cloud with management server
>> built from modified CloudStack 4.0 source code, or should I create a
>> new host?
>>
>> In short, can a XenServer host be switched between two or more
>> management servers? I think giving same IP to both management servers
>> and using one at a time should be okay. Correct me if I am wrong.
>>
>
>The xenserver host you add to cloudstack should be fresh. It shouldn't
>have any previously running VMs that you may have deployed using the
>older source. Why not use devcloud for this purpose? You can restore
>to a base snapshot as you please. DevCloud isn't very different from a
>real physical  xenserver.

RE: [DISCUSS] Affinity / Anti-affinity Rules

[Sangeetha Hariharan]

Had the following questions after going over the spec - 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups
 :

1.In the API changes section - For all the API calls listed can we include if 
the parameters are optional/Required?

2.Can new DB changes introduced by this feature be included  ?

3.CreateAffinityGroup API: 
Using this API can admin/domain admin user be able to create an affinity group 
for other accounts to use ? Or can it be created only for its own consumption?

4.ListAffinityGroups API:
Will admin/domain admin user be able to list the affinity groups of the other 
regular users ?

5.DeleteAffinityGroup API:
Can this API  be used to delete an Affinity Group / Can it be used to only to 
remove an existing VM from an affinity group? 
Why is there a need to pass AccountName and DomainId for this API call ? Is 
Affinity Group Id not enough to uniquely identify this entity ?

6. Will Affinity rules apply when we stop and start the Vms? Will it be applied 
when CloudStack performs HA ? 

7.When User tries to migrate/live migrate the VM , will the action to migrate a 
VM to a host that does not satisfy the affinity rule fail ?

8.DeployVirtualMachine API:  It is mentioned in the FS that  affinitygroupids 
OR affinitygroupnames need to be passed. If both are passed will the API fail ?

9.Is there any difference in implementation for this feature between a Basic 
Zone and Advanced zone ?  

10. In cases where there is a mismatch between service offering (host tag and 
storage tag) and anti-affinity group that the Vm is being deployed with , will 
the error message provided to the user be informative enough to know why the 
deployment failure happened (due to service offerings vs anti-affinity group) 
to take corrective actions?

11. When a VM is in "Stopped" state , will it continue to be  part of the 
affinity group ?  Or will its association with the affinity group be released 
immediately, so that new Vms deployed as part of this anti-affinity group can 
use the host on which the Vm was running before it was stopped.  

Thanks
Sangeetha

-Original Message-
From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] 
Sent: Friday, March 01, 2013 2:12 PM
To: cloudstack-...@incubator.apache.org
Cc: Manan Shah; Alex Huang
Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules



On 3/1/13 7:14 AM, "Chip Childers"  wrote:

>On Thu, Feb 28, 2013 at 03:18:51PM -0800, Prachi Damle wrote:
>> So far per the scope of the feature, Affinity groups is an entity 
>>created by an individual account and can be used, listed only by that 
>>account.
>> 
>> Wanted to know if we see any use case where one would need to create 
>>domain-level affinity groups that  all accounts in that domain can 
>>access? I can see that this may not be useful, since users would want 
>>to have VM placement preferences exclusive to their accounts and not 
>>shared with other accounts.
>> 
>> Any thoughts?
>
>I spent time thinking about this, and I'm not sure I see a use-case for 
>it.  Others might though...
Me neither

Review Request: Change StatsCollector to be of a normal CloudStack manager component to ensure proper initialization order

[Kelven Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10232/
---

Review request for cloudstack and Chip Childers.


Description
---

Change StatsCollector to be of a normal CloudStack manager component to ensure 
proper initialization order


This addresses bug CLOUDSTACK-1865.


Diffs
-

  server/src/com/cloud/server/StatsCollector.java 4ae4999 

Diff: https://reviews.apache.org/r/10232/diff/


Testing
---

Manual test to check its initialization order


Thanks,

Kelven Yang

[VMWARE]Any way to allow concurrent operations?

[Musayev, Ilya]

If I need to do a mass operation on my vmware cluster, all jobs will be 
schedule and executed one at a time.

If you need to do a mass migration from one storage to another, this may take 
enormous amount of time. vSphere allows at least 4 concurrent operations and 
others are scheduled until the queue frees up.

Is there any possible method of allowing concurrent operations and not waiting 
for job X to complete before another one executes?

Thanks
ilya

Re: [DISCUSS] VMWare Snapshot Issue

[Francois Gaudreault]

Fang told me the snapshot feature works on her environment(which is on 4.1), so 
I need more mgt server logs to identify the issue.

I'll get you the log first thing tomorrow morning.



Thanks!

-chip

On Mon, Apr 01, 2013 at 11:42:32AM -0400, Francois Gaudreault wrote:

Hi,

I think you should consider to "upgrade" this bug to critical for 4.1:
https://issues.apache.org/jira/browse/CLOUDSTACK-1224

We just replicated the exact same thing on our setup today.  We use
up-to-date 4.1 with couple commit behind maybe.  This bug kills a key
feature (snapshots!).

Thanks!

--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC  H3J 1S6
www.cloudops.com
@CloudOps_





--
Francois Gaudreault
Architecte de Solution Cloud | Cloud Solutions Architect
fgaudrea...@cloudops.com
514-629-6775
- - -
CloudOps
420 rue Guy
Montréal QC  H3J 1S6
www.cloudops.com
@CloudOps_

Re: Review Request: (CLOUDSTACK-1638) Network plugins won't be notified VM migration.

[Hiroaki KAWAI]

If we spread a new implementation all over the network elements,
then we might silently break third party network elements that
is not included in apache repository. IMHO, that's the impact.

(2013/04/02 5:30), Chiradeep Vittal wrote:

Yes, please. If you add a no-op implementation for all the existing
network elements then there is no impact.

On 4/1/13 12:28 AM, "Hiroaki Kawai"  wrote:





On March 29, 2013, 8 p.m., Chiradeep Vittal wrote:

I do think an explicit migration interface on NetworkElement is the

right way to do it. This way, network elements can decide explicitly
when and how to handle this state.

Sprinkling
if(!nic.getReservationId().equals(context.getReservationId())){
 // migration operation
 return;
 }
everywhere is error prone:
- Implementors of new NetworkElements are not aware of this

indirectly expressed dependency

- This snippet of code (except for the comment) does not in any way

indicate the operation.


I agree that introducing a new interface is a good solution. But the kind
of interface changes seems to happen in the next cloudstack refactoring,
so I implemented as shown not to change the interface as possible as I
can. If we add a new interface, we must spread that implementation for
that interface to every plugins anyway.

If you do want to add a new interface right now, I'll create another
patch.


- Hiroaki


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9871/#review18531
---


On March 29, 2013, 1:49 a.m., Hiroaki Kawai wrote:


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9871/
---

(Updated March 29, 2013, 1:49 a.m.)


Review request for cloudstack, Hugo Trippaers and Chiradeep Vittal.


Description
---

The location of the virtual machine is provided by DeployDestination,
which will be passed in NetworkGuru#reserve and NetworkElement#prepare.

During the virtual machine migration, it actually changes
DeployDestination and it looks like that it will tell that event to
network components as it has NetworkManager#prepareNicForMigration. The
problem is that althogh the interface has that method,
NetworkManagerImpl does not tell the DeployDestination changes to
network components.

So IMHO, we need to add calls of NetworkGuru#reserve and
NetworkElement#prepare in NetworkManagerImpl#prepareNicForMigration .
And then, we also need to add calls NetworkGuru#release and
NetworkElement#release after the migration, otherwise the network
resources that plugin reserved will be kept even when the vm leaves off.

Created a first minimum patch to show the concept.


This addresses bug CLOUDSTACK-1638.


Diffs
-

   docs/en-US/plugin-niciranvp-tables.xml 4f81655

plugins/network-elements/nicira-nvp/src/com/cloud/network/NiciraNvpNicMap
pingVO.java 0779e69

plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraN
vpElement.java 1fcccdb
   server/src/com/cloud/network/NetworkManager.java 4124b19
   server/src/com/cloud/network/NetworkManagerImpl.java a98bdd4
   server/src/com/cloud/network/guru/ControlNetworkGuru.java 934cd70
   server/src/com/cloud/network/guru/DirectNetworkGuru.java ee824af
   server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java
354d7ed
   server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java
24d24f8
   server/src/com/cloud/network/guru/GuestNetworkGuru.java cebfb08
   server/src/com/cloud/network/guru/PodBasedNetworkGuru.java b513325
   server/src/com/cloud/network/guru/StorageNetworkGuru.java 879d0cd
   server/src/com/cloud/vm/VirtualMachineManagerImpl.java 9230f4a
   setup/db/create-schema.sql 5b6dc04

Diff: https://reviews.apache.org/r/9871/diff/


Testing
---


Thanks,

Hiroaki Kawai

RE: [DISCUSS] Affinity / Anti-affinity Rules

[Prachi Damle]

Comments inline. 

Thanks,
Prachi

-Original Message-
From: Sangeetha Hariharan [mailto:sangeetha.hariha...@citrix.com] 
Sent: Monday, April 01, 2013 5:45 PM
To: cloudstack-...@incubator.apache.org
Subject: RE: [DISCUSS] Affinity / Anti-affinity Rules

Had the following questions after going over the spec - 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/FS+-+Affinity-Anti-affinity+groups
 :

1.In the API changes section - For all the API calls listed can we include if 
the parameters are optional/Required?

2.Can new DB changes introduced by this feature be included  ?

[Prachi] Will update FS with this info.

3.CreateAffinityGroup API: 
Using this API can admin/domain admin user be able to create an affinity group 
for other accounts to use ? Or can it be created only for its own consumption?

[Prachi] No, affinity groups can be created and used by individual account 
only. Affinity groups are a way for the user to specify her/his deployment 
preference, so the visibility/usage is limited to the account only.

4.ListAffinityGroups API:
Will admin/domain admin user be able to list the affinity groups of the other 
regular users ?

[Prachi] No, same as above.

5.DeleteAffinityGroup API:
Can this API  be used to delete an Affinity Group / Can it be used to only to 
remove an existing VM from an affinity group? 
Why is there a need to pass AccountName and DomainId for this API call ? Is 
Affinity Group Id not enough to uniquely identify this entity ?

[Prachi] It is for deleting groups of an account. The API also takes in the 
user-friendly group name too, in which case the account info is needed.
To delete VM's association to the group, updateVMAffinityGroup API will serve 
the purpose.

6. Will Affinity rules apply when we stop and start the Vms? Will it be applied 
when CloudStack performs HA ? 

[Prachi] Yes. 

7.When User tries to migrate/live migrate the VM , will the action to migrate a 
VM to a host that does not satisfy the affinity rule fail ?

[Prachi] When root Admin lists hosts available for live migration, CS will 
indicate the hosts that do not fit the affinity rules as 'Not Suitable'. If 
admin still goes ahead and migrates to such a host, migration will proceed.

8.DeployVirtualMachine API:  It is mentioned in the FS that  affinitygroupids 
OR affinitygroupnames need to be passed. If both are passed will the API fail ?

[Prachi] yes

9.Is there any difference in implementation for this feature between a Basic 
Zone and Advanced zone ?  

[Prachi] Nope

10. In cases where there is a mismatch between service offering (host tag and 
storage tag) and anti-affinity group that the Vm is being deployed with , will 
the error message provided to the user be informative enough to know why the 
deployment failure happened (due to service offerings vs anti-affinity group) 
to take corrective actions?

[Prachi] No, it will still throw out the generic InsufficentCapacity error - 
and this is correct, since the underlying error is still not enough compute or 
storage capacity in the datacenter to _match_ your needs.
The logs will however show that the affinity rules + tags are being applied - 
causing no match found.

11. When a VM is in "Stopped" state , will it continue to be  part of the 
affinity group ?  Or will its association with the affinity group be released 
immediately, so that new Vms deployed as part of this anti-affinity group can 
use the host on which the Vm was running before it was stopped.  

[Prachi] Good point. The association with affinity group is maintained. We 
never remove it unless user removes. But yes, it makes sense to avoid deploying 
new VMs in the same group, on that host. So host-anti-affinity should also 
consider last_host_id of the VM's that are in stopped state.

Thanks
Sangeetha

-Original Message-
From: Chiradeep Vittal [mailto:chiradeep.vit...@citrix.com] 
Sent: Friday, March 01, 2013 2:12 PM
To: cloudstack-...@incubator.apache.org
Cc: Manan Shah; Alex Huang
Subject: Re: [DISCUSS] Affinity / Anti-affinity Rules



On 3/1/13 7:14 AM, "Chip Childers"  wrote:

>On Thu, Feb 28, 2013 at 03:18:51PM -0800, Prachi Damle wrote:
>> So far per the scope of the feature, Affinity groups is an entity 
>>created by an individual account and can be used, listed only by that 
>>account.
>> 
>> Wanted to know if we see any use case where one would need to create 
>>domain-level affinity groups that  all accounts in that domain can 
>>access? I can see that this may not be useful, since users would want 
>>to have VM placement preferences exclusive to their accounts and not 
>>shared with other accounts.
>> 
>> Any thoughts?
>
>I spent time thinking about this, and I'm not sure I see a use-case for 
>it.  Others might though...
Me neither

Re: cloudmonkey printing enhancements proposal

[Justin Grudzien]

I created CLOUDSTACK-1875 for the JSON enhancements. I am going to begin
filling out the ticket with the work that I have done and I will submit the
first patch as soon as I figure out the rest of the non-comitter steps. I
should be able to make my first patch submission attempt tomorrow. Thanks
for all the help so far.

Justin


On Mon, Apr 1, 2013 at 8:52 AM, Chip Childers wrote:

> On Mon, Apr 01, 2013 at 05:56:11AM -0500, Justin Grudzien wrote:
> > Thanks for the information. If you haven't already put the patch for
> review
> > I would like to do so and follow the directions so I can get used to the
> > process. Let me know.
>
> Go for it Justin.
>
> Also, give me your Jira ID once you have one, so that I can give you
> karma to assign the "improvement" bug to yourself (and resolve it once
> the patch is committed).
>
> > Also, I saw the lambda functions and chose not to use
> > them because I think they make the code harder to read for the more
> novice
> > python programmer. I can go back and add it if the goal is to have
> shorter
> > code but if it were up to me I would have the code more readable. :) I
> hope
> > people find this work to be useful, I appreciate the guidance through the
> > submission process.
> >
> > Justin
> >
> >
> > On Mon, Apr 1, 2013 at 3:07 AM, Rohit Yadav  wrote:
> >
> > > Hi Justin, thanks for your first patch.
> > >
> > > As per our workflows, we use git for our version control and have a
> > > reviewboard for reviewing patches and an issue tracker for tracking
> > > bugs/features.
> > >
> > > Git:
> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git
> > >
> > > Review board:
> > > https://reviews.apache.org
> > >
> > > For filtering only passed key names, we can cut down the whole if-else
> tree
> > > using filter (which you may see I'd used at several places) and reduce
> > > list.
> > >
> > > Something like; filter(lamba key:  like
> > > key in my set of filters>, dictionary.keys()), using this filtered
> failsafe
> > > list of keys, we can generate dictionary, using a map; map(lambda x:
> > > myjson[x] = jsondictionary[x], failsafekeys) etc.
> > >
> > > I can help with this, thanks for your patch. Since it's small I can
> help
> > > apply that for future please use review board or share git formatted
> patch
> > > which would apply cleanly on master.
> > >
> > > Regards.
> > >
> > > On Mon, Apr 1, 2013 at 10:42 AM, Justin Grudzien 
> > > wrote:
> > >
> > > > Greetings,
> > > >
> > > > I am posting my first patch. I am not sure exactly how to submit a
> patch
> > > > but I figure I will paste it here and someone will tell me the right
> > > > method. I am keeping track of the work that I am doing and as Rohit
> > > > suggested I will update the mailing list as I progress. This first
> patch
> > > > adds basic JSON output with filtering. I have tested it on
> cloudmonkey
> > > > 4.1.0-0 and 4.1.0-snapshot3 and all seems well. Comments are
> appreciated.
> > > >
> > > > --- README ---
> > > > Added
> > > > 1. display = [default|json|tabularize] has been added in the config
> to
> > > > replace
> > > > tabularize = [true|false]
> > > > 2. tabularize is deprecated but we will still set it as "false" once
> the
> > > > user
> > > > removes it out of their config to avoid throwing an error. This
> will
> > > be
> > > > removed in the next major version.
> > > > 3. display = "default" is added to the [ui] section of the config if
> it
> > > is
> > > > not
> > > > present.
> > > > 4. You can now output JSON formatted text by setting the config
> display =
> > > > json
> > > > 5. You can now filter text in JSON output mode. (i.e. list users
> > > > account=grudzien filter=account,id,email). Filtered output
> returns a
> > > > properly formatted JSON document.
> > > >
> > > > Removed
> > > > 1. Removed the printing of attr keys in read_config().
> > > >
> > > > Deprecated
> > > > 1. tabularize = [true|false] is now messaged as deprecated.
> > > >
> > > > ToDo
> > > > 1. Need to format empty return messages with proper JSON messaging.
> > > > 2. Need to standardize JSON output messaging.
> > > > A. Add return code [Error|Success] to all calls.
> > > > B. Add called API verb.
> > > > 3. Count is not decreased in results when filtering completely
> > > eliminates a
> > > > result.
> > > > 4. JSON print needs to be implemented manually to support colors.
> Right
> > > now
> > > > json.dumps() pretty prints the output.
> > > > 5. Color tagging needs to be added to JSON printing.
> > > > 6. Error messages need to have proper JSON formatting.
> > > > 7. Various help text has grammar or consistency errors that need
> fixing.
> > > > 8. Make color a passable option to a command instead of a config
> > > parameter.
> > > > (i.e. list users account=grudzien color=true)
> > > > A. This will require reworking the result_filter passable
> argument to
> > > > the
> > > > various print methods since they only expect filter= in

RE: [IP Reservation] Is VM creation using the reserved IPs on cloudstack managed hosts supported?

[Abhinav Roy]

Hi,

Is creation of VMs using the reserved IP (after doing IP reservation) on 
cloudstack managed hosts supported? Because when I tried to do that the VM got 
destroyed as soon as it was created.
While on hosts which are not managed by cloudstack VM creation using reserved 
IPs went through.



Thanks and regards,
Abhinav

Re: Review Request: Make SHA256Salt the default password encoding and authentication mechanism for cloudstack

[Venkata Siva Vijayendra Bhamidipati]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10039/
---

(Updated April 2, 2013, 5:42 a.m.)


Review request for cloudstack, Hugo Trippaers, Kelven Yang, and Min Chen.


Changes
---

Attaching latest diffs on top of master, incorporating changes suggested by Min.


Description
---

Changing default password encoding mechanism from MD5 to SHA256Salted.


This addresses bug CS-1734.


Diffs (updated)
-

  api/src/org/apache/cloudstack/api/command/admin/account/CreateAccountCmd.java 
89673ea 
  api/src/org/apache/cloudstack/api/command/admin/user/CreateUserCmd.java 
fb29e1a 
  api/src/org/apache/cloudstack/api/command/admin/user/UpdateUserCmd.java 
1f31662 
  client/tomcatconf/applicationContext.xml.in 636eac2 
  client/tomcatconf/componentContext.xml.in fea1d0f 
  client/tomcatconf/nonossComponentContext.xml.in 0b02eb6 
  developer/developer-prefill.sql 6300d35 
  
plugins/user-authenticators/ldap/src/com/cloud/server/auth/LDAPUserAuthenticator.java
 61eebe5 
  
plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
 026125e 
  
plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
 52e7cb3 
  
plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
 1b29f69 
  server/src/com/cloud/server/ManagementServerImpl.java d0904e1 
  server/src/com/cloud/user/AccountManagerImpl.java 40db4ed 

Diff: https://reviews.apache.org/r/10039/diff/


Testing
---

Manual testing done for both oss and nonoss components. Both admin and users 
added later are encoded according to the scheme configured, and authenticated 
by the same scheme.

To change the order of the schemes, modify the following list properties in 
client/tomcatconf/nonossComponentContext.xml.in or 
client/tomcatconf/componentContext.xml.in as applicable, to the desired order:


 










 
 

 


Thanks,

Venkata Siva Vijayendra Bhamidipati

[ACS41][QA]Regarding 4.0.1 to 4.1 upgrade path verification

[Sailaja Mada]

Hi,

Sangeetha & Myself validated the upgrade path from 4.0 to 4.1 [KVM & Ubuntu 
Management server, KVM agent ].

It would be good if any one interested to pick and validate 4.0.1 to 4.1 
upgrade path if not done already.  Please comment.

Thanks,
Sailaja.M

RE: [ACS41] Final push to an RC?

[Sailaja Mada]

Hi,

I have one more issue with upgrade parh from 4.0 to 4.1 [Ubuntu]:

CLOUDSTACK-1877

Thanks,
Sailaja.M

-Original Message-
From: Sudha Ponnaganti [mailto:sudha.ponnaga...@citrix.com] 
Sent: Tuesday, April 02, 2013 1:46 AM
To: dev@cloudstack.apache.org
Cc: Edison Su; bhais...@apache.org; Abhinandan Prateek; shadow...@gmail.com
Subject: RE: [ACS41] Final push to an RC?

Upgrade resolved defects are being validated tonight and tomorrow morning. 
1839/1842 are  required but they can be validated in RC Candidate drop as it 
seem to be isolated fix and may not impact anything else. 

I need to cleanup some of the New and Improved feature list i.e mark them 
closed based on resolved items and testing done. Haven't done in last couple of 
weeks. Will do after I finish one more task for day job. 

-Original Message-
From: Chip Childers [mailto:chip.child...@sungard.com] 
Sent: Monday, April 01, 2013 1:09 PM
To: dev@cloudstack.apache.org
Cc: Edison Su; bhais...@apache.org; Abhinandan Prateek; shadow...@gmail.com
Subject: [ACS41] Final push to an RC?

Hi all,

We're getting there...  Here's what I see as outstanding:

* Edison - CLOUDSTACK-1224 Volume snapshot creation failing This bug was 
recently bumped from major to Critical.  Edison, please shout if you disagree 
with the priority bump.

* Rohit - CLOUDSTACK-1839 Upgrade 4.0 -> 4.1 - Upgraded DB has lot more keys 
and indexes for many tables compare to the fresh installed 4.1 DB.
We may need someone to review and fix if Rohit isn't around to fix this one in 
the next day or so.  Pending his response to a query about availability.

* Abhinandan - CLOUDSTACK-1842 ASF 4.0 to 4.1 Upgrade: Missing Ubuntu 12.04 
Guest OS Types on the Upgraded Setup   Abhinandan Prateek
This bug was assigned to Abhinandan when I went to look at it.  I sent an email 
out to the group asking a couple of questions as well.  Let's see if we can 
wrap this one up (as I believe the solution should be fairly straight forward).

* Marcus - CLOUDSTACK-1845 KVM - storage migration often fails Marcus Sorensen
* Marcus - CLOUDSTACK-1846 KVM - storage pools can silently fail to be 
unregistered, leading to failure to register later  Marcus Sorensen Marcus, I 
applied the fix for these to 4.1.  Can you solve the 4.0 branch version of the 
problem so that the bugs can be resolved?  If it's going to take awhile to get 
back to 4.0, I'd actually say that it's OK to not worry about it much.  Unless 
you were trying to get the 4.0 fix into 4.0.2, we weren't planning to do 
another bug-fix release for 4.0.x after 4.0.2.  Feel free to resolve the bugs 
if that makes sense please.

* Unassigned - CLOUDSTACK-1868 GetVmStatsCommand throws NullPointerException 
with VMWare We need someone to pick this one up.  Anyone?

* Joe - Release Notes
I think that the upgrade portion of the docs probably needs to be updated to 
reflect Wido's notes on the wiki for how to perform the upgrade process itself. 
 Do you have this one under control, or do you want help?  I'd love to have the 
upgrade instructions in the release itself (and will probably hold off on an RC 
until we have them).

* QA Team - Upgrade testing finalization I see that there was some traffic 
about upgrade testing over the weekend.  Where do we stand?  Are we done now 
that Wido provided the fixes he provided?

Any finally...  does anyone else have any outstanding work / items / bugs that 
you believe need to be addressed before we start the voting process for 4.1.0's 
first RC?

-chip

Re: Review Request: (CLOUDSTACK-1638) Network plugins won't be notified VM migration.

[Chiradeep Vittal]

Understood. However it is hard to hold an interface steady in such an
early project. 
I hope there are not too many of these non-public implementations (at
least, I have not heard of any).

On 4/1/13 7:22 PM, "Hiroaki KAWAI"  wrote:

>If we spread a new implementation all over the network elements,
>then we might silently break third party network elements that
>is not included in apache repository. IMHO, that's the impact.
>
>(2013/04/02 5:30), Chiradeep Vittal wrote:
>> Yes, please. If you add a no-op implementation for all the existing
>> network elements then there is no impact.
>>
>> On 4/1/13 12:28 AM, "Hiroaki Kawai"  wrote:
>>
>>>
>>>
 On March 29, 2013, 8 p.m., Chiradeep Vittal wrote:
> I do think an explicit migration interface on NetworkElement is the
 right way to do it. This way, network elements can decide explicitly
 when and how to handle this state.
> Sprinkling
> if(!nic.getReservationId().equals(context.getReservationId())){
>  // migration operation
>  return;  
>  }
> everywhere is error prone:
> - Implementors of new NetworkElements are not aware of this
 indirectly expressed dependency
> - This snippet of code (except for the comment) does not in any way
 indicate the operation.
>>>
>>> I agree that introducing a new interface is a good solution. But the
>>>kind
>>> of interface changes seems to happen in the next cloudstack
>>>refactoring,
>>> so I implemented as shown not to change the interface as possible as I
>>> can. If we add a new interface, we must spread that implementation for
>>> that interface to every plugins anyway.
>>>
>>> If you do want to add a new interface right now, I'll create another
>>> patch.
>>>
>>>
>>> - Hiroaki
>>>
>>>
>>> ---
>>> This is an automatically generated e-mail. To reply, visit:
>>> https://reviews.apache.org/r/9871/#review18531
>>> ---
>>>
>>>
>>> On March 29, 2013, 1:49 a.m., Hiroaki Kawai wrote:

 ---
 This is an automatically generated e-mail. To reply, visit:
 https://reviews.apache.org/r/9871/
 ---

 (Updated March 29, 2013, 1:49 a.m.)


 Review request for cloudstack, Hugo Trippaers and Chiradeep Vittal.


 Description
 ---

 The location of the virtual machine is provided by DeployDestination,
 which will be passed in NetworkGuru#reserve and
NetworkElement#prepare.

 During the virtual machine migration, it actually changes
 DeployDestination and it looks like that it will tell that event to
 network components as it has NetworkManager#prepareNicForMigration.
The
 problem is that althogh the interface has that method,
 NetworkManagerImpl does not tell the DeployDestination changes to
 network components.

 So IMHO, we need to add calls of NetworkGuru#reserve and
 NetworkElement#prepare in NetworkManagerImpl#prepareNicForMigration .
 And then, we also need to add calls NetworkGuru#release and
 NetworkElement#release after the migration, otherwise the network
 resources that plugin reserved will be kept even when the vm leaves
off.

 Created a first minimum patch to show the concept.


 This addresses bug CLOUDSTACK-1638.


 Diffs
 -

docs/en-US/plugin-niciranvp-tables.xml 4f81655

 
plugins/network-elements/nicira-nvp/src/com/cloud/network/NiciraNvpNicM
ap
 pingVO.java 0779e69

 
plugins/network-elements/nicira-nvp/src/com/cloud/network/element/Nicir
aN
 vpElement.java 1fcccdb
server/src/com/cloud/network/NetworkManager.java 4124b19
server/src/com/cloud/network/NetworkManagerImpl.java a98bdd4
server/src/com/cloud/network/guru/ControlNetworkGuru.java 934cd70
server/src/com/cloud/network/guru/DirectNetworkGuru.java ee824af
server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java
 354d7ed
server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java
 24d24f8
server/src/com/cloud/network/guru/GuestNetworkGuru.java cebfb08
server/src/com/cloud/network/guru/PodBasedNetworkGuru.java b513325
server/src/com/cloud/network/guru/StorageNetworkGuru.java 879d0cd
server/src/com/cloud/vm/VirtualMachineManagerImpl.java 9230f4a
setup/db/create-schema.sql 5b6dc04

 Diff: https://reviews.apache.org/r/9871/diff/


 Testing
 ---


 Thanks,

 Hiroaki Kawai


>>>
>>
>

Re: [DISCUSS] Palo Alto Integration

[Murali Reddy]

The 'Network' has a life cycle associated with it. Network goes from 
'allocated' state (after the design phase) to 'implemented' (after implement 
phase). Unless a network is implemented it is not ready for use in 'isolated 
network' case. Only after network is implemented, it gets full identity. Can 
you please deploy a Vm into the network and confirm you see that 
non-overlapping CIDR's is allocated? 10.1.1.0/24 you see is the default CIDR 
network gets after design phase which will be replaced once network is 
implemented.

From: Will Stevens mailto:wstev...@cloudops.com>>
Reply-To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Date: Tuesday, 2 April 2013 12:33 AM
To: "dev@cloudstack.apache.org" 
mailto:dev@cloudstack.apache.org>>
Subject: Re: [DISCUSS] Palo Alto Integration

So I have been stepping through the code and I can confirm that the 'design' 
method of ExternalGuestNetworkGuru is being hit, but it doesn't do anything, so 
it passes off work of creating the network to the 'design' method of 
GuestNetworkGuru which assigns 10.1.1.0/24 to the network 
every time I create a network.

Something I am finding strange is that 'config.getId()' gives -1, so the new 
network that is being created while in the 'design' method of 
ExternalGuestNetworkGuru does not hit the only logic in the function:

NetworkVO config = (NetworkVO) super.design(offering, plan, userSpecified, 
owner);
if (config == null) {
return null;
} else if 
(_networkModel.networkIsConfiguredForExternalNetworking(plan.getDataCenterId(), 
config.getId())) {
/* In order to revert userSpecified network setup */
config.setState(State.Allocated);
}

So the config.setState(State.Allocated) is not getting hit.

There does seem to be some logic for updating the cidr in the 'implement' 
function of ExternalGuestNetworkGuru, but that is not run until a VM is added 
to the network (from what I understand), so that is a bit strange to me.

Are the non-overlapping cidrs implemented only when a VM is added to the 
network and the same placeholder cidr is used until then?

Thanks,

Will


On Mon, Apr 1, 2013 at 11:22 AM, Will Stevens 
mailto:wstev...@cloudops.com>> wrote:
Thank you for all your help Murali...

So my Provider has been setup with isExternal = true this whole time.
public static final Provider PaloAlto = new Provider("PaloAlto", true);

If I run a debugger and then create a guest network, I see it enter the 
'design' function of the ExternalGuestNetworkGuru, but it does not do anything 
in there because the config is not null, but the config.getId() = -1, so it 
just returns the config (Network object) and doesn't really do anything.

Apparently the 'implement' method doesn't get called until a VM is attempted to 
be launched on the network.

I must be missing something because, every Isolated guest network I create on 
my provider is defaulting to the cidr of 10.1.1.0/24.  Even 
if I have multiple Isolated networks associated with the same account, they all 
by default have that cidr.

If the default behaviour of the ExternalGuestNetworkGuru is to create 
non-overlapping guest cidrs, why does it always default to the 
10.1.1.0/24 cidr when I create a new network?  I can not 
specify a gateway or netmask because it is an external network (as you can see 
from the included screenshot).
[Inline image 1]

What am I missing here?  Why am I unable to create non-overlapping cidrs with 
the ExternalGuestNetworkGuru?

Thanks,

Will


On Fri, Mar 29, 2013 at 1:23 AM, Murali Reddy 
mailto:murali.re...@citrix.com>> wrote:
On 28/03/13 10:59 PM, "Will Stevens" 
mailto:wstev...@cloudops.com>> wrote:

>I am trying to implement the non-overlapping cidrs right now and I have
>some questions.  Does the ExternalGuestNetworkGuru create networks with
>non-overlapping cidrs by default?  Or do I need to override it's 'design'
>and 'implement' methods to implement non overlapping cidrs?

Will, yes, it does by default. You can just use
'ExternalGuestNetworkGuru'. Just so that you know, there is check
'networkIsConfiguredForExternalNetworking' in ExternalGuestNetworkGuru.
Which basically checks if provider is configured as service provider using
external physical appliances. So when you declare provider, mark
'isExternal' as true in the provider constructor.

>
>If I have to write my own methods, I think I understand how to
>override ExternalGuestNetworkGuru and then get it to run by adding it to
>the components.xml (or nonoss-components.xml) as well as the
>componentContext.xml.in.
>
>If I do not have to actually write the logic for the non-overlapping cidrs
>(which i am hoping is the case), and the ExternalGuestNetworkGuru actually
>implements that logic, how would I get the ExternalGuestNetworkGuru into
>my
>flow without actually overriding the class?  I understand that th

Re: [IP clearance] merge IP cleared features into master

[Murali Reddy]

On 01/04/13 7:03 PM, "Chip Childers"  wrote:

>On Mon, Apr 01, 2013 at 07:00:07AM +, Murali Reddy wrote:
>> I revoked the code for the features 'optional public IP assignment for
>>EIP' [1] and 'network services in shared network' [2] features from
>>master due to IP clearance issues. Both the features have passed IP
>>clearance window [3][4]. Due to multiple refactors that happened at 4.1
>>code freeze time, patches are invalid now. I did not get time to work on
>>new patch and test them out. I would like work (its almost rework now)
>>on these features now and merge them back into master.
>> 
>> Let me know if there are any concerns.
>
>+1
>
>I'd suggest creating a feature branch, and then requesting a merge once
>you have things sorted out (WRT the refactoring that forces changes to
>the previously approved patches).

Ok, will go through the regular process.

>
>> 
>> [1] https://issues.apache.org/jira/browse/CLOUDSTACK-265
>> [2] https://issues.apache.org/jira/browse/CLOUDSTACK-312
>> [3] 
>>http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.
>>mbox/%3ccakprhvzyidegdeu3ptb_dctzjc5xyutsof_p+gexkzhnt_+...@mail.gmail.co
>>m%3E
>> [4] 
>>http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201301.
>>mbox/%3ccakprhvyp0xozlwuj7yomboi12a02c+woe0f9omke0k2uuxs...@mail.gmail.co
>>m%3E
>> 
>> Thanks,
>> Murali
>