Re: Managed storage with KVM

[Marcus Sorensen]

OK.  Most other storage types interrogate the storage for the
capacitywhethwr directly or through the hypervisor. This makes it dynamic
(user could add capacity and cloudstack notices), and provides accurate
accounting things like thin provisioning. I would be surprised if edison
didn't allow for this in the new storage framework.
On Sep 17, 2013 10:34 PM, "Mike Tutkowski" 
wrote:

> This should answer your question, I believe:
>
> * When you add primary storage that is based on the SolidFire plug-in, you
> specify info like host, port, number of bytes from the SAN that CS can use,
> number of IOPS from the SAN that CS can use, among other info.
>
> * When a volume is attached for the first time and the storage framework
> asks my plug-in to create a volume (LUN) on the SAN, my plug-in increments
> the used_bytes field of the cloud.storage_pool table. If the used_bytes
> would go above the capacity_bytes, then the allocator would not have
> selected my plug-in to back the storage. Additionally, if the required IOPS
> would bring the SolidFire SAN above the number of IOPS that were dedicated
> to CS, the allocator would not have selected my plug-in to back the
> storage.
>
> * When a CS volume is deleted that uses my plug-in, the storage framework
> asks my plug-in to delete the volume (LUN) on the SAN. My plug-in
> decrements the used_bytes field of the cloud.storage_pool table.
>
> So, it just boils down to we don't require the accounting of space and IOPS
> to take place on the hypervisor side.
>
>
> On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen  >wrote:
>
> > Ok, on most storage pools it shows how many GB free/used when listing
> > the pool both via API and in the UI. I'm guessing those are empty then
> > for the solid fire storage, but it seems like the user should have to
> > define some sort of pool that the luns get carved out of, and you
> > should be able to get the stats for that, right? Or is a solid fire
> > appliance only one pool per appliance? This isn't about billing, but
> > just so cloudstack itself knows whether or not there is space left on
> > the storage device, so cloudstack can go on allocating from a
> > different primary storage as this one fills up. There are also
> > notifications and things. It seems like there should be a call you can
> > handle for this, maybe Edison knows.
> >
> > On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
> > wrote:
> > > You respond to more than attach and detach, right? Don't you create
> luns
> > as
> > > well? Or are you just referring to the hypervisor stuff?
> > >
> > > On Sep 17, 2013 7:51 PM, "Mike Tutkowski" <
> mike.tutkow...@solidfire.com>
> > > wrote:
> > >>
> > >> Hi Marcus,
> > >>
> > >> I never need to respond to a CreateStoragePool call for either
> XenServer
> > >> or
> > >> VMware.
> > >>
> > >> What happens is I respond only to the Attach- and Detach-volume
> > commands.
> > >>
> > >> Let's say an attach comes in:
> > >>
> > >> In this case, I check to see if the storage is "managed." Talking
> > >> XenServer
> > >> here, if it is, I log in to the LUN that is the disk we want to
> attach.
> > >> After, if this is the first time attaching this disk, I create an SR
> > and a
> > >> VDI within the SR. If it is not the first time attaching this disk,
> the
> > >> LUN
> > >> already has the SR and VDI on it.
> > >>
> > >> Once this is done, I let the normal "attach" logic run because this
> > logic
> > >> expected an SR and a VDI and now it has it.
> > >>
> > >> It's the same thing for VMware: Just substitute datastore for SR and
> > VMDK
> > >> for VDI.
> > >>
> > >> Does that make sense?
> > >>
> > >> Thanks!
> > >>
> > >>
> > >> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
> > >> wrote:
> > >>
> > >> > What do you do with Xen? I imagine the user enter the SAN details
> when
> > >> > registering the pool? A the pool details are basically just
> > instructions
> > >> > on
> > >> > how to log into a target, correct?
> > >> >
> > >> > You can choose to log in a KVM host to the target during
> > >> > createStoragePool
> > >> > and save the pool in a map, or just save the pool info in a map for
> > >> > future
> > >> > reference by uuid, for when you do need to log in. The
> > createStoragePool
> > >> > then just becomes a way to save the pool info to the agent.
> > Personally,
> > >> > I'd
> > >> > log in on the pool create and look/scan for specific luns when
> they're
> > >> > needed, but I haven't thought it through thoroughly. I just say that
> > >> > mainly
> > >> > because login only happens once, the first time the pool is used,
> and
> > >> > every
> > >> > other storage command is about discovering new luns or maybe
> > >> > deleting/disconnecting luns no longer needed. On the other hand, you
> > >> > could
> > >> > do all of the above: log in on pool create, then also check if
> you're
> > >> > logged in on other commands and log in if you've lost connection.
> > >> >
> > >> > With Xen, what does your registered pool   s

RE: Call for 4.3 and 4.2.1 Release Managers!

[Animesh Chaturvedi]

> -Original Message-
> From: Abhinandan Prateek [mailto:abhinandan.prat...@citrix.com]
> Sent: Monday, September 16, 2013 10:18 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Call for 4.3 and 4.2.1 Release Managers!
> 
> If it is fine with community I am willing to take up 4.2.1 maintenance
> release.
> 
> -abhi
[Animesh>] Abhi looks like you are the only one for 4.2.1. Let me know if you 
need any help. 
> 
> On 16/09/13 5:05 pm, "Daan Hoogland"  wrote:
> 
> >Animesh, I am passing till next time, I am willing though if really no
> >one can be found.
> >
> >Daan
> >
> >On Wed, Sep 11, 2013 at 7:57 AM, Animesh Chaturvedi
> > wrote:
> >> Folks
> >>
> >> Now that 4.2.0 VOTE has passed I want to call out for any committers
> >>who are interested in taking over as the release manager for the
> >>upcoming 4.2.1 maintenance release and the 4.3.0 release.
> >>
> >>
> >> Following our 4 month release cycle the feature freeze for 4.3.0
> >>should be by end of October and as community we need to figure out the
> >>timelines for 4.2.1 maintenance release.
> >>
> >> I will update my experience and learning from the 4.2.0 release in
> >>the wiki shortly after the formal release announcement  later this
> week.
> >>
> >> Thanks
> >> Animesh
> >>
> >>
> >>
> >>

RE: Call for 4.3 and 4.2.1 Release Managers!

[Animesh Chaturvedi]

> -Original Message-
> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
> Sent: Monday, September 16, 2013 10:30 AM
> To: dev@cloudstack.apache.org
> Subject: RE: Call for 4.3 and 4.2.1 Release Managers!
> 
> 
> 
> > -Original Message-
> > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > Sent: Monday, September 16, 2013 4:35 AM
> > To: dev
> > Subject: Re: Call for 4.3 and 4.2.1 Release Managers!
> >
> > Animesh, I am passing till next time, I am willing though if really no
> > one can be found.
> >
> > Dann
> 
> [Animesh>] Daan thanks for the offer, I was getting worried that no one
> wants to be RM. Let's see who else turns up.
> 
[Animesh>] Daan do you want to take up 4.3.0? I have added  a 4.3.0 release 
page with tentative dates for now. If you don't want to do it now, I can 
continue for 4.3.0.
> >
> > On Wed, Sep 11, 2013 at 7:57 AM, Animesh Chaturvedi
> >  wrote:
> > > Folks
> > >
> > > Now that 4.2.0 VOTE has passed I want to call out for any committers
> > who are interested in taking over as the release manager for the
> > upcoming 4.2.1 maintenance release and the 4.3.0 release.
> > >
> > >
> > > Following our 4 month release cycle the feature freeze for 4.3.0
> > should be by end of October and as community we need to figure out the
> > timelines for 4.2.1 maintenance release.
> > >
> > > I will update my experience and learning from the 4.2.0 release in
> > > the
> > wiki shortly after the formal release announcement  later this week.
> > >
> > > Thanks
> > > Animesh
> > >
> > >
> > >
> > >

Re: Managed storage with KVM

[Mike Tutkowski]

I see where you're coming from.

John Burwell and I took a different approach for this kind of storage.

If you want to add capacity and/or IOPS to primary storage that's based on
my plug-in, you invoke the updateStoragePool API command and pass in the
new capacity and/or IOPS.

Your existing volumes - the ones in use by the hypervisor - are another
story, though.

I have a JIRA ticket to look into how to modify via CS the capacity and
IOPS of a volume that's based on the SolidFire SAN.

Changing IOPS is no problem. Changing the size of the volume is a bit more
involved.


On Tue, Sep 17, 2013 at 11:03 PM, Marcus Sorensen wrote:

> OK.  Most other storage types interrogate the storage for the
> capacitywhethwr directly or through the hypervisor. This makes it dynamic
> (user could add capacity and cloudstack notices), and provides accurate
> accounting things like thin provisioning. I would be surprised if edison
> didn't allow for this in the new storage framework.
> On Sep 17, 2013 10:34 PM, "Mike Tutkowski" 
> wrote:
>
> > This should answer your question, I believe:
> >
> > * When you add primary storage that is based on the SolidFire plug-in,
> you
> > specify info like host, port, number of bytes from the SAN that CS can
> use,
> > number of IOPS from the SAN that CS can use, among other info.
> >
> > * When a volume is attached for the first time and the storage framework
> > asks my plug-in to create a volume (LUN) on the SAN, my plug-in
> increments
> > the used_bytes field of the cloud.storage_pool table. If the used_bytes
> > would go above the capacity_bytes, then the allocator would not have
> > selected my plug-in to back the storage. Additionally, if the required
> IOPS
> > would bring the SolidFire SAN above the number of IOPS that were
> dedicated
> > to CS, the allocator would not have selected my plug-in to back the
> > storage.
> >
> > * When a CS volume is deleted that uses my plug-in, the storage framework
> > asks my plug-in to delete the volume (LUN) on the SAN. My plug-in
> > decrements the used_bytes field of the cloud.storage_pool table.
> >
> > So, it just boils down to we don't require the accounting of space and
> IOPS
> > to take place on the hypervisor side.
> >
> >
> > On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen  > >wrote:
> >
> > > Ok, on most storage pools it shows how many GB free/used when listing
> > > the pool both via API and in the UI. I'm guessing those are empty then
> > > for the solid fire storage, but it seems like the user should have to
> > > define some sort of pool that the luns get carved out of, and you
> > > should be able to get the stats for that, right? Or is a solid fire
> > > appliance only one pool per appliance? This isn't about billing, but
> > > just so cloudstack itself knows whether or not there is space left on
> > > the storage device, so cloudstack can go on allocating from a
> > > different primary storage as this one fills up. There are also
> > > notifications and things. It seems like there should be a call you can
> > > handle for this, maybe Edison knows.
> > >
> > > On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
> > > wrote:
> > > > You respond to more than attach and detach, right? Don't you create
> > luns
> > > as
> > > > well? Or are you just referring to the hypervisor stuff?
> > > >
> > > > On Sep 17, 2013 7:51 PM, "Mike Tutkowski" <
> > mike.tutkow...@solidfire.com>
> > > > wrote:
> > > >>
> > > >> Hi Marcus,
> > > >>
> > > >> I never need to respond to a CreateStoragePool call for either
> > XenServer
> > > >> or
> > > >> VMware.
> > > >>
> > > >> What happens is I respond only to the Attach- and Detach-volume
> > > commands.
> > > >>
> > > >> Let's say an attach comes in:
> > > >>
> > > >> In this case, I check to see if the storage is "managed." Talking
> > > >> XenServer
> > > >> here, if it is, I log in to the LUN that is the disk we want to
> > attach.
> > > >> After, if this is the first time attaching this disk, I create an SR
> > > and a
> > > >> VDI within the SR. If it is not the first time attaching this disk,
> > the
> > > >> LUN
> > > >> already has the SR and VDI on it.
> > > >>
> > > >> Once this is done, I let the normal "attach" logic run because this
> > > logic
> > > >> expected an SR and a VDI and now it has it.
> > > >>
> > > >> It's the same thing for VMware: Just substitute datastore for SR and
> > > VMDK
> > > >> for VDI.
> > > >>
> > > >> Does that make sense?
> > > >>
> > > >> Thanks!
> > > >>
> > > >>
> > > >> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
> > > >> wrote:
> > > >>
> > > >> > What do you do with Xen? I imagine the user enter the SAN details
> > when
> > > >> > registering the pool? A the pool details are basically just
> > > instructions
> > > >> > on
> > > >> > how to log into a target, correct?
> > > >> >
> > > >> > You can choose to log in a KVM host to the target during
> > > >> > createStoragePool
> > > >> > and save the pool in a map, or just save the pool info in a 

Re: Managed storage with KVM

[Marcus Sorensen]

I'm surprised there's no mention of pool on the SAN in your description of
the framework. I had assumed this was specific to your implementation,
because normally SANs host multiple disk pools, maybe multiple RAID 50s and
10s, or however the SAN admin wants to split it up. Maybe a pool intended
for root disks and a separate one for data disks. Or one pool for
cloudstack and one dedicated to some other internal db application. But it
sounds as though there's no place to specify which disks or pool on the SAN
to use.

We implemented our own internal storage SAN plugin based on 4.1. We used
the 'path' attribute of the primary storage pool object to specify which
pool name on the back end SAN to use, so we could create all-ssd pools and
slower spindle pools, then differentiate between them based on storage
tags. Normally the path attribute would be the mount point for NFS, but its
just a string. So when registering ours we enter San dns host name, the
san's rest api port, and the pool name. Then luns created from that primary
storage come from the matching disk pool on the SAN. We can create and
register multiple pools of different types and purposes on the same SAN. We
haven't yet gotten to porting it to the 4.2 frame work, so it will be
interesting to see what we can come up with to make it work similarly.
 On Sep 17, 2013 10:43 PM, "Mike Tutkowski" 
wrote:

> What you're saying here is definitely something we should talk about.
>
> Hopefully my previous e-mail has clarified how this works a bit.
>
> It mainly comes down to this:
>
> For the first time in CS history, primary storage is no longer required to
> be preallocated by the admin and then handed to CS. CS volumes don't have
> to share a preallocated volume anymore.
>
> As of 4.2, primary storage can be based on a SAN (or some other storage
> device). You can tell CS how many bytes and IOPS to use from this storage
> device and CS invokes the appropriate plug-in to carve out LUNs
> dynamically.
>
> Each LUN is home to one and only one data disk. Data disks - in this model
> - never share a LUN.
>
> The main use case for this is so a CS volume can deliver guaranteed IOPS if
> the storage device (ex. SolidFire SAN) delivers guaranteed IOPS on a
> LUN-by-LUN basis.
>
>
> On Tue, Sep 17, 2013 at 10:16 PM, Marcus Sorensen  >wrote:
>
> > I guess whether or not a solidfire device is capable of hosting
> > multiple disk pools is irrelevant, we'd hope that we could get the
> > stats (maybe 30TB availabie, and 15TB allocated in LUNs). But if these
> > stats aren't collected, I can't as an admin define multiple pools and
> > expect cloudstack to allocate evenly from them or fill one up and move
> > to the next, because it doesn't know how big it is.
> >
> > Ultimately this discussion has nothing to do with the KVM stuff
> > itself, just a tangent, but something to think about.
> >
> > On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen 
> > wrote:
> > > Ok, on most storage pools it shows how many GB free/used when listing
> > > the pool both via API and in the UI. I'm guessing those are empty then
> > > for the solid fire storage, but it seems like the user should have to
> > > define some sort of pool that the luns get carved out of, and you
> > > should be able to get the stats for that, right? Or is a solid fire
> > > appliance only one pool per appliance? This isn't about billing, but
> > > just so cloudstack itself knows whether or not there is space left on
> > > the storage device, so cloudstack can go on allocating from a
> > > different primary storage as this one fills up. There are also
> > > notifications and things. It seems like there should be a call you can
> > > handle for this, maybe Edison knows.
> > >
> > > On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
> > wrote:
> > >> You respond to more than attach and detach, right? Don't you create
> > luns as
> > >> well? Or are you just referring to the hypervisor stuff?
> > >>
> > >> On Sep 17, 2013 7:51 PM, "Mike Tutkowski" <
> mike.tutkow...@solidfire.com
> > >
> > >> wrote:
> > >>>
> > >>> Hi Marcus,
> > >>>
> > >>> I never need to respond to a CreateStoragePool call for either
> > XenServer
> > >>> or
> > >>> VMware.
> > >>>
> > >>> What happens is I respond only to the Attach- and Detach-volume
> > commands.
> > >>>
> > >>> Let's say an attach comes in:
> > >>>
> > >>> In this case, I check to see if the storage is "managed." Talking
> > >>> XenServer
> > >>> here, if it is, I log in to the LUN that is the disk we want to
> attach.
> > >>> After, if this is the first time attaching this disk, I create an SR
> > and a
> > >>> VDI within the SR. If it is not the first time attaching this disk,
> the
> > >>> LUN
> > >>> already has the SR and VDI on it.
> > >>>
> > >>> Once this is done, I let the normal "attach" logic run because this
> > logic
> > >>> expected an SR and a VDI and now it has it.
> > >>>
> > >>> It's the same thing for VMware: Just substitute datastore for SR and
> > VMDK
> > >>> 

RE: [VOTE] Apache CloudStack CloudMonkey 5.0.0 (first round)

[Sateesh Chodapuneedi]

+1 (binding)
Tested over VMware going through VM life cycle.

Regards,
Sateesh


> -Original Message-
> From: David Nalley [mailto:da...@gnsa.us]
> Sent: 17 September 2013 23:23
> To: dev@cloudstack.apache.org
> Subject: Re: [VOTE] Apache CloudStack CloudMonkey 5.0.0 (first round)
> 
> On Fri, Sep 13, 2013 at 9:23 AM, Chip Childers  
> wrote:
> > I've created a 5.0.0 release of cloudmonkey, with the following
> > artifacts up for a
> > vote:
> >
> > Git Branch and Commit SH:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a
> > =shortlog;h=refs/heads/5.0
> > Commit: 767bfbe084e24d441f1ad73ace183c09f26a276b
> >
> > List of changes:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a
> > =blob;f=CHANGES;h=54f2e32357b0a726dc6f202e4897ed4af531f8ab;hb=refs/hea
> > ds/5.0
> >
> > Source release (checksums and signatures are available at the same
> > location):
> > https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.0.0/
> >
> > PGP release keys (signed using 94BE0D7C):
> > https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >
> > I have not created testing instructions for this release artifact, but
> > would appreciate help documenting it as people to it.
> >
> > Vote will be open for at least 72 hours.
> >
> > For sanity in tallying the vote, can PMC members please be sure to
> > indicate "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> 
> 
> +1 (binding)
> 
> Verified checksums/signatures
> Built and tested
> 
> --David

Re: [jira] [Commented] (CLOUDSTACK-4690) KVM Router - to many ethernet devices created

[Wei ZHOU]

Did you build the source code in Windows?

2013/9/18, Paul Edwards (JIRA) :
>
> [
> https://issues.apache.org/jira/browse/CLOUDSTACK-4690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770450#comment-13770450
> ]
>
> Paul Edwards commented on CLOUDSTACK-4690:
> --
>
> As I said above, there was no activity in the agent log. I was tailing it at
> the time. Below I have pasted the entire log from the host where the vm was
> running, as you can see there was no action at the period expected. Now it
> maybe that I need to turn some logging on somewhere. If so please tell me
> where.
>
> Thanks
> P.
>
> 2013-09-17 06:59:14,628 INFO  [cloud.agent.AgentShell] (main:null) Agent
> started
> 2013-09-17 06:59:14,630 INFO  [cloud.agent.AgentShell] (main:null)
> Implementation Version is 4.1.1
> 2013-09-17 06:59:14,632 INFO  [cloud.agent.AgentShell] (main:null)
> agent.properties found at /etc/cloudstack/agent/agent.properties
> 2013-09-17 06:59:14,634 INFO  [cloud.agent.AgentShell] (main:null)
> Defaulting to using properties file for storage
> 2013-09-17 06:59:14,635 INFO  [cloud.agent.AgentShell] (main:null)
> Defaulting to the constant time backoff algorithm
> 2013-09-17 06:59:14,704 INFO  [cloud.agent.Agent] (main:null) id is 17
> 2013-09-17 06:59:14,724 INFO
> [resource.virtualnetwork.VirtualRoutingResource] (main:null)
> VirtualRoutingResource _scriptDir to use: scripts/network/domr/kvm
> 2013-09-17 06:59:15,313 INFO  [kvm.resource.LibvirtComputingResource]
> (main:null) No libvirt.vif.driver specififed. Defaults to BridgeVifDriver.
> 2013-09-17 06:59:15,432 INFO  [cloud.agent.Agent] (main:null) Agent [id = 17
> : type = LibvirtComputingResource : zone = 1 : pod = 1 : workers = 5 : host
> = xxx.xxx.245.48 : port = 8250
> 2013-09-17 06:59:15,450 INFO  [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:18,456 ERROR [utils.nio.NioConnection]
> (Agent-Selector:null) Unable to initialize the threads.
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:530)
> at com.cloud.utils.nio.NioClient.init(NioClient.java:64)
> at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
> at java.lang.Thread.run(Thread.java:679)
> 2013-09-17 06:59:23,475 INFO  [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:26,476 ERROR [utils.nio.NioConnection]
> (Agent-Selector:null) Unable to initialize the threads.
> java.net.NoRouteToHostException: No route to host
> at sun.nio.ch.Net.connect(Native Method)
> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:530)
> at com.cloud.utils.nio.NioClient.init(NioClient.java:64)
> at com.cloud.utils.nio.NioConnection.run(NioConnection.java:108)
> at java.lang.Thread.run(Thread.java:679)
> 2013-09-17 06:59:31,482 INFO  [utils.nio.NioClient] (Agent-Selector:null)
> Connecting to xxx.xxx.245.48:8250
> 2013-09-17 06:59:31,656 INFO  [utils.nio.NioClient] (Agent-Selector:null)
> SSL: Handshake done
> 2013-09-17 06:59:31,911 INFO  [cloud.serializer.GsonHelper]
> (Agent-Handler-1:null) Default Builder inited.
> 2013-09-17 06:59:31,953 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
> Proccess agent startup answer, agent id = 0
> 2013-09-17 06:59:31,953 INFO  [cloud.agent.Agent] (Agent-Handler-2:null) Set
> agent id 0
> 2013-09-17 06:59:31,960 INFO  [cloud.agent.Agent] (Agent-Handler-2:null)
> Startup Response Received: agent id = 0
> 2013-09-17 07:01:06,324 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Proccess agent ready command, agent id = 17
> 2013-09-17 07:01:06,325 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Set agent id 17
> 2013-09-17 07:01:06,326 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-1:null) Ready command is processed: agent id = 17
> 2013-09-17 07:01:06,576 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Proccess agent ready command, agent id = 17
> 2013-09-17 07:01:06,577 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Set agent id 17
> 2013-09-17 07:01:06,585 INFO  [cloud.agent.Agent]
> (agentRequest-Handler-2:null) Ready command is processed: agent id = 17
> 2013-09-17 07:39:57,937 WARN  [kvm.resource.KVMHAMonitor] (Script-7:null)
> Interrupting script.
> 2013-09-17 07:39:57,940 WARN  [kvm.resource.KVMHAMonitor] (Thread-42:null)
> Timed out:
> /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/kvmheartbeat.sh -i
> 172.16.16.16 -p /cassandra2 -m /mnt/18c08b3e-251e-325e-b910-3063fa00756f -h
> 172.16.16.14 .  Output is:
> 2013-09-17 07:39:57,940 WARN  [kvm.resource.KVMHAMonitor] (Thread-42:null)
> write heartbeat failed: timeout, retry: 0
> 2013-09-17 07:40:57,949 WARN  [kvm.resource.KVMHAMonitor] (Script-6:null)
> Interrupting script.
> 2013-09-17 07

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Abhinandan Prateek]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/#review26209
---


In client/tomcatconf/applicationContext.xml.in we are referring passing name 
properties to the authentication adapters e.g.:  





There adapters never make use of this property. We should just remove the 
property from applicationContext.xml.in too ?

- Abhinandan Prateek


On Sept. 16, 2013, 4:56 p.m., Darren Shepherd wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14148/
> ---
> 
> (Updated Sept. 16, 2013, 4:56 p.m.)
> 
> 
> Review request for cloudstack and Abhinandan Prateek.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
> making the setName() method not work as expected.  This patch cleans up the 
> code such that getName() will be getClass().getSimpleName() unless overridden 
> in the Spring configuration.
> 
> 
> Diffs
> -
> 
>   
> plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
>  e62a3d8 
>   
> plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
>  e5b169f 
>   
> plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
>  f102275 
>   
> plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
>  91be922 
>   server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 
> 
> Diff: https://reviews.apache.org/r/14148/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Darren Shepherd
> 
>

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/
---

(Updated Sept. 18, 2013, 5:38 a.m.)


Review request for cloudstack and Abhinandan Prateek.


Changes
---

adding kelven for spring changes.


Repository: cloudstack-git


Description
---

DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
making the setName() method not work as expected.  This patch cleans up the 
code such that getName() will be getClass().getSimpleName() unless overridden 
in the Spring configuration.


Diffs
-

  
plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
 e62a3d8 
  
plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
 e5b169f 
  
plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
 f102275 
  
plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
 91be922 
  server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 

Diff: https://reviews.apache.org/r/14148/diff/


Testing
---


Thanks,

Darren Shepherd

Re: Review Request 14148: Cleanup DefaultUserAuthenticator and removed masking _name variable

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14148/
---

(Updated Sept. 18, 2013, 5:39 a.m.)


Review request for cloudstack, Abhinandan Prateek and Kelven Yang.


Repository: cloudstack-git


Description
---

DefaultUserAuthenticator maskes the _name varible in ComponentLifecycleBase 
making the setName() method not work as expected.  This patch cleans up the 
code such that getName() will be getClass().getSimpleName() unless overridden 
in the Spring configuration.


Diffs
-

  
plugins/user-authenticators/ldap/src/org/apache/cloudstack/ldap/LdapAuthenticator.java
 e62a3d8 
  
plugins/user-authenticators/md5/src/com/cloud/server/auth/MD5UserAuthenticator.java
 e5b169f 
  
plugins/user-authenticators/plain-text/src/com/cloud/server/auth/PlainTextUserAuthenticator.java
 f102275 
  
plugins/user-authenticators/sha256salted/src/com/cloud/server/auth/SHA256SaltedUserAuthenticator.java
 91be922 
  server/src/com/cloud/server/auth/DefaultUserAuthenticator.java 952f724 

Diff: https://reviews.apache.org/r/14148/diff/


Testing
---


Thanks,

Darren Shepherd

RE: security around api.log

[Rajesh Battala]

If anybody got access to the api.log using the session details we can do 
execute api's and cause harm.
But the api.log is present in the mgmt server and if anybody got access to it, 
he can corrupt anything. 
Not just accessing api.log, any other services logs and get the data. I feel 
it's up to admin how to protect his system and services.

Thanks
Rajesh Battala

-Original Message-
From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com] 
Sent: Saturday, September 14, 2013 2:10 AM
To: dev@cloudstack.apache.org
Subject: security around api.log

I just noticed api.log which seems to log all the API access in a form like

2013-09-13 00:02:09,451 INFO  [a.c.c.a.ApiServer]
(2011638958@qtp-657397168-0:ctx-81b1e088 ctx-174e4a62) (userId=2
accountId=2 sessionId=7asvmtwoesbc6ia3e4kxtzrl) 127.0.0.1 -- GET
command=listZones&response=json&sessionkey=ec6h46Om8a1y3d%2BhrdIpQ85cAfc%3D&_=1379055729422
200 { "listzonesresponse" : { "count":1 ,"zone" : [ 
{"id":"cdaf82f1-3b57-4aa4-b3ce-b60173ed45f2","name":"zone1","dns1":"8.8.8.8","dns2":"8.8.4.4","internaldns1":"8.8.4.4","networktype":"Basic","securitygroupsenabled":true,"allocationstate":"Enabled","zonetoken":"6dce94e8-e8dc-3077-bfde-c6e8594bd449","dhcpprovider":"VirtualRouter","localstorageenabled":false}
] } }

The sessionId and sessionKey is logged in the file.  I haven't tried it yet, 
but can't I use that info to hijack the session?  That introduces a security 
issue in that any server operator can now hijack anybody's session.  So that 
api.log file really needs to be protected in the same way a file with a 
password in it would be.

I would suggest that we just don't log the sessionId or sessionKey.

Darren

Re: [DISCUSS] CSS framework for CloudStack UI

[Amit Das]

Hi Brian,

I agree with Edison on usage of grunt & using maven-exec to call grunt.

Will wait for your repository that has your experiments.
I believe setting up the Maven tasks will be a one-time setting & should
work without issues.


Regards,
Amit
*CloudByte Inc.* 


On Wed, Sep 18, 2013 at 5:49 AM, Edison Su  wrote:

> How about use maven-exec->call grunt->compile CSS?
> http://addyosmani.com/blog/making-maven-grunt/
> This is more flexible than sass-maven-plugin, as we can do lot of stuff in
> grunt, such unit test.
>
> > -Original Message-
> > From: Brian Federle [mailto:brian.fede...@citrix.com]
> > Sent: Tuesday, September 17, 2013 3:53 PM
> > To: 'dev@cloudstack.apache.org'
> > Cc: Rayees Namathponnan; Frank Zhang; Animesh Chaturvedi
> > Subject: RE: [DISCUSS] CSS framework for CloudStack UI
> >
> > Following up with this thread I started last month.
> >
> > For the first step (hopefully ready by 4.3), I'm planning on getting the
> > SCSS/SASS compiler working with the UI build  ->
> > https://github.com/Jasig/sass-maven-plugin. I've experimented with it
> using
> > the existing CSS in our UI, and it will be pretty easy to use right
> away, then
> > will gradually refactor over time into a more robust framework.
> Hopefully I
> > can set up a work-in-progress branch up soon once I've got something
> > tangible.
> >
> > Amit/Rayees/Frank/anyone else, let me know if you are able to find out if
> > this is fairly easy to setup; I'm not familiar at all with Maven, so
> will need a
> > fellow expert to help out here :)
> >
> > -Brian
> >
> > -Original Message-
> > From: Amit Das [mailto:amit@cloudbyte.com]
> > Sent: Monday, August 12, 2013 10:11 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] CSS framework for CloudStack UI
> >
> > Hi Brian,
> >
> > I should be able to help on the build tools part.
> >
> > However, I am new to ACS & ACS UI. So I might take some time to
> > understand the existing build process of ACS UI.
> >
> > We can surely coordinate on this.
> >
> >
> > Regards,
> > Amit
> > *CloudByte Inc.* 
> >
> >
> > On Thu, Aug 8, 2013 at 11:11 PM, Brian Federle
> > wrote:
> >
> > > That sounds awesome, though I'm not too much of a guru with either
> > > maven or grunt -- I would be glad to coordinate with anyone who wants
> > > to take that on.
> > >
> > > -Brian
> > >
> > > -Original Message-
> > > From: Amit Das [mailto:amit@cloudbyte.com]
> > > Sent: Tuesday, August 06, 2013 10:16 PM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: [DISCUSS] CSS framework for CloudStack UI
> > >
> > > This is definitely a great idea.
> > >
> > > Will it be feasible to decouple the front-end build from server-side
> > > build as mentioned in the following link:
> > > http://addyosmani.com/blog/making-maven-grunt/
> > >
> > >
> > > Regards,
> > > Amit Kumar Das
> > > *CloudByte Inc.* 
> > >
> > >
> > > On Wed, Aug 7, 2013 at 5:48 AM, Ian Duffy  wrote:
> > >
> > > > +1 to that definitely!
> > > >
> > > >
> > > > > The only problem I cannot handle is integrating the SASS compiler
> > > > > (which
> > > > converts .scss->.css) into the UI build phase. I've found a maven
> > > > plugin at https://github.com/Jasig/sass-maven-plugin, so if anyone
> > > > wants to tackle that  it would be awesome.
> > > >
> > > >
> > > > Below is a meer suggestion, I've only read about it and never done
> it.
> > > > I believe you are able to chain grunt into the maven life cycle.
> > > > This would allow us to do various things with the front end code.
> > > > This includes but is not limited to:
> > > >
> > > > - JSHint
> > > > - Unit tests via xyz(suggesting QUnit or Jasmine) framework
> > > > - Static analysis of the javascript with Plato
> > > > - Compiling SASS + LESS
> > > > - Uglifying/Minifying
> > > > - Optimise images(png crush etc.)
> > > > - Dependency management of 3rd party js libs
> > > >
> > > > It would supply us with a bunch nicer build process for the UI
> > > > elements of Cloudstack. However it would add a dependency of nodejs
> > > > to the build process.
> > > >
> > >
>

[ACS42] Resolved Defects

[Sudha Ponnaganti]

Please validate and close the following as release is getting closed now

Reporter  Total
Abhinandan Prateek  2
Ahmad Emneina   2
Alena Prokharchyk  12
angeline shen5
Anthony Xu4
Ashutosk Kelkar   1
Bharat Kumar12
Brian Angus1
Brian Federle 2
Brian Spindler2
Chandan Purushothama   16
Chiradeep Vittal   1
Dave Brosius  1
Dave Cahill  1
David Noland 1
david vz1
Devdeep Singh 3
Diogo Monteiro1
Fang Wang  4
Francois Gaudreault   1
Gaurav Aradhye   1
Harikrishna Patnala 7
Hiroaki Kawai 2
Hongtu Zang  2
Hugo Trippaers 1
ilya musayev  3
Isaac Chiang   3
Jayapal Reddy   4
Jessica Tomechak2
Jessica Wang  2
John Burwell  3
Kelven Yang   2
Kishan Kavala 9
Koushik Das2
Likitha Shetty 4
Max Clark1
Mice Xia   1
Milamber2
Min Chen1
Minying Bao   2
Murali Reddy 8
Nick Wales  1
Nitin Mehta18
Parth Jagirdar1
Paul Angus  1
Prachi Damle  9
Pranav Saxena  1
Prasanna Santhanam 4
Radhika Nair   8
Rajesh Battala   3
Ram Ganesh  2
Rayees Namathponnan6
roxanne chang  1
sadhu suresh 4
Sailaja Mada   4
Saksham Srivastava 8
Sangeetha Hariharan  2
Sanjay Tripathi  1
Sanjeev N   5
Sateesh Chodapuneedi 11
sebastien goasguen1
Shane Witbeck  1
Shanker Balan   1
Sheng Yang 3
shweta agarwal1
Simon Waterhouse 1
Simon Weller 1
Sowmya Krishnan3
Sreekanth Challa  1
Srikanteswararao Talluri2
Sudha Ponnaganti   1
Toshiaki Hatano2
Venkata Siva Vijayendra Bhamidipati  4
venkata swamybabu budumuru   2
XxYton  1
Grand Total250

Review Request 14197: CLOUDSTACK-4696 Handle Netscaler provider enable for multiple zones

[Sowmya Krishnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14197/
---

Review request for cloudstack and Prasanna Santhanam.


Bugs: CLOUDSTACK-4696


Repository: cloudstack-git


Description
---

Handle Netscaler service provider enable for multiple zones


Diffs
-

  test/integration/component/test_netscaler_nw_off.py cb49dbe 

Diff: https://reviews.apache.org/r/14197/diff/


Testing
---

Tested locally.
nosetests -v --with-marvin --marvin-config=datacenter.cfg --load 
test/integration/component/test_netscaler_nw_off.py:TestAddMultipleNSDiffZone

Test add netscaler devices in different zones ... ok

--
Ran 1 test in 20.605s

OK


Thanks,

Sowmya Krishnan

Re: HA redundant virtual router

[Sten Spans]

On Mon, 16 Sep 2013, Sheng Yang wrote:


The reason for no HA as I said before, due to the complexity. E.g, if there
can be 3 routers in the network(which control network is down but not the
guest network), and it would cause two of them with the same priority(at
certain time). The doc is mainly for describing the current policy and
reason, as the basic for possible improvement.

I haven't thought much about redundant router with HA, but many times we're
dealing with intermittent network issue, so you try to plug off then plug
in the network cable to see if HA works as expect.

The priority cannot changed on the fly, it's a parameter of keepalived
process, which is running. So at least both router need to be stopped
before priority reset. And it's not reset to minimum, since the value can
go up or down based on the different cases.


Looking at the doc you wrote I see no mention of HAProxy.
Is this intentional?

A failure of HAProxy (which we've observed in practice) would
result in a loss of service for loadbalanced ports.

Currently I'm thinking of adding something like the following:

 diff -Nru keepalived.conf.templ.orig keepalived.conf.templ
--- keepalived.conf.templ.orig  2013-09-17 09:02:28.410646521 +0200
+++ keepalived.conf.templ   2013-09-17 09:03:34.131434084 +0200
@@ -19,6 +19,12 @@
router_id [ROUTER_ID]
 }

+vrrp_script check_haproxy {
+script "/usr/bin/killall -0 haproxy"
+interval 5
+weight 10
+}
+
 vrrp_script check_bumpup {
 script "[RROUTER_BIN_PATH]/check_bumpup.sh"
 interval 5
@@ -47,6 +53,7 @@
 }

 track_script {
+check_haproxy
 check_bumpup
 heartbeat
 }


This would boost vrrp priorities if haproxy is running, trigger
a failover if it fails, and should be harmless on hosts not running 
haproxy.


--
Sten Spans

"There is a crack in everything, that's how the light gets in."
Leonard Cohen - Anthem

Re: [PROPOSAL][SIMPLIFY] Future ACS RC testing

[Daan Hoogland]

so hijackit, are we on for wednesday morning/afternoon/evening/night?

On Mon, Sep 16, 2013 at 11:52 PM, David Nalley  wrote:
> On Mon, Sep 16, 2013 at 1:17 PM, Daan Hoogland  
> wrote:
>> On Mon, Sep 16, 2013 at 7:12 PM, David Nalley  wrote:
>>> I do favor (extensive) unit tests as criteria.
>>>
>>> Disagreeing is fine - no one (least of all me) says that my opinion is
>>> right all of the time. :)
>>>
>>> Over time I've become less concerned about a feature working, and more
>>> concerned about changes in a feature breaking the rest of CloudStack.
>>
>> I will interpret this as a 'I concur' ;)
>
> :) Yes - that.
>
> --David

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14167/#review26169
---



api/src/com/cloud/network/Networks.java


Did you base this on an old version? please rebase your code so only the 
essentials of your code remain.



plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java


can you remove the commented out code? It is still in git so there is no 
problem with that



server/src/com/cloud/network/NetworkModelImpl.java


remove code in comment, please.



server/src/com/cloud/network/element/VirtualRouterElement.java


remove code in comment, please


Do you have any (unit-)tests or #!human test scenarios to validate your code?

- daan Hoogland


On Sept. 17, 2013, 3:03 a.m., tuna wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14167/
> ---
> 
> (Updated Sept. 17, 2013, 3:03 a.m.)
> 
> 
> Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
> LoadBalancing) to GRE controller
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java aea496d 
>   api/src/com/cloud/network/Networks.java 5aede05 
>   api/src/org/apache/cloudstack/api/ResponseGenerator.java b8ecef3 
>   
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>  9741763 
>   plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java 
> 3824669 
>   
> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>  6ad6e83 
>   scripts/vm/hypervisor/xenserver/ovstunnel 2b26ed6 
>   server/src/com/cloud/network/NetworkModelImpl.java d7ca639 
>   server/src/com/cloud/network/element/VirtualRouterElement.java ecf6473 
>   ui/scripts/system.js 18c3df4 
> 
> Diff: https://reviews.apache.org/r/14167/diff/
> 
> 
> Testing
> ---
> 
> Testing done. I will make a screencast demo asap.
> 
> 
> Thanks,
> 
> tuna
> 
>

xenserver tools

[Daan Hoogland]

H,

In the present template for systemvms on xen the xenserver tools are
not installed due to distribution rights issues. Now that xenserver is
open-sourced, can anyone say if they can be added again?

regards,
Daan

Re: Apache CloudStack 4.2.0 (fifth round)

[Daan Hoogland]

let me change my vote to +1,

we still have some monitor scripts that are based on the xenserver
tools, but migrations of system vms works indeed. sorry for kicking
trouble on this thread.

Daan

On Tue, Sep 17, 2013 at 3:31 AM, Marcus Sorensen  wrote:
> Some people might be expecting that they still have time.
> On Sep 16, 2013 6:55 PM, "Animesh Chaturvedi" 
> wrote:
>
>>
>>
>> 72 hours for this VOTE has passed, I had kept this VOTE open until
>> Wednesday  because of weekend, but many community members tested this over
>> weekend and we have 12 (+1) Votes and one (+0). Should I keep the VOTE open
>> until Wednesday or is the community OK to close it out.
>>
>>
>> Thanks
>> Animesh
>>
>>
>> 
>> From: Animesh Chaturvedi
>> Sent: Friday, September 13, 2013 4:12 PM
>> To: dev@cloudstack.apache.org
>> Subject: Apache CloudStack 4.2.0 (fifth round)
>>
>> I've created a 4.2.0 release, with the following artifacts up for a vote:
>>
>> Git Branch and Commit SH:
>>
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.2
>> Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f
>>
>> List of changes:
>>
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;f=CHANGES;hb=4.2
>>
>> Source release (checksums and signatures are available at the same
>> location):
>> https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/
>>
>> PGP release keys (signed using 94BE0D7C):
>> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>>
>> Testing instructions are here:
>>
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+procedure
>>
>> Vote will be open for 72 hours (Wednesday 9/18 End of Day PST).
>>
>> For sanity in tallying the vote, can PMC members please be sure to
>> indicate "(binding)" with their vote?
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>>
>>

RE: Should VPC VR go to Stop after all VMs are deleted?

[Sanjeev Neelarapu]

Vpc router is not stopped even all the instances in a VPC are deleted. Router 
will remain in running state until we explicitly stops/destroys it.

-Original Message-
From: Sowmya Krishnan [mailto:sowmya.krish...@citrix.com] 
Sent: Tuesday, September 17, 2013 11:51 AM
To: dev@cloudstack.apache.org
Subject: Should VPC VR go to Stop after all VMs are deleted?

When all instances in a VPC are deleted, I find the VPC router isn't stopped 
unlike normal isolated networks. Is this expected? 
I think it is, since initially when we create a VPC, when no VMs are created 
yet, we still start up the VPC VR. 

But we have a test (test_vpc_network. test_01_wait_network_gc) which checks for 
Router to be stopped after stopping all VMs and waits for gc to kick in. 
Expected result is that, the router should be stopped. I don't find this 
happening. I am not even sure this is the expected behavior.

RE: Should VPC VR go to Stop after all VMs are deleted?

[Sowmya Krishnan]

So seems like this test is invalid: test_vpc_network. test_01_wait_network_gc 
and rest of the 2 in that suite needs to be re done as well without the 
network.gc

> -Original Message-
> From: Sanjeev Neelarapu [mailto:sanjeev.neelar...@citrix.com]
> Sent: Tuesday, September 17, 2013 2:36 PM
> To: dev@cloudstack.apache.org
> Subject: RE: Should VPC VR go to Stop after all VMs are deleted?
> 
> Vpc router is not stopped even all the instances in a VPC are deleted. Router 
> will
> remain in running state until we explicitly stops/destroys it.
> 
> -Original Message-
> From: Sowmya Krishnan [mailto:sowmya.krish...@citrix.com]
> Sent: Tuesday, September 17, 2013 11:51 AM
> To: dev@cloudstack.apache.org
> Subject: Should VPC VR go to Stop after all VMs are deleted?
> 
> When all instances in a VPC are deleted, I find the VPC router isn't stopped 
> unlike
> normal isolated networks. Is this expected?
> I think it is, since initially when we create a VPC, when no VMs are created 
> yet,
> we still start up the VPC VR.
> 
> But we have a test (test_vpc_network. test_01_wait_network_gc) which checks
> for Router to be stopped after stopping all VMs and waits for gc to kick in.
> Expected result is that, the router should be stopped. I don't find this 
> happening.
> I am not even sure this is the expected behavior.

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[tuna]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14167/#review26170
---



api/src/com/cloud/network/Networks.java


Yes, I'm basing on an unofficial 4.1 version. Will rebase!



plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java


yes, sure



plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java


yes, sure



server/src/com/cloud/network/NetworkModelImpl.java


sure



server/src/com/cloud/network/element/VirtualRouterElement.java


will do


- tuna


On Sept. 17, 2013, 3:03 a.m., tuna wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14167/
> ---
> 
> (Updated Sept. 17, 2013, 3:03 a.m.)
> 
> 
> Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
> LoadBalancing) to GRE controller
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java aea496d 
>   api/src/com/cloud/network/Networks.java 5aede05 
>   api/src/org/apache/cloudstack/api/ResponseGenerator.java b8ecef3 
>   
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>  9741763 
>   plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java 
> 3824669 
>   
> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>  6ad6e83 
>   scripts/vm/hypervisor/xenserver/ovstunnel 2b26ed6 
>   server/src/com/cloud/network/NetworkModelImpl.java d7ca639 
>   server/src/com/cloud/network/element/VirtualRouterElement.java ecf6473 
>   ui/scripts/system.js 18c3df4 
> 
> Diff: https://reviews.apache.org/r/14167/diff/
> 
> 
> Testing
> ---
> 
> Testing done. I will make a screencast demo asap.
> 
> 
> Thanks,
> 
> tuna
> 
>

Re: Review Request 13934: CLOUDSTACK-4347 provisioning of a nicira based vpc router

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13934/
---

(Updated Sept. 17, 2013, 9:32 a.m.)


Review request for cloudstack, Chiradeep Vittal, Hugo Trippaers, and Sheng Yang.


Changes
---

rebase while still testing. no functional change, just trying to keep up to date


Bugs: 4347


Repository: cloudstack-git


Description
---

provisioning of a nicira based vpc router


Diffs (updated)
-

  api/src/com/cloud/agent/api/to/IpAddressTO.java 82c7d99 
  api/src/com/cloud/agent/api/to/NetworkTO.java 3edd4c0 
  api/src/com/cloud/network/NetworkService.java 87fecb0 
  api/src/com/cloud/network/Networks.java f8166c6 
  api/src/com/cloud/network/vpc/PrivateIp.java eb68433 
  api/src/com/cloud/network/vpc/StaticRouteProfile.java 54aa6e4 
  api/src/com/cloud/network/vpc/VpcGateway.java 9652b4b 
  api/src/com/cloud/network/vpc/VpcService.java f772879 
  
api/src/org/apache/cloudstack/api/command/admin/vpc/CreatePrivateGatewayCmd.java
 be7e784 
  api/src/org/apache/cloudstack/api/response/PrivateGatewayResponse.java 
c5c7df5 
  api/test/com/cloud/network/NetworksTest.java 07b55d2 
  core/src/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResource.java 
1fb86e0 
  engine/schema/src/com/cloud/network/vpc/VpcGatewayVO.java 2c592cd 
  engine/schema/src/com/cloud/upgrade/DatabaseUpgradeChecker.java eb987ea 
  
plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetaNetworkGuru.java
 56bbb28 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
 c94856d 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 203587a 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java 
119f117 
  plugins/hypervisors/ovm/src/com/cloud/ovm/hypervisor/OvmResourceBase.java 
8f21c13 
  
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 450c5f0 
  
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
 3cc60db 
  
plugins/network-elements/bigswitch-vns/src/com/cloud/network/guru/BigSwitchVnsGuestNetworkGuru.java
 daf65a1 
  
plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java
 c053856 
  
plugins/network-elements/f5/src/com/cloud/network/resource/F5BigIpResource.java 
ffddd30 
  
plugins/network-elements/juniper-srx/src/com/cloud/network/resource/JuniperSrxResource.java
 46ef332 
  
plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java
 58541c6 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraNvpElement.java
 686f79c 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/guru/NiciraNvpGuestNetworkGuru.java
 fc1ecd0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
 e8ff1a0 
  server/src/com/cloud/api/ApiResponseHelper.java 1ad0eab 
  server/src/com/cloud/configuration/ConfigurationManagerImpl.java fb727a1 
  server/src/com/cloud/network/ExternalDeviceUsageManagerImpl.java 3db5111 
  server/src/com/cloud/network/ExternalFirewallDeviceManagerImpl.java e4625a4 
  server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java 
b86f4ae 
  server/src/com/cloud/network/NetworkServiceImpl.java eb63fe0 
  server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java 67ebef1 
  server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java 55a33cc 
  server/src/com/cloud/network/guru/GuestNetworkGuru.java e90b84b 
  server/src/com/cloud/network/guru/PrivateNetworkGuru.java 12dce85 
  
server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java 
566f1e4 
  server/src/com/cloud/network/vpc/PrivateGatewayProfile.java 74ce002 
  server/src/com/cloud/network/vpc/PrivateIpAddress.java 2f3cf53 
  server/src/com/cloud/network/vpc/VpcManagerImpl.java 6cace7e 
  server/test/com/cloud/network/CreatePrivateNetworkTest.java ed9efd6 
  server/test/com/cloud/vpc/MockNetworkManagerImpl.java 003b774 
  utils/src/com/cloud/utils/net/NetUtils.java 7a89103 

Diff: https://reviews.apache.org/r/13934/diff/


Testing
---

This is a rebase/port of code that has been tested intensively. It is here for 
review only at this time. re-tests are needed.


Thanks,

daan Hoogland

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[tuna]

On Sept. 17, 2013, 8:24 a.m., tuna wrote:
> > Do you have any (unit-)tests or #!human test scenarios to validate your 
> > code?

I haven't made unit tests yet. Will do asap. Just try with my own scenarios.


- tuna


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14167/#review26169
---


On Sept. 17, 2013, 3:03 a.m., tuna wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14167/
> ---
> 
> (Updated Sept. 17, 2013, 3:03 a.m.)
> 
> 
> Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
> LoadBalancing) to GRE controller
> 
> 
> Diffs
> -
> 
>   api/src/com/cloud/network/Network.java aea496d 
>   api/src/com/cloud/network/Networks.java 5aede05 
>   api/src/org/apache/cloudstack/api/ResponseGenerator.java b8ecef3 
>   
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>  9741763 
>   plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java 
> 3824669 
>   
> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>  6ad6e83 
>   scripts/vm/hypervisor/xenserver/ovstunnel 2b26ed6 
>   server/src/com/cloud/network/NetworkModelImpl.java d7ca639 
>   server/src/com/cloud/network/element/VirtualRouterElement.java ecf6473 
>   ui/scripts/system.js 18c3df4 
> 
> Diff: https://reviews.apache.org/r/14167/diff/
> 
> 
> Testing
> ---
> 
> Testing done. I will make a screencast demo asap.
> 
> 
> Thanks,
> 
> tuna
> 
>

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[tuna]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14167/
---

(Updated Sept. 17, 2013, 10:03 a.m.)


Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.


Changes
---

The update patch aim to remove commented code and old files


Repository: cloudstack-git


Description
---

I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
LoadBalancing) to GRE controller


Diffs (updated)
-

  api/src/com/cloud/network/Network.java aea496d 
  api/src/com/cloud/network/Networks.java 5aede05 
  
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
 9741763 
  
plugins/network-elements/ovs/src/com/cloud/api/response/OvsDeviceResponse.java 
c0901b2 
  
plugins/network-elements/ovs/src/com/cloud/network/commands/AddOvsDeviceCmd.java
 1abc324 
  
plugins/network-elements/ovs/src/com/cloud/network/commands/DeleteOvsDeviceCmd.java
 87eedfb 
  
plugins/network-elements/ovs/src/com/cloud/network/commands/ListOvsDevicesCmd.java
 2adb33a 
  plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java 
3824669 
  plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsApi.java b533312 
  plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsApiException.java 
20603e0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsCreateGreTunnelAnswer.java
 5f0f8c1 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsCreateGreTunnelCommand.java
 e2cd2d8 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsCreateTunnelAnswer.java
 fc2eb8a 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsCreateTunnelCommand.java
 1ececa0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsDeleteFlowCommand.java
 2a6d5d7 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsDestroyBridgeCommand.java
 8be5586 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsDestroyTunnelCommand.java
 4594d99 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsFetchInterfaceAnswer.java
 1ee6606 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsFetchInterfaceCommand.java
 c27daf0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsSetTagAndFlowAnswer.java
 ba16839 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsSetTagAndFlowCommand.java
 17121a0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsSetupBridgeCommand.java
 29cce15 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
 6ad6e83 
  plugins/network-elements/ovs/src/com/cloud/network/ovs/StartupOvsCommand.java 
b85331e 
  plugins/network-elements/ovs/src/com/cloud/network/ovs/dao/OvsDeviceDao.java 
794e45e 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/dao/OvsDeviceDaoImpl.java
 11a4d48 
  plugins/network-elements/ovs/src/com/cloud/network/ovs/dao/OvsDeviceVO.java 
cab63f6 
  plugins/network-elements/ovs/src/com/cloud/network/resource/OvsResource.java 
a94e4f8 
  server/src/com/cloud/network/NetworkModelImpl.java d7ca639 
  server/src/com/cloud/network/element/VirtualRouterElement.java ecf6473 
  ui/scripts/system.js 18c3df4 

Diff: https://reviews.apache.org/r/14167/diff/


Testing
---

Testing done. I will make a screencast demo asap.


Thanks,

tuna

Re: Review Request 14167: [GSoC] Adding LB, PF service to GRE controller

[Nguyen Anh Tu]

Hi guys, I made an update patch, which aim to remove commented code and old
files. Please review it


2013/9/17 tuna 

>This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14167/
>
> On September 17th, 2013, 8:24 a.m. UTC, *daan Hoogland* wrote:
>
> Do you have any (unit-)tests or #!human test scenarios to validate your code?
>
>  I haven't made unit tests yet. Will do asap. Just try with my own scenarios.
>
>
> - tuna
>
> On September 17th, 2013, 3:03 a.m. UTC, tuna wrote:
>   Review request for cloudstack, Sebastien Goasguen and Hugo Trippaers.
> By tuna.
>
> *Updated Sept. 17, 2013, 3:03 a.m.*
>  *Repository: * cloudstack-git
> Description
>
> I add a final patch for supporting L3 services (staticNAT, PortForwarding, 
> LoadBalancing) to GRE controller
>
>   Testing
>
> Testing done. I will make a screencast demo asap.
>
>   Diffs
>
>- api/src/com/cloud/network/Network.java (aea496d)
>- api/src/com/cloud/network/Networks.java (5aede05)
>- api/src/org/apache/cloudstack/api/ResponseGenerator.java (b8ecef3)
>- 
> plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
>(9741763)
>- 
> plugins/network-elements/ovs/src/com/cloud/network/element/OvsElement.java
>(3824669)
>- 
> plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
>(6ad6e83)
>- scripts/vm/hypervisor/xenserver/ovstunnel (2b26ed6)
>- server/src/com/cloud/network/NetworkModelImpl.java (d7ca639)
>- server/src/com/cloud/network/element/VirtualRouterElement.java
>(ecf6473)
>- ui/scripts/system.js (18c3df4)
>
> View Diff 
>



-- 

N.g.U.y.e.N.A.n.H.t.U

Problem executing async jobs explicitly

[Gaurav Aradhye]

Hi all,

I am trying to run test cases, in which I am creating snapshots of root
disk volumes of vms belonging to an account. The jobs (commands) to create
snapshots are supposed to run asynchronously.

Marvin code >> For this I am executing "submitCmdsAndWait" in
cloudstackTestClient which in turn calls "asyncJobMgr.submitCmdsAndWait"
and so on

The problem is I can't see APIs getting called when jobs are submitted to
async manager. When I debugged the code, I found an error "CloudConnection"
object has no attribute make_request. That should have been
"marvin_request" (Prasanna had changed code for POST request. See >>
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201304.mbox/%3C20130423171058.GA12552@cloud-2.local%3E)

So After changing this code, the errors have gone, but still API commands
are not executed and can;t be seen in management server log.

Has anybody used asyncJobMgr before? If yes, please share how you are
passing the jobs to be run asynchronously.


Regards,
Gaurav

ROOT disk resize

[Antonio Petrocelli]

Hello,

is there a way to resize the ROOT disk of a vm?

API reply is: *Can only resize DATA volumes*

Does anyone know a method to bypass that control?

Thanks a lot.

Regards.


smime.p7s
Description: Firma crittografica S/MIME

Review Request 14171: fix test_networks.py to reduce time taken by the tests

[SrikanteswaraRao Talluri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14171/
---

Review request for cloudstack, venkata swamy babu  budumuru and Prasanna 
Santhanam.


Bugs: CLOUDSTACK-4344


Repository: cloudstack-git


Description
---

pass delay and retries explicitly to remoteSSHclient to reduce time taken by 
the tests


Diffs
-

  test/integration/smoke/test_network.py 042ac84 

Diff: https://reviews.apache.org/r/14171/diff/


Testing
---

tested


Thanks,

SrikanteswaraRao Talluri

Re: xenserver tools

[Abhinandan Prateek]

Daan,
  Do you know what all packages we need to add to get the tools working. I
see that current scripts are installing xenstore-utils.


On 17/09/13 1:58 pm, "Daan Hoogland"  wrote:

>H,
>
>In the present template for systemvms on xen the xenserver tools are
>not installed due to distribution rights issues. Now that xenserver is
>open-sourced, can anyone say if they can be added again?
>
>regards,
>Daan

securitygroup confusion

[sebgoa]

Hi,

I am looking at 
http://cloudstack.apache.org/docs/api/apidocs-4.1/root_admin/authorizeSecurityGroupIngress.html

you can pass a cidrlist and a securitygroupname (exclusive with 
securitygroupid).

Are those exclusive ?

Say I want to open port 22 for 1.1.1.1/0 and all VMs in securitygroup foobar.

Can I create this rule with a single call by specifying foobar for 
securitygroupname ?

I am asking because listSecurityGroups:
http://cloudstack.apache.org/docs/api/apidocs-4.1/root_admin/listSecurityGroups.html

returns the name of the sg and a securitygroupname under the 
ingress/egress/rule 

And also on one cloud I am using some of the rules don't have a cidr key 
because they are set for sg internals.

thoughts ?

-sebastien

Re: ROOT disk resize

[Marcus Sorensen]

You'd need to make minor code changes to remove the check in the code and
recompile. It should otherwise work, its just that traditionally root disk
size is based on template size, while data disks are based on disk
offerings. Not everyone wants root resize, e.g. if they are billing based
on disk offering rather than checking actual vol size.

You could request an enhancement where root resize can be enabled via
config parameter.
On Sep 17, 2013 4:28 AM, "Antonio Petrocelli" 
wrote:

>  Hello,
>
> is there a way to resize the ROOT disk of a vm?
>
> API reply is: *Can only resize DATA volumes*
>
> Does anyone know a method to bypass that control?
>
> Thanks a lot.
>
> Regards.
>

Re: xenserver tools

[Daan Hoogland]

Hi Abhinandan,

What we have seen so far is that two files in the xe-guest-utilities
package are missing, where CS assumes that they are there.

/usr/bin/xenstore  => used in script /usr/sbin/xe-update-guest-attrs
(this script is also included in the package but overwritten by the CS
systemvm.iso scripts)
And
/etc/init.d/xe-linux-distribution => calls /usr/sbin/xe-daemon (this
script is also included in the package but overwritten by the CS
systemvm.iso scripts)

This is the package we installed to fix it and the files included in
that package.

root@r-8691-VM:~# dpkg -L xe-guest-utilities
/.
/usr
/usr/bin
/usr/bin/xenstore-write
/usr/bin/xenstore
/usr/bin/xenstore-read
/usr/bin/xenstore-exists
/usr/bin/xenstore-list
/usr/bin/xenstore-ls
/usr/bin/xenstore-chmod
/usr/bin/xenstore-rm
/usr/share
/usr/share/xe-guest-utilities
/usr/share/xe-guest-utilities/citrix.list
/usr/share/doc
/usr/share/doc/xe-guest-utilities
/usr/share/doc/xe-guest-utilities/COPYING.LGPL.gz
/usr/share/doc/xe-guest-utilities/copyright
/usr/share/doc/xe-guest-utilities/COPYING.gz
/usr/sbin
/usr/sbin/xe-update-guest-attrs
/usr/sbin/xe-linux-distribution
/usr/sbin/xe-daemon
/etc
/etc/udev
/etc/udev/rules.d
/etc/udev/rules.d/z10_xen-vcpu-hotplug.rules
/etc/init.d
/etc/init.d/xe-linux-distribution
root@r-8691-VM:~# ls xe-guest-utilities_6.0.2-766_i386.deb

Thanks,

On Tue, Sep 17, 2013 at 1:11 PM, Abhinandan Prateek
 wrote:
> Daan,
>   Do you know what all packages we need to add to get the tools working. I
> see that current scripts are installing xenstore-utils.
>
>
> On 17/09/13 1:58 pm, "Daan Hoogland"  wrote:
>
>>H,
>>
>>In the present template for systemvms on xen the xenserver tools are
>>not installed due to distribution rights issues. Now that xenserver is
>>open-sourced, can anyone say if they can be added again?
>>
>>regards,
>>Daan
>

Re: ROOT disk resize

[Antonio Petrocelli]

Thanks a lot for reply.

Do you know how change the code?

Il 17/09/2013 14:22, Marcus Sorensen ha scritto:

You'd need to make minor code changes to remove the check in the code and
recompile. It should otherwise work, its just that traditionally root disk
size is based on template size, while data disks are based on disk
offerings. Not everyone wants root resize, e.g. if they are billing based
on disk offering rather than checking actual vol size.

You could request an enhancement where root resize can be enabled via
config parameter.
On Sep 17, 2013 4:28 AM, "Antonio Petrocelli" 
wrote:


  Hello,

is there a way to resize the ROOT disk of a vm?

API reply is: *Can only resize DATA volumes*

Does anyone know a method to bypass that control?

Thanks a lot.

Regards.





smime.p7s
Description: Firma crittografica S/MIME

Re: [jira] [Commented] (CLOUDSTACK-3565) Restarting libvirtd service leading to destroy storage pool

[Marcus Sorensen]

Since this is dependent on specific versions, I'm not clear on whether
there was any difference between persistent and non persistent pools. It
was framed as though it was the cloudstack change that broke things, but
perhaps it was the libvirt update. If an NFS mount point is in use when a
*defined* NFS pool is started with an affected version, does it also fail?
I'm just trying to consider the case where we go back to persistently
defined storage and whether or not that will fix the issue. I will know
when I get a broken one to try (i will attempt stock centos next time).
Either way, we could perhaps assume that whether it works or not for
persistent, if someone on the libvirt dev team saw fit to check in the
pool.create() case, they could likely add the same in starting persistent
pools in the future.

Just brainstorming, it seems we can handle the libvirt error of 'mount
point in use' somehow.

One idea as mentioned is to 'umount -l', which should keep existing stuff
working and allow the pool to be recreated, but just seems messy/hackish to
me. I'd be OK with it though because I think it would make things 'just
work' for the end user.

Another option to consider might be to handle the error by switching to a
dir based pool like local uses, when the already mounted error is caught. I
don't think that will care if something is mounted, and should also allow
things to just work. The downsides are that the NFS mount won't clean up
when the pool is removed (small problem compared to now) and it may cause
confusion if someone goes inspecting their pool details.
On Sep 17, 2013 6:28 AM, "Marcus Sorensen (JIRA)"  wrote:

>
> [
> https://issues.apache.org/jira/browse/CLOUDSTACK-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769475#comment-13769475]
>
> Marcus Sorensen commented on CLOUDSTACK-3565:
> -
>
> Ok, now I understand why it wasn't caught as an issue before. So is this
> preemptive, preparing for when newer libvirtd are included in supported
> platforms, or does it affect maybe CentOS 6.4? What's the criteria to
> reproduce?
> On Sep 17, 2013 3:59 AM, "Wido den Hollander (JIRA)" 
>
>
>
> > Restarting libvirtd service leading to destroy storage pool
> > ---
> >
> > Key: CLOUDSTACK-3565
> > URL:
> https://issues.apache.org/jira/browse/CLOUDSTACK-3565
> > Project: CloudStack
> >  Issue Type: Bug
> >  Security Level: Public(Anyone can view this level - this is the
> default.)
> >  Components: KVM
> >Affects Versions: 4.2.0
> > Environment: KVM
> > Branch 4.2
> >Reporter: Rayees Namathponnan
> >Assignee: Marcus Sorensen
> >Priority: Blocker
> >  Labels: documentation
> > Fix For: 4.2.0
> >
> >
> > Steps to reproduce
> > Step 1 : Create cloudstack step in kvm
> > Step 2 : From kvm host check "virsh pool-list"
> > Step 3:  Stop and start libvirtd service
> > Step 4 : Check "virsh pool-list"
> > Actual Result
> > "virsh pool-list"  is blank after restart libvird service
> > [root@Rack2Host12 agent]# virsh pool-list
> > Name State  Autostart
> > -
> > 41b632b5-40b3-3024-a38b-ea259c72579f active no
> > 469da865-0712-4d4b-a4cf-a2d68f99f1b6 active no
> > fff90cb5-06dd-33b3-8815-d78c08ca01d9 active no
> > [root@Rack2Host12 agent]# service cloudstack-agent stop
> > Stopping Cloud Agent:
> > [root@Rack2Host12 agent]# virsh pool-list
> > Name State  Autostart
> > -
> > 41b632b5-40b3-3024-a38b-ea259c72579f active no
> > 469da865-0712-4d4b-a4cf-a2d68f99f1b6 active no
> > fff90cb5-06dd-33b3-8815-d78c08ca01d9 active no
> > [root@Rack2Host12 agent]# virsh list
> >  IdName   State
> > 
> > [root@Rack2Host12 agent]# service libvirtd stop
> > Stopping libvirtd daemon:  [  OK  ]
> > [root@Rack2Host12 agent]# service libvirtd start
> > Starting libvirtd daemon:  [  OK  ]
> > [root@Rack2Host12 agent]# virsh pool-list
> > Name State  Autostart
> > -
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>

(CLOUDSTACK-692) The CleanupSnapshotBackup process on SSVM deletes snapshots that are still in the process of being copied to secondary storage

[Joris van Lieshout]

Hi there,

I was wondering if anyone can help us with this issue? There seems to be a 
situation where the CleanupSnapshotBackup process deletes vhd files belonging 
to an active BackupSnapshot process. I've created CLOUDSTACK-692 for it and 
logged as much info as possible, including the steps I use to clean the mess up 
after we have hit this. We have seen it happen in CS 4.0 and 4.1.1, and from 
what I have seen in the code it probably also exists in 4.2.
I haven't reproduced the issue in a lab because we are hitting it quite often 
in production and uat so I have all the examples I need. :) But I guess the 
best way to reproduce it is to create a vm with quite some io activity (so 
snapshots will be big), enable hourly snapshot and shorten the 
storage.cleanup.interval global setting so the cleanup process gets trigger 
more frequently.
We are hitting this on XenServer 6.0.2 but if this snapshot cleanup and backup 
process is generally the same across other HVs type I would image this is 
relevant for that as well...

Kind regards,
Joris van Lieshout

Re: Review Request 14060: CLOUDSTACK-4634 - CPU Limits/ test_maximum_limits.py - -Fixed issues arising due to wrong indentation

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14060/#review26173
---


Commit 51da534cce7bd07dcbf1e828a32e27a716b5b0f8 in branch refs/heads/disk-cache 
from Gaurav Aradhye
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=51da534 ]

CLOUDSTACK-4634: Fixed issue arising due to wrong indentation


- ASF Subversion and Git Services


On Sept. 10, 2013, 11:22 a.m., Gaurav Aradhye wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14060/
> ---
> 
> (Updated Sept. 10, 2013, 11:22 a.m.)
> 
> 
> Review request for cloudstack and Prasanna Santhanam.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> The indentation in createInstance function was wrong. Whenever we would pass 
> value to api_client parameter, the instance would not get created. Fixed the 
> indentation.
> 
> 
> Diffs
> -
> 
>   test/integration/component/cpu_limits/test_maximum_limits.py 2302504 
> 
> Diff: https://reviews.apache.org/r/14060/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Gaurav Aradhye
> 
>

Re: Review Request 14171: fix test_networks.py to reduce time taken by the tests

[Rayees Namathponnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14171/#review26174
---


New tag "test" added,  it is required ?

@attr(tags = ["advanced", "advancedns", "smoke","test"])

- Rayees Namathponnan


On Sept. 17, 2013, 10:33 a.m., SrikanteswaraRao Talluri wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14171/
> ---
> 
> (Updated Sept. 17, 2013, 10:33 a.m.)
> 
> 
> Review request for cloudstack, venkata swamy babu  budumuru and Prasanna 
> Santhanam.
> 
> 
> Bugs: CLOUDSTACK-4344
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> pass delay and retries explicitly to remoteSSHclient to reduce time taken by 
> the tests
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_network.py 042ac84 
> 
> Diff: https://reviews.apache.org/r/14171/diff/
> 
> 
> Testing
> ---
> 
> tested
> 
> 
> Thanks,
> 
> SrikanteswaraRao Talluri
> 
>

Re: Review Request 13934: CLOUDSTACK-4347 provisioning of a nicira based vpc router

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13934/
---

(Updated Sept. 17, 2013, 2:59 p.m.)


Review request for cloudstack, Chiradeep Vittal, Hugo Trippaers, and Sheng Yang.


Changes
---

tested with nicira and conflicts in vmwarresource removed


Bugs: 4347


Repository: cloudstack-git


Description
---

provisioning of a nicira based vpc router


Diffs (updated)
-

  api/src/com/cloud/agent/api/to/IpAddressTO.java 82c7d99 
  api/src/com/cloud/agent/api/to/NetworkTO.java 3edd4c0 
  api/src/com/cloud/network/NetworkService.java 87fecb0 
  api/src/com/cloud/network/Networks.java f8166c6 
  api/src/com/cloud/network/vpc/PrivateIp.java eb68433 
  api/src/com/cloud/network/vpc/StaticRouteProfile.java 54aa6e4 
  api/src/com/cloud/network/vpc/VpcGateway.java 9652b4b 
  api/src/com/cloud/network/vpc/VpcService.java f772879 
  
api/src/org/apache/cloudstack/api/command/admin/vpc/CreatePrivateGatewayCmd.java
 be7e784 
  api/src/org/apache/cloudstack/api/response/PrivateGatewayResponse.java 
c5c7df5 
  api/test/com/cloud/network/NetworksTest.java 07b55d2 
  core/src/com/cloud/agent/resource/virtualnetwork/VirtualRoutingResource.java 
1fb86e0 
  engine/schema/src/com/cloud/network/dao/NetworkDao.java d0a1a25 
  engine/schema/src/com/cloud/network/dao/NetworkDaoImpl.java 0f83815 
  engine/schema/src/com/cloud/network/vpc/VpcGatewayVO.java 2c592cd 
  engine/schema/src/com/cloud/upgrade/DatabaseUpgradeChecker.java eb987ea 
  
plugins/hypervisors/baremetal/src/com/cloud/baremetal/networkservice/BaremetaNetworkGuru.java
 56bbb28 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/BridgeVifDriver.java
 c94856d 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
 203587a 
  
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/OvsVifDriver.java 
119f117 
  plugins/hypervisors/ovm/src/com/cloud/ovm/hypervisor/OvmResourceBase.java 
8f21c13 
  
plugins/hypervisors/vmware/src/com/cloud/hypervisor/vmware/resource/VmwareResource.java
 450c5f0 
  
plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/CitrixResourceBase.java
 3cc60db 
  
plugins/network-elements/bigswitch-vns/src/com/cloud/network/guru/BigSwitchVnsGuestNetworkGuru.java
 daf65a1 
  
plugins/network-elements/cisco-vnmc/src/com/cloud/network/element/CiscoVnmcElement.java
 c053856 
  
plugins/network-elements/f5/src/com/cloud/network/resource/F5BigIpResource.java 
ffddd30 
  
plugins/network-elements/juniper-srx/src/com/cloud/network/resource/JuniperSrxResource.java
 46ef332 
  
plugins/network-elements/netscaler/src/com/cloud/network/resource/NetscalerResource.java
 58541c6 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/element/NiciraNvpElement.java
 686f79c 
  
plugins/network-elements/nicira-nvp/src/com/cloud/network/guru/NiciraNvpGuestNetworkGuru.java
 fc1ecd0 
  
plugins/network-elements/ovs/src/com/cloud/network/ovs/OvsTunnelManagerImpl.java
 e8ff1a0 
  server/src/com/cloud/api/ApiResponseHelper.java 1ad0eab 
  server/src/com/cloud/configuration/ConfigurationManagerImpl.java fb727a1 
  server/src/com/cloud/network/ExternalDeviceUsageManagerImpl.java 3db5111 
  server/src/com/cloud/network/ExternalFirewallDeviceManagerImpl.java e4625a4 
  server/src/com/cloud/network/ExternalLoadBalancerDeviceManagerImpl.java 
b86f4ae 
  server/src/com/cloud/network/NetworkServiceImpl.java eb63fe0 
  server/src/com/cloud/network/guru/DirectPodBasedNetworkGuru.java 67ebef1 
  server/src/com/cloud/network/guru/ExternalGuestNetworkGuru.java 55a33cc 
  server/src/com/cloud/network/guru/GuestNetworkGuru.java e90b84b 
  server/src/com/cloud/network/guru/PrivateNetworkGuru.java 12dce85 
  
server/src/com/cloud/network/router/VpcVirtualNetworkApplianceManagerImpl.java 
566f1e4 
  server/src/com/cloud/network/vpc/PrivateGatewayProfile.java 74ce002 
  server/src/com/cloud/network/vpc/PrivateIpAddress.java 2f3cf53 
  server/src/com/cloud/network/vpc/VpcManagerImpl.java 6cace7e 
  server/test/com/cloud/network/CreatePrivateNetworkTest.java ed9efd6 
  server/test/com/cloud/vpc/MockNetworkManagerImpl.java 003b774 
  server/test/com/cloud/vpc/dao/MockNetworkDaoImpl.java ec1a017 
  utils/src/com/cloud/utils/net/NetUtils.java 7a89103 

Diff: https://reviews.apache.org/r/13934/diff/


Testing
---

This is a rebase/port of code that has been tested intensively. It is here for 
review only at this time. re-tests are needed.


Thanks,

daan Hoogland

Re: contrail virtual network controller

[Pedro Roque Marques]

Chip,
A couple of questions...

On Sep 17, 2013, at 9:54 AM, Chip Childers wrote:

> I think the best way for this to happen, is for the code to be published
> in a couple of logical patches to reviews.apache.org as a way to get
> community agreement that we should bring it into the codebase.
> 
> See:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Review+Board+Guidelines
> 
> You should re-base against master first.

Should we submit the review as feature branch, rather than a patch against 
mainline ?
Would this allow an incremental review / comment process... ?

> 
> Once we have the review(s) available, I can start a formal VOTE to
> accept the donation (I assume that this is a large chunk of work), and
> then I can work with you as the sponsor of the donation to the ASF.
> 
> Once the IP assignment is handled, we would likely commit the code into a 
> feature branch, so that tests could be added (if they aren't in the initial 
> donation) before we bring it into master.

We have an integration test which requires spawning a "mysql" db, initialize it 
and then go through the process of creating a zone, etc.
This allows us to test the plugin against the API. The drawback is that this 
test takes a bit to spawn the mysql db...
Does anyone have a recipe to run against an in-memory database for testing 
purposes ? 

> 
> Pedro, does this work for you?
Do we need to submit a design doc for the plugin ? Or is the documentation 
available at opencontrail.org sufficient ?

> 
> -chip
> 
> On Tue, Sep 17, 2013 at 05:05:33AM +, Animesh Chaturvedi wrote:
>> Pedro
>> 
>> We are excited to learn about Contrail's plugin for CloudStack. Let me pass 
>> on few links to get you started
>> 
>> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/OnBoarding
>> [2] 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Adding+new+features+and+design+documents
>> 
>> 
>> It seems the plugin is already under development and would have to go 
>> through IP Clearance to get accepted by Apache. The process is described 
>> here http://incubator.apache.org/ip-clearance/. 
>> 
>> 
>> Which release are you targeting the Contrail plugin? Apache CloudStack is on 
>> a 4 month release cycle and we are just concluding the current release. The 
>> next release is ACS 4.3.0 and for which tentative feature freeze date is 
>> October 31st. Let us know if you have any questions
>> 
>> 
>> Thanks
>> Animesh
>> 
>> 
>> 
>>> -Original Message-
>>> From: Pedro Roque Marques [mailto:pedro.r.marq...@gmail.com]
>>> Sent: Monday, September 16, 2013 3:12 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: contrail virtual network controller
>>> 
>>> Myself and a few others in the Contrail team have been working on a
>>> cloudstack network plugin that interfaces with the contrail VNC
>>> (http://www.opencontrail.org). At the moment the plugin provides
>>> isolated guest networks (and a virtualized "Public" network) as an
>>> overlay that is capable of interconnecting directly with
>>> routers/switches that implement the existing standard for network
>>> virtualization (RFC 4364).
>>> 
>>> We would like to contribute this code to CloudStack. We intend to
>>> continue to enhance it in order to add full support for functionality
>>> one would expect out of the EC2 VPN API.
>>> 
>>> What is the process to do that ?
>>> 
>>> thanks,
>>>  Pedro.
>> 

Re: contrail virtual network controller

[Chip Childers]

I think the best way for this to happen, is for the code to be published
in a couple of logical patches to reviews.apache.org as a way to get
community agreement that we should bring it into the codebase.

See:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Review+Board+Guidelines

You should re-base against master first.

Once we have the review(s) available, I can start a formal VOTE to
accept the donation (I assume that this is a large chunk of work), and
then I can work with you as the sponsor of the donation to the ASF.

Once the IP assignment is handled, we would likely commit the code into a 
feature branch, so that tests could be added (if they aren't in the initial 
donation) before we bring it into master.

Pedro, does this work for you?

-chip

On Tue, Sep 17, 2013 at 05:05:33AM +, Animesh Chaturvedi wrote:
> Pedro
> 
> We are excited to learn about Contrail's plugin for CloudStack. Let me pass 
> on few links to get you started
> 
> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/OnBoarding
> [2] 
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Adding+new+features+and+design+documents
> 
> 
> It seems the plugin is already under development and would have to go through 
> IP Clearance to get accepted by Apache. The process is described here 
> http://incubator.apache.org/ip-clearance/. 
> 
> 
> Which release are you targeting the Contrail plugin? Apache CloudStack is on 
> a 4 month release cycle and we are just concluding the current release. The 
> next release is ACS 4.3.0 and for which tentative feature freeze date is 
> October 31st. Let us know if you have any questions
> 
> 
> Thanks
> Animesh
> 
> 
> 
> > -Original Message-
> > From: Pedro Roque Marques [mailto:pedro.r.marq...@gmail.com]
> > Sent: Monday, September 16, 2013 3:12 PM
> > To: dev@cloudstack.apache.org
> > Subject: contrail virtual network controller
> > 
> > Myself and a few others in the Contrail team have been working on a
> > cloudstack network plugin that interfaces with the contrail VNC
> > (http://www.opencontrail.org). At the moment the plugin provides
> > isolated guest networks (and a virtualized "Public" network) as an
> > overlay that is capable of interconnecting directly with
> > routers/switches that implement the existing standard for network
> > virtualization (RFC 4364).
> > 
> > We would like to contribute this code to CloudStack. We intend to
> > continue to enhance it in order to add full support for functionality
> > one would expect out of the EC2 VPN API.
> > 
> > What is the process to do that ?
> > 
> > thanks,
> >   Pedro.
> 

Re: ROOT disk resize

[Mike Tutkowski]

Just checking on something.

If you spin up a VM based on an ISO, the ROOT disk size is based on the
selected disk offering, right?


On Tue, Sep 17, 2013 at 6:33 AM, Antonio Petrocelli <
a.petroce...@netsons.com> wrote:

> Thanks a lot for reply.
>
> Do you know how change the code?
>
> Il 17/09/2013 14:22, Marcus Sorensen ha scritto:
>
>  You'd need to make minor code changes to remove the check in the code and
>> recompile. It should otherwise work, its just that traditionally root disk
>> size is based on template size, while data disks are based on disk
>> offerings. Not everyone wants root resize, e.g. if they are billing based
>> on disk offering rather than checking actual vol size.
>>
>> You could request an enhancement where root resize can be enabled via
>> config parameter.
>> On Sep 17, 2013 4:28 AM, "Antonio Petrocelli" 
>> wrote:
>>
>>Hello,
>>>
>>> is there a way to resize the ROOT disk of a vm?
>>>
>>> API reply is: *Can only resize DATA volumes*
>>>
>>> Does anyone know a method to bypass that control?
>>>
>>> Thanks a lot.
>>>
>>> Regards.
>>>
>>>
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: ROOT disk resize

[Marcus Sorensen]

Yes
On Sep 17, 2013 11:01 AM, "Mike Tutkowski" 
wrote:

> Just checking on something.
>
> If you spin up a VM based on an ISO, the ROOT disk size is based on the
> selected disk offering, right?
>
>
> On Tue, Sep 17, 2013 at 6:33 AM, Antonio Petrocelli <
> a.petroce...@netsons.com> wrote:
>
> > Thanks a lot for reply.
> >
> > Do you know how change the code?
> >
> > Il 17/09/2013 14:22, Marcus Sorensen ha scritto:
> >
> >  You'd need to make minor code changes to remove the check in the code
> and
> >> recompile. It should otherwise work, its just that traditionally root
> disk
> >> size is based on template size, while data disks are based on disk
> >> offerings. Not everyone wants root resize, e.g. if they are billing
> based
> >> on disk offering rather than checking actual vol size.
> >>
> >> You could request an enhancement where root resize can be enabled via
> >> config parameter.
> >> On Sep 17, 2013 4:28 AM, "Antonio Petrocelli"  >
> >> wrote:
> >>
> >>Hello,
> >>>
> >>> is there a way to resize the ROOT disk of a vm?
> >>>
> >>> API reply is: *Can only resize DATA volumes*
> >>>
> >>> Does anyone know a method to bypass that control?
> >>>
> >>> Thanks a lot.
> >>>
> >>> Regards.
> >>>
> >>>
> >
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*
>

Re: contrail virtual network controller

[Chip Childers]

On Tue, Sep 17, 2013 at 10:18:29AM -0700, Pedro Roque Marques wrote:
> Chip,
> A couple of questions...
> 
> On Sep 17, 2013, at 9:54 AM, Chip Childers wrote:
> 
> > I think the best way for this to happen, is for the code to be published
> > in a couple of logical patches to reviews.apache.org as a way to get
> > community agreement that we should bring it into the codebase.
> > 
> > See:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Review+Board+Guidelines
> > 
> > You should re-base against master first.
> 
> Should we submit the review as feature branch, rather than a patch against 
> mainline ?
> Would this allow an incremental review / comment process... ?

Reviewboard is rather basic on this front.  Tag the reviews as belonging
to master, so that they are tested against master.  We'll just commit to a
feature branch when we have dealt with the legal acceptance of the code.

> 
> > 
> > Once we have the review(s) available, I can start a formal VOTE to
> > accept the donation (I assume that this is a large chunk of work), and
> > then I can work with you as the sponsor of the donation to the ASF.
> > 
> > Once the IP assignment is handled, we would likely commit the code into a 
> > feature branch, so that tests could be added (if they aren't in the initial 
> > donation) before we bring it into master.
> 
> We have an integration test which requires spawning a "mysql" db, initialize 
> it and then go through the process of creating a zone, etc.
> This allows us to test the plugin against the API. The drawback is that this 
> test takes a bit to spawn the mysql db...
> Does anyone have a recipe to run against an in-memory database for testing 
> purposes ? 

Prasanna - cc'ed you to see if you have any suggestions on this front.

> 
> > 
> > Pedro, does this work for you?
> Do we need to submit a design doc for the plugin ? Or is the documentation 
> available at opencontrail.org sufficient ?

Ideally, we would have a design document on the wiki that would help
everyone understand the implementation.  We're not looking for a design
doc for Contrail / OpenContrail...  we're looking for information about
the integration itself.  This is both for other devs, tech writers that
may choose to help document this, and for users to get a better feel for 
the feature.

AFAICT, I'm not sure if the opencontrail.org site has anything that
answers the questions that the CloudStack community needs to have
documented...  but I may just not have found it.

-chip

> 
> > 
> > -chip
> > 
> > On Tue, Sep 17, 2013 at 05:05:33AM +, Animesh Chaturvedi wrote:
> >> Pedro
> >> 
> >> We are excited to learn about Contrail's plugin for CloudStack. Let me 
> >> pass on few links to get you started
> >> 
> >> [1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/OnBoarding
> >> [2] 
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Adding+new+features+and+design+documents
> >> 
> >> 
> >> It seems the plugin is already under development and would have to go 
> >> through IP Clearance to get accepted by Apache. The process is described 
> >> here http://incubator.apache.org/ip-clearance/. 
> >> 
> >> 
> >> Which release are you targeting the Contrail plugin? Apache CloudStack is 
> >> on a 4 month release cycle and we are just concluding the current release. 
> >> The next release is ACS 4.3.0 and for which tentative feature freeze date 
> >> is October 31st. Let us know if you have any questions
> >> 
> >> 
> >> Thanks
> >> Animesh
> >> 
> >> 
> >> 
> >>> -Original Message-
> >>> From: Pedro Roque Marques [mailto:pedro.r.marq...@gmail.com]
> >>> Sent: Monday, September 16, 2013 3:12 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: contrail virtual network controller
> >>> 
> >>> Myself and a few others in the Contrail team have been working on a
> >>> cloudstack network plugin that interfaces with the contrail VNC
> >>> (http://www.opencontrail.org). At the moment the plugin provides
> >>> isolated guest networks (and a virtualized "Public" network) as an
> >>> overlay that is capable of interconnecting directly with
> >>> routers/switches that implement the existing standard for network
> >>> virtualization (RFC 4364).
> >>> 
> >>> We would like to contribute this code to CloudStack. We intend to
> >>> continue to enhance it in order to add full support for functionality
> >>> one would expect out of the EC2 VPN API.
> >>> 
> >>> What is the process to do that ?
> >>> 
> >>> thanks,
> >>>  Pedro.
> >> 
> 
> 

Re: Build failure on master (KVM related)

[Marcus Sorensen]

I made the last commit, but I know it  wasn't that. My change was a
simple string change, it wouldn't affect compilation. I'm looking at
it to see if I can determine the source.

On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
 wrote:
> Just updated and ran mvn -P developer,systemvm clean install (twice - with
> the same results)
>
> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS [2.299s]
> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS [6.294s]
> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS [3.454s]
> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS [3.893s]
> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS [8.444s]
> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE [6.461s]
>
> [INFO]
> 
> [INFO] BUILD FAILURE
> [INFO]
> 
> [INFO] Total time: 3:41.533s
> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
> [INFO] Final Memory: 38M/101M
> [INFO]
> 
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on
> project cloud-plugin-hypervisor-kvm: There are test failures.
> [ERROR]
> [ERROR] Please refer to
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
> for the individual test results.
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*

Re: Build failure on master (KVM related)

[Marcus Sorensen]

See email thread 'Libvirt-java 0.5.0 has been released', it's due to that.

On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen  wrote:
> I made the last commit, but I know it  wasn't that. My change was a
> simple string change, it wouldn't affect compilation. I'm looking at
> it to see if I can determine the source.
>
> On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
>  wrote:
>> Just updated and ran mvn -P developer,systemvm clean install (twice - with
>> the same results)
>>
>> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS [2.299s]
>> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS [6.294s]
>> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS [3.454s]
>> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS [3.893s]
>> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS [8.444s]
>> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE [6.461s]
>>
>> [INFO]
>> 
>> [INFO] BUILD FAILURE
>> [INFO]
>> 
>> [INFO] Total time: 3:41.533s
>> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
>> [INFO] Final Memory: 38M/101M
>> [INFO]
>> 
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on
>> project cloud-plugin-hypervisor-kvm: There are test failures.
>> [ERROR]
>> [ERROR] Please refer to
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
>> for the individual test results.
>> [ERROR] -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
>> switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
>>
>>
>> --
>> *Mike Tutkowski*
>> *Senior CloudStack Developer, SolidFire Inc.*
>> e: mike.tutkow...@solidfire.com
>> o: 303.746.7302
>> Advancing the way the world uses the
>> cloud
>> *™*

Re: Build failure on master (KVM related)

[Mike Tutkowski]

Got that e-mail lying around somewhere.

Thanks


On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen wrote:

> See email thread 'Libvirt-java 0.5.0 has been released', it's due to that.
>
> On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen 
> wrote:
> > I made the last commit, but I know it  wasn't that. My change was a
> > simple string change, it wouldn't affect compilation. I'm looking at
> > it to see if I can determine the source.
> >
> > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
> >  wrote:
> >> Just updated and ran mvn -P developer,systemvm clean install (twice -
> with
> >> the same results)
> >>
> >> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS
> [2.299s]
> >> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS
> [6.294s]
> >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS
> [3.454s]
> >> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS
> [3.893s]
> >> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS
> [8.444s]
> >> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE
> [6.461s]
> >>
> >> [INFO]
> >> 
> >> [INFO] BUILD FAILURE
> >> [INFO]
> >> 
> >> [INFO] Total time: 3:41.533s
> >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
> >> [INFO] Final Memory: 38M/101M
> >> [INFO]
> >> 
> >> [ERROR] Failed to execute goal
> >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test)
> on
> >> project cloud-plugin-hypervisor-kvm: There are test failures.
> >> [ERROR]
> >> [ERROR] Please refer to
> >>
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
> >> for the individual test results.
> >> [ERROR] -> [Help 1]
> >> [ERROR]
> >> [ERROR] To see the full stack trace of the errors, re-run Maven with
> the -e
> >> switch.
> >> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >> [ERROR]
> >> [ERROR] For more information about the errors and possible solutions,
> >> please read the following articles:
> >> [ERROR] [Help 1]
> >> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> >> [ERROR]
> >> [ERROR] After correcting the problems, you can resume the build with the
> >> command
> >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
> >>
> >>
> >> --
> >> *Mike Tutkowski*
> >> *Senior CloudStack Developer, SolidFire Inc.*
> >> e: mike.tutkow...@solidfire.com
> >> o: 303.746.7302
> >> Advancing the way the world uses the
> >> cloud
> >> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: [VOTE] Apache CloudStack CloudMonkey 5.0.0 (first round)

[David Nalley]

On Fri, Sep 13, 2013 at 9:23 AM, Chip Childers
 wrote:
> I've created a 5.0.0 release of cloudmonkey, with the following artifacts
> up for a
> vote:
>
> Git Branch and Commit SH:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/5.0
> Commit: 767bfbe084e24d441f1ad73ace183c09f26a276b
>
> List of changes:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=blob;f=CHANGES;h=54f2e32357b0a726dc6f202e4897ed4af531f8ab;hb=refs/heads/5.0
>
> Source release (checksums and signatures are available at the same
> location):
> https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.0.0/
>
> PGP release keys (signed using 94BE0D7C):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
> I have not created testing instructions for this release artifact, but
> would appreciate help documenting it as people to it.
>
> Vote will be open for at least 72 hours.
>
> For sanity in tallying the vote, can PMC members please be sure to indicate
> "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>


+1 (binding)

Verified checksums/signatures
Built and tested

--David

Re: Libvirt-java 0.5.0 has been released

[Mike Tutkowski]

I'm not familiar with how we package these binding classes in CloudStack.

Is there a new JAR I need to download or source code?

Thanks!


On Mon, Sep 16, 2013 at 2:19 PM, Wido den Hollander  wrote:

>
>
> On 09/16/2013 07:46 PM, Min Chen wrote:
>
>> I got the following test failure in building master:
>>
>> Running com.cloud.hypervisor.kvm.**resource.**
>> LibvirtComputingResourceTest
>> Tests run: 3, Failures: 0, Errors: 2, Skipped: 1, Time elapsed: 0.059 sec
>> <<< FAILURE!
>> testCreateVMFromSpecLegacy(**com.cloud.hypervisor.kvm.**
>> resource.LibvirtComputi
>> ngResourceTest)  Time elapsed: 0.018 sec  <<< ERROR!
>> java.lang.**UnsupportedClassVersionError: org/libvirt/LibvirtException :
>> Unsupported major.minor version 51.0
>> at java.lang.ClassLoader.**defineClass1(Native Method)
>> at java.lang.ClassLoader.**defineClassCond(ClassLoader.**
>> java:631)
>> at java.lang.ClassLoader.**defineClass(ClassLoader.java:**615)
>> at java.security.**SecureClassLoader.defineClass(**
>> SecureClassLoader.java:141)
>> at java.net.URLClassLoader.**defineClass(URLClassLoader.**
>> java:283)
>> at java.net.URLClassLoader.**access$000(URLClassLoader.**java:58)
>> at java.net.URLClassLoader$1.run(**URLClassLoader.java:197)
>> at java.security.**AccessController.doPrivileged(**Native Method)
>> at java.net.URLClassLoader.**findClass(URLClassLoader.java:**190)
>> at java.lang.ClassLoader.**loadClass(ClassLoader.java:**306)
>> at sun.misc.Launcher$**AppClassLoader.loadClass(**
>> Launcher.java:301)
>> at java.lang.ClassLoader.**loadClass(ClassLoader.java:**247)
>> at
>> com.cloud.hypervisor.kvm.**resource.**LibvirtComputingResourceTest.**
>> testCreateVM
>> FromSpecLegacy(**LibvirtComputingResourceTest.**java:64)
>>
>>
>> Wido, is this related to this libvirt change?
>>
>>
> I think so. I'll test this tomorrow, but I think the RedHat guys build the
> bindings with Java 7 since they all work on a recent Fedora version.
>
> Wido
>
>
>
>> Thanks
>> -min
>>
>>
>>
>> On 9/16/13 4:28 AM, "Wido den Hollander"  wrote:
>>
>>  On 09/16/2013 12:51 PM, Wei ZHOU wrote:
>>>
 Thanks Wido.

 Do you know the minimal requirement of libvirt if we use libvirt-java
 0.5.0
 ?


>>> There shouldn't be a difference in the required libvirt version. I
>>> implemented a couple of methods in the bindings which were already in
>>> libvirt for a long time, but the bindings was just missing them.
>>>
>>> Wido
>>>
>>>
 2013/9/16 Wido den Hollander 

  Hi,
>
> I worked with the RedHat guys last week to get libvirt-java 0.5.0
> released
> which has some nice new features for us:
>
> - Supports different XML on destination host during migration
> - Supports resizing volumes
> - Supports more snapshot functionalities
>
> I took the liberty to depend on 0.5.0 in master and also merge in the
> code
> for the VNC listen, we now no longer listen on 0.0.0.0 for VNC, which
> was a
> security issue imho.
>
> The reason I'm bringing this to the list is that we can simplify some
> code
> around resizing volumes and snapshotting where we currently have some
> nasty
> scripts to do the work.
>
> See my commits in libvirt-java: 
> http://www.libvirt.org/git/?p=
>
> libvirt-java.git;a=summary libvirt-java.gi 
> t;a=summary>
>
> So keep in mind libvirt-java has some new features!
>
> Wido
>
>

>>>
>>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Build failure on master (KVM related)

[Mike Tutkowski]

OK, thanks! I can try that.


On Tue, Sep 17, 2013 at 12:34 PM, Wei ZHOU  wrote:

> build successfully after install java-1.7.0-openjdk.x86_64 and
> java-1.7.0-openjdk-devel.x86_64, and set the JAVA_HOME
>
>
> 2013/9/17 Mike Tutkowski 
>
> > Got that e-mail lying around somewhere.
> >
> > Thanks
> >
> >
> > On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen  > >wrote:
> >
> > > See email thread 'Libvirt-java 0.5.0 has been released', it's due to
> > that.
> > >
> > > On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen  >
> > > wrote:
> > > > I made the last commit, but I know it  wasn't that. My change was a
> > > > simple string change, it wouldn't affect compilation. I'm looking at
> > > > it to see if I can determine the source.
> > > >
> > > > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
> > > >  wrote:
> > > >> Just updated and ran mvn -P developer,systemvm clean install (twice
> -
> > > with
> > > >> the same results)
> > > >>
> > > >> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS
> > > [2.299s]
> > > >> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS
> > > [6.294s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS
> > > [3.454s]
> > > >> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS
> > > [3.893s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS
> > > [8.444s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE
> > > [6.461s]
> > > >>
> > > >> [INFO]
> > > >>
> > 
> > > >> [INFO] BUILD FAILURE
> > > >> [INFO]
> > > >>
> > 
> > > >> [INFO] Total time: 3:41.533s
> > > >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
> > > >> [INFO] Final Memory: 38M/101M
> > > >> [INFO]
> > > >>
> > 
> > > >> [ERROR] Failed to execute goal
> > > >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
> > (default-test)
> > > on
> > > >> project cloud-plugin-hypervisor-kvm: There are test failures.
> > > >> [ERROR]
> > > >> [ERROR] Please refer to
> > > >>
> > >
> >
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
> > > >> for the individual test results.
> > > >> [ERROR] -> [Help 1]
> > > >> [ERROR]
> > > >> [ERROR] To see the full stack trace of the errors, re-run Maven with
> > > the -e
> > > >> switch.
> > > >> [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> > > >> [ERROR]
> > > >> [ERROR] For more information about the errors and possible
> solutions,
> > > >> please read the following articles:
> > > >> [ERROR] [Help 1]
> > > >>
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> > > >> [ERROR]
> > > >> [ERROR] After correcting the problems, you can resume the build with
> > the
> > > >> command
> > > >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkow...@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the
> > > >> cloud
> > > >> *™*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud
> > *™*
> >
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Build failure on master (KVM related)

[Mike Tutkowski]

Just updated and ran mvn -P developer,systemvm clean install (twice - with
the same results)

[INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS [2.299s]
[INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS [6.294s]
[INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS [3.454s]
[INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS [3.893s]
[INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS [8.444s]
[INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE [6.461s]

[INFO]

[INFO] BUILD FAILURE
[INFO]

[INFO] Total time: 3:41.533s
[INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
[INFO] Final Memory: 38M/101M
[INFO]

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.14:test (default-test) on
project cloud-plugin-hypervisor-kvm: There are test failures.
[ERROR]
[ERROR] Please refer to
/Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

[PROPOSAL] standardize on or at least allow use of slf4j going forward

[Darren Shepherd]

Currently all logging in ACS is done using log4j APIs.  slf4j is already 
packaged as a transitive dependency in ACS.  I propose that going 
forward we starting using slf4j APIs as opposed to log4j APIs.


For those who don't know, slf4j is a logging abstraction API.  In the 
world of java logging abstractions there are basically three frameworks 
java.util.logging (JUL), jakarta commons logging (JCL), and simple 
logging facade for java (slf4j).  Most people gravitate towards slf4j 
these days [1].  JUL isn't really that usable and JCL requires you to do 
"if (log.isDebug())" everywhere to avoid the overhead of string 
concatenation.


Its important to note that I'm not saying we stop using log4j.  At 
runtime log4j will be still used as the implementation, I'm merely 
talking about the API that we code against.  It's a generally good 
practice to use a logging abstraction API and its some what of shame 
that one was never put in place to start with.


At one point we can decide if we want to switch 100% to slf4j.  There is 
a utility to convert your source code from log4j to slf4j.  Right now I 
just want to "bless" using slf4j APIs as okay.


Darren

[1] Projects depending on SLF4J at http://www.slf4j.org/

Re: Build failure on master (KVM related)

[Mike Tutkowski]

That fixed it.

Thanks!


On Tue, Sep 17, 2013 at 12:36 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> OK, thanks! I can try that.
>
>
> On Tue, Sep 17, 2013 at 12:34 PM, Wei ZHOU  wrote:
>
>> build successfully after install java-1.7.0-openjdk.x86_64 and
>> java-1.7.0-openjdk-devel.x86_64, and set the JAVA_HOME
>>
>>
>> 2013/9/17 Mike Tutkowski 
>>
>> > Got that e-mail lying around somewhere.
>> >
>> > Thanks
>> >
>> >
>> > On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen > > >wrote:
>> >
>> > > See email thread 'Libvirt-java 0.5.0 has been released', it's due to
>> > that.
>> > >
>> > > On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen <
>> shadow...@gmail.com>
>> > > wrote:
>> > > > I made the last commit, but I know it  wasn't that. My change was a
>> > > > simple string change, it wouldn't affect compilation. I'm looking at
>> > > > it to see if I can determine the source.
>> > > >
>> > > > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
>> > > >  wrote:
>> > > >> Just updated and ran mvn -P developer,systemvm clean install
>> (twice -
>> > > with
>> > > >> the same results)
>> > > >>
>> > > >> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS
>> > > [2.299s]
>> > > >> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS
>> > > [6.294s]
>> > > >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS
>> > > [3.454s]
>> > > >> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS
>> > > [3.893s]
>> > > >> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS
>> > > [8.444s]
>> > > >> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE
>> > > [6.461s]
>> > > >>
>> > > >> [INFO]
>> > > >>
>> > 
>> > > >> [INFO] BUILD FAILURE
>> > > >> [INFO]
>> > > >>
>> > 
>> > > >> [INFO] Total time: 3:41.533s
>> > > >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
>> > > >> [INFO] Final Memory: 38M/101M
>> > > >> [INFO]
>> > > >>
>> > 
>> > > >> [ERROR] Failed to execute goal
>> > > >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
>> > (default-test)
>> > > on
>> > > >> project cloud-plugin-hypervisor-kvm: There are test failures.
>> > > >> [ERROR]
>> > > >> [ERROR] Please refer to
>> > > >>
>> > >
>> >
>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
>> > > >> for the individual test results.
>> > > >> [ERROR] -> [Help 1]
>> > > >> [ERROR]
>> > > >> [ERROR] To see the full stack trace of the errors, re-run Maven
>> with
>> > > the -e
>> > > >> switch.
>> > > >> [ERROR] Re-run Maven using the -X switch to enable full debug
>> logging.
>> > > >> [ERROR]
>> > > >> [ERROR] For more information about the errors and possible
>> solutions,
>> > > >> please read the following articles:
>> > > >> [ERROR] [Help 1]
>> > > >>
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>> > > >> [ERROR]
>> > > >> [ERROR] After correcting the problems, you can resume the build
>> with
>> > the
>> > > >> command
>> > > >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
>> > > >>
>> > > >>
>> > > >> --
>> > > >> *Mike Tutkowski*
>> > > >> *Senior CloudStack Developer, SolidFire Inc.*
>> > > >> e: mike.tutkow...@solidfire.com
>> > > >> o: 303.746.7302
>> > > >> Advancing the way the world uses the
>> > > >> cloud
>> > > >> *™*
>> > >
>> >
>> >
>> >
>> > --
>> > *Mike Tutkowski*
>> > *Senior CloudStack Developer, SolidFire Inc.*
>> > e: mike.tutkow...@solidfire.com
>> > o: 303.746.7302
>> > Advancing the way the world uses the
>> > cloud
>> > *™*
>> >
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the 
> cloud
> *™*
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the
cloud
*™*

Re: Build failure on master (KVM related)

[Wei ZHOU]

Agree.
I prefer to use libvirt 0.5.0 from next major version like 4.3.0 or 5.0.0


2013/9/17 Edison Su 

> We should be cautious to advance java version, as most of our code
> run/tested on java 6.
> Is it a must to upgrade to Libvirt 0.5.0?
>
> > -Original Message-
> > From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
> > Sent: Tuesday, September 17, 2013 11:34 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Build failure on master (KVM related)
> >
> > build successfully after install java-1.7.0-openjdk.x86_64 and
> java-1.7.0-
> > openjdk-devel.x86_64, and set the JAVA_HOME
> >
> >
> > 2013/9/17 Mike Tutkowski 
> >
> > > Got that e-mail lying around somewhere.
> > >
> > > Thanks
> > >
> > >
> > > On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen
> >  > > >wrote:
> > >
> > > > See email thread 'Libvirt-java 0.5.0 has been released', it's due to
> > > that.
> > > >
> > > > On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen
> > > > 
> > > > wrote:
> > > > > I made the last commit, but I know it  wasn't that. My change was
> > > > > a simple string change, it wouldn't affect compilation. I'm
> > > > > looking at it to see if I can determine the source.
> > > > >
> > > > > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
> > > > >  wrote:
> > > > >> Just updated and ran mvn -P developer,systemvm clean install
> > > > >> (twice -
> > > > with
> > > > >> the same results)
> > > > >>
> > > > >> [INFO] Apache CloudStack Plugin - Host Allocator Random ..
> > > > >> SUCCESS
> > > > [2.299s]
> > > > >> [INFO] Apache CloudStack Plugin - Dedicated Resources 
> > > > >> SUCCESS
> > > > [6.294s]
> > > > >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM 
> > > > >> SUCCESS
> > > > [3.454s]
> > > > >> [INFO] Apache CloudStack Plugin - Open vSwitch ...
> > > > >> SUCCESS
> > > > [3.893s]
> > > > >> [INFO] Apache CloudStack Plugin - Hypervisor Xen .
> > > > >> SUCCESS
> > > > [8.444s]
> > > > >> [INFO] Apache CloudStack Plugin - Hypervisor KVM .
> > > > >> FAILURE
> > > > [6.461s]
> > > > >>
> > > > >> [INFO]
> > > > >>
> > > --
> > > --
> > > > >> [INFO] BUILD FAILURE
> > > > >> [INFO]
> > > > >>
> > > --
> > > --
> > > > >> [INFO] Total time: 3:41.533s
> > > > >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013 [INFO] Final
> > > > >> Memory: 38M/101M [INFO]
> > > > >>
> > > --
> > > --
> > > > >> [ERROR] Failed to execute goal
> > > > >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
> > > (default-test)
> > > > on
> > > > >> project cloud-plugin-hypervisor-kvm: There are test failures.
> > > > >> [ERROR]
> > > > >> [ERROR] Please refer to
> > > > >>
> > > >
> > >
> > /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervi
> > s
> > > ors/kvm/target/surefire-reports
> > > > >> for the individual test results.
> > > > >> [ERROR] -> [Help 1]
> > > > >> [ERROR]
> > > > >> [ERROR] To see the full stack trace of the errors, re-run Maven
> > > > >> with
> > > > the -e
> > > > >> switch.
> > > > >> [ERROR] Re-run Maven using the -X switch to enable full debug
> > logging.
> > > > >> [ERROR]
> > > > >> [ERROR] For more information about the errors and possible
> > > > >> solutions, please read the following articles:
> > > > >> [ERROR] [Help 1]
> > > > >>
> > http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExcep
> > > > >> tion
> > > > >> [ERROR]
> > > > >> [ERROR] After correcting the problems, you can resume the build
> > > > >> with
> > > the
> > > > >> command
> > > > >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
> > > > >>
> > > > >>
> > > > >> --
> > > > >> *Mike Tutkowski*
> > > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > > >> e: mike.tutkow...@solidfire.com
> > > > >> o: 303.746.7302
> > > > >> Advancing the way the world uses the
> > > > >> cloud
> > > > >> *(tm)*
> > > >
> > >
> > >
> > >
> > > --
> > > *Mike Tutkowski*
> > > *Senior CloudStack Developer, SolidFire Inc.*
> > > e: mike.tutkow...@solidfire.com
> > > o: 303.746.7302
> > > Advancing the way the world uses the
> > > cloud
> > > *(tm)*
> > >
>

Re: Build failure on master (KVM related)

[Wido den Hollander]

I'm in contact with the RedHat guys to get 0.5.1 released since they compiled 
it for Java 7.

I got a patch in at libvirt-java which sets the target version for javac to 1.6

Wido

Op 17 sep. 2013 om 22:10 heeft "Wei ZHOU"  het volgende 
geschreven:

> Agree.
> I prefer to use libvirt 0.5.0 from next major version like 4.3.0 or 5.0.0
> 
> 
> 2013/9/17 Edison Su 
> 
>> We should be cautious to advance java version, as most of our code
>> run/tested on java 6.
>> Is it a must to upgrade to Libvirt 0.5.0?
>> 
>>> -Original Message-
>>> From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
>>> Sent: Tuesday, September 17, 2013 11:34 AM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: Build failure on master (KVM related)
>>> 
>>> build successfully after install java-1.7.0-openjdk.x86_64 and
>> java-1.7.0-
>>> openjdk-devel.x86_64, and set the JAVA_HOME
>>> 
>>> 
>>> 2013/9/17 Mike Tutkowski 
>>> 
 Got that e-mail lying around somewhere.
 
 Thanks
 
 
 On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen
>>>  wrote:
 
> See email thread 'Libvirt-java 0.5.0 has been released', it's due to
 that.
> 
> On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen
> 
> wrote:
>> I made the last commit, but I know it  wasn't that. My change was
>> a simple string change, it wouldn't affect compilation. I'm
>> looking at it to see if I can determine the source.
>> 
>> On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
>>  wrote:
>>> Just updated and ran mvn -P developer,systemvm clean install
>>> (twice -
> with
>>> the same results)
>>> 
>>> [INFO] Apache CloudStack Plugin - Host Allocator Random ..
>>> SUCCESS
> [2.299s]
>>> [INFO] Apache CloudStack Plugin - Dedicated Resources 
>>> SUCCESS
> [6.294s]
>>> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM 
>>> SUCCESS
> [3.454s]
>>> [INFO] Apache CloudStack Plugin - Open vSwitch ...
>>> SUCCESS
> [3.893s]
>>> [INFO] Apache CloudStack Plugin - Hypervisor Xen .
>>> SUCCESS
> [8.444s]
>>> [INFO] Apache CloudStack Plugin - Hypervisor KVM .
>>> FAILURE
> [6.461s]
>>> 
>>> [INFO]
 --
 --
>>> [INFO] BUILD FAILURE
>>> [INFO]
 --
 --
>>> [INFO] Total time: 3:41.533s
>>> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013 [INFO] Final
>>> Memory: 38M/101M [INFO]
 --
 --
>>> [ERROR] Failed to execute goal
>>> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
 (default-test)
> on
>>> project cloud-plugin-hypervisor-kvm: There are test failures.
>>> [ERROR]
>>> [ERROR] Please refer to
>>> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervi
>>> s
 ors/kvm/target/surefire-reports
>>> for the individual test results.
>>> [ERROR] -> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven
>>> with
> the -e
>>> switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug
>>> logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible
>>> solutions, please read the following articles:
>>> [ERROR] [Help 1]
>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExcep
>>> tion
>>> [ERROR]
>>> [ERROR] After correcting the problems, you can resume the build
>>> with
 the
>>> command
>>> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
>>> 
>>> 
>>> --
>>> *Mike Tutkowski*
>>> *Senior CloudStack Developer, SolidFire Inc.*
>>> e: mike.tutkow...@solidfire.com
>>> o: 303.746.7302
>>> Advancing the way the world uses the
>>> cloud
>>> *(tm)*
 
 
 
 --
 *Mike Tutkowski*
 *Senior CloudStack Developer, SolidFire Inc.*
 e: mike.tutkow...@solidfire.com
 o: 303.746.7302
 Advancing the way the world uses the
 cloud
 *(tm)*
>> 

RE: Build failure on master (KVM related)

[Edison Su]

We should be cautious to advance java version, as most of our code run/tested 
on java 6.
Is it a must to upgrade to Libvirt 0.5.0? 

> -Original Message-
> From: Wei ZHOU [mailto:ustcweiz...@gmail.com]
> Sent: Tuesday, September 17, 2013 11:34 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Build failure on master (KVM related)
> 
> build successfully after install java-1.7.0-openjdk.x86_64 and java-1.7.0-
> openjdk-devel.x86_64, and set the JAVA_HOME
> 
> 
> 2013/9/17 Mike Tutkowski 
> 
> > Got that e-mail lying around somewhere.
> >
> > Thanks
> >
> >
> > On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen
>  > >wrote:
> >
> > > See email thread 'Libvirt-java 0.5.0 has been released', it's due to
> > that.
> > >
> > > On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen
> > > 
> > > wrote:
> > > > I made the last commit, but I know it  wasn't that. My change was
> > > > a simple string change, it wouldn't affect compilation. I'm
> > > > looking at it to see if I can determine the source.
> > > >
> > > > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
> > > >  wrote:
> > > >> Just updated and ran mvn -P developer,systemvm clean install
> > > >> (twice -
> > > with
> > > >> the same results)
> > > >>
> > > >> [INFO] Apache CloudStack Plugin - Host Allocator Random ..
> > > >> SUCCESS
> > > [2.299s]
> > > >> [INFO] Apache CloudStack Plugin - Dedicated Resources 
> > > >> SUCCESS
> > > [6.294s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM 
> > > >> SUCCESS
> > > [3.454s]
> > > >> [INFO] Apache CloudStack Plugin - Open vSwitch ...
> > > >> SUCCESS
> > > [3.893s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor Xen .
> > > >> SUCCESS
> > > [8.444s]
> > > >> [INFO] Apache CloudStack Plugin - Hypervisor KVM .
> > > >> FAILURE
> > > [6.461s]
> > > >>
> > > >> [INFO]
> > > >>
> > --
> > --
> > > >> [INFO] BUILD FAILURE
> > > >> [INFO]
> > > >>
> > --
> > --
> > > >> [INFO] Total time: 3:41.533s
> > > >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013 [INFO] Final
> > > >> Memory: 38M/101M [INFO]
> > > >>
> > --
> > --
> > > >> [ERROR] Failed to execute goal
> > > >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
> > (default-test)
> > > on
> > > >> project cloud-plugin-hypervisor-kvm: There are test failures.
> > > >> [ERROR]
> > > >> [ERROR] Please refer to
> > > >>
> > >
> >
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervi
> s
> > ors/kvm/target/surefire-reports
> > > >> for the individual test results.
> > > >> [ERROR] -> [Help 1]
> > > >> [ERROR]
> > > >> [ERROR] To see the full stack trace of the errors, re-run Maven
> > > >> with
> > > the -e
> > > >> switch.
> > > >> [ERROR] Re-run Maven using the -X switch to enable full debug
> logging.
> > > >> [ERROR]
> > > >> [ERROR] For more information about the errors and possible
> > > >> solutions, please read the following articles:
> > > >> [ERROR] [Help 1]
> > > >>
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureExcep
> > > >> tion
> > > >> [ERROR]
> > > >> [ERROR] After correcting the problems, you can resume the build
> > > >> with
> > the
> > > >> command
> > > >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
> > > >>
> > > >>
> > > >> --
> > > >> *Mike Tutkowski*
> > > >> *Senior CloudStack Developer, SolidFire Inc.*
> > > >> e: mike.tutkow...@solidfire.com
> > > >> o: 303.746.7302
> > > >> Advancing the way the world uses the
> > > >> cloud
> > > >> *(tm)*
> > >
> >
> >
> >
> > --
> > *Mike Tutkowski*
> > *Senior CloudStack Developer, SolidFire Inc.*
> > e: mike.tutkow...@solidfire.com
> > o: 303.746.7302
> > Advancing the way the world uses the
> > cloud
> > *(tm)*
> >

Re: Build failure on master (KVM related)

[Wei ZHOU]

build successfully after install java-1.7.0-openjdk.x86_64 and
java-1.7.0-openjdk-devel.x86_64, and set the JAVA_HOME


2013/9/17 Mike Tutkowski 

> Got that e-mail lying around somewhere.
>
> Thanks
>
>
> On Tue, Sep 17, 2013 at 11:49 AM, Marcus Sorensen  >wrote:
>
> > See email thread 'Libvirt-java 0.5.0 has been released', it's due to
> that.
> >
> > On Tue, Sep 17, 2013 at 11:45 AM, Marcus Sorensen 
> > wrote:
> > > I made the last commit, but I know it  wasn't that. My change was a
> > > simple string change, it wouldn't affect compilation. I'm looking at
> > > it to see if I can determine the source.
> > >
> > > On Tue, Sep 17, 2013 at 11:41 AM, Mike Tutkowski
> > >  wrote:
> > >> Just updated and ran mvn -P developer,systemvm clean install (twice -
> > with
> > >> the same results)
> > >>
> > >> [INFO] Apache CloudStack Plugin - Host Allocator Random .. SUCCESS
> > [2.299s]
> > >> [INFO] Apache CloudStack Plugin - Dedicated Resources  SUCCESS
> > [6.294s]
> > >> [INFO] Apache CloudStack Plugin - Hypervisor OracleVM  SUCCESS
> > [3.454s]
> > >> [INFO] Apache CloudStack Plugin - Open vSwitch ... SUCCESS
> > [3.893s]
> > >> [INFO] Apache CloudStack Plugin - Hypervisor Xen . SUCCESS
> > [8.444s]
> > >> [INFO] Apache CloudStack Plugin - Hypervisor KVM . FAILURE
> > [6.461s]
> > >>
> > >> [INFO]
> > >>
> 
> > >> [INFO] BUILD FAILURE
> > >> [INFO]
> > >>
> 
> > >> [INFO] Total time: 3:41.533s
> > >> [INFO] Finished at: Tue Sep 17 11:40:04 MDT 2013
> > >> [INFO] Final Memory: 38M/101M
> > >> [INFO]
> > >>
> 
> > >> [ERROR] Failed to execute goal
> > >> org.apache.maven.plugins:maven-surefire-plugin:2.14:test
> (default-test)
> > on
> > >> project cloud-plugin-hypervisor-kvm: There are test failures.
> > >> [ERROR]
> > >> [ERROR] Please refer to
> > >>
> >
> /Users/mtutkowski/Documents/CloudStack/src/CloudStack/plugins/hypervisors/kvm/target/surefire-reports
> > >> for the individual test results.
> > >> [ERROR] -> [Help 1]
> > >> [ERROR]
> > >> [ERROR] To see the full stack trace of the errors, re-run Maven with
> > the -e
> > >> switch.
> > >> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> > >> [ERROR]
> > >> [ERROR] For more information about the errors and possible solutions,
> > >> please read the following articles:
> > >> [ERROR] [Help 1]
> > >> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> > >> [ERROR]
> > >> [ERROR] After correcting the problems, you can resume the build with
> the
> > >> command
> > >> [ERROR]   mvn  -rf :cloud-plugin-hypervisor-kvm
> > >>
> > >>
> > >> --
> > >> *Mike Tutkowski*
> > >> *Senior CloudStack Developer, SolidFire Inc.*
> > >> e: mike.tutkow...@solidfire.com
> > >> o: 303.746.7302
> > >> Advancing the way the world uses the
> > >> cloud
> > >> *™*
> >
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the
> cloud
> *™*
>

RE: [PROPOSAL] standardize on or at least allow use of slf4j going forward

[Alex Huang]

Sure.  Like you said, it's already pulled in as a dependency from other third 
parties so there's no real penalties in using it.

--Alex

> -Original Message-
> From: Darren Shepherd [mailto:darren.s.sheph...@gmail.com]
> Sent: Tuesday, September 17, 2013 11:56 AM
> To: dev@cloudstack.apache.org
> Subject: [PROPOSAL] standardize on or at least allow use of slf4j going
> forward
> 
> Currently all logging in ACS is done using log4j APIs.  slf4j is already 
> packaged
> as a transitive dependency in ACS.  I propose that going forward we starting
> using slf4j APIs as opposed to log4j APIs.
> 
> For those who don't know, slf4j is a logging abstraction API.  In the world of
> java logging abstractions there are basically three frameworks
> java.util.logging (JUL), jakarta commons logging (JCL), and simple logging
> facade for java (slf4j).  Most people gravitate towards slf4j these days [1].
> JUL isn't really that usable and JCL requires you to do "if (log.isDebug())"
> everywhere to avoid the overhead of string concatenation.
> 
> Its important to note that I'm not saying we stop using log4j.  At runtime 
> log4j
> will be still used as the implementation, I'm merely talking about the API 
> that
> we code against.  It's a generally good practice to use a logging abstraction
> API and its some what of shame that one was never put in place to start with.
> 
> At one point we can decide if we want to switch 100% to slf4j.  There is a
> utility to convert your source code from log4j to slf4j.  Right now I just 
> want
> to "bless" using slf4j APIs as okay.
> 
> Darren
> 
> [1] Projects depending on SLF4J at http://www.slf4j.org/

Re: Build failure on master (KVM related)

[David Nalley]

On Tue, Sep 17, 2013 at 4:01 PM, Edison Su  wrote:
> We should be cautious to advance java version, as most of our code run/tested 
> on java 6.
> Is it a must to upgrade to Libvirt 0.5.0?
>


We've discussed this before:

http://cloudstack.markmail.org/thread/ucos7mpgfu5juskb

[PROPOSAL] Modularize Spring

[Darren Shepherd]

Currently ACS code is fairly modular in that you can add plug-ins to ACS to
extend most functionality.  Unfortunately ACS is not packaged in a modular
way.  It is still delivered essentially as one large unit.  There are many
reason for this but one large barrier to modularizing ACS is that the
Spring configuration is managed as one large unit.

I propose that we break apart the Spring XML configuration such that each
component contributes its own configuration.  Additionally each component
will be loaded into its own Spring ApplicationContext such that its beans
will not conflict with the wiring of other beans in ACS.  This change will
lay the foundation for a richer plugin model and additionally a more
distributed architecture.

The technical details for this proposal can be found on the wiki [1].

Darren

[1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Modularize+Spring

Re: Apache CloudStack 4.2.0 (fifth round)

[Amogh Vasekar]

+1
Tested using the release procedure.

Thanks,
Amogh

On 9/16/13 5:54 PM, "Animesh Chaturvedi" 
wrote:

>
>
>72 hours for this VOTE has passed, I had kept this VOTE open until
>Wednesday  because of weekend, but many community members tested this
>over weekend and we have 12 (+1) Votes and one (+0). Should I keep the
>VOTE open until Wednesday or is the community OK to close it out.
>
>
>Thanks
>Animesh
>
>
>
>From: Animesh Chaturvedi
>Sent: Friday, September 13, 2013 4:12 PM
>To: dev@cloudstack.apache.org
>Subject: Apache CloudStack 4.2.0 (fifth round)
>
>I've created a 4.2.0 release, with the following artifacts up for a vote:
>
>Git Branch and Commit SH:
>https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs
>/heads/4.2
>Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f
>
>List of changes:
>https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;f=CH
>ANGES;hb=4.2
>
>Source release (checksums and signatures are available at the same
>location):
>https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/
>
>PGP release keys (signed using 94BE0D7C):
>https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
>Testing instructions are here:
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+proced
>ure
>
>Vote will be open for 72 hours (Wednesday 9/18 End of Day PST).
>
>For sanity in tallying the vote, can PMC members please be sure to
>indicate "(binding)" with their vote?
>
>[ ] +1  approve
>[ ] +0  no opinion
>[ ] -1  disapprove (and reason why)
>

Re: Apache CloudStack 4.2.0 (fifth round)

[Simon Weller]

+1 

Built nonoss RPMS and deployed fresh installation on RHEL 6.3. 
Tested CLVM on KVM, adding users, adding projects, adding templates/isos and 
deploying VMs. 

I have not tested an upgrade as of yet. 

- Original Message -

From: "Animesh Chaturvedi"  
To: dev@cloudstack.apache.org 
Sent: Friday, September 13, 2013 6:12:39 PM 
Subject: Apache CloudStack 4.2.0 (fifth round) 


I've created a 4.2.0 release, with the following artifacts up for a vote: 

Git Branch and Commit SH: 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.2
 
Commit: c1e24ff89f6d14d6ae74d12dbca108c35449030f 

List of changes: 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;f=CHANGES;hb=4.2
 

Source release (checksums and signatures are available at the same 
location): 
https://dist.apache.org/repos/dist/dev/cloudstack/4.2.0/ 

PGP release keys (signed using 94BE0D7C): 
https://dist.apache.org/repos/dist/release/cloudstack/KEYS 

Testing instructions are here: 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+test+procedure 

Vote will be open for 72 hours (Wednesday 9/18 End of Day PST). 

For sanity in tallying the vote, can PMC members please be sure to indicate 
"(binding)" with their vote? 

[ ] +1 approve 
[ ] +0 no opinion 
[ ] -1 disapprove (and reason why) 

Review Request 14185: Add managed context library

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14185/
---

Review request for cloudstack and Alex Huang.


Repository: cloudstack-git


Description
---

This is a small framework to manage CloudStack's thread context information.  
This patch commits the framework but it is not included in the build.


Diffs
-

  framework/managed-context/pom.xml PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/AbstractManagedContextListener.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/ManagedContext.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/ManagedContextListener.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/ManagedContextRunnable.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/ManagedContextTimerTask.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/ManagedContextUtils.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/context/impl/DefaultManagedContext.java
 PRE-CREATION 
  
framework/managed-context/src/main/java/org/apache/cloudstack/managed/threadlocal/ManagedThreadLocal.java
 PRE-CREATION 
  
framework/managed-context/src/test/java/org/apache/cloudstack/managed/context/impl/DefaultManagedContextTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/14185/diff/


Testing
---


Thanks,

Darren Shepherd

Re: HA redundant virtual router

[Sheng Yang]

No, it's not intentional.

the HAproxy is a part of services that redundant router would
enable/disabled according to the MASTER/BACKUP status. All the services
related to redundant router are controlled by services.sh.

What's the failure of HAproxy exactly in your case? And what's the root
cause?

Also, I think just yield due to haproxy failure won't help much since
effort still needed for CS to recover the situation, at least it would need
to notify admin. Better transiting to FAULT state if it's a critical error.

--Sheng


On Tue, Sep 17, 2013 at 12:07 AM, Sten Spans  wrote:

> On Mon, 16 Sep 2013, Sheng Yang wrote:
>
>  The reason for no HA as I said before, due to the complexity. E.g, if
>> there
>> can be 3 routers in the network(which control network is down but not the
>> guest network), and it would cause two of them with the same priority(at
>> certain time). The doc is mainly for describing the current policy and
>> reason, as the basic for possible improvement.
>>
>> I haven't thought much about redundant router with HA, but many times
>> we're
>> dealing with intermittent network issue, so you try to plug off then plug
>> in the network cable to see if HA works as expect.
>>
>> The priority cannot changed on the fly, it's a parameter of keepalived
>> process, which is running. So at least both router need to be stopped
>> before priority reset. And it's not reset to minimum, since the value can
>> go up or down based on the different cases.
>>
>
> Looking at the doc you wrote I see no mention of HAProxy.
> Is this intentional?
>
> A failure of HAProxy (which we've observed in practice) would
> result in a loss of service for loadbalanced ports.
>
> Currently I'm thinking of adding something like the following:
>
>  diff -Nru keepalived.conf.templ.orig keepalived.conf.templ
> --- keepalived.conf.templ.orig  2013-09-17 09:02:28.410646521 +0200
> +++ keepalived.conf.templ   2013-09-17 09:03:34.131434084 +0200
> @@ -19,6 +19,12 @@
> router_id [ROUTER_ID]
>  }
>
> +vrrp_script check_haproxy {
> +script "/usr/bin/killall -0 haproxy"
> +interval 5
> +weight 10
> +}
> +
>  vrrp_script check_bumpup {
>  script "[RROUTER_BIN_PATH]/check_**bumpup.sh"
>  interval 5
> @@ -47,6 +53,7 @@
>  }
>
>  track_script {
> +check_haproxy
>  check_bumpup
>  heartbeat
>  }
>
>
> This would boost vrrp priorities if haproxy is running, trigger
> a failover if it fails, and should be harmless on hosts not running
> haproxy.
>
> --
> Sten Spans
>
> "There is a crack in everything, that's how the light gets in."
> Leonard Cohen - Anthem
>

Re: contrail virtual network controller

[Pedro Roque Marques]

On Sep 17, 2013, at 10:30 AM, Chip Childers wrote:
> 
> Ideally, we would have a design document on the wiki that would help
> everyone understand the implementation.  We're not looking for a design
> doc for Contrail / OpenContrail...  we're looking for information about
> the integration itself.  This is both for other devs, tech writers that
> may choose to help document this, and for users to get a better feel for 
> the feature.

Is the following a bit more reasonable ?
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Contrail+network+plugin

  Pedro.

Review Request 14186: Create parent pom to allow maven standard paths in project

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14186/
---

Review request for cloudstack, Alex Huang and Kelven Yang.


Repository: cloudstack-git


Description
---

Currently all cloudstack poms use non-standard maven paths for artifacts.  The 
overrides are defined in the parent pom.  This parent pom extends the 
cloudstack top level POM but then overrides the non-standard paths back to the 
standard paths.  This patch adds the parent pom but does not add it to the 
build.  That will come in another patch.


Diffs
-

  maven-standard/pom.xml PRE-CREATION 

Diff: https://reviews.apache.org/r/14186/diff/


Testing
---


Thanks,

Darren Shepherd

RE: [DISCUSS] CSS framework for CloudStack UI

[Brian Federle]

Following up with this thread I started last month.

For the first step (hopefully ready by 4.3), I'm planning on getting the 
SCSS/SASS compiler working with the UI build  -> 
https://github.com/Jasig/sass-maven-plugin. I've experimented with it using the 
existing CSS in our UI, and it will be pretty easy to use right away, then will 
gradually refactor over time into a more robust framework. Hopefully I can set 
up a work-in-progress branch up soon once I've got something tangible.

Amit/Rayees/Frank/anyone else, let me know if you are able to find out if this 
is fairly easy to setup; I'm not familiar at all with Maven, so will need a 
fellow expert to help out here :)

-Brian

-Original Message-
From: Amit Das [mailto:amit@cloudbyte.com] 
Sent: Monday, August 12, 2013 10:11 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] CSS framework for CloudStack UI

Hi Brian,

I should be able to help on the build tools part.

However, I am new to ACS & ACS UI. So I might take some time to understand the 
existing build process of ACS UI.

We can surely coordinate on this.


Regards,
Amit
*CloudByte Inc.* 


On Thu, Aug 8, 2013 at 11:11 PM, Brian Federle wrote:

> That sounds awesome, though I'm not too much of a guru with either 
> maven or grunt -- I would be glad to coordinate with anyone who wants 
> to take that on.
>
> -Brian
>
> -Original Message-
> From: Amit Das [mailto:amit@cloudbyte.com]
> Sent: Tuesday, August 06, 2013 10:16 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] CSS framework for CloudStack UI
>
> This is definitely a great idea.
>
> Will it be feasible to decouple the front-end build from server-side 
> build as mentioned in the following link:
> http://addyosmani.com/blog/making-maven-grunt/
>
>
> Regards,
> Amit Kumar Das
> *CloudByte Inc.* 
>
>
> On Wed, Aug 7, 2013 at 5:48 AM, Ian Duffy  wrote:
>
> > +1 to that definitely!
> >
> >
> > > The only problem I cannot handle is integrating the SASS compiler 
> > > (which
> > converts .scss->.css) into the UI build phase. I've found a maven 
> > plugin at https://github.com/Jasig/sass-maven-plugin, so if anyone 
> > wants to tackle that  it would be awesome.
> >
> >
> > Below is a meer suggestion, I've only read about it and never done it.
> > I believe you are able to chain grunt into the maven life cycle. 
> > This would allow us to do various things with the front end code. 
> > This includes but is not limited to:
> >
> > - JSHint
> > - Unit tests via xyz(suggesting QUnit or Jasmine) framework
> > - Static analysis of the javascript with Plato
> > - Compiling SASS + LESS
> > - Uglifying/Minifying
> > - Optimise images(png crush etc.)
> > - Dependency management of 3rd party js libs
> >
> > It would supply us with a bunch nicer build process for the UI 
> > elements of Cloudstack. However it would add a dependency of nodejs 
> > to the build process.
> >
>

Review Request 14187: Framework to discover and load spring contexts in a hierarchy

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14187/
---

Review request for cloudstack, Alex Huang and Kelven Yang.


Repository: cloudstack-git


Description
---

This is a small framework to discover Spring XML files and then load then in a 
hierarchy of application contexts.  This patch adds the new project but does 
not add it to the build.  That will come later.


Diffs
-

  framework/spring/module/pom.xml PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/context/ResourceApplicationContext.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/factory/CloudStackSpringContext.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/factory/ModuleBasedContextFactory.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/locator/ModuleDefinitionLocator.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/locator/impl/ClasspathModuleDefinitionLocator.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/ModuleDefinition.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/ModuleDefinitionSet.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinition.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinitionSet.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/util/Main.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/util/ModuleLocationUtils.java
 PRE-CREATION 
  
framework/spring/module/src/main/java/org/apache/cloudstack/spring/module/web/CloudStackContextLoaderListener.java
 PRE-CREATION 
  
framework/spring/module/src/main/resources/org/apache/cloudstack/spring/module/model/impl/defaults-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/factory/ModuleBasedContextFactoryTest.java
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/locator/impl/ClasspathModuleDefinitionSetLocatorTest.java
 PRE-CREATION 
  
framework/spring/module/src/test/java/org/apache/cloudstack/spring/module/model/impl/DefaultModuleDefinitionTest.java
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/defaults.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/empty-context-inheritable.xml
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/empty-context.xml 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/empty2-context-inheritable.xml
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/empty2-context.xml 
PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/all/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/all/test2-defaults.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/badname/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/blankname/module.properties
 PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/good/empty-context.xml 
PRE-CREATION 
  framework/spring/module/src/test/resources/testfiles/good/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/missingname/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testfiles/wrongname/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/module.properties 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/test-context-inheritable.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/base/test-context.xml 
PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1-1/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1-1/test-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child1/test-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child2/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/child2/test-context.xml
 PRE-CREATION 
  
framework/spring/module/src/test/resources/testhierarchy/orphan1/module.properties
 PRE-CREATION 
  
framework/spring/module/src/test/resource

Re: HA redundant virtual router

[Simon Weller]

I think monit is installed currently in the system vm image. Sounds like it 
might make more sense to manage haproxy via monit, and allow it to recover the 
service should it fail. 

- Original Message -

From: "Sheng Yang"  
To: ""  
Cc: "Daan Hoogland" , "int-cloud" 
 
Sent: Tuesday, September 17, 2013 6:10:42 PM 
Subject: Re: HA redundant virtual router 

No, it's not intentional. 

the HAproxy is a part of services that redundant router would 
enable/disabled according to the MASTER/BACKUP status. All the services 
related to redundant router are controlled by services.sh. 

What's the failure of HAproxy exactly in your case? And what's the root 
cause? 

Also, I think just yield due to haproxy failure won't help much since 
effort still needed for CS to recover the situation, at least it would need 
to notify admin. Better transiting to FAULT state if it's a critical error. 

--Sheng 


On Tue, Sep 17, 2013 at 12:07 AM, Sten Spans  wrote: 

> On Mon, 16 Sep 2013, Sheng Yang wrote: 
> 
> The reason for no HA as I said before, due to the complexity. E.g, if 
>> there 
>> can be 3 routers in the network(which control network is down but not the 
>> guest network), and it would cause two of them with the same priority(at 
>> certain time). The doc is mainly for describing the current policy and 
>> reason, as the basic for possible improvement. 
>> 
>> I haven't thought much about redundant router with HA, but many times 
>> we're 
>> dealing with intermittent network issue, so you try to plug off then plug 
>> in the network cable to see if HA works as expect. 
>> 
>> The priority cannot changed on the fly, it's a parameter of keepalived 
>> process, which is running. So at least both router need to be stopped 
>> before priority reset. And it's not reset to minimum, since the value can 
>> go up or down based on the different cases. 
>> 
> 
> Looking at the doc you wrote I see no mention of HAProxy. 
> Is this intentional? 
> 
> A failure of HAProxy (which we've observed in practice) would 
> result in a loss of service for loadbalanced ports. 
> 
> Currently I'm thinking of adding something like the following: 
> 
> diff -Nru keepalived.conf.templ.orig keepalived.conf.templ 
> --- keepalived.conf.templ.orig 2013-09-17 09:02:28.410646521 +0200 
> +++ keepalived.conf.templ 2013-09-17 09:03:34.131434084 +0200 
> @@ -19,6 +19,12 @@ 
> router_id [ROUTER_ID] 
> } 
> 
> +vrrp_script check_haproxy { 
> + script "/usr/bin/killall -0 haproxy" 
> + interval 5 
> + weight 10 
> +} 
> + 
> vrrp_script check_bumpup { 
> script "[RROUTER_BIN_PATH]/check_**bumpup.sh" 
> interval 5 
> @@ -47,6 +53,7 @@ 
> } 
> 
> track_script { 
> + check_haproxy 
> check_bumpup 
> heartbeat 
> } 
> 
> 
> This would boost vrrp priorities if haproxy is running, trigger 
> a failover if it fails, and should be harmless on hosts not running 
> haproxy. 
> 
> -- 
> Sten Spans 
> 
> "There is a crack in everything, that's how the light gets in." 
> Leonard Cohen - Anthem 
> 

Review Request 14189: UserVmDomRInvestigator and ManagementIPSystemVMInvestigator mask adapter name

[Darren Shepherd]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14189/
---

Review request for cloudstack, Alex Huang and Kelven Yang.


Repository: cloudstack-git


Description
---

UserVmDomRInvestigator and ManagementIPSystemVMInvestigator mask _name from 
ComponentLifeCycleBase making set/getName() not work as expected


Diffs
-

  server/src/com/cloud/ha/ManagementIPSystemVMInvestigator.java 2b6d261 
  server/src/com/cloud/ha/UserVmDomRInvestigator.java 195deff 

Diff: https://reviews.apache.org/r/14189/diff/


Testing
---


Thanks,

Darren Shepherd

Re: security around api.log

[Ian Duffy]

Just after doing a installation of Cloudstack 4.1.1

apilog.log was created with the following permissions:

-rw-rw-r--. 1 cloud cloud  95449 Sep 18 01:05 apilog.log

Owner..rw
Group..rw
Nobody/everybodyr

Considering what was discussed above this is not good. Anybody with
shell access to the box could possibly get admin login to Cloudstack
if they happen to pull out the correct session information within the
1 hour expire time. Fairly easy if you just run tail -f and grep out
some admin command.

On 13 September 2013 22:14, Ian Duffy  wrote:
>>  I haven't tried it yet, but can't I use that info to hijack the session?
>
> You can...
>
> Create a cookie: (please excuse the full stops as spaces, didn't trust it to
> render correctly)
>
> Key... Value
> JSESSIONID  7asvmtwoesbc6ia3e4kxtzrl
> sessionKey ... ec6h46Om8a1y3d%252BhrdIpQ85cAfc%253D
>
> and pass all requests with a parameter of:
> sessionkey = ec6h46Om8a1y3d%2BhrdIpQ85cAfc%3D
>
>>  So that api.log file really needs to be protected in the same way a file
>> with a password in it would be
>
> I don't have the manager deployed anywhere to test this but I would hope the
> log file is read/write only to the owner user.
>
>> I would suggest that we just don't log the sessionId or sessionKey.
>
> +1 to that.
>
>
> On 13 September 2013 21:40, Darren Shepherd 
> wrote:
>>
>> I just noticed api.log which seems to log all the API access in a form
>> like
>>
>> 2013-09-13 00:02:09,451 INFO  [a.c.c.a.ApiServer]
>> (2011638958@qtp-657397168-0:ctx-81b1e088 ctx-174e4a62) (userId=2 accountId=2
>> sessionId=7asvmtwoesbc6ia3e4kxtzrl) 127.0.0.1 -- GET
>> command=listZones&response=json&sessionkey=ec6h46Om8a1y3d%2BhrdIpQ85cAfc%3D&_=1379055729422
>> 200 { "listzonesresponse" : { "count":1 ,"zone" : [
>> {"id":"cdaf82f1-3b57-4aa4-b3ce-b60173ed45f2","name":"zone1","dns1":"8.8.8.8","dns2":"8.8.4.4","internaldns1":"8.8.4.4","networktype":"Basic","securitygroupsenabled":true,"allocationstate":"Enabled","zonetoken":"6dce94e8-e8dc-3077-bfde-c6e8594bd449","dhcpprovider":"VirtualRouter","localstorageenabled":false}
>> ] } }
>>
>> The sessionId and sessionKey is logged in the file.  I haven't tried it
>> yet, but can't I use that info to hijack the session?  That introduces a
>> security issue in that any server operator can now hijack anybody's session.
>> So that api.log file really needs to be protected in the same way a file
>> with a password in it would be.
>>
>> I would suggest that we just don't log the sessionId or sessionKey.
>>
>> Darren
>
>

Re: contrail virtual network controller

[Chip Childers]

On Sep 17, 2013, at 7:05 PM, Pedro Roque Marques 
wrote:

On Sep 17, 2013, at 10:30 AM, Chip Childers wrote:


Ideally, we would have a design document on the wiki that would help
everyone understand the implementation.  We're not looking for a design
doc for Contrail / OpenContrail...  we're looking for information about
the integration itself.  This is both for other devs, tech writers that
may choose to help document this, and for users to get a better feel for
the feature.


Is the following a bit more reasonable ?
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Contrail+network+plugin

  Pedro.


LGTM. Can you get the code up on review board so that we can look at it
(and have something to start the donation process with)?

IMO design questions / debate can happen after we have the initial code
agreed as something to accept (into a feature branch). Work can continue
here in the community from that point on to get it ready for master.

RE: [DISCUSS] CSS framework for CloudStack UI

[Edison Su]

How about use maven-exec->call grunt->compile CSS?
http://addyosmani.com/blog/making-maven-grunt/
This is more flexible than sass-maven-plugin, as we can do lot of stuff in 
grunt, such unit test.

> -Original Message-
> From: Brian Federle [mailto:brian.fede...@citrix.com]
> Sent: Tuesday, September 17, 2013 3:53 PM
> To: 'dev@cloudstack.apache.org'
> Cc: Rayees Namathponnan; Frank Zhang; Animesh Chaturvedi
> Subject: RE: [DISCUSS] CSS framework for CloudStack UI
> 
> Following up with this thread I started last month.
> 
> For the first step (hopefully ready by 4.3), I'm planning on getting the
> SCSS/SASS compiler working with the UI build  ->
> https://github.com/Jasig/sass-maven-plugin. I've experimented with it using
> the existing CSS in our UI, and it will be pretty easy to use right away, then
> will gradually refactor over time into a more robust framework. Hopefully I
> can set up a work-in-progress branch up soon once I've got something
> tangible.
> 
> Amit/Rayees/Frank/anyone else, let me know if you are able to find out if
> this is fairly easy to setup; I'm not familiar at all with Maven, so will 
> need a
> fellow expert to help out here :)
> 
> -Brian
> 
> -Original Message-
> From: Amit Das [mailto:amit@cloudbyte.com]
> Sent: Monday, August 12, 2013 10:11 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] CSS framework for CloudStack UI
> 
> Hi Brian,
> 
> I should be able to help on the build tools part.
> 
> However, I am new to ACS & ACS UI. So I might take some time to
> understand the existing build process of ACS UI.
> 
> We can surely coordinate on this.
> 
> 
> Regards,
> Amit
> *CloudByte Inc.* 
> 
> 
> On Thu, Aug 8, 2013 at 11:11 PM, Brian Federle
> wrote:
> 
> > That sounds awesome, though I'm not too much of a guru with either
> > maven or grunt -- I would be glad to coordinate with anyone who wants
> > to take that on.
> >
> > -Brian
> >
> > -Original Message-
> > From: Amit Das [mailto:amit@cloudbyte.com]
> > Sent: Tuesday, August 06, 2013 10:16 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] CSS framework for CloudStack UI
> >
> > This is definitely a great idea.
> >
> > Will it be feasible to decouple the front-end build from server-side
> > build as mentioned in the following link:
> > http://addyosmani.com/blog/making-maven-grunt/
> >
> >
> > Regards,
> > Amit Kumar Das
> > *CloudByte Inc.* 
> >
> >
> > On Wed, Aug 7, 2013 at 5:48 AM, Ian Duffy  wrote:
> >
> > > +1 to that definitely!
> > >
> > >
> > > > The only problem I cannot handle is integrating the SASS compiler
> > > > (which
> > > converts .scss->.css) into the UI build phase. I've found a maven
> > > plugin at https://github.com/Jasig/sass-maven-plugin, so if anyone
> > > wants to tackle that  it would be awesome.
> > >
> > >
> > > Below is a meer suggestion, I've only read about it and never done it.
> > > I believe you are able to chain grunt into the maven life cycle.
> > > This would allow us to do various things with the front end code.
> > > This includes but is not limited to:
> > >
> > > - JSHint
> > > - Unit tests via xyz(suggesting QUnit or Jasmine) framework
> > > - Static analysis of the javascript with Plato
> > > - Compiling SASS + LESS
> > > - Uglifying/Minifying
> > > - Optimise images(png crush etc.)
> > > - Dependency management of 3rd party js libs
> > >
> > > It would supply us with a bunch nicer build process for the UI
> > > elements of Cloudstack. However it would add a dependency of nodejs
> > > to the build process.
> > >
> >

Re: contrail virtual network controller

[Pedro Roque Marques]

On Sep 17, 2013, at 5:17 PM, Chip Childers wrote:

> 
> 
> On Sep 17, 2013, at 7:05 PM, Pedro Roque Marques  
> wrote:
> 
>> On Sep 17, 2013, at 10:30 AM, Chip Childers wrote:
>>> 
>>> Ideally, we would have a design document on the wiki that would help
>>> everyone understand the implementation.  We're not looking for a design
>>> doc for Contrail / OpenContrail...  we're looking for information about
>>> the integration itself.  This is both for other devs, tech writers that
>>> may choose to help document this, and for users to get a better feel for 
>>> the feature.
>> 
>> Is the following a bit more reasonable ?
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Contrail+network+plugin
>> 
>>   Pedro.
>> 
> 
> LGTM. Can you get the code up on review board so that we can look at it (and 
> have something to start the donation process with)?

Will do. It will probably take us a few days to to sync from from our 4.2 based 
branch to master.

> 
> IMO design questions / debate can happen after we have the initial code 
> agreed as something to accept (into a feature branch). Work can continue here 
> in the community from that point on to get it ready for master.

Great.

Thanks,
  Pedro.

Re: Managed storage with KVM

[Mike Tutkowski]

Hey Marcus,

I'm reviewing your e-mails as I implement the necessary methods in new
classes.

"So, referencing StorageAdaptor.java, createStoragePool accepts all of
the pool data (host, port, name, path) which would be used to log the
host into the initiator."

Can you tell me, in my case, since a storage pool (primary storage) is
actually the SAN, I wouldn't really be logging into anything at this point,
correct?

Also, what kind of capacity, available, and used bytes make sense to report
for KVMStoragePool (since KVMStoragePool represents the SAN in my case and
not an individual LUN)?

Thanks!


On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen wrote:

> Ok, KVM will be close to that, of course, because only the hypervisor
> classes differ, the rest is all mgmt server. Creating a volume is just
> a db entry until it's deployed for the first time. AttachVolumeCommand
> on the agent side (LibvirtStorageAdaptor.java is analogous to
> CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> StorageAdaptor) to log in the host to the target and then you have a
> block device.  Maybe libvirt will do that for you, but my quick read
> made it sound like the iscsi libvirt pool type is actually a pool, not
> a lun or volume, so you'll need to figure out if that works or if
> you'll have to use iscsiadm commands.
>
> If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> doesn't really manage your pool the way you want), you're going to
> have to create a version of KVMStoragePool class and a StorageAdaptor
> class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> implementing all of the methods, then in KVMStorageManager.java
> there's a "_storageMapper" map. This is used to select the correct
> adaptor, you can see in this file that every call first pulls the
> correct adaptor out of this map via getStorageAdaptor. So you can see
> a comment in this file that says "add other storage adaptors here",
> where it puts to this map, this is where you'd register your adaptor.
>
> So, referencing StorageAdaptor.java, createStoragePool accepts all of
> the pool data (host, port, name, path) which would be used to log the
> host into the initiator. I *believe* the method getPhysicalDisk will
> need to do the work of attaching the lun.  AttachVolumeCommand calls
> this and then creates the XML diskdef and attaches it to the VM. Now,
> one thing you need to know is that createStoragePool is called often,
> sometimes just to make sure the pool is there. You may want to create
> a map in your adaptor class and keep track of pools that have been
> created, LibvirtStorageAdaptor doesn't have to do this because it asks
> libvirt about which storage pools exist. There are also calls to
> refresh the pool stats, and all of the other calls can be seen in the
> StorageAdaptor as well. There's a createPhysical disk, clone, etc, but
> it's probably a hold-over from 4.1, as I have the vague idea that
> volumes are created on the mgmt server via the plugin now, so whatever
> doesn't apply can just be stubbed out (or optionally
> extended/reimplemented here, if you don't mind the hosts talking to
> the san api).
>
> There is a difference between attaching new volumes and launching a VM
> with existing volumes.  In the latter case, the VM definition that was
> passed to the KVM agent includes the disks, (StartCommand).
>
> I'd be interested in how your pool is defined for Xen, I imagine it
> would need to be kept the same. Is it just a definition to the SAN
> (ip address or some such, port number) and perhaps a volume pool name?
>
> > If there is a way for me to update the ACL list on the SAN to have only a
> > single KVM host have access to the volume, that would be ideal.
>
> That depends on your SAN API.  I was under the impression that the
> storage plugin framework allowed for acls, or for you to do whatever
> you want for create/attach/delete/snapshot, etc. You'd just call your
> SAN API with the host info for the ACLs prior to when the disk is
> attached (or the VM is started).  I'd have to look more at the
> framework to know the details, in 4.1 I would do this in
> getPhysicalDisk just prior to connecting up the LUN.
>
>
> On Fri, Sep 13, 2013 at 10:27 PM, Mike Tutkowski
>  wrote:
> > OK, yeah, the ACL part will be interesting. That is a bit different from
> how
> > it works with XenServer and VMware.
> >
> > Just to give you an idea how it works in 4.2 with XenServer:
> >
> > * The user creates a CS volume (this is just recorded in the
> cloud.volumes
> > table).
> >
> > * The user attaches the volume as a disk to a VM for the first time (if
> the
> > storage allocator picks the SolidFire plug-in, the storage framework
> invokes
> > a method on the plug-in that creates a volume on the SAN...info like the
> IQN
> > of the SAN volume is recorded in the DB).
> >
> > * CitrixResourceBase's execute(AttachVolumeCommand) is executed. It
> > determines based on a flag passed in that the storage in que

Re: Managed storage with KVM

[Marcus Sorensen]

What do you do with Xen? I imagine the user enter the SAN details when
registering the pool? A the pool details are basically just instructions on
how to log into a target, correct?

You can choose to log in a KVM host to the target during createStoragePool
and save the pool in a map, or just save the pool info in a map for future
reference by uuid, for when you do need to log in. The createStoragePool
then just becomes a way to save the pool info to the agent. Personally, I'd
log in on the pool create and look/scan for specific luns when they're
needed, but I haven't thought it through thoroughly. I just say that mainly
because login only happens once, the first time the pool is used, and every
other storage command is about discovering new luns or maybe
deleting/disconnecting luns no longer needed. On the other hand, you could
do all of the above: log in on pool create, then also check if you're
logged in on other commands and log in if you've lost connection.

With Xen, what does your registered pool   show in the UI for avail/used
capacity, and how does it get that info? I assume there is some sort of
disk pool that the luns are carved from, and that your plugin is called to
talk to the SAN and expose to the user how much of that pool has been
allocated. Knowing how you already solves these problems with Xen will help
figure out what to do with KVM.

If this is the case, I think the plugin can continue to handle it rather
than getting details from the agent. I'm not sure if that means nulls are
OK for these on the agent side or what, I need to look at the storage
plugin arch more closely.
On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
wrote:

> Hey Marcus,
>
> I'm reviewing your e-mails as I implement the necessary methods in new
> classes.
>
> "So, referencing StorageAdaptor.java, createStoragePool accepts all of
> the pool data (host, port, name, path) which would be used to log the
> host into the initiator."
>
> Can you tell me, in my case, since a storage pool (primary storage) is
> actually the SAN, I wouldn't really be logging into anything at this point,
> correct?
>
> Also, what kind of capacity, available, and used bytes make sense to report
> for KVMStoragePool (since KVMStoragePool represents the SAN in my case and
> not an individual LUN)?
>
> Thanks!
>
>
> On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen  >wrote:
>
> > Ok, KVM will be close to that, of course, because only the hypervisor
> > classes differ, the rest is all mgmt server. Creating a volume is just
> > a db entry until it's deployed for the first time. AttachVolumeCommand
> > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > StorageAdaptor) to log in the host to the target and then you have a
> > block device.  Maybe libvirt will do that for you, but my quick read
> > made it sound like the iscsi libvirt pool type is actually a pool, not
> > a lun or volume, so you'll need to figure out if that works or if
> > you'll have to use iscsiadm commands.
> >
> > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > doesn't really manage your pool the way you want), you're going to
> > have to create a version of KVMStoragePool class and a StorageAdaptor
> > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> > implementing all of the methods, then in KVMStorageManager.java
> > there's a "_storageMapper" map. This is used to select the correct
> > adaptor, you can see in this file that every call first pulls the
> > correct adaptor out of this map via getStorageAdaptor. So you can see
> > a comment in this file that says "add other storage adaptors here",
> > where it puts to this map, this is where you'd register your adaptor.
> >
> > So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > the pool data (host, port, name, path) which would be used to log the
> > host into the initiator. I *believe* the method getPhysicalDisk will
> > need to do the work of attaching the lun.  AttachVolumeCommand calls
> > this and then creates the XML diskdef and attaches it to the VM. Now,
> > one thing you need to know is that createStoragePool is called often,
> > sometimes just to make sure the pool is there. You may want to create
> > a map in your adaptor class and keep track of pools that have been
> > created, LibvirtStorageAdaptor doesn't have to do this because it asks
> > libvirt about which storage pools exist. There are also calls to
> > refresh the pool stats, and all of the other calls can be seen in the
> > StorageAdaptor as well. There's a createPhysical disk, clone, etc, but
> > it's probably a hold-over from 4.1, as I have the vague idea that
> > volumes are created on the mgmt server via the plugin now, so whatever
> > doesn't apply can just be stubbed out (or optionally
> > extended/reimplemented here, if you don't mind the hosts talking to
> > the san api).
> >
> > There is a difference between 

Re: Managed storage with KVM

[Mike Tutkowski]

Hi Marcus,

I never need to respond to a CreateStoragePool call for either XenServer or
VMware.

What happens is I respond only to the Attach- and Detach-volume commands.

Let's say an attach comes in:

In this case, I check to see if the storage is "managed." Talking XenServer
here, if it is, I log in to the LUN that is the disk we want to attach.
After, if this is the first time attaching this disk, I create an SR and a
VDI within the SR. If it is not the first time attaching this disk, the LUN
already has the SR and VDI on it.

Once this is done, I let the normal "attach" logic run because this logic
expected an SR and a VDI and now it has it.

It's the same thing for VMware: Just substitute datastore for SR and VMDK
for VDI.

Does that make sense?

Thanks!


On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen wrote:

> What do you do with Xen? I imagine the user enter the SAN details when
> registering the pool? A the pool details are basically just instructions on
> how to log into a target, correct?
>
> You can choose to log in a KVM host to the target during createStoragePool
> and save the pool in a map, or just save the pool info in a map for future
> reference by uuid, for when you do need to log in. The createStoragePool
> then just becomes a way to save the pool info to the agent. Personally, I'd
> log in on the pool create and look/scan for specific luns when they're
> needed, but I haven't thought it through thoroughly. I just say that mainly
> because login only happens once, the first time the pool is used, and every
> other storage command is about discovering new luns or maybe
> deleting/disconnecting luns no longer needed. On the other hand, you could
> do all of the above: log in on pool create, then also check if you're
> logged in on other commands and log in if you've lost connection.
>
> With Xen, what does your registered pool   show in the UI for avail/used
> capacity, and how does it get that info? I assume there is some sort of
> disk pool that the luns are carved from, and that your plugin is called to
> talk to the SAN and expose to the user how much of that pool has been
> allocated. Knowing how you already solves these problems with Xen will help
> figure out what to do with KVM.
>
> If this is the case, I think the plugin can continue to handle it rather
> than getting details from the agent. I'm not sure if that means nulls are
> OK for these on the agent side or what, I need to look at the storage
> plugin arch more closely.
> On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
> wrote:
>
> > Hey Marcus,
> >
> > I'm reviewing your e-mails as I implement the necessary methods in new
> > classes.
> >
> > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > the pool data (host, port, name, path) which would be used to log the
> > host into the initiator."
> >
> > Can you tell me, in my case, since a storage pool (primary storage) is
> > actually the SAN, I wouldn't really be logging into anything at this
> point,
> > correct?
> >
> > Also, what kind of capacity, available, and used bytes make sense to
> report
> > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
> and
> > not an individual LUN)?
> >
> > Thanks!
> >
> >
> > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen  > >wrote:
> >
> > > Ok, KVM will be close to that, of course, because only the hypervisor
> > > classes differ, the rest is all mgmt server. Creating a volume is just
> > > a db entry until it's deployed for the first time. AttachVolumeCommand
> > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > StorageAdaptor) to log in the host to the target and then you have a
> > > block device.  Maybe libvirt will do that for you, but my quick read
> > > made it sound like the iscsi libvirt pool type is actually a pool, not
> > > a lun or volume, so you'll need to figure out if that works or if
> > > you'll have to use iscsiadm commands.
> > >
> > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > > doesn't really manage your pool the way you want), you're going to
> > > have to create a version of KVMStoragePool class and a StorageAdaptor
> > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> > > implementing all of the methods, then in KVMStorageManager.java
> > > there's a "_storageMapper" map. This is used to select the correct
> > > adaptor, you can see in this file that every call first pulls the
> > > correct adaptor out of this map via getStorageAdaptor. So you can see
> > > a comment in this file that says "add other storage adaptors here",
> > > where it puts to this map, this is where you'd register your adaptor.
> > >
> > > So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > > the pool data (host, port, name, path) which would be used to log the
> > > host into the initiator. I *believe* the method getPhysicalDisk will
>

Re: Managed storage with KVM

[Mike Tutkowski]

"I imagine the user enter the SAN details when
registering the pool?"

When primary storage is added to CS that is based on the SolidFire plug-in,
these details (host, port, etc.) are provided. The primary storage then
represents the SAN and not a preallocated volume (i.e. not a particular
LUN).


On Tue, Sep 17, 2013 at 7:51 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Hi Marcus,
>
> I never need to respond to a CreateStoragePool call for either XenServer
> or VMware.
>
> What happens is I respond only to the Attach- and Detach-volume commands.
>
> Let's say an attach comes in:
>
> In this case, I check to see if the storage is "managed." Talking
> XenServer here, if it is, I log in to the LUN that is the disk we want to
> attach. After, if this is the first time attaching this disk, I create an
> SR and a VDI within the SR. If it is not the first time attaching this
> disk, the LUN already has the SR and VDI on it.
>
> Once this is done, I let the normal "attach" logic run because this logic
> expected an SR and a VDI and now it has it.
>
> It's the same thing for VMware: Just substitute datastore for SR and VMDK
> for VDI.
>
> Does that make sense?
>
> Thanks!
>
>
> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen wrote:
>
>> What do you do with Xen? I imagine the user enter the SAN details when
>> registering the pool? A the pool details are basically just instructions
>> on
>> how to log into a target, correct?
>>
>> You can choose to log in a KVM host to the target during createStoragePool
>> and save the pool in a map, or just save the pool info in a map for future
>> reference by uuid, for when you do need to log in. The createStoragePool
>> then just becomes a way to save the pool info to the agent. Personally,
>> I'd
>> log in on the pool create and look/scan for specific luns when they're
>> needed, but I haven't thought it through thoroughly. I just say that
>> mainly
>> because login only happens once, the first time the pool is used, and
>> every
>> other storage command is about discovering new luns or maybe
>> deleting/disconnecting luns no longer needed. On the other hand, you could
>> do all of the above: log in on pool create, then also check if you're
>> logged in on other commands and log in if you've lost connection.
>>
>> With Xen, what does your registered pool   show in the UI for avail/used
>> capacity, and how does it get that info? I assume there is some sort of
>> disk pool that the luns are carved from, and that your plugin is called to
>> talk to the SAN and expose to the user how much of that pool has been
>> allocated. Knowing how you already solves these problems with Xen will
>> help
>> figure out what to do with KVM.
>>
>> If this is the case, I think the plugin can continue to handle it rather
>> than getting details from the agent. I'm not sure if that means nulls are
>> OK for these on the agent side or what, I need to look at the storage
>> plugin arch more closely.
>> On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
>> wrote:
>>
>> > Hey Marcus,
>> >
>> > I'm reviewing your e-mails as I implement the necessary methods in new
>> > classes.
>> >
>> > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
>> > the pool data (host, port, name, path) which would be used to log the
>> > host into the initiator."
>> >
>> > Can you tell me, in my case, since a storage pool (primary storage) is
>> > actually the SAN, I wouldn't really be logging into anything at this
>> point,
>> > correct?
>> >
>> > Also, what kind of capacity, available, and used bytes make sense to
>> report
>> > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
>> and
>> > not an individual LUN)?
>> >
>> > Thanks!
>> >
>> >
>> > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen > > >wrote:
>> >
>> > > Ok, KVM will be close to that, of course, because only the hypervisor
>> > > classes differ, the rest is all mgmt server. Creating a volume is just
>> > > a db entry until it's deployed for the first time. AttachVolumeCommand
>> > > on the agent side (LibvirtStorageAdaptor.java is analogous to
>> > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
>> > > StorageAdaptor) to log in the host to the target and then you have a
>> > > block device.  Maybe libvirt will do that for you, but my quick read
>> > > made it sound like the iscsi libvirt pool type is actually a pool, not
>> > > a lun or volume, so you'll need to figure out if that works or if
>> > > you'll have to use iscsiadm commands.
>> > >
>> > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
>> > > doesn't really manage your pool the way you want), you're going to
>> > > have to create a version of KVMStoragePool class and a StorageAdaptor
>> > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
>> > > implementing all of the methods, then in KVMStorageManager.java
>> > > there's a "_storageMapper" map. This is used to select the correct
>> > > adapt

Re: Managed storage with KVM

[Mike Tutkowski]

Plus, when you log in to a LUN, you need the CHAP info and this info is
required for each LUN (as opposed to being for the SAN).

This is how my createStoragePool currently looks, so I think we're on the
same page.


public KVMStoragePool createStoragePool(String name, String host, int port,
String path, String userInfo, StoragePoolType type) {

iScsiAdmStoragePool storagePool = new iScsiAdmStoragePool(name,
host, port, this);

_mapUuidToAdaptor.put(name, storagePool);

return storagePool;

}


On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen wrote:

> What do you do with Xen? I imagine the user enter the SAN details when
> registering the pool? A the pool details are basically just instructions on
> how to log into a target, correct?
>
> You can choose to log in a KVM host to the target during createStoragePool
> and save the pool in a map, or just save the pool info in a map for future
> reference by uuid, for when you do need to log in. The createStoragePool
> then just becomes a way to save the pool info to the agent. Personally, I'd
> log in on the pool create and look/scan for specific luns when they're
> needed, but I haven't thought it through thoroughly. I just say that mainly
> because login only happens once, the first time the pool is used, and every
> other storage command is about discovering new luns or maybe
> deleting/disconnecting luns no longer needed. On the other hand, you could
> do all of the above: log in on pool create, then also check if you're
> logged in on other commands and log in if you've lost connection.
>
> With Xen, what does your registered pool   show in the UI for avail/used
> capacity, and how does it get that info? I assume there is some sort of
> disk pool that the luns are carved from, and that your plugin is called to
> talk to the SAN and expose to the user how much of that pool has been
> allocated. Knowing how you already solves these problems with Xen will help
> figure out what to do with KVM.
>
> If this is the case, I think the plugin can continue to handle it rather
> than getting details from the agent. I'm not sure if that means nulls are
> OK for these on the agent side or what, I need to look at the storage
> plugin arch more closely.
> On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
> wrote:
>
> > Hey Marcus,
> >
> > I'm reviewing your e-mails as I implement the necessary methods in new
> > classes.
> >
> > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > the pool data (host, port, name, path) which would be used to log the
> > host into the initiator."
> >
> > Can you tell me, in my case, since a storage pool (primary storage) is
> > actually the SAN, I wouldn't really be logging into anything at this
> point,
> > correct?
> >
> > Also, what kind of capacity, available, and used bytes make sense to
> report
> > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
> and
> > not an individual LUN)?
> >
> > Thanks!
> >
> >
> > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen  > >wrote:
> >
> > > Ok, KVM will be close to that, of course, because only the hypervisor
> > > classes differ, the rest is all mgmt server. Creating a volume is just
> > > a db entry until it's deployed for the first time. AttachVolumeCommand
> > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > StorageAdaptor) to log in the host to the target and then you have a
> > > block device.  Maybe libvirt will do that for you, but my quick read
> > > made it sound like the iscsi libvirt pool type is actually a pool, not
> > > a lun or volume, so you'll need to figure out if that works or if
> > > you'll have to use iscsiadm commands.
> > >
> > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > > doesn't really manage your pool the way you want), you're going to
> > > have to create a version of KVMStoragePool class and a StorageAdaptor
> > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> > > implementing all of the methods, then in KVMStorageManager.java
> > > there's a "_storageMapper" map. This is used to select the correct
> > > adaptor, you can see in this file that every call first pulls the
> > > correct adaptor out of this map via getStorageAdaptor. So you can see
> > > a comment in this file that says "add other storage adaptors here",
> > > where it puts to this map, this is where you'd register your adaptor.
> > >
> > > So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > > the pool data (host, port, name, path) which would be used to log the
> > > host into the initiator. I *believe* the method getPhysicalDisk will
> > > need to do the work of attaching the lun.  AttachVolumeCommand calls
> > > this and then creates the XML diskdef and attaches it to the VM. Now,
> > > one thing you need to know is that createStoragePool is called often,
> > > sometimes jus

Re: Managed storage with KVM

[Mike Tutkowski]

Woops...I named this incorrectly:

_mapUuidToAdaptor.put(name, storagePool);

should be

_mapUuidToPool.put(name, storagePool);


On Tue, Sep 17, 2013 at 8:01 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Plus, when you log in to a LUN, you need the CHAP info and this info is
> required for each LUN (as opposed to being for the SAN).
>
> This is how my createStoragePool currently looks, so I think we're on the
> same page.
>
>
> public KVMStoragePool createStoragePool(String name, String host, intport, 
> String path, String userInfo, StoragePoolType type) {
>
> iScsiAdmStoragePool storagePool = new iScsiAdmStoragePool(name,
> host, port, this);
>
> _mapUuidToAdaptor.put(name, storagePool);
>
> return storagePool;
>
> }
>
>
> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen wrote:
>
>> What do you do with Xen? I imagine the user enter the SAN details when
>> registering the pool? A the pool details are basically just instructions
>> on
>> how to log into a target, correct?
>>
>> You can choose to log in a KVM host to the target during createStoragePool
>> and save the pool in a map, or just save the pool info in a map for future
>> reference by uuid, for when you do need to log in. The createStoragePool
>> then just becomes a way to save the pool info to the agent. Personally,
>> I'd
>> log in on the pool create and look/scan for specific luns when they're
>> needed, but I haven't thought it through thoroughly. I just say that
>> mainly
>> because login only happens once, the first time the pool is used, and
>> every
>> other storage command is about discovering new luns or maybe
>> deleting/disconnecting luns no longer needed. On the other hand, you could
>> do all of the above: log in on pool create, then also check if you're
>> logged in on other commands and log in if you've lost connection.
>>
>> With Xen, what does your registered pool   show in the UI for avail/used
>> capacity, and how does it get that info? I assume there is some sort of
>> disk pool that the luns are carved from, and that your plugin is called to
>> talk to the SAN and expose to the user how much of that pool has been
>> allocated. Knowing how you already solves these problems with Xen will
>> help
>> figure out what to do with KVM.
>>
>> If this is the case, I think the plugin can continue to handle it rather
>> than getting details from the agent. I'm not sure if that means nulls are
>> OK for these on the agent side or what, I need to look at the storage
>> plugin arch more closely.
>> On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
>> wrote:
>>
>> > Hey Marcus,
>> >
>> > I'm reviewing your e-mails as I implement the necessary methods in new
>> > classes.
>> >
>> > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
>> > the pool data (host, port, name, path) which would be used to log the
>> > host into the initiator."
>> >
>> > Can you tell me, in my case, since a storage pool (primary storage) is
>> > actually the SAN, I wouldn't really be logging into anything at this
>> point,
>> > correct?
>> >
>> > Also, what kind of capacity, available, and used bytes make sense to
>> report
>> > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
>> and
>> > not an individual LUN)?
>> >
>> > Thanks!
>> >
>> >
>> > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen > > >wrote:
>> >
>> > > Ok, KVM will be close to that, of course, because only the hypervisor
>> > > classes differ, the rest is all mgmt server. Creating a volume is just
>> > > a db entry until it's deployed for the first time. AttachVolumeCommand
>> > > on the agent side (LibvirtStorageAdaptor.java is analogous to
>> > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
>> > > StorageAdaptor) to log in the host to the target and then you have a
>> > > block device.  Maybe libvirt will do that for you, but my quick read
>> > > made it sound like the iscsi libvirt pool type is actually a pool, not
>> > > a lun or volume, so you'll need to figure out if that works or if
>> > > you'll have to use iscsiadm commands.
>> > >
>> > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
>> > > doesn't really manage your pool the way you want), you're going to
>> > > have to create a version of KVMStoragePool class and a StorageAdaptor
>> > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
>> > > implementing all of the methods, then in KVMStorageManager.java
>> > > there's a "_storageMapper" map. This is used to select the correct
>> > > adaptor, you can see in this file that every call first pulls the
>> > > correct adaptor out of this map via getStorageAdaptor. So you can see
>> > > a comment in this file that says "add other storage adaptors here",
>> > > where it puts to this map, this is where you'd register your adaptor.
>> > >
>> > > So, referencing StorageAdaptor.java, createStoragePool accepts all of
>> > > the pool data (host, port, name, path) which

conflicting dependencies between CloudStack and Whirr

[Han,Meng]

Dear all,

I am adding an API to CloudStack which utilizes Whirr to launch various 
clusters on CloudStack. Now I am facing a dependency conflicting issue.


Whirr 0.8.2 requires gson 2.2.2 while CloudStack API requires gson 
1.7.1. If I use gson 1.7.1 for Whirr, the following error will happen:


com.google.common.util.concurrent.ExecutionError: 
java.lang.NoClassDefFoundError: com/google/gson/TypeAdapter


This TypeAdapter class can be found inside gson-2.2.2.jar. However if I 
modify CloudStack to use gson 2.2.2 CloudStack will not build 
successfully due to the following test error.


[INFO] Building Apache CloudStack Core 4.1.1
[INFO] 


[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ cloud-core 
---

[INFO] Deleting /home/meng/cloudstack/core/target
[INFO]
[INFO] --- maven-remote-resources-plugin:1.3:process (default) @ 
cloud-core ---

[INFO]
[INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ 
cloud-core ---

[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/home/meng/cloudstack/core/src/main/resources

[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ 
cloud-core ---
[INFO] Compiling 156 source files to 
/home/meng/cloudstack/core/target/classes

[INFO]
[INFO] --- maven-resources-plugin:2.5:testResources 
(default-testResources) @ cloud-core ---

[debug] execute contextualize
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/home/meng/cloudstack/core/src/test/resources

[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile 
(default-testCompile) @ cloud-core ---
[INFO] Compiling 1 source file to 
/home/meng/cloudstack/core/target/test-classes

[INFO]
[INFO] --- maven-surefire-plugin:2.12:test (default-test) @ cloud-core 
---
[INFO] Surefire report directory: 
/home/meng/cloudstack/core/target/surefire-reports


---
 T E S T S
---
Running com.cloud.agent.transport.RequestTest
log4j:WARN No appenders could be found for logger 
(com.cloud.agent.transport.RequestTest).

log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig 
for more info.
Tests run: 4, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 3.054 
sec <<< FAILURE!


Results :

Failed tests:   testSerDeser(com.cloud.agent.transport.RequestTest)

Tests in error:
  testLogging(com.cloud.agent.transport.RequestTest)

Tests run: 4, Failures: 1, Errors: 1, Skipped: 0


I ran "mvn -P developer clean install -DskipTests" , the CloudStack 
building process went through. But when I issued an API --launchCluster 
to CloudStack, the following error related to gson popped up on 
CloudStack Management Server:


ERROR [agent.transport.Request] (AgentManager-Handler-2:) Caught 
problem with 
[{"StartupRoutingCommand":{"cpus":2,"speed":3000,"memory":3844370432,"dom0MinMemory":384437043,"poolSync":false,"vms":{"v-2-VM":{"state":"Running"},"s-1-VM":{"state":"Running"},"r-4-VM":{"state":"Running"}},"caps":"hvm,snapshot","pool":"/root","hypervisorType":"KVM","hostDetails":{"com.cloud.network.Networks.RouterPrivateIpStrategy":"HostLocal","Host.OS":"CentOS","Host.OS.Kernel.Version":"2.6.32-358.el6.x86_64","Host.OS.Version":"6.4"},"type":"Routing","dataCenter":"1","pod":"1","cluster":"1","guid":"a7320748-6346-3c9a-975e-90ac4ae4a986-LibvirtComputingResource","name":"meng.acis.ufl.edu","id":2,"version":"4.1.1","publicIpAddress":"10.244.18.55","publicNetmask":"255.0.0.0","publicMacAddress":"00:23:ae:94:f7:22","privateIpAddress":"10.244.18.55","privateMacAddress":"00:23:ae:94:f7:22","privateNetmask":"255.0.0.0","storageIpAddress":"10.244.18.55","storageNetmask":"255.0.0.0","storageMacAddress":"00:23:ae:94:f7:22","resourceName":"LibvirtComputingResource","gatewayIpAddress":"!

10.244.18
.1","contextMap":{},"wait":0}},{"StartupStorageCommand":{"totalSize":0,"poolInfo":{"uuid":"9447c0b1-cc3f-439f-85f6-13d35539a9ed","host":"10.244.18.55","localPath":"/var/lib/libvirt/images/","hostPath":"/var/lib/libvirt/images/","poolType":"Filesystem","capacityBytes":52844687360,"availableBytes":41535332352},"resourceType":"STORAGE_POOL","hostDetails":{},"type":"Storage","dataCenter":"1","pod":"1","guid":"a7320748-6346-3c9a-975e-90ac4ae4a986-LibvirtComputingResource","name":"meng.acis.ufl.edu","id":2,"version":"4.1.1","resourceName":"LibvirtComputingResource","contextMap":{},"wait":0}}]
java.lang.ClassCastException: 
com.google.gson.internal.$Gson$Types$GenericArrayTypeImpl cannot be cast 
to java.lang.Class
	at 
com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:84)
	at 
com.cloud.agent.transport.ArrayTypeAdaptor.deserialize(ArrayTypeAdaptor.java:37)

at com.go

Re: Managed storage with KVM

[Marcus Sorensen]

OK, if you log in per lun, then just saving the info for future reference
is fine.

Does CS provide storage stats at all, then, for other platforms?
On Sep 17, 2013 8:01 PM, "Mike Tutkowski" 
wrote:

> Plus, when you log in to a LUN, you need the CHAP info and this info is
> required for each LUN (as opposed to being for the SAN).
>
> This is how my createStoragePool currently looks, so I think we're on the
> same page.
>
>
> public KVMStoragePool createStoragePool(String name, String host, int port,
> String path, String userInfo, StoragePoolType type) {
>
> iScsiAdmStoragePool storagePool = new iScsiAdmStoragePool(name,
> host, port, this);
>
> _mapUuidToAdaptor.put(name, storagePool);
>
> return storagePool;
>
> }
>
>
> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen  >wrote:
>
> > What do you do with Xen? I imagine the user enter the SAN details when
> > registering the pool? A the pool details are basically just instructions
> on
> > how to log into a target, correct?
> >
> > You can choose to log in a KVM host to the target during
> createStoragePool
> > and save the pool in a map, or just save the pool info in a map for
> future
> > reference by uuid, for when you do need to log in. The createStoragePool
> > then just becomes a way to save the pool info to the agent. Personally,
> I'd
> > log in on the pool create and look/scan for specific luns when they're
> > needed, but I haven't thought it through thoroughly. I just say that
> mainly
> > because login only happens once, the first time the pool is used, and
> every
> > other storage command is about discovering new luns or maybe
> > deleting/disconnecting luns no longer needed. On the other hand, you
> could
> > do all of the above: log in on pool create, then also check if you're
> > logged in on other commands and log in if you've lost connection.
> >
> > With Xen, what does your registered pool   show in the UI for avail/used
> > capacity, and how does it get that info? I assume there is some sort of
> > disk pool that the luns are carved from, and that your plugin is called
> to
> > talk to the SAN and expose to the user how much of that pool has been
> > allocated. Knowing how you already solves these problems with Xen will
> help
> > figure out what to do with KVM.
> >
> > If this is the case, I think the plugin can continue to handle it rather
> > than getting details from the agent. I'm not sure if that means nulls are
> > OK for these on the agent side or what, I need to look at the storage
> > plugin arch more closely.
> > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
> > wrote:
> >
> > > Hey Marcus,
> > >
> > > I'm reviewing your e-mails as I implement the necessary methods in new
> > > classes.
> > >
> > > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > > the pool data (host, port, name, path) which would be used to log the
> > > host into the initiator."
> > >
> > > Can you tell me, in my case, since a storage pool (primary storage) is
> > > actually the SAN, I wouldn't really be logging into anything at this
> > point,
> > > correct?
> > >
> > > Also, what kind of capacity, available, and used bytes make sense to
> > report
> > > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
> > and
> > > not an individual LUN)?
> > >
> > > Thanks!
> > >
> > >
> > > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen  > > >wrote:
> > >
> > > > Ok, KVM will be close to that, of course, because only the hypervisor
> > > > classes differ, the rest is all mgmt server. Creating a volume is
> just
> > > > a db entry until it's deployed for the first time.
> AttachVolumeCommand
> > > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > > StorageAdaptor) to log in the host to the target and then you have a
> > > > block device.  Maybe libvirt will do that for you, but my quick read
> > > > made it sound like the iscsi libvirt pool type is actually a pool,
> not
> > > > a lun or volume, so you'll need to figure out if that works or if
> > > > you'll have to use iscsiadm commands.
> > > >
> > > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > > > doesn't really manage your pool the way you want), you're going to
> > > > have to create a version of KVMStoragePool class and a StorageAdaptor
> > > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> > > > implementing all of the methods, then in KVMStorageManager.java
> > > > there's a "_storageMapper" map. This is used to select the correct
> > > > adaptor, you can see in this file that every call first pulls the
> > > > correct adaptor out of this map via getStorageAdaptor. So you can see
> > > > a comment in this file that says "add other storage adaptors here",
> > > > where it puts to this map, this is where you'd register your adaptor.
> > > >
> > > > So, referencing StorageAdaptor.java, createStorag

Re: Managed storage with KVM

[Marcus Sorensen]

You respond to more than attach and detach, right? Don't you create luns as
well? Or are you just referring to the hypervisor stuff?
On Sep 17, 2013 7:51 PM, "Mike Tutkowski" 
wrote:

> Hi Marcus,
>
> I never need to respond to a CreateStoragePool call for either XenServer or
> VMware.
>
> What happens is I respond only to the Attach- and Detach-volume commands.
>
> Let's say an attach comes in:
>
> In this case, I check to see if the storage is "managed." Talking XenServer
> here, if it is, I log in to the LUN that is the disk we want to attach.
> After, if this is the first time attaching this disk, I create an SR and a
> VDI within the SR. If it is not the first time attaching this disk, the LUN
> already has the SR and VDI on it.
>
> Once this is done, I let the normal "attach" logic run because this logic
> expected an SR and a VDI and now it has it.
>
> It's the same thing for VMware: Just substitute datastore for SR and VMDK
> for VDI.
>
> Does that make sense?
>
> Thanks!
>
>
> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen  >wrote:
>
> > What do you do with Xen? I imagine the user enter the SAN details when
> > registering the pool? A the pool details are basically just instructions
> on
> > how to log into a target, correct?
> >
> > You can choose to log in a KVM host to the target during
> createStoragePool
> > and save the pool in a map, or just save the pool info in a map for
> future
> > reference by uuid, for when you do need to log in. The createStoragePool
> > then just becomes a way to save the pool info to the agent. Personally,
> I'd
> > log in on the pool create and look/scan for specific luns when they're
> > needed, but I haven't thought it through thoroughly. I just say that
> mainly
> > because login only happens once, the first time the pool is used, and
> every
> > other storage command is about discovering new luns or maybe
> > deleting/disconnecting luns no longer needed. On the other hand, you
> could
> > do all of the above: log in on pool create, then also check if you're
> > logged in on other commands and log in if you've lost connection.
> >
> > With Xen, what does your registered pool   show in the UI for avail/used
> > capacity, and how does it get that info? I assume there is some sort of
> > disk pool that the luns are carved from, and that your plugin is called
> to
> > talk to the SAN and expose to the user how much of that pool has been
> > allocated. Knowing how you already solves these problems with Xen will
> help
> > figure out what to do with KVM.
> >
> > If this is the case, I think the plugin can continue to handle it rather
> > than getting details from the agent. I'm not sure if that means nulls are
> > OK for these on the agent side or what, I need to look at the storage
> > plugin arch more closely.
> > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
> > wrote:
> >
> > > Hey Marcus,
> > >
> > > I'm reviewing your e-mails as I implement the necessary methods in new
> > > classes.
> > >
> > > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
> > > the pool data (host, port, name, path) which would be used to log the
> > > host into the initiator."
> > >
> > > Can you tell me, in my case, since a storage pool (primary storage) is
> > > actually the SAN, I wouldn't really be logging into anything at this
> > point,
> > > correct?
> > >
> > > Also, what kind of capacity, available, and used bytes make sense to
> > report
> > > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
> > and
> > > not an individual LUN)?
> > >
> > > Thanks!
> > >
> > >
> > > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen  > > >wrote:
> > >
> > > > Ok, KVM will be close to that, of course, because only the hypervisor
> > > > classes differ, the rest is all mgmt server. Creating a volume is
> just
> > > > a db entry until it's deployed for the first time.
> AttachVolumeCommand
> > > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > > StorageAdaptor) to log in the host to the target and then you have a
> > > > block device.  Maybe libvirt will do that for you, but my quick read
> > > > made it sound like the iscsi libvirt pool type is actually a pool,
> not
> > > > a lun or volume, so you'll need to figure out if that works or if
> > > > you'll have to use iscsiadm commands.
> > > >
> > > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > > > doesn't really manage your pool the way you want), you're going to
> > > > have to create a version of KVMStoragePool class and a StorageAdaptor
> > > > class (see LibvirtStoragePool.java and LibvirtStorageAdaptor.java),
> > > > implementing all of the methods, then in KVMStorageManager.java
> > > > there's a "_storageMapper" map. This is used to select the correct
> > > > adaptor, you can see in this file that every call first pulls the
> > > > correct adaptor out of this map via getStorageAda

Re: Review Request 14079: marvin tests for IP Address reservation within a network

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14079/#review26201
---

Ship it!


master, 4.2-forward

- Prasanna Santhanam


On Sept. 16, 2013, 11:11 a.m., Anshul Gangwar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/14079/
> ---
> 
> (Updated Sept. 16, 2013, 11:11 a.m.)
> 
> 
> Review request for cloudstack, Saksham Srivastava and Prasanna Santhanam.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-2266
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> marvin tests for IP Address reservation within a network 
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_ip_reservation.py PRE-CREATION 
>   tools/marvin/marvin/integration/lib/base.py fa4cc82 
> 
> Diff: https://reviews.apache.org/r/14079/diff/
> 
> 
> Testing
> ---
> 
> ran successfully on  my local advanced zone setup
> 
> 
> Thanks,
> 
> Anshul Gangwar
> 
>

Re: Review Request 12545: CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12545/#review26202
---


Commit dd4b1cbf449b63cd2b34197c7c9b19ec1cc4822e in branch refs/heads/master 
from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=dd4b1cb ]

CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM

Reviewed-By: Harikrishna Patnala 
Signed-off-by: Prasanna Santhanam 


- ASF Subversion and Git Services


On Sept. 14, 2013, 7:46 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12545/
> ---
> 
> (Updated Sept. 14, 2013, 7:46 a.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-1800
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12545/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 12545: CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12545/#review26203
---


Commit 8cc37c24705dfcc2ddb973673522fa39bf246ef5 in branch 
refs/heads/4.2-forward from Girish Shilamkar
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=8cc37c2 ]

CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM

Reviewed-By: Harikrishna Patnala 
Signed-off-by: Prasanna Santhanam 
(cherry picked from commit dd4b1cbf449b63cd2b34197c7c9b19ec1cc4822e)


- ASF Subversion and Git Services


On Sept. 14, 2013, 7:46 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12545/
> ---
> 
> (Updated Sept. 14, 2013, 7:46 a.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-1800
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12545/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Managed storage with KVM

[Mike Tutkowski]

I think the way people bill for this kind of storage is simply by seeing
how many volumes are in use for a given CS account and tracing a volume
back to the Disk Offering it was created from, which contains info about
guaranteed IOPS.

I am not aware of what stats may be collected for this for XenServer and
VMware.


On Tue, Sep 17, 2013 at 8:17 PM, Marcus Sorensen wrote:

> OK, if you log in per lun, then just saving the info for future reference
> is fine.
>
> Does CS provide storage stats at all, then, for other platforms?
> On Sep 17, 2013 8:01 PM, "Mike Tutkowski" 
> wrote:
>
> > Plus, when you log in to a LUN, you need the CHAP info and this info is
> > required for each LUN (as opposed to being for the SAN).
> >
> > This is how my createStoragePool currently looks, so I think we're on the
> > same page.
> >
> >
> > public KVMStoragePool createStoragePool(String name, String host, int
> port,
> > String path, String userInfo, StoragePoolType type) {
> >
> > iScsiAdmStoragePool storagePool = new iScsiAdmStoragePool(name,
> > host, port, this);
> >
> > _mapUuidToAdaptor.put(name, storagePool);
> >
> > return storagePool;
> >
> > }
> >
> >
> > On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen  > >wrote:
> >
> > > What do you do with Xen? I imagine the user enter the SAN details when
> > > registering the pool? A the pool details are basically just
> instructions
> > on
> > > how to log into a target, correct?
> > >
> > > You can choose to log in a KVM host to the target during
> > createStoragePool
> > > and save the pool in a map, or just save the pool info in a map for
> > future
> > > reference by uuid, for when you do need to log in. The
> createStoragePool
> > > then just becomes a way to save the pool info to the agent. Personally,
> > I'd
> > > log in on the pool create and look/scan for specific luns when they're
> > > needed, but I haven't thought it through thoroughly. I just say that
> > mainly
> > > because login only happens once, the first time the pool is used, and
> > every
> > > other storage command is about discovering new luns or maybe
> > > deleting/disconnecting luns no longer needed. On the other hand, you
> > could
> > > do all of the above: log in on pool create, then also check if you're
> > > logged in on other commands and log in if you've lost connection.
> > >
> > > With Xen, what does your registered pool   show in the UI for
> avail/used
> > > capacity, and how does it get that info? I assume there is some sort of
> > > disk pool that the luns are carved from, and that your plugin is called
> > to
> > > talk to the SAN and expose to the user how much of that pool has been
> > > allocated. Knowing how you already solves these problems with Xen will
> > help
> > > figure out what to do with KVM.
> > >
> > > If this is the case, I think the plugin can continue to handle it
> rather
> > > than getting details from the agent. I'm not sure if that means nulls
> are
> > > OK for these on the agent side or what, I need to look at the storage
> > > plugin arch more closely.
> > > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" <
> mike.tutkow...@solidfire.com>
> > > wrote:
> > >
> > > > Hey Marcus,
> > > >
> > > > I'm reviewing your e-mails as I implement the necessary methods in
> new
> > > > classes.
> > > >
> > > > "So, referencing StorageAdaptor.java, createStoragePool accepts all
> of
> > > > the pool data (host, port, name, path) which would be used to log the
> > > > host into the initiator."
> > > >
> > > > Can you tell me, in my case, since a storage pool (primary storage)
> is
> > > > actually the SAN, I wouldn't really be logging into anything at this
> > > point,
> > > > correct?
> > > >
> > > > Also, what kind of capacity, available, and used bytes make sense to
> > > report
> > > > for KVMStoragePool (since KVMStoragePool represents the SAN in my
> case
> > > and
> > > > not an individual LUN)?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen <
> shadow...@gmail.com
> > > > >wrote:
> > > >
> > > > > Ok, KVM will be close to that, of course, because only the
> hypervisor
> > > > > classes differ, the rest is all mgmt server. Creating a volume is
> > just
> > > > > a db entry until it's deployed for the first time.
> > AttachVolumeCommand
> > > > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > > > StorageAdaptor) to log in the host to the target and then you have
> a
> > > > > block device.  Maybe libvirt will do that for you, but my quick
> read
> > > > > made it sound like the iscsi libvirt pool type is actually a pool,
> > not
> > > > > a lun or volume, so you'll need to figure out if that works or if
> > > > > you'll have to use iscsiadm commands.
> > > > >
> > > > > If you're NOT going to use LibvirtStorageAdaptor (because Libvirt
> > > > > doesn't really manage your pool the way you want), you're g

Re: Review Request 12545: CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12545/#review26204
---

Ship it!


master, 4.2-forward

- Prasanna Santhanam


On Sept. 14, 2013, 7:46 a.m., Girish Shilamkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12545/
> ---
> 
> (Updated Sept. 14, 2013, 7:46 a.m.)
> 
> 
> Review request for cloudstack, Harikrishna Patnala and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-1800
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-1800: Add automation tests for reset sshkey to access VM
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_reset_ssh_keypair.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/12545/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Girish Shilamkar
> 
>

Re: Review Request 13682: CLOUDSTACK-2247: Automation - Reset a VM on Reboot

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13682/#review26205
---


Commit 62b0ad03c871c7100433b39593a82d393879124e in branch refs/heads/master 
from Harikrishna Patnala
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=62b0ad0 ]

CLOUDSTACK-2247: Automation - Reset a VM on Reboot

Signed-off-by: Prasanna Santhanam 


- ASF Subversion and Git Services


On Aug. 20, 2013, 11:04 a.m., Harikrishna Patnala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13682/
> ---
> 
> (Updated Aug. 20, 2013, 11:04 a.m.)
> 
> 
> Review request for cloudstack and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-2247
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-2247: Automation - Reset a VM on Reboot
> 
> Created a vm with service offering (marked as volatile) and rebooted it.
> Checking if the volumes are different before and after reboot.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_reset_vm_on_reboot.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/13682/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Harikrishna Patnala
> 
>

Re: Managed storage with KVM

[Mike Tutkowski]

Yeah, I should have clarified what I was referring to.

As you mentioned in your last sentence, I was just talking about on the
hypervisor side (responding to attach and detach commands).

On the storage side, the storage framework invokes my plug-in when it needs
a volume created, deleted, etc. By the time we get to the hypervisor side
to attach the volume, it already exists on the SAN.

As an example, if you go to attach a volume to a VM for the first time, the
storage framework will ask my plug-in to create a volume on its SAN.


On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen wrote:

> You respond to more than attach and detach, right? Don't you create luns as
> well? Or are you just referring to the hypervisor stuff?
> On Sep 17, 2013 7:51 PM, "Mike Tutkowski" 
> wrote:
>
> > Hi Marcus,
> >
> > I never need to respond to a CreateStoragePool call for either XenServer
> or
> > VMware.
> >
> > What happens is I respond only to the Attach- and Detach-volume commands.
> >
> > Let's say an attach comes in:
> >
> > In this case, I check to see if the storage is "managed." Talking
> XenServer
> > here, if it is, I log in to the LUN that is the disk we want to attach.
> > After, if this is the first time attaching this disk, I create an SR and
> a
> > VDI within the SR. If it is not the first time attaching this disk, the
> LUN
> > already has the SR and VDI on it.
> >
> > Once this is done, I let the normal "attach" logic run because this logic
> > expected an SR and a VDI and now it has it.
> >
> > It's the same thing for VMware: Just substitute datastore for SR and VMDK
> > for VDI.
> >
> > Does that make sense?
> >
> > Thanks!
> >
> >
> > On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen  > >wrote:
> >
> > > What do you do with Xen? I imagine the user enter the SAN details when
> > > registering the pool? A the pool details are basically just
> instructions
> > on
> > > how to log into a target, correct?
> > >
> > > You can choose to log in a KVM host to the target during
> > createStoragePool
> > > and save the pool in a map, or just save the pool info in a map for
> > future
> > > reference by uuid, for when you do need to log in. The
> createStoragePool
> > > then just becomes a way to save the pool info to the agent. Personally,
> > I'd
> > > log in on the pool create and look/scan for specific luns when they're
> > > needed, but I haven't thought it through thoroughly. I just say that
> > mainly
> > > because login only happens once, the first time the pool is used, and
> > every
> > > other storage command is about discovering new luns or maybe
> > > deleting/disconnecting luns no longer needed. On the other hand, you
> > could
> > > do all of the above: log in on pool create, then also check if you're
> > > logged in on other commands and log in if you've lost connection.
> > >
> > > With Xen, what does your registered pool   show in the UI for
> avail/used
> > > capacity, and how does it get that info? I assume there is some sort of
> > > disk pool that the luns are carved from, and that your plugin is called
> > to
> > > talk to the SAN and expose to the user how much of that pool has been
> > > allocated. Knowing how you already solves these problems with Xen will
> > help
> > > figure out what to do with KVM.
> > >
> > > If this is the case, I think the plugin can continue to handle it
> rather
> > > than getting details from the agent. I'm not sure if that means nulls
> are
> > > OK for these on the agent side or what, I need to look at the storage
> > > plugin arch more closely.
> > > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" <
> mike.tutkow...@solidfire.com>
> > > wrote:
> > >
> > > > Hey Marcus,
> > > >
> > > > I'm reviewing your e-mails as I implement the necessary methods in
> new
> > > > classes.
> > > >
> > > > "So, referencing StorageAdaptor.java, createStoragePool accepts all
> of
> > > > the pool data (host, port, name, path) which would be used to log the
> > > > host into the initiator."
> > > >
> > > > Can you tell me, in my case, since a storage pool (primary storage)
> is
> > > > actually the SAN, I wouldn't really be logging into anything at this
> > > point,
> > > > correct?
> > > >
> > > > Also, what kind of capacity, available, and used bytes make sense to
> > > report
> > > > for KVMStoragePool (since KVMStoragePool represents the SAN in my
> case
> > > and
> > > > not an individual LUN)?
> > > >
> > > > Thanks!
> > > >
> > > >
> > > > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen <
> shadow...@gmail.com
> > > > >wrote:
> > > >
> > > > > Ok, KVM will be close to that, of course, because only the
> hypervisor
> > > > > classes differ, the rest is all mgmt server. Creating a volume is
> > just
> > > > > a db entry until it's deployed for the first time.
> > AttachVolumeCommand
> > > > > on the agent side (LibvirtStorageAdaptor.java is analogous to
> > > > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
> > > > > StorageAdaptor) to log in the host to the

Re: Review Request 13682: CLOUDSTACK-2247: Automation - Reset a VM on Reboot

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13682/#review26206
---


Commit a9218f341186dcc5914dc23b283636652f83416a in branch 
refs/heads/4.2-forward from Harikrishna Patnala
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a9218f3 ]

CLOUDSTACK-2247: Automation - Reset a VM on Reboot

Signed-off-by: Prasanna Santhanam 
(cherry picked from commit 62b0ad03c871c7100433b39593a82d393879124e)


- ASF Subversion and Git Services


On Aug. 20, 2013, 11:04 a.m., Harikrishna Patnala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13682/
> ---
> 
> (Updated Aug. 20, 2013, 11:04 a.m.)
> 
> 
> Review request for cloudstack and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-2247
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-2247: Automation - Reset a VM on Reboot
> 
> Created a vm with service offering (marked as volatile) and rebooted it.
> Checking if the volumes are different before and after reboot.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_reset_vm_on_reboot.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/13682/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Harikrishna Patnala
> 
>

Re: Review Request 13682: CLOUDSTACK-2247: Automation - Reset a VM on Reboot

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13682/#review26207
---

Ship it!


master, 4.2-forward

- Prasanna Santhanam


On Aug. 20, 2013, 11:04 a.m., Harikrishna Patnala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13682/
> ---
> 
> (Updated Aug. 20, 2013, 11:04 a.m.)
> 
> 
> Review request for cloudstack and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-2247
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> CLOUDSTACK-2247: Automation - Reset a VM on Reboot
> 
> Created a vm with service offering (marked as volatile) and rebooted it.
> Checking if the volumes are different before and after reboot.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_reset_vm_on_reboot.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/13682/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Harikrishna Patnala
> 
>

Re: Review Request 13991: Made misc changes in test_multiple_ip_ranges.py

[Prasanna Santhanam]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13991/#review26208
---


patch fails to apply on master

- Prasanna Santhanam


On Sept. 5, 2013, 10:12 a.m., sanjeev n wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/13991/
> ---
> 
> (Updated Sept. 5, 2013, 10:12 a.m.)
> 
> 
> Review request for cloudstack, venkata swamy babu  budumuru, SrikanteswaraRao 
> Talluri, and Prasanna Santhanam.
> 
> 
> Bugs: CLOUDSTACK-702
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> This patch is a combination of two commits. In first commit
> 1. Removed advanced tag from all the tests. At the movement scripts
>  support only basic zone
> 2. Added a test to deploy guest vm after adding ip range in new CIDR
> In second commit:
> Removed test for deploying vm after adding new cidr
> 1.This test would block parallel execution of remianing tests
> 2.Removed list_Routers proc and unused code
> 3.Added few more debug statements
> 
> 
> Diffs
> -
> 
>   test/integration/component/test_multiple_ip_ranges.py 18409c5 
> 
> Diff: https://reviews.apache.org/r/13991/diff/
> 
> 
> Testing
> ---
> 
> Yes.
> 
> 
> Thanks,
> 
> sanjeev n
> 
>

Re: Managed storage with KVM

[Marcus Sorensen]

Ok, on most storage pools it shows how many GB free/used when listing
the pool both via API and in the UI. I'm guessing those are empty then
for the solid fire storage, but it seems like the user should have to
define some sort of pool that the luns get carved out of, and you
should be able to get the stats for that, right? Or is a solid fire
appliance only one pool per appliance? This isn't about billing, but
just so cloudstack itself knows whether or not there is space left on
the storage device, so cloudstack can go on allocating from a
different primary storage as this one fills up. There are also
notifications and things. It seems like there should be a call you can
handle for this, maybe Edison knows.

On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen  wrote:
> You respond to more than attach and detach, right? Don't you create luns as
> well? Or are you just referring to the hypervisor stuff?
>
> On Sep 17, 2013 7:51 PM, "Mike Tutkowski" 
> wrote:
>>
>> Hi Marcus,
>>
>> I never need to respond to a CreateStoragePool call for either XenServer
>> or
>> VMware.
>>
>> What happens is I respond only to the Attach- and Detach-volume commands.
>>
>> Let's say an attach comes in:
>>
>> In this case, I check to see if the storage is "managed." Talking
>> XenServer
>> here, if it is, I log in to the LUN that is the disk we want to attach.
>> After, if this is the first time attaching this disk, I create an SR and a
>> VDI within the SR. If it is not the first time attaching this disk, the
>> LUN
>> already has the SR and VDI on it.
>>
>> Once this is done, I let the normal "attach" logic run because this logic
>> expected an SR and a VDI and now it has it.
>>
>> It's the same thing for VMware: Just substitute datastore for SR and VMDK
>> for VDI.
>>
>> Does that make sense?
>>
>> Thanks!
>>
>>
>> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
>> wrote:
>>
>> > What do you do with Xen? I imagine the user enter the SAN details when
>> > registering the pool? A the pool details are basically just instructions
>> > on
>> > how to log into a target, correct?
>> >
>> > You can choose to log in a KVM host to the target during
>> > createStoragePool
>> > and save the pool in a map, or just save the pool info in a map for
>> > future
>> > reference by uuid, for when you do need to log in. The createStoragePool
>> > then just becomes a way to save the pool info to the agent. Personally,
>> > I'd
>> > log in on the pool create and look/scan for specific luns when they're
>> > needed, but I haven't thought it through thoroughly. I just say that
>> > mainly
>> > because login only happens once, the first time the pool is used, and
>> > every
>> > other storage command is about discovering new luns or maybe
>> > deleting/disconnecting luns no longer needed. On the other hand, you
>> > could
>> > do all of the above: log in on pool create, then also check if you're
>> > logged in on other commands and log in if you've lost connection.
>> >
>> > With Xen, what does your registered pool   show in the UI for avail/used
>> > capacity, and how does it get that info? I assume there is some sort of
>> > disk pool that the luns are carved from, and that your plugin is called
>> > to
>> > talk to the SAN and expose to the user how much of that pool has been
>> > allocated. Knowing how you already solves these problems with Xen will
>> > help
>> > figure out what to do with KVM.
>> >
>> > If this is the case, I think the plugin can continue to handle it rather
>> > than getting details from the agent. I'm not sure if that means nulls
>> > are
>> > OK for these on the agent side or what, I need to look at the storage
>> > plugin arch more closely.
>> > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
>> > wrote:
>> >
>> > > Hey Marcus,
>> > >
>> > > I'm reviewing your e-mails as I implement the necessary methods in new
>> > > classes.
>> > >
>> > > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
>> > > the pool data (host, port, name, path) which would be used to log the
>> > > host into the initiator."
>> > >
>> > > Can you tell me, in my case, since a storage pool (primary storage) is
>> > > actually the SAN, I wouldn't really be logging into anything at this
>> > point,
>> > > correct?
>> > >
>> > > Also, what kind of capacity, available, and used bytes make sense to
>> > report
>> > > for KVMStoragePool (since KVMStoragePool represents the SAN in my case
>> > and
>> > > not an individual LUN)?
>> > >
>> > > Thanks!
>> > >
>> > >
>> > > On Fri, Sep 13, 2013 at 11:42 PM, Marcus Sorensen > > > >wrote:
>> > >
>> > > > Ok, KVM will be close to that, of course, because only the
>> > > > hypervisor
>> > > > classes differ, the rest is all mgmt server. Creating a volume is
>> > > > just
>> > > > a db entry until it's deployed for the first time.
>> > > > AttachVolumeCommand
>> > > > on the agent side (LibvirtStorageAdaptor.java is analogous to
>> > > > CitrixResourceBase.java) will do the iscsiadm commands (via a KVM
>> 

Re: Managed storage with KVM

[Marcus Sorensen]

I guess whether or not a solidfire device is capable of hosting
multiple disk pools is irrelevant, we'd hope that we could get the
stats (maybe 30TB availabie, and 15TB allocated in LUNs). But if these
stats aren't collected, I can't as an admin define multiple pools and
expect cloudstack to allocate evenly from them or fill one up and move
to the next, because it doesn't know how big it is.

Ultimately this discussion has nothing to do with the KVM stuff
itself, just a tangent, but something to think about.

On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen  wrote:
> Ok, on most storage pools it shows how many GB free/used when listing
> the pool both via API and in the UI. I'm guessing those are empty then
> for the solid fire storage, but it seems like the user should have to
> define some sort of pool that the luns get carved out of, and you
> should be able to get the stats for that, right? Or is a solid fire
> appliance only one pool per appliance? This isn't about billing, but
> just so cloudstack itself knows whether or not there is space left on
> the storage device, so cloudstack can go on allocating from a
> different primary storage as this one fills up. There are also
> notifications and things. It seems like there should be a call you can
> handle for this, maybe Edison knows.
>
> On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen  wrote:
>> You respond to more than attach and detach, right? Don't you create luns as
>> well? Or are you just referring to the hypervisor stuff?
>>
>> On Sep 17, 2013 7:51 PM, "Mike Tutkowski" 
>> wrote:
>>>
>>> Hi Marcus,
>>>
>>> I never need to respond to a CreateStoragePool call for either XenServer
>>> or
>>> VMware.
>>>
>>> What happens is I respond only to the Attach- and Detach-volume commands.
>>>
>>> Let's say an attach comes in:
>>>
>>> In this case, I check to see if the storage is "managed." Talking
>>> XenServer
>>> here, if it is, I log in to the LUN that is the disk we want to attach.
>>> After, if this is the first time attaching this disk, I create an SR and a
>>> VDI within the SR. If it is not the first time attaching this disk, the
>>> LUN
>>> already has the SR and VDI on it.
>>>
>>> Once this is done, I let the normal "attach" logic run because this logic
>>> expected an SR and a VDI and now it has it.
>>>
>>> It's the same thing for VMware: Just substitute datastore for SR and VMDK
>>> for VDI.
>>>
>>> Does that make sense?
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
>>> wrote:
>>>
>>> > What do you do with Xen? I imagine the user enter the SAN details when
>>> > registering the pool? A the pool details are basically just instructions
>>> > on
>>> > how to log into a target, correct?
>>> >
>>> > You can choose to log in a KVM host to the target during
>>> > createStoragePool
>>> > and save the pool in a map, or just save the pool info in a map for
>>> > future
>>> > reference by uuid, for when you do need to log in. The createStoragePool
>>> > then just becomes a way to save the pool info to the agent. Personally,
>>> > I'd
>>> > log in on the pool create and look/scan for specific luns when they're
>>> > needed, but I haven't thought it through thoroughly. I just say that
>>> > mainly
>>> > because login only happens once, the first time the pool is used, and
>>> > every
>>> > other storage command is about discovering new luns or maybe
>>> > deleting/disconnecting luns no longer needed. On the other hand, you
>>> > could
>>> > do all of the above: log in on pool create, then also check if you're
>>> > logged in on other commands and log in if you've lost connection.
>>> >
>>> > With Xen, what does your registered pool   show in the UI for avail/used
>>> > capacity, and how does it get that info? I assume there is some sort of
>>> > disk pool that the luns are carved from, and that your plugin is called
>>> > to
>>> > talk to the SAN and expose to the user how much of that pool has been
>>> > allocated. Knowing how you already solves these problems with Xen will
>>> > help
>>> > figure out what to do with KVM.
>>> >
>>> > If this is the case, I think the plugin can continue to handle it rather
>>> > than getting details from the agent. I'm not sure if that means nulls
>>> > are
>>> > OK for these on the agent side or what, I need to look at the storage
>>> > plugin arch more closely.
>>> > On Sep 17, 2013 7:08 PM, "Mike Tutkowski" 
>>> > wrote:
>>> >
>>> > > Hey Marcus,
>>> > >
>>> > > I'm reviewing your e-mails as I implement the necessary methods in new
>>> > > classes.
>>> > >
>>> > > "So, referencing StorageAdaptor.java, createStoragePool accepts all of
>>> > > the pool data (host, port, name, path) which would be used to log the
>>> > > host into the initiator."
>>> > >
>>> > > Can you tell me, in my case, since a storage pool (primary storage) is
>>> > > actually the SAN, I wouldn't really be logging into anything at this
>>> > point,
>>> > > correct?
>>> > >
>>> > > Also, what kind of capacity, availabl

Re: Managed storage with KVM

[Mike Tutkowski]

This should answer your question, I believe:

* When you add primary storage that is based on the SolidFire plug-in, you
specify info like host, port, number of bytes from the SAN that CS can use,
number of IOPS from the SAN that CS can use, among other info.

* When a volume is attached for the first time and the storage framework
asks my plug-in to create a volume (LUN) on the SAN, my plug-in increments
the used_bytes field of the cloud.storage_pool table. If the used_bytes
would go above the capacity_bytes, then the allocator would not have
selected my plug-in to back the storage. Additionally, if the required IOPS
would bring the SolidFire SAN above the number of IOPS that were dedicated
to CS, the allocator would not have selected my plug-in to back the storage.

* When a CS volume is deleted that uses my plug-in, the storage framework
asks my plug-in to delete the volume (LUN) on the SAN. My plug-in
decrements the used_bytes field of the cloud.storage_pool table.

So, it just boils down to we don't require the accounting of space and IOPS
to take place on the hypervisor side.


On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen wrote:

> Ok, on most storage pools it shows how many GB free/used when listing
> the pool both via API and in the UI. I'm guessing those are empty then
> for the solid fire storage, but it seems like the user should have to
> define some sort of pool that the luns get carved out of, and you
> should be able to get the stats for that, right? Or is a solid fire
> appliance only one pool per appliance? This isn't about billing, but
> just so cloudstack itself knows whether or not there is space left on
> the storage device, so cloudstack can go on allocating from a
> different primary storage as this one fills up. There are also
> notifications and things. It seems like there should be a call you can
> handle for this, maybe Edison knows.
>
> On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
> wrote:
> > You respond to more than attach and detach, right? Don't you create luns
> as
> > well? Or are you just referring to the hypervisor stuff?
> >
> > On Sep 17, 2013 7:51 PM, "Mike Tutkowski" 
> > wrote:
> >>
> >> Hi Marcus,
> >>
> >> I never need to respond to a CreateStoragePool call for either XenServer
> >> or
> >> VMware.
> >>
> >> What happens is I respond only to the Attach- and Detach-volume
> commands.
> >>
> >> Let's say an attach comes in:
> >>
> >> In this case, I check to see if the storage is "managed." Talking
> >> XenServer
> >> here, if it is, I log in to the LUN that is the disk we want to attach.
> >> After, if this is the first time attaching this disk, I create an SR
> and a
> >> VDI within the SR. If it is not the first time attaching this disk, the
> >> LUN
> >> already has the SR and VDI on it.
> >>
> >> Once this is done, I let the normal "attach" logic run because this
> logic
> >> expected an SR and a VDI and now it has it.
> >>
> >> It's the same thing for VMware: Just substitute datastore for SR and
> VMDK
> >> for VDI.
> >>
> >> Does that make sense?
> >>
> >> Thanks!
> >>
> >>
> >> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
> >> wrote:
> >>
> >> > What do you do with Xen? I imagine the user enter the SAN details when
> >> > registering the pool? A the pool details are basically just
> instructions
> >> > on
> >> > how to log into a target, correct?
> >> >
> >> > You can choose to log in a KVM host to the target during
> >> > createStoragePool
> >> > and save the pool in a map, or just save the pool info in a map for
> >> > future
> >> > reference by uuid, for when you do need to log in. The
> createStoragePool
> >> > then just becomes a way to save the pool info to the agent.
> Personally,
> >> > I'd
> >> > log in on the pool create and look/scan for specific luns when they're
> >> > needed, but I haven't thought it through thoroughly. I just say that
> >> > mainly
> >> > because login only happens once, the first time the pool is used, and
> >> > every
> >> > other storage command is about discovering new luns or maybe
> >> > deleting/disconnecting luns no longer needed. On the other hand, you
> >> > could
> >> > do all of the above: log in on pool create, then also check if you're
> >> > logged in on other commands and log in if you've lost connection.
> >> >
> >> > With Xen, what does your registered pool   show in the UI for
> avail/used
> >> > capacity, and how does it get that info? I assume there is some sort
> of
> >> > disk pool that the luns are carved from, and that your plugin is
> called
> >> > to
> >> > talk to the SAN and expose to the user how much of that pool has been
> >> > allocated. Knowing how you already solves these problems with Xen will
> >> > help
> >> > figure out what to do with KVM.
> >> >
> >> > If this is the case, I think the plugin can continue to handle it
> rather
> >> > than getting details from the agent. I'm not sure if that means nulls
> >> > are
> >> > OK for these on the agent side or what, I need to look

Re: Managed storage with KVM

[Mike Tutkowski]

What you're saying here is definitely something we should talk about.

Hopefully my previous e-mail has clarified how this works a bit.

It mainly comes down to this:

For the first time in CS history, primary storage is no longer required to
be preallocated by the admin and then handed to CS. CS volumes don't have
to share a preallocated volume anymore.

As of 4.2, primary storage can be based on a SAN (or some other storage
device). You can tell CS how many bytes and IOPS to use from this storage
device and CS invokes the appropriate plug-in to carve out LUNs dynamically.

Each LUN is home to one and only one data disk. Data disks - in this model
- never share a LUN.

The main use case for this is so a CS volume can deliver guaranteed IOPS if
the storage device (ex. SolidFire SAN) delivers guaranteed IOPS on a
LUN-by-LUN basis.


On Tue, Sep 17, 2013 at 10:16 PM, Marcus Sorensen wrote:

> I guess whether or not a solidfire device is capable of hosting
> multiple disk pools is irrelevant, we'd hope that we could get the
> stats (maybe 30TB availabie, and 15TB allocated in LUNs). But if these
> stats aren't collected, I can't as an admin define multiple pools and
> expect cloudstack to allocate evenly from them or fill one up and move
> to the next, because it doesn't know how big it is.
>
> Ultimately this discussion has nothing to do with the KVM stuff
> itself, just a tangent, but something to think about.
>
> On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen 
> wrote:
> > Ok, on most storage pools it shows how many GB free/used when listing
> > the pool both via API and in the UI. I'm guessing those are empty then
> > for the solid fire storage, but it seems like the user should have to
> > define some sort of pool that the luns get carved out of, and you
> > should be able to get the stats for that, right? Or is a solid fire
> > appliance only one pool per appliance? This isn't about billing, but
> > just so cloudstack itself knows whether or not there is space left on
> > the storage device, so cloudstack can go on allocating from a
> > different primary storage as this one fills up. There are also
> > notifications and things. It seems like there should be a call you can
> > handle for this, maybe Edison knows.
> >
> > On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
> wrote:
> >> You respond to more than attach and detach, right? Don't you create
> luns as
> >> well? Or are you just referring to the hypervisor stuff?
> >>
> >> On Sep 17, 2013 7:51 PM, "Mike Tutkowski"  >
> >> wrote:
> >>>
> >>> Hi Marcus,
> >>>
> >>> I never need to respond to a CreateStoragePool call for either
> XenServer
> >>> or
> >>> VMware.
> >>>
> >>> What happens is I respond only to the Attach- and Detach-volume
> commands.
> >>>
> >>> Let's say an attach comes in:
> >>>
> >>> In this case, I check to see if the storage is "managed." Talking
> >>> XenServer
> >>> here, if it is, I log in to the LUN that is the disk we want to attach.
> >>> After, if this is the first time attaching this disk, I create an SR
> and a
> >>> VDI within the SR. If it is not the first time attaching this disk, the
> >>> LUN
> >>> already has the SR and VDI on it.
> >>>
> >>> Once this is done, I let the normal "attach" logic run because this
> logic
> >>> expected an SR and a VDI and now it has it.
> >>>
> >>> It's the same thing for VMware: Just substitute datastore for SR and
> VMDK
> >>> for VDI.
> >>>
> >>> Does that make sense?
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
> >>> wrote:
> >>>
> >>> > What do you do with Xen? I imagine the user enter the SAN details
> when
> >>> > registering the pool? A the pool details are basically just
> instructions
> >>> > on
> >>> > how to log into a target, correct?
> >>> >
> >>> > You can choose to log in a KVM host to the target during
> >>> > createStoragePool
> >>> > and save the pool in a map, or just save the pool info in a map for
> >>> > future
> >>> > reference by uuid, for when you do need to log in. The
> createStoragePool
> >>> > then just becomes a way to save the pool info to the agent.
> Personally,
> >>> > I'd
> >>> > log in on the pool create and look/scan for specific luns when
> they're
> >>> > needed, but I haven't thought it through thoroughly. I just say that
> >>> > mainly
> >>> > because login only happens once, the first time the pool is used, and
> >>> > every
> >>> > other storage command is about discovering new luns or maybe
> >>> > deleting/disconnecting luns no longer needed. On the other hand, you
> >>> > could
> >>> > do all of the above: log in on pool create, then also check if you're
> >>> > logged in on other commands and log in if you've lost connection.
> >>> >
> >>> > With Xen, what does your registered pool   show in the UI for
> avail/used
> >>> > capacity, and how does it get that info? I assume there is some sort
> of
> >>> > disk pool that the luns are carved from, and that your plugin is
> called
> >>> > to
> >>> >

Re: Managed storage with KVM

[Mike Tutkowski]

For what it's worth, OpenStack is quite a bit different.

All storage volumes are dynamically created (like what was enabled in 4.2)
and these volumes are directly attached to VMs (without going through the
hypervisor).

Since we go through the hypervisor, to enable a 1:1 mapping between a CS
volume and a SAN LUN required - for XenServer - the creation of an SR with
one VDI that took up as much space in the SR as it could.

Same idea with ESX(i), but - as we chatted about a bit - it's one datastore
having a VMDK file that took up as much space in the datastore as it could.

That's why - initially - I was under the impression we'd take the same
approach with KVM.

I'm glad, though, you presented the option about discovering the iSCSI
target and then attaching the resultant device as a raw device to the VM.

Since CS handles HA for KVM, that makes sense (you don't need the
equivalent then of the overhead we incur with an SR or datastore).


On Tue, Sep 17, 2013 at 10:41 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> What you're saying here is definitely something we should talk about.
>
> Hopefully my previous e-mail has clarified how this works a bit.
>
> It mainly comes down to this:
>
> For the first time in CS history, primary storage is no longer required to
> be preallocated by the admin and then handed to CS. CS volumes don't have
> to share a preallocated volume anymore.
>
> As of 4.2, primary storage can be based on a SAN (or some other storage
> device). You can tell CS how many bytes and IOPS to use from this storage
> device and CS invokes the appropriate plug-in to carve out LUNs dynamically.
>
> Each LUN is home to one and only one data disk. Data disks - in this model
> - never share a LUN.
>
> The main use case for this is so a CS volume can deliver guaranteed IOPS
> if the storage device (ex. SolidFire SAN) delivers guaranteed IOPS on a
> LUN-by-LUN basis.
>
>
> On Tue, Sep 17, 2013 at 10:16 PM, Marcus Sorensen wrote:
>
>> I guess whether or not a solidfire device is capable of hosting
>> multiple disk pools is irrelevant, we'd hope that we could get the
>> stats (maybe 30TB availabie, and 15TB allocated in LUNs). But if these
>> stats aren't collected, I can't as an admin define multiple pools and
>> expect cloudstack to allocate evenly from them or fill one up and move
>> to the next, because it doesn't know how big it is.
>>
>> Ultimately this discussion has nothing to do with the KVM stuff
>> itself, just a tangent, but something to think about.
>>
>> On Tue, Sep 17, 2013 at 10:13 PM, Marcus Sorensen 
>> wrote:
>> > Ok, on most storage pools it shows how many GB free/used when listing
>> > the pool both via API and in the UI. I'm guessing those are empty then
>> > for the solid fire storage, but it seems like the user should have to
>> > define some sort of pool that the luns get carved out of, and you
>> > should be able to get the stats for that, right? Or is a solid fire
>> > appliance only one pool per appliance? This isn't about billing, but
>> > just so cloudstack itself knows whether or not there is space left on
>> > the storage device, so cloudstack can go on allocating from a
>> > different primary storage as this one fills up. There are also
>> > notifications and things. It seems like there should be a call you can
>> > handle for this, maybe Edison knows.
>> >
>> > On Tue, Sep 17, 2013 at 8:46 PM, Marcus Sorensen 
>> wrote:
>> >> You respond to more than attach and detach, right? Don't you create
>> luns as
>> >> well? Or are you just referring to the hypervisor stuff?
>> >>
>> >> On Sep 17, 2013 7:51 PM, "Mike Tutkowski" <
>> mike.tutkow...@solidfire.com>
>> >> wrote:
>> >>>
>> >>> Hi Marcus,
>> >>>
>> >>> I never need to respond to a CreateStoragePool call for either
>> XenServer
>> >>> or
>> >>> VMware.
>> >>>
>> >>> What happens is I respond only to the Attach- and Detach-volume
>> commands.
>> >>>
>> >>> Let's say an attach comes in:
>> >>>
>> >>> In this case, I check to see if the storage is "managed." Talking
>> >>> XenServer
>> >>> here, if it is, I log in to the LUN that is the disk we want to
>> attach.
>> >>> After, if this is the first time attaching this disk, I create an SR
>> and a
>> >>> VDI within the SR. If it is not the first time attaching this disk,
>> the
>> >>> LUN
>> >>> already has the SR and VDI on it.
>> >>>
>> >>> Once this is done, I let the normal "attach" logic run because this
>> logic
>> >>> expected an SR and a VDI and now it has it.
>> >>>
>> >>> It's the same thing for VMware: Just substitute datastore for SR and
>> VMDK
>> >>> for VDI.
>> >>>
>> >>> Does that make sense?
>> >>>
>> >>> Thanks!
>> >>>
>> >>>
>> >>> On Tue, Sep 17, 2013 at 7:34 PM, Marcus Sorensen
>> >>> wrote:
>> >>>
>> >>> > What do you do with Xen? I imagine the user enter the SAN details
>> when
>> >>> > registering the pool? A the pool details are basically just
>> instructions
>> >>> > on
>> >>> > how to log into a target, correct?
>> >>> >
>> >>> > You