Jetty server fail to start in master branch

[Isaac Chiang]

Hi all:
  I checked out lastest master code. I successfully built the code and
deployed the database, but failed to start the jetty server. Did anyone
happen to see the error?

ERROR [web.context.ContextLoader] (main:) Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating
bean with name 'actionEventUtils': Injection of autowired depen
dencies failed; nested exception is
org.springframework.beans.factory.BeanCreationException: Could not autowire
field: com.cloud.event.
dao.EventDao com.cloud.event.ActionEventUtils.eventDao; nested exception is
org.springframework.beans.factory.CannotLoadBeanClassExcept
ion: Cannot find class
[org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy] for bean
with name 'vmwareStorageMotionStrate
gy' defined in class path resource [applicationContext.xml]; nested
exception is java.lang.ClassNotFoundException: org.apache.cloudstac
k.storage.motion.VmwareStorageMotionStrategy
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotat
ionBeanPostProcessor.java:287)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory
.java:1106)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory
.java:517)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.j
ava:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.jav
a:609)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.ja
va:918)


Regards

Isaac

Re: [MERGE] disk_io_throttling to MASTER

[John Burwell]

Wei,

My opinion is that no features should be merged until all functional issues
have been resolved and it is ready to turn over to test.  Until the total
Ops vs discrete read/write ops issue is addressed and re-reviewed by Wido,
I don't think this criteria has been satisfied.

Also, how does this work intersect/compliment the SolidFire patch (
https://reviews.apache.org/r/11479/)?  As I understand it that work is also
involves provisioned IOPS.  I would like to ensure we don't have a scenario
where provisioned IOPS in KVM and SolidFire are unnecessarily incompatible.

Thanks,
-John

On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:

Wido,


Sure. I will change it next week.


-Wei



2013/6/1 Wido den Hollander 


Hi Wei,



On 06/01/2013 08:24 AM, Wei ZHOU wrote:


Wido,


Exactly. I have pushed the features into master.


If anyone object thems for technical reason till Monday, I will revert

them.


For the sake of clarity I just want to mention again that we should change

the total IOps to R/W IOps asap so that we never release a version with

only total IOps.


You laid the groundwork for the I/O throttling and that's great! We should

however prevent that we create legacy from day #1.


Wido


-Wei



2013/5/31 Wido den Hollander 


On 05/31/2013 03:59 PM, John Burwell wrote:


Wido,


+1 -- this enhancement must to discretely support read and write IOPS.

I

don't see how it could be fixed later because I don't see how we

correctly

split total IOPS into read and write.  Therefore, we would be stuck

with a

total unless/until we decided to break backwards compatibility.



What Wei meant was merging it into master now so that it will go in the

4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2

will be released with Read and Write instead of Total IOps.


This is to make the May 31st feature freeze date. But if the window moves

(see other threads) then it won't be necessary to do that.


Wido



I also completely agree that there is no association between network

and


disk I/O.


Thanks,

-John


On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:


Hi Wei,



On 05/31/2013 03:13 PM, Wei ZHOU wrote:


Hi Wido,


Thanks. Good question.


I  thought about at the beginning. Finally I decided to ignore the

difference of read and write mainly because the network throttling did

not

care the difference of sent and received bytes as well.

That reasoning seems odd. Networking and disk I/O completely different.


Disk I/O is much more expensive in most situations then network

bandwith.


Implementing it will be some copy-paste work. It could be

implemented in


few days. For the deadline of feature freeze, I will implement it

after

that , if needed.



It think it's a feature we can't miss. But if it goes into the 4.2

window we have to make sure we don't release with only total IOps and

fix

it in 4.3, that will confuse users.


Wido


-Wei






2013/5/31 Wido den Hollander 


Hi Wei,




On 05/30/2013 06:03 PM, Wei ZHOU wrote:


Hi,


I would like to merge disk_io_throttling branch into master.

If nobody object, I will merge into master in 48 hours.

The purpose is :


Virtual machines are running on the same storage device (local

storage or

share strage). Because of the rate limitation of device (such as

iops), if

one VM has large disk operation, it may affect the disk performance

of

other VMs running on the same storage device.

It is neccesary to set the maximum rate and limit the disk I/O of

VMs.



Looking at the code I see you make no difference between Read and

Write

IOps.


Qemu and libvirt support setting both a different rate for Read and

Write

IOps which could benefit a lot of users.


It's also strange, in the polling side you collect both the Read and

Write

IOps, but on the throttling side you only go for a global value.


Write IOps are usually much more expensive then Read IOps, so it

seems

like a valid use-case where that an admin would set a lower value for

write

IOps vs Read IOps.


Since this only supports KVM at this point I think it would be of

great

value to at least have the mechanism in place to support both,

implementing

this later would be a lot of work.


If a hypervisor doesn't support setting different values for read and

write you can always sum both up and set that as the total limit.


Can you explain why you implemented it this way?


Wido


 The feature includes:



(1) set the maximum rate of VMs (in disk_offering, and global

configuration)

(2) change the maximum rate of VMs

(3) limit the disk rate (total bps and iops)

JIRA ticket: https://issues.apache.org/

jira/browse/CLOUDSTACK-1192https://issues.apache.org/**jira/browse/CLOUDSTACK-1192>

http://issues.apache.org/jira/**browse/CLOUDSTACK-1192>

<**
https://issues.apache.org/**jira/browse/CLOUDSTACK-1192



FS (I will update later) :

https://cwiki.apache.org/**confluence/display/CLOUDSTACK/**

Re: [MERGE] disk_io_throttling to MASTER

[Mike Tutkowski]

Hi John,

I believe Wei's feature deals with controlling the max number of IOPS from
the hypervisor side.

My feature is focused on controlling IOPS from the storage system side.

I hope that helps. :)


On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:

> Wei,
>
> My opinion is that no features should be merged until all functional
> issues have been resolved and it is ready to turn over to test.  Until the
> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
> Wido, I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)?  As I understand it that work is
> also involves provisioned IOPS.  I would like to ensure we don't have a
> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander 
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write.  Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
>  I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
>
>
>  Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I  thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
>  Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
> that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
>  window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
>  -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
>  Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
>  Hi,
>
>
>  I would like to merge disk_io_throttling branch into master.
>
>  If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
>  share strage). Because of the rate limitation of device (such as
>
> iops), if
>
>  one VM has large disk operation, it may affect the disk performance
>
>  of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
>  VMs.
>
>
>
>  Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.
>
>
> Qemu and libvirt support setting both a different rate for Read and
>
>  Write
>
> IOps which could benefit a lot of users.
>
>
>  It's also strange, in the polling side you collect both the Read and
>
>  Write
>
> IOps, but on the throttling side you only go for a global value.
>
>
> Write IOps are usually much more expensive then Read IOps, so it
>
> seems
>
> like a valid use-case where that an admin would set a lower value for
>
>  write
>
> IOps vs Read IOps.
>
>
>  Since this only supports KVM at this point I think it would be of
>
> great
>
> value to at least have the mechanism in place to support both,
>
> implementing
>
> this later would be a lot of work.
>
>
>  If a hypervisor doesn't support setting different values for read and
>
> write you can always sum both up and set that a

Re: [MERGE] disk_io_throttling to MASTER

[Mike Tutkowski]

Perhaps I should add more detail. :)

The SolidFire plug-in is based on and requires Edison's new storage
framework (also going in with 4.2).

Leveraging Edison's framework, the plug-in is invoked at certain times to
create and delete volumes on the SAN (and also to create and delete SRs in
XenServer or Datastores in VMware to make use of those volumes).

The Min, Max, and Burst IOPS can be filled in by an admin when he/she
creates a Disk Offering or he/she can pass these IOPS fields onto end users
to fill in (in the Add Volume dialog). These fields are - ultimately -
optional as most storage systems do not yet support them (but they are
intentionally meant for future use by other storage systems per an earlier
discussion with the CloudStack e-mail list).

Talk to you later! :)


On Sun, Jun 2, 2013 at 6:38 PM, Mike Tutkowski  wrote:

> Hi John,
>
> I believe Wei's feature deals with controlling the max number of IOPS from
> the hypervisor side.
>
> My feature is focused on controlling IOPS from the storage system side.
>
> I hope that helps. :)
>
>
> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:
>
>> Wei,
>>
>> My opinion is that no features should be merged until all functional
>> issues have been resolved and it is ready to turn over to test.  Until the
>> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
>> Wido, I don't think this criteria has been satisfied.
>>
>> Also, how does this work intersect/compliment the SolidFire patch (
>> https://reviews.apache.org/r/11479/)?  As I understand it that work is
>> also involves provisioned IOPS.  I would like to ensure we don't have a
>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>> incompatible.
>>
>> Thanks,
>> -John
>>
>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
>>
>> Wido,
>>
>>
>> Sure. I will change it next week.
>>
>>
>> -Wei
>>
>>
>>
>> 2013/6/1 Wido den Hollander 
>>
>>
>> Hi Wei,
>>
>>
>>
>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>
>>
>> Wido,
>>
>>
>> Exactly. I have pushed the features into master.
>>
>>
>> If anyone object thems for technical reason till Monday, I will revert
>>
>> them.
>>
>>
>> For the sake of clarity I just want to mention again that we should change
>>
>> the total IOps to R/W IOps asap so that we never release a version with
>>
>> only total IOps.
>>
>>
>> You laid the groundwork for the I/O throttling and that's great! We should
>>
>> however prevent that we create legacy from day #1.
>>
>>
>> Wido
>>
>>
>> -Wei
>>
>>
>>
>> 2013/5/31 Wido den Hollander 
>>
>>
>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>
>>
>> Wido,
>>
>>
>> +1 -- this enhancement must to discretely support read and write IOPS.
>>
>> I
>>
>> don't see how it could be fixed later because I don't see how we
>>
>> correctly
>>
>> split total IOPS into read and write.  Therefore, we would be stuck
>>
>> with a
>>
>> total unless/until we decided to break backwards compatibility.
>>
>>
>>
>> What Wei meant was merging it into master now so that it will go in the
>>
>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>
>> will be released with Read and Write instead of Total IOps.
>>
>>
>> This is to make the May 31st feature freeze date. But if the window moves
>>
>> (see other threads) then it won't be necessary to do that.
>>
>>
>> Wido
>>
>>
>>
>>  I also completely agree that there is no association between network
>>
>> and
>>
>>
>> disk I/O.
>>
>>
>> Thanks,
>>
>> -John
>>
>>
>> On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
>>
>>
>>  Hi Wei,
>>
>>
>>
>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>
>>
>> Hi Wido,
>>
>>
>> Thanks. Good question.
>>
>>
>> I  thought about at the beginning. Finally I decided to ignore the
>>
>> difference of read and write mainly because the network throttling did
>>
>> not
>>
>> care the difference of sent and received bytes as well.
>>
>> That reasoning seems odd. Networking and disk I/O completely different.
>>
>>
>> Disk I/O is much more expensive in most situations then network
>>
>> bandwith.
>>
>>
>>  Implementing it will be some copy-paste work. It could be
>>
>> implemented in
>>
>>
>> few days. For the deadline of feature freeze, I will implement it
>>
>> after
>>
>>  that , if needed.
>>
>>
>>
>> It think it's a feature we can't miss. But if it goes into the 4.2
>>
>>  window we have to make sure we don't release with only total IOps and
>>
>> fix
>>
>> it in 4.3, that will confuse users.
>>
>>
>> Wido
>>
>>
>>  -Wei
>>
>>
>>
>>
>>
>>
>> 2013/5/31 Wido den Hollander 
>>
>>
>>  Hi Wei,
>>
>>
>>
>>
>> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>>
>>
>>  Hi,
>>
>>
>>  I would like to merge disk_io_throttling branch into master.
>>
>>  If nobody object, I will merge into master in 48 hours.
>>
>> The purpose is :
>>
>>
>> Virtual machines are running on the same storage device (local
>>
>> storage or
>>
>>  share strage). Because of the rate limitation of device (such as
>>
>> iops), if
>>
>>  one VM has large disk operat

Re: [MERGE] disk_io_throttling to MASTER

[John Burwell]

Mike,

The things I want to understand are the following:

   1) Is there value in capturing IOPS policies be captured in a common
data model (e.g. for billing/usage purposes, expressing offerings).
2) Should there be a common interface model for reasoning about IOP
provisioning at runtime?
3) How are conflicting provisioned IOPS configurations between a
hypervisor and storage device reconciled?  In particular, a scenario where
is lead to believe (and billed) for more IOPS configured for a VM than a
storage device has been configured to deliver.  Another scenario could a
consistent configuration between a VM and a storage device at creation
time, but a later modification to storage device introduces logical
inconsistency.

Thanks,
-John

On Jun 2, 2013, at 8:38 PM, Mike Tutkowski 
wrote:

Hi John,

I believe Wei's feature deals with controlling the max number of IOPS from
the hypervisor side.

My feature is focused on controlling IOPS from the storage system side.

I hope that helps. :)


On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:

> Wei,
>
> My opinion is that no features should be merged until all functional
> issues have been resolved and it is ready to turn over to test.  Until the
> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
> Wido, I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)?  As I understand it that work is
> also involves provisioned IOPS.  I would like to ensure we don't have a
> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander 
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write.  Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
>  I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
>
>
>  Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I  thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
>  Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
>  that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
>  window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
>  -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
>  Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
>  Hi,
>
>
>  I would like to merge disk_io_throttling branch into master.
>
>  If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
>  share strage). Because of the rate limitation of device (such as
>
> iops), if
>
>  one VM has large disk operation, it may affect the disk performance
>
>  of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
>  VMs.
>
>
>
>  Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.

Re: [MERGE] disk_io_throttling to MASTER

[Mike Tutkowski]

Sending to the CS e-mail list (I accidentally only sent this to John
before).


On Sun, Jun 2, 2013 at 7:10 PM, Mike Tutkowski  wrote:

> Hi John,
>
> Let me see if I can answer your questions.
>
> 1) If we mean provide the ability to charge more for higher IOPS Disk
> Offerings, then I would say 'yes.'
>
> 2) I might need more detail on what you're thinking here.
>
> 3) The IOPS in my patch code are from the storage system's point of view
> only. If the hypervisor has a Max that is, say, below the Max of the
> storage system, the storage system will never hit its Max. That being the
> case, it probably makes sense to provide an error message to the user in
> such a situation. I haven't actually seen Wei's code yet, but we should
> coordinate on activities like this.
>
>
> On Sun, Jun 2, 2013 at 6:54 PM, John Burwell  wrote:
>
>> Mike,
>>
>> The things I want to understand are the following:
>>
>>1) Is there value in capturing IOPS policies be captured in a common
>> data model (e.g. for billing/usage purposes, expressing offerings).
>> 2) Should there be a common interface model for reasoning about IOP
>> provisioning at runtime?
>> 3) How are conflicting provisioned IOPS configurations between a
>> hypervisor and storage device reconciled?  In particular, a scenario where
>> is lead to believe (and billed) for more IOPS configured for a VM than a
>> storage device has been configured to deliver.  Another scenario could a
>> consistent configuration between a VM and a storage device at creation
>> time, but a later modification to storage device introduces logical
>> inconsistency.
>>
>> Thanks,
>> -John
>>
>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski 
>> wrote:
>>
>>  Hi John,
>>
>> I believe Wei's feature deals with controlling the max number of IOPS
>> from the hypervisor side.
>>
>> My feature is focused on controlling IOPS from the storage system side.
>>
>> I hope that helps. :)
>>
>>
>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:
>>
>>> Wei,
>>>
>>> My opinion is that no features should be merged until all functional
>>> issues have been resolved and it is ready to turn over to test.  Until the
>>> total Ops vs discrete read/write ops issue is addressed and re-reviewed by
>>> Wido, I don't think this criteria has been satisfied.
>>>
>>> Also, how does this work intersect/compliment the SolidFire patch (
>>> https://reviews.apache.org/r/11479/)?  As I understand it that work is
>>> also involves provisioned IOPS.  I would like to ensure we don't have a
>>> scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
>>> incompatible.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
>>>
>>> Wido,
>>>
>>>
>>> Sure. I will change it next week.
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/6/1 Wido den Hollander 
>>>
>>>
>>> Hi Wei,
>>>
>>>
>>>
>>> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> Exactly. I have pushed the features into master.
>>>
>>>
>>> If anyone object thems for technical reason till Monday, I will revert
>>>
>>> them.
>>>
>>>
>>> For the sake of clarity I just want to mention again that we should
>>> change
>>>
>>> the total IOps to R/W IOps asap so that we never release a version with
>>>
>>> only total IOps.
>>>
>>>
>>> You laid the groundwork for the I/O throttling and that's great! We
>>> should
>>>
>>> however prevent that we create legacy from day #1.
>>>
>>>
>>> Wido
>>>
>>>
>>> -Wei
>>>
>>>
>>>
>>> 2013/5/31 Wido den Hollander 
>>>
>>>
>>> On 05/31/2013 03:59 PM, John Burwell wrote:
>>>
>>>
>>> Wido,
>>>
>>>
>>> +1 -- this enhancement must to discretely support read and write IOPS.
>>>
>>> I
>>>
>>> don't see how it could be fixed later because I don't see how we
>>>
>>> correctly
>>>
>>> split total IOPS into read and write.  Therefore, we would be stuck
>>>
>>> with a
>>>
>>> total unless/until we decided to break backwards compatibility.
>>>
>>>
>>>
>>> What Wei meant was merging it into master now so that it will go in the
>>>
>>> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>>>
>>> will be released with Read and Write instead of Total IOps.
>>>
>>>
>>> This is to make the May 31st feature freeze date. But if the window moves
>>>
>>> (see other threads) then it won't be necessary to do that.
>>>
>>>
>>> Wido
>>>
>>>
>>>
>>>  I also completely agree that there is no association between network
>>>
>>> and
>>>
>>>
>>> disk I/O.
>>>
>>>
>>> Thanks,
>>>
>>> -John
>>>
>>>
>>> On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
>>>
>>>
>>>  Hi Wei,
>>>
>>>
>>>
>>> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>>>
>>>
>>> Hi Wido,
>>>
>>>
>>> Thanks. Good question.
>>>
>>>
>>> I  thought about at the beginning. Finally I decided to ignore the
>>>
>>> difference of read and write mainly because the network throttling did
>>>
>>> not
>>>
>>> care the difference of sent and received bytes as well.
>>>
>>> That reasoning seems odd. Networking and disk I/O completely different.
>>>
>

Re: [MERGE] disk_io_throttling to MASTER

[Mike Tutkowski]

I should also point out, John, that I am only supporting Disk Offerings for
4.2 (not Compute Offerings). I plan to support Compute Offerings in 4.3.

Also, the patch I submitted is for XenServer only. Depending on when we
freeze 4.2, I could submit code for VMware.

The XenServer and VMware code, though, is technically not a part of the
plug-in, but rather the storage framework (enhancing what Edison built). It
could be leveraged by any dynamic, zone-wide primary storage.


On Sun, Jun 2, 2013 at 7:14 PM, Mike Tutkowski  wrote:

> Sending to the CS e-mail list (I accidentally only sent this to John
> before).
>
>
>
> On Sun, Jun 2, 2013 at 7:10 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Hi John,
>>
>> Let me see if I can answer your questions.
>>
>> 1) If we mean provide the ability to charge more for higher IOPS Disk
>> Offerings, then I would say 'yes.'
>>
>> 2) I might need more detail on what you're thinking here.
>>
>> 3) The IOPS in my patch code are from the storage system's point of view
>> only. If the hypervisor has a Max that is, say, below the Max of the
>> storage system, the storage system will never hit its Max. That being the
>> case, it probably makes sense to provide an error message to the user in
>> such a situation. I haven't actually seen Wei's code yet, but we should
>> coordinate on activities like this.
>>
>>
>> On Sun, Jun 2, 2013 at 6:54 PM, John Burwell  wrote:
>>
>>> Mike,
>>>
>>> The things I want to understand are the following:
>>>
>>>1) Is there value in capturing IOPS policies be captured in a common
>>> data model (e.g. for billing/usage purposes, expressing offerings).
>>> 2) Should there be a common interface model for reasoning about IOP
>>> provisioning at runtime?
>>> 3) How are conflicting provisioned IOPS configurations between a
>>> hypervisor and storage device reconciled?  In particular, a scenario where
>>> is lead to believe (and billed) for more IOPS configured for a VM than a
>>> storage device has been configured to deliver.  Another scenario could a
>>> consistent configuration between a VM and a storage device at creation
>>> time, but a later modification to storage device introduces logical
>>> inconsistency.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski 
>>> wrote:
>>>
>>>  Hi John,
>>>
>>> I believe Wei's feature deals with controlling the max number of IOPS
>>> from the hypervisor side.
>>>
>>> My feature is focused on controlling IOPS from the storage system side.
>>>
>>> I hope that helps. :)
>>>
>>>
>>> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:
>>>
 Wei,

 My opinion is that no features should be merged until all functional
 issues have been resolved and it is ready to turn over to test.  Until the
 total Ops vs discrete read/write ops issue is addressed and re-reviewed by
 Wido, I don't think this criteria has been satisfied.

 Also, how does this work intersect/compliment the SolidFire patch (
 https://reviews.apache.org/r/11479/)?  As I understand it that work is
 also involves provisioned IOPS.  I would like to ensure we don't have a
 scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
 incompatible.

 Thanks,
 -John

 On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:

 Wido,


 Sure. I will change it next week.


 -Wei



 2013/6/1 Wido den Hollander 


 Hi Wei,



 On 06/01/2013 08:24 AM, Wei ZHOU wrote:


 Wido,


 Exactly. I have pushed the features into master.


 If anyone object thems for technical reason till Monday, I will revert

 them.


 For the sake of clarity I just want to mention again that we should
 change

 the total IOps to R/W IOps asap so that we never release a version with

 only total IOps.


 You laid the groundwork for the I/O throttling and that's great! We
 should

 however prevent that we create legacy from day #1.


 Wido


 -Wei



 2013/5/31 Wido den Hollander 


 On 05/31/2013 03:59 PM, John Burwell wrote:


 Wido,


 +1 -- this enhancement must to discretely support read and write IOPS.

 I

 don't see how it could be fixed later because I don't see how we

 correctly

 split total IOPS into read and write.  Therefore, we would be stuck

 with a

 total unless/until we decided to break backwards compatibility.



 What Wei meant was merging it into master now so that it will go in the

 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2

 will be released with Read and Write instead of Total IOps.


 This is to make the May 31st feature freeze date. But if the window
 moves

 (see oth

Re: Jetty server fail to start in master branch

[Isaac Chiang]

Hi all:
   I found that commit : f24e81fe0da let the jetty fail to start in
master. Did anyone see this error?

Here's my steps :

   1. mvn clean install -P developer,systemvm
   2. mvn -pl developer,tools/devcloud -Ddeploydb -P developer
   3. mvn -pl client jetty:run




Regards

Isaac


On Sun, Jun 2, 2013 at 10:37 PM, Isaac Chiang  wrote:

> Hi all:
>   I checked out lastest master code. I successfully built the code and
> deployed the database, but failed to start the jetty server. Did anyone
> happen to see the error?
>
> ERROR [web.context.ContextLoader] (main:) Context initialization failed
> org.springframework.beans.factory.BeanCreationException: Error creating
> bean with name 'actionEventUtils': Injection of autowired depen
> dencies failed; nested exception is
> org.springframework.beans.factory.BeanCreationException: Could not autowire
> field: com.cloud.event.
> dao.EventDao com.cloud.event.ActionEventUtils.eventDao; nested exception
> is org.springframework.beans.factory.CannotLoadBeanClassExcept
> ion: Cannot find class
> [org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy] for bean
> with name 'vmwareStorageMotionStrate
> gy' defined in class path resource [applicationContext.xml]; nested
> exception is java.lang.ClassNotFoundException: org.apache.cloudstac
> k.storage.motion.VmwareStorageMotionStrategy
> at
> org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotat
> ionBeanPostProcessor.java:287)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory
> .java:1106)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory
> .java:517)
> at
> org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.j
> ava:456)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
> at
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
> at
> org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.jav
> a:609)
> at
> org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.ja
> va:918)
>
>
> Regards
>
> Isaac
>
>

Re: [MERGE] disk_io_throttling to MASTER

[Wei ZHOU]

John,
Thanks for your reply.

I will revert the commit for the opnion from Wido and you, and ask you
reviewing it after modification.

-Wei

2013/6/3, John Burwell :
> Wei,
>
> My opinion is that no features should be merged until all functional issues
> have been resolved and it is ready to turn over to test.  Until the total
> Ops vs discrete read/write ops issue is addressed and re-reviewed by Wido,
> I don't think this criteria has been satisfied.
>
> Also, how does this work intersect/compliment the SolidFire patch (
> https://reviews.apache.org/r/11479/)?  As I understand it that work is also
> involves provisioned IOPS.  I would like to ensure we don't have a scenario
> where provisioned IOPS in KVM and SolidFire are unnecessarily incompatible.
>
> Thanks,
> -John
>
> On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
>
> Wido,
>
>
> Sure. I will change it next week.
>
>
> -Wei
>
>
>
> 2013/6/1 Wido den Hollander 
>
>
> Hi Wei,
>
>
>
> On 06/01/2013 08:24 AM, Wei ZHOU wrote:
>
>
> Wido,
>
>
> Exactly. I have pushed the features into master.
>
>
> If anyone object thems for technical reason till Monday, I will revert
>
> them.
>
>
> For the sake of clarity I just want to mention again that we should change
>
> the total IOps to R/W IOps asap so that we never release a version with
>
> only total IOps.
>
>
> You laid the groundwork for the I/O throttling and that's great! We should
>
> however prevent that we create legacy from day #1.
>
>
> Wido
>
>
> -Wei
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
> On 05/31/2013 03:59 PM, John Burwell wrote:
>
>
> Wido,
>
>
> +1 -- this enhancement must to discretely support read and write IOPS.
>
> I
>
> don't see how it could be fixed later because I don't see how we
>
> correctly
>
> split total IOPS into read and write.  Therefore, we would be stuck
>
> with a
>
> total unless/until we decided to break backwards compatibility.
>
>
>
> What Wei meant was merging it into master now so that it will go in the
>
> 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
>
> will be released with Read and Write instead of Total IOps.
>
>
> This is to make the May 31st feature freeze date. But if the window moves
>
> (see other threads) then it won't be necessary to do that.
>
>
> Wido
>
>
>
> I also completely agree that there is no association between network
>
> and
>
>
> disk I/O.
>
>
> Thanks,
>
> -John
>
>
> On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
>
>
> Hi Wei,
>
>
>
> On 05/31/2013 03:13 PM, Wei ZHOU wrote:
>
>
> Hi Wido,
>
>
> Thanks. Good question.
>
>
> I  thought about at the beginning. Finally I decided to ignore the
>
> difference of read and write mainly because the network throttling did
>
> not
>
> care the difference of sent and received bytes as well.
>
> That reasoning seems odd. Networking and disk I/O completely different.
>
>
> Disk I/O is much more expensive in most situations then network
>
> bandwith.
>
>
> Implementing it will be some copy-paste work. It could be
>
> implemented in
>
>
> few days. For the deadline of feature freeze, I will implement it
>
> after
>
> that , if needed.
>
>
>
> It think it's a feature we can't miss. But if it goes into the 4.2
>
> window we have to make sure we don't release with only total IOps and
>
> fix
>
> it in 4.3, that will confuse users.
>
>
> Wido
>
>
> -Wei
>
>
>
>
>
>
> 2013/5/31 Wido den Hollander 
>
>
> Hi Wei,
>
>
>
>
> On 05/30/2013 06:03 PM, Wei ZHOU wrote:
>
>
> Hi,
>
>
> I would like to merge disk_io_throttling branch into master.
>
> If nobody object, I will merge into master in 48 hours.
>
> The purpose is :
>
>
> Virtual machines are running on the same storage device (local
>
> storage or
>
> share strage). Because of the rate limitation of device (such as
>
> iops), if
>
> one VM has large disk operation, it may affect the disk performance
>
> of
>
> other VMs running on the same storage device.
>
> It is neccesary to set the maximum rate and limit the disk I/O of
>
> VMs.
>
>
>
> Looking at the code I see you make no difference between Read and
>
> Write
>
> IOps.
>
>
> Qemu and libvirt support setting both a different rate for Read and
>
> Write
>
> IOps which could benefit a lot of users.
>
>
> It's also strange, in the polling side you collect both the Read and
>
> Write
>
> IOps, but on the throttling side you only go for a global value.
>
>
> Write IOps are usually much more expensive then Read IOps, so it
>
> seems
>
> like a valid use-case where that an admin would set a lower value for
>
> write
>
> IOps vs Read IOps.
>
>
> Since this only supports KVM at this point I think it would be of
>
> great
>
> value to at least have the mechanism in place to support both,
>
> implementing
>
> this later would be a lot of work.
>
>
> If a hypervisor doesn't support setting different values for read and
>
> write you can always sum both up and set that as the total limit.
>
>
> Can you explain why you implemented it this way?
>
>
> Wido
>
>
>  The feature includes:

RE: Jetty server fail to start in master branch

[Sateesh Chodapuneedi]

Hi Isaac,
Fixed this issue.
Can you please pull the latest code and re-try?

Regards,
> -Original Message-
> From: Isaac Chiang [mailto:isaacchi...@gmail.com]
> Sent: 03 June 2013 09:21
> To: dev@cloudstack.apache.org
> Subject: Re: Jetty server fail to start in master branch
> 
> Hi all:
>I found that commit : f24e81fe0da let the jetty fail to start in 
> master. Did anyone see this error?
> 
> Here's my steps :
> 
>1. mvn clean install -P developer,systemvm
>2. mvn -pl developer,tools/devcloud -Ddeploydb -P developer
>3. mvn -pl client jetty:run
> 
> 
> 
> 
> Regards
> 
> Isaac
> 
> 
> On Sun, Jun 2, 2013 at 10:37 PM, Isaac Chiang  wrote:
> 
> > Hi all:
> >   I checked out lastest master code. I successfully built the code
> > and deployed the database, but failed to start the jetty server. Did
> > anyone happen to see the error?
> >
> > ERROR [web.context.ContextLoader] (main:) Context initialization
> > failed
> > org.springframework.beans.factory.BeanCreationException: Error
> > creating bean with name 'actionEventUtils': Injection of autowired
> > depen dencies failed; nested exception is
> > org.springframework.beans.factory.BeanCreationException: Could not
> > autowire
> > field: com.cloud.event.
> > dao.EventDao com.cloud.event.ActionEventUtils.eventDao; nested
> > exception is
> > org.springframework.beans.factory.CannotLoadBeanClassExcept
> > ion: Cannot find class
> > [org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy] for
> > bean with name 'vmwareStorageMotionStrate gy' defined in class path
> > resource [applicationContext.xml]; nested exception is
> > java.lang.ClassNotFoundException: org.apache.cloudstac
> > k.storage.motion.VmwareStorageMotionStrategy
> > at
> > org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPo
> > stProcessor.postProcessPropertyValues(AutowiredAnnotat
> > ionBeanPostProcessor.java:287)
> > at
> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > actory.populateBean(AbstractAutowireCapableBeanFactory
> > .java:1106)
> > at
> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > actory.doCreateBean(AbstractAutowireCapableBeanFactory
> > .java:517)
> > at
> > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > actory.createBean(AbstractAutowireCapableBeanFactory.j
> > ava:456)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
> > at
> > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
> > at
> > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
> > at
> > org.springframework.beans.factory.support.DefaultListableBeanFactory.p
> > reInstantiateSingletons(DefaultListableBeanFactory.jav
> > a:609)
> > at
> > org.springframework.context.support.AbstractApplicationContext.finishB
> > eanFactoryInitialization(AbstractApplicationContext.ja
> > va:918)
> >
> >
> > Regards
> >
> > Isaac
> >
> >

Re: VMware API Question

[Mike Tutkowski]

Hi Vijay,

I have a quick question for you. :)

I have about 99% of the VMware-related code written.

This is what it does and how it fits in with my storage plug-in.

* Let's say an admin creates a Disk Offering for a volume that is 1 GB and
has certain Min, Max, and Burst IOPS set.

* An end user later executes this Disk Offering and a row is added to the
volumes table.

* Next the end user wants to attach this disk to a VM running in a VMware
cluster.

* Edison's storage framework realizes that the SAN volume needs to be
created and calls into my plug-in, which creates a 1 GB volume with the
appropriate Min, Max, and Burst IOPS.

* Next VmResource's execute(AttachVolumeCmd) method is invoked.

* My new code recognizes that the type of storage involved is dynamic,
zone-wide storage (dynamic in the sense that a datastore needs to be
created because one does not yet exist for the storage system's volume).

* My new code creates a datastore (basing it on the storage IP address of
the SAN and the IQN of the volume).

* By default, the new datastore will take up all of the space provided by
the SAN volume (this is ideal).

So, here's my question:

The time comes to attach the disk to the VM. I see when we are preparing
the VirtualDisk object that we do not call the setCapacityInKB(long) method
on the disk object.

Do you know what the default size of the disk will be? Ideally it will
consume as much of the datastore as remains (which is probably not the
entire datastore since a little of it is probably already consumed by
VMware metadata).

I'm not sure of the behavior of setCapacityInKB and the docs I've seen are
vague. Perhaps if I call this method, it fixes the size of the disk? I
expect that whatever the behavior, the disk must have a size anyways that
the VM sees.

Thanks!



On Sun, Jun 2, 2013 at 12:01 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Hi Vijay,
>
> I think I understand how to translate this code.
>
> Perhaps you might be willing to do a quick review of what I have when it's
> ready?
>
> Talk to you later,
>
>
> On Sat, Jun 1, 2013 at 8:07 PM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
>> Hi Vijay,
>>
>> I was wondering - if you had a few moments - if you might give me some
>> pointers on how to translate this code I've written in the VI Java API to
>> the VI API, which we use in CloudStack?
>>
>> Below is my createVmfsDatastore method.
>>
>> Thanks for any time you might be able to spend on this!
>>
>>  public static void createVmfsDatastore(String url, String username,
>> String password, String datacenterName, String clusterName,
>>
>>  String datastoreName, String storageIpAddress, Integer portNumber,
>> String iqn) throws Exception
>>
>> {
>>
>>  ServiceInstance si = new ServiceInstance(new URL(url), username,
>> password, true);
>>
>>   ManagedEntity datacenter =
>> si.getSearchIndex().findByInventoryPath(datacenterName);
>>
>>   if (datacenter == null)
>>
>>  {
>>
>>  throw new Exception("Datacenter '" + datacenterName + "' was not found."
>> );
>>
>>  }
>>
>>   InventoryNavigator inventoryNavigator = 
>> newInventoryNavigator(si.getRootFolder());
>>
>>   ClusterComputeResource cluster =
>> (ClusterComputeResource)inventoryNavigator.searchManagedEntity(
>> "ClusterComputeResource", clusterName);
>>
>>   if (cluster == null)
>>
>>  {
>>
>>  throw new Exception("Cluster '" + clusterName + "' was not found.");
>>
>>  }
>>
>>   HostSystem[] aHostSystems = cluster.getHosts();
>>
>>   if (aHostSystems == null || aHostSystems.length == 0)
>>
>>  {
>>
>>  throw new Exception("There are no hosts in cluster '" + clusterName +
>> "'.");
>>
>>  }
>>
>>   HostInternetScsiHbaStaticTarget target = 
>> newHostInternetScsiHbaStaticTarget();
>>
>>   target.setAddress(storageIpAddress);
>>
>>  target.setPort(portNumber);
>>
>>  target.setIScsiName(iqn);
>>
>>   HostInternetScsiHbaAuthenticationProperties auth = 
>> newHostInternetScsiHbaAuthenticationProperties();
>>
>>   String strAuthType = "chapRequired";
>>
>>   auth.setChapAuthEnabled(true);
>>
>>  auth.setChapInherited(false);
>>
>>  auth.setChapAuthenticationType(strAuthType);
>>
>>  auth.setChapName("admin");
>>
>>  auth.setChapSecret("solidfire");
>>
>>  auth.setMutualChapInherited(false);
>>
>>  auth.setMutualChapAuthenticationType(strAuthType);
>>
>>  auth.setMutualChapName("admin2");
>>
>>  auth.setMutualChapSecret("solidfire2");
>>
>>   target.setAuthenticationProperties(auth);
>>
>>   HostInternetScsiHbaStaticTarget[] aTargets = 
>> newHostInternetScsiHbaStaticTarget[1];
>>
>>   aTargets[0] = target;
>>
>>   for (HostSystem hostSystem : aHostSystems)
>>
>>  {
>>
>>  boolean iScsiHbaConfigured = false;
>>
>>HostStorageSystem hostStorageSystem =
>> hostSystem.getHostStorageSystem();
>>
>>for (HostHostBusAdapter hba :
>> hostStorageSystem.getStorageDeviceInfo().getHostBusAdapter())
>>
>>  {
>>
>>   if (hba instanceof HostInternetScsiHba)
>>
>>   {
>>
>>   // just finding an instance of HostInternetScsiHb

Re: Inconsistent error while creating instances

[Nitin Mehta]

Refer to this - 
http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-dev/201303.mb
ox/%3c67ef18fdca335f489b366120481ab6c5f6be740...@banpmailbox01.citrite.net%
3E 

On 01/06/13 4:18 AM, "Ahmad Emneina"  wrote:

>I've hit this in the past when there was a mismatch between the actual
>guest os and what i provided as the guest os type (in cloudstack).
>
>
>On Fri, May 31, 2013 at 8:30 AM, Pranav Saxena
>wrote:
>
>> Are your systemVms up and running fine ? Is your XS host a brand new
>>setup
>> ? Do you have the vhd-util file present at /opt/xensource/bin in your
>>host ?
>>
>> -Original Message-
>> From: Gaurav Aradhye [mailto:gaurav.arad...@clogeny.com]
>> Sent: Friday, May 31, 2013 7:11 PM
>> To: dev@cloudstack.apache.org
>> Subject: Inconsistent error while creating instances
>>
>> Hi,
>>
>> I am getting following error while creating instances through test
>>cases.
>> The error is not reproducible each time, but the typical scenario is
>>while
>> creating two or more instances consecutively.
>>
>> Error info says "*Bootloader.Bad_error*"
>>
>> The error is produced for each available host, eventually adding that
>>host
>> to "Avoid Set" and finally it fails to create the instance because every
>> host is in avoid set and the management log gives exception -
>> "InsufficientServerCapacityException".
>>
>> Has anybody encountered this error before? It will be great if someone
>>can
>> provide directions on this.
>>
>> *Error details:*
>> *
>> *
>> WARN [xen.resource.CitrixResourceBase] (DirectAgent-155:null) Unable to
>> start i-553-469-QA due to
>> com.cloud.utils.exception.CloudRuntimeException: Unable to start
>> VM(i-553-469-QA) on host(35d9b483-9a19-4a91-a0a6-bfe138e79db4) due to
>>Task
>> failed! Task record:
>>  uuid: 9e350b27-f9a5-30c0-bad8-d23bc7f80568
>>nameLabel: Async.VM.start_on
>>  nameDescription:
>>allowedOperations: []
>>currentOperations: {}
>>  created: Sun May 12 17:58:14 PDT 2013
>> finished: Sun May 12 17:58:19 PDT 2013
>>   status: failure
>>   residentOn: com.xensource.xenapi.Host@5848f813
>> progress: 1.0
>> type: 
>>   result:
>>errorInfo: [INTERNAL_ERROR, xenopsd internal error: VM =
>> 1110c003-1d45-5ab5-226f-952aa95630a0; domid = 393; Bootloader.Bad_error
>> Traceback (most rece nt call last):
>>   File "/usr/bin/pygrub", line 900, in ?
>> fs = fsimage.open(file, part_offs[0], bootfsoptions)
>> IOError: [Errno 95] Operation not supported ]
>>  otherConfig: {}
>>subtaskOf: com.xensource.xenapi.Task@aaf13f6f
>> subtasks: []
>>
>>
>> Regards,
>> Gaurav Aradhye
>>

Re: Jetty server fail to start in master branch

[Isaac Chiang]

It works, thanks :)




Regards

Isaac



On Mon, Jun 3, 2013 at 1:14 PM, Sateesh Chodapuneedi <
sateesh.chodapune...@citrix.com> wrote:

> Hi Isaac,
> Fixed this issue.
> Can you please pull the latest code and re-try?
>
> Regards,
> > -Original Message-
> > From: Isaac Chiang [mailto:isaacchi...@gmail.com]
> > Sent: 03 June 2013 09:21
> > To: dev@cloudstack.apache.org
> > Subject: Re: Jetty server fail to start in master branch
> >
> > Hi all:
> >I found that commit : f24e81fe0da let the jetty fail to start in
> master. Did anyone see this error?
> >
> > Here's my steps :
> >
> >1. mvn clean install -P developer,systemvm
> >2. mvn -pl developer,tools/devcloud -Ddeploydb -P developer
> >3. mvn -pl client jetty:run
> >
> >
> >
> >
> > Regards
> >
> > Isaac
> >
> >
> > On Sun, Jun 2, 2013 at 10:37 PM, Isaac Chiang 
> wrote:
> >
> > > Hi all:
> > >   I checked out lastest master code. I successfully built the code
> > > and deployed the database, but failed to start the jetty server. Did
> > > anyone happen to see the error?
> > >
> > > ERROR [web.context.ContextLoader] (main:) Context initialization
> > > failed
> > > org.springframework.beans.factory.BeanCreationException: Error
> > > creating bean with name 'actionEventUtils': Injection of autowired
> > > depen dencies failed; nested exception is
> > > org.springframework.beans.factory.BeanCreationException: Could not
> > > autowire
> > > field: com.cloud.event.
> > > dao.EventDao com.cloud.event.ActionEventUtils.eventDao; nested
> > > exception is
> > > org.springframework.beans.factory.CannotLoadBeanClassExcept
> > > ion: Cannot find class
> > > [org.apache.cloudstack.storage.motion.VmwareStorageMotionStrategy] for
> > > bean with name 'vmwareStorageMotionStrate gy' defined in class path
> > > resource [applicationContext.xml]; nested exception is
> > > java.lang.ClassNotFoundException: org.apache.cloudstac
> > > k.storage.motion.VmwareStorageMotionStrategy
> > > at
> > > org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPo
> > > stProcessor.postProcessPropertyValues(AutowiredAnnotat
> > > ionBeanPostProcessor.java:287)
> > > at
> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > > actory.populateBean(AbstractAutowireCapableBeanFactory
> > > .java:1106)
> > > at
> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > > actory.doCreateBean(AbstractAutowireCapableBeanFactory
> > > .java:517)
> > > at
> > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanF
> > > actory.createBean(AbstractAutowireCapableBeanFactory.j
> > > ava:456)
> > > at
> > >
> org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
> > > at
> > >
> org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
> > > at
> > >
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
> > > at
> > >
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
> > > at
> > > org.springframework.beans.factory.support.DefaultListableBeanFactory.p
> > > reInstantiateSingletons(DefaultListableBeanFactory.jav
> > > a:609)
> > > at
> > > org.springframework.context.support.AbstractApplicationContext.finishB
> > > eanFactoryInitialization(AbstractApplicationContext.ja
> > > va:918)
> > >
> > >
> > > Regards
> > >
> > > Isaac
> > >
> > >
>

Re: [MERGE] disk_io_throttling to MASTER

[Wei ZHOU]

Hi John, Mike

I hope Mike's aswer helps you. I am trying to adding more.

(1) I think billing should depend on IO statistics rather than IOPS
limitation. Please review disk_io_stat if you have time.   disk_io_stat can
get the IO statistics including bytes/iops read/write for an individual
virtual machine.

(2) Do you mean IOPS runtime change? KVM supports setting IOPS/BPS
limitation for a running virtual machine through command line. However,
CloudStack does not support changing the parameters of a created offering
(computer offering or disk offering).

(3) It is a good question. Maybe it is better to commit Mike's patch after
disk_io_throttling as Mike needs to consider the limitation in hypervisor
type, I think.

- Wei


2013/6/3 John Burwell 

> Mike,
>
> The things I want to understand are the following:
>
>1) Is there value in capturing IOPS policies be captured in a common
> data model (e.g. for billing/usage purposes, expressing offerings).
> 2) Should there be a common interface model for reasoning about IOP
> provisioning at runtime?
> 3) How are conflicting provisioned IOPS configurations between a
> hypervisor and storage device reconciled?  In particular, a scenario where
> is lead to believe (and billed) for more IOPS configured for a VM than a
> storage device has been configured to deliver.  Another scenario could a
> consistent configuration between a VM and a storage device at creation
> time, but a later modification to storage device introduces logical
> inconsistency.
>
> Thanks,
> -John
>
> On Jun 2, 2013, at 8:38 PM, Mike Tutkowski 
> wrote:
>
> Hi John,
>
> I believe Wei's feature deals with controlling the max number of IOPS from
> the hypervisor side.
>
> My feature is focused on controlling IOPS from the storage system side.
>
> I hope that helps. :)
>
>
> On Sun, Jun 2, 2013 at 6:35 PM, John Burwell  wrote:
>
> > Wei,
> >
> > My opinion is that no features should be merged until all functional
> > issues have been resolved and it is ready to turn over to test.  Until
> the
> > total Ops vs discrete read/write ops issue is addressed and re-reviewed
> by
> > Wido, I don't think this criteria has been satisfied.
> >
> > Also, how does this work intersect/compliment the SolidFire patch (
> > https://reviews.apache.org/r/11479/)?  As I understand it that work is
> > also involves provisioned IOPS.  I would like to ensure we don't have a
> > scenario where provisioned IOPS in KVM and SolidFire are unnecessarily
> > incompatible.
> >
> > Thanks,
> > -John
> >
> > On Jun 1, 2013, at 6:47 AM, Wei ZHOU  wrote:
> >
> > Wido,
> >
> >
> > Sure. I will change it next week.
> >
> >
> > -Wei
> >
> >
> >
> > 2013/6/1 Wido den Hollander 
> >
> >
> > Hi Wei,
> >
> >
> >
> > On 06/01/2013 08:24 AM, Wei ZHOU wrote:
> >
> >
> > Wido,
> >
> >
> > Exactly. I have pushed the features into master.
> >
> >
> > If anyone object thems for technical reason till Monday, I will revert
> >
> > them.
> >
> >
> > For the sake of clarity I just want to mention again that we should
> change
> >
> > the total IOps to R/W IOps asap so that we never release a version with
> >
> > only total IOps.
> >
> >
> > You laid the groundwork for the I/O throttling and that's great! We
> should
> >
> > however prevent that we create legacy from day #1.
> >
> >
> > Wido
> >
> >
> > -Wei
> >
> >
> >
> > 2013/5/31 Wido den Hollander 
> >
> >
> > On 05/31/2013 03:59 PM, John Burwell wrote:
> >
> >
> > Wido,
> >
> >
> > +1 -- this enhancement must to discretely support read and write IOPS.
> >
> > I
> >
> > don't see how it could be fixed later because I don't see how we
> >
> > correctly
> >
> > split total IOPS into read and write.  Therefore, we would be stuck
> >
> > with a
> >
> > total unless/until we decided to break backwards compatibility.
> >
> >
> >
> > What Wei meant was merging it into master now so that it will go in the
> >
> > 4.2 branch and add Read / Write IOps before the 4.2 release so that 4.2
> >
> > will be released with Read and Write instead of Total IOps.
> >
> >
> > This is to make the May 31st feature freeze date. But if the window moves
> >
> > (see other threads) then it won't be necessary to do that.
> >
> >
> > Wido
> >
> >
> >
> >  I also completely agree that there is no association between network
> >
> > and
> >
> >
> > disk I/O.
> >
> >
> > Thanks,
> >
> > -John
> >
> >
> > On May 31, 2013, at 9:51 AM, Wido den Hollander  wrote:
> >
> >
> >  Hi Wei,
> >
> >
> >
> > On 05/31/2013 03:13 PM, Wei ZHOU wrote:
> >
> >
> > Hi Wido,
> >
> >
> > Thanks. Good question.
> >
> >
> > I  thought about at the beginning. Finally I decided to ignore the
> >
> > difference of read and write mainly because the network throttling did
> >
> > not
> >
> > care the difference of sent and received bytes as well.
> >
> > That reasoning seems odd. Networking and disk I/O completely different.
> >
> >
> > Disk I/O is much more expensive in most situations then network
> >
> > bandwith.
> >
> >
> >  Implementing i

Is it nessesary to test getBase64Keystore 100000 times in ConfigurationServerImplTest.java

[Wei ZHOU]

The build of latest master branch spent much more time on
ConfigurationServerImplTest.java, which includes test getBase64Keystore
10 times.

Is it neccesary?

[root@weizhou-centos incubator-cloudstack]# git diff
2f29185943ac0412aa501b59493837b4055642e0
4894187991d581b72807b4282b7a29a48a8031e5
+@Test
+public void testGetBase64KeystoreZillionTimes() throws IOException {
+File temp = File.createTempFile("keystore", "");
+try {
+// may cause IOException with the original implementation
because of too many open files
+for (int i = 0; i < 10; i++) {
+FileUtils.writeStringToFile(temp,
Base64.encodeBase64String(TEST.getBytes()));
+final String keystore =
ConfigurationServerImpl.getBase64Keystore(temp.getPath());
+// let's decode it to make sure it makes sense
+Base64.decodeBase64(keystore);
+}
+} finally {
+temp.delete();
+}
+}

Re: Is it nessesary to test getBase64Keystore 100000 times in ConfigurationServerImplTest.java

[Prasanna Santhanam]

On Mon, Jun 03, 2013 at 08:19:41AM +0200, Wei ZHOU wrote:
> The build of latest master branch spent much more time on
> ConfigurationServerImplTest.java, which includes test getBase64Keystore
> 10 times.
> 
> Is it neccesary?
> 
> [root@weizhou-centos incubator-cloudstack]# git diff
> 2f29185943ac0412aa501b59493837b4055642e0
> 4894187991d581b72807b4282b7a29a48a8031e5
> +@Test
> +public void testGetBase64KeystoreZillionTimes() throws IOException {
> +File temp = File.createTempFile("keystore", "");
> +try {
> +// may cause IOException with the original implementation
> because of too many open files
> +for (int i = 0; i < 10; i++) {
> +FileUtils.writeStringToFile(temp,
> Base64.encodeBase64String(TEST.getBytes()));
> +final String keystore =
> ConfigurationServerImpl.getBase64Keystore(temp.getPath());
> +// let's decode it to make sure it makes sense
> +Base64.decodeBase64(keystore);
> +}
> +} finally {
> +temp.delete();
> +}
> +}

My bad - feel free to fix the test. I see it's taking up a minute on
the build server too -

http://jenkins.buildacloud.org/view/master/job/cloudstack-master/org.apache.cloudstack$cloud-server/488/testReport/com.cloud.server/ConfigurationServerImplTest/


-- 
Prasanna.,


Powered by BigRock.com