Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?

[Hieu LE]

Hi Mike,

Done that, I have added new FAQ section.

In addition, I have tested that volume can take a snapshot, create volume
from that snapshot and attach back to VM normally. Currently I am testing
with volume migration.


On Fri, Jun 6, 2014 at 11:11 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Hi Hieu,
>
> Would you be able to place these questions and answers in your design doc
> so that we can more easily track them?
>
> Thanks!
> Mike
>
>
> On Thu, Jun 5, 2014 at 9:55 PM, Hieu LE  wrote:
>
> > Hi Todd,
> >
> >
> > On Fri, Jun 6, 2014 at 9:17 AM, Todd Pigram  wrote:
> >
> > > Hieu,
> > >
> > > I assume you are using MCS for you golden image? What version of XD?
> > Given
> > > you are using pooled desktops, have you thought about using a PVS BDM
> iso
> > > and mount it with in your 1000 VMs? This way you can stagger reboots
> via
> > > PVS console or Studio. This would require a change to your delivery
> > group.
> > >
> > >
> > Sorry but I did not use MCS or XenDesktop in my company :-)
> >
> >
> > >
> > > On Thu, Jun 5, 2014 at 9:28 PM, Mike Tutkowski <
> > > mike.tutkow...@solidfire.com
> > > > wrote:
> > >
> > > > 6) The copy_vhd_from_secondarystorage XenServer plug-in is not used
> > when
> > > > you're using XenServer + XS62ESP1 + XS62ESP1004. In that case, please
> > > refer
> > > > to copyTemplateToPrimaryStorage(CopyCommand) method in the
> > > > Xenserver625StorageProcessor class.
> > > >
> > >
> >
> > Thank Mike, I will take note of that.
> >
> >
> > >  >
> > > > On Thu, Jun 5, 2014 at 1:56 PM, Mike Tutkowski <
> > > > mike.tutkow...@solidfire.com
> > > > > wrote:
> > > >
> > > > > Other than going through a "for" loop and deploying VM after VM, I
> > > don't
> > > > > think CloudStack currently supports a bulk-VM-deploy operation.
> > > > >
> > > > > It would be nice if CS did so at some point in the future; however,
> > > that
> > > > > is probably a separate proposal from Hieu's.
> > > > >
> > > > >
> > > > > On Thu, Jun 5, 2014 at 12:13 AM, Amit Das 
> > > > wrote:
> > > > >
> > > > >> Hi Hieu,
> > > > >>
> > > > >> Will it be good to include bulk operation of this feature? In
> > > addition,
> > > > >> does Xen support parallel execution of these operations ?
> > > > >>
> > > > >> Regards,
> > > > >> Amit
> > > > >> *CloudByte Inc.* 
> > > > >>
> > > > >>
> > > > >> On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE 
> wrote:
> > > > >>
> > > > >> > Mike, Punith,
> > > > >> >
> > > > >> > Please review "Golden Primary Storage" proposal. [1]
> > > > >> >
> > > > >> > Thank you.
> > > > >> >
> > > > >> > [1]:
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage
> > > > >> >
> > > > >> >
> > > > >> > On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski <
> > > > >> > mike.tutkow...@solidfire.com> wrote:
> > > > >> >
> > > > >> >> Daan helped out with this. You should be good to go now.
> > > > >> >>
> > > > >> >>
> > > > >> >> On Tue, Jun 3, 2014 at 8:50 PM, Hieu LE 
> > > wrote:
> > > > >> >>
> > > > >> >> > Hi Mike,
> > > > >> >> >
> > > > >> >> > Could you please give edit/create permission on ASF Jira/Wiki
> > > > >> >> confluence ?
> > > > >> >> > I can not add a new Wiki page.
> > > > >> >> >
> > > > >> >> > My Jira ID: hieulq
> > > > >> >> > Wiki: hieulq89
> > > > >> >> > Review Board: hieulq
> > > > >> >> >
> > > > >> >> > Thanks !
> > > > >> >> >
> > > > >> >> >
> > > > >> >> > On Wed, Jun 4, 2014 at 9:17 AM, Mike Tutkowski <
> > > > >> >> > mike.tutkow...@solidfire.com
> > > > >> >> > > wrote:
> > > > >> >> >
> > > > >> >> > > Hi,
> > > > >> >> > >
> > > > >> >> > > Yes, please feel free to add a new Wiki page for your
> design.
> > > > >> >> > >
> > > > >> >> > > Here is a link to applicable design info:
> > > > >> >> > >
> > > > >> >> > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Design
> > > > >> >> > >
> > > > >> >> > > Also, feel free to ask more questions and have me review
> your
> > > > >> design.
> > > > >> >> > >
> > > > >> >> > > Thanks!
> > > > >> >> > > Mike
> > > > >> >> > >
> > > > >> >> > >
> > > > >> >> > > On Tue, Jun 3, 2014 at 7:29 PM, Hieu LE <
> hieul...@gmail.com>
> > > > >> wrote:
> > > > >> >> > >
> > > > >> >> > > > Hi Mike,
> > > > >> >> > > >
> > > > >> >> > > > You are right, performance will be decreased over time
> > > because
> > > > >> >> writes
> > > > >> >> > > IOPS
> > > > >> >> > > > will always end up on slower storage pool.
> > > > >> >> > > >
> > > > >> >> > > > In our case, we are using CloudStack integrated in VDI
> > > solution
> > > > >> to
> > > > >> >> > > provived
> > > > >> >> > > > pooled VM type[1]. So may be my approach can bring better
> > UX
> > > > for
> > > > >> >> user
> > > > >> >> > > with
> > > > >> >> > > > lower bootime ...
> > > > >> >> > > >
> > > > >> >> > > > A short change in design are followings
> > > > >> >> > > > - VM will be deployed with golden primary storage if
> > primary
> > > > >> >> storag

Re: SNAT iptable entry on VirtualRouter

[Sachchidanand Vaidya]

Hi,
   I'm seeing the same issue with 4.4 code.  After further debug, I see
that CS mgmt server is sending
following command to XenHost,
xensource.log:  /opt/cloud/bin/ipassoc.sh -A -s -f -l
10.84.59.131/24 -c eth0 -g 10.84.59.254 
VirtualRouter's public interface is eth2. Also as per dump of VIF list on
XenHost, deviceid for public interface
of domainRouter is 2.
As part of VirtualRoutingResource.java:generateConfig(), CS mgmt server
generates this command.
It generates publicNic = "eth" + ip.getNicDevId()?
Which deviceId does it refer to? Shouldn't it be the deviceid as per the
XenHost dump?
Does anyone have input on what could he happening here ?

Thanks,
Sachin



>Hi,
>I have an isolated network (192.168.3.x/24) being served by
>VirtualRouter, where 10.84.59.131 is SourceNAT address
>and eth0 is VN interface of VirtualRouter & eth2 is  the public interface
>of VirtualRouter.
>
> When I look at the nat table entries on the VirtualRouter, it shows
>following :
>
>root@r-6-VM:~# iptables -L -t nat -n -v
>..
>..
>Chain POSTROUTING (policy ACCEPT 330 packets, 22113 bytes)
> pkts bytes target prot opt in out source
>destination
>0 0 SNAT   all  --  *  eth00.0.0.0/0
>0.0.0.0/0to:10.84.59.131
>
>--> Why the "out" interface for the SNAT entry is VN interface (eth0)
>instead of Public interface (eth2) ?
>
>I'm using "Cloudstack Release 4.3.0 (64-bit) Thu Apr 10 20:27:11 UTC
>2014" cloudstack-release template.
>
>Thanks,
>Sachin
>
>---
>root@r-6-VM:~# ifconfig
>eth0  Link encap:Ethernet  HWaddr 02:13:87:88:e6:dd
>  inet addr:192.168.3.226  Bcast:192.168.3.255  Mask:255.255.255.0
>  inet6 addr: fe80::13:87ff:fe88:e6dd/64 Scope:Link
>  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>  RX packets:350 errors:0 dropped:0 overruns:0 frame:0
>  TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
>  collisions:0 txqueuelen:1000
>  RX bytes:29400 (28.7 KiB)  TX bytes:602 (602.0 B)
>  Interrupt:25
>
>eth1  Link encap:Ethernet  HWaddr 0e:00:a9:fe:02:6b
>  inet addr:169.254.2.107  Bcast:169.254.255.255  Mask:255.255.0.0
>  inet6 addr: fe80::c00:a9ff:fefe:26b/64 Scope:Link
>  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>  RX packets:3293 errors:0 dropped:0 overruns:0 frame:0
>  TX packets:2934 errors:0 dropped:0 overruns:0 carrier:0
>  collisions:0 txqueuelen:1000
>  RX bytes:444768 (434.3 KiB)  TX bytes:539100 (526.4 KiB)
>  Interrupt:26
>
>eth2  Link encap:Ethernet  HWaddr 06:d5:1c:00:00:0b
>  inet addr:10.84.59.131  Bcast:10.84.59.255  Mask:255.255.255.0
>  inet6 addr: fe80::4d5:1cff:fe00:b/64 Scope:Link
>  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>  RX packets:133 errors:0 dropped:0 overruns:0 frame:0
>  TX packets:1451 errors:0 dropped:0 overruns:0 carrier:0
>  collisions:0 txqueuelen:1000
>  RX bytes:4636 (4.5 KiB)  TX bytes:113282 (110.6 KiB)
>  Interrupt:27
>
>

Re: Review Request 20316: CLOUDSTACK-1466: Adding automation test cases for Primary Storage Limits

[Gaurav Aradhye]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20316/
---

(Updated June 6, 2014, 2:15 p.m.)


Review request for cloudstack, Girish Shilamkar and Santhosh Edukulla.


Changes
---

Review changes.


Bugs: CLOUDSTACK-1466
https://issues.apache.org/jira/browse/CLOUDSTACK-1466


Repository: cloudstack-git


Description
---

Adding test suits in Primary Storage Limits test cases.
1)Root/Domain admin limits
2)Domain Limits
3)Resize volume
4)Project Limits
5)Maximum Limits


Diffs (updated)
-

  test/integration/component/test_ps_domain_limits.py PRE-CREATION 
  test/integration/component/test_ps_limits.py PRE-CREATION 
  test/integration/component/test_ps_max_limits.py PRE-CREATION 
  test/integration/component/test_ps_project_limits.py PRE-CREATION 
  test/integration/component/test_ps_resize_volume.py PRE-CREATION 
  tools/marvin/marvin/codes.py ef49c0c 
  tools/marvin/marvin/lib/base.py 2681724 
  tools/marvin/marvin/lib/common.py 91fe053 

Diff: https://reviews.apache.org/r/20316/diff/


Testing
---

Yes


Thanks,

Gaurav Aradhye

Re: Realhostip service extended till Sep 30th

[Andrija Panic]

That's:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replace+realhostip.com+with+Your+Own+Domain+Name
thanks


On 6 June 2014 04:03, Nitin Mehta  wrote:

> Please find wiki [1] containing the procedure to replace realhostip.com
> with Your own custom domain name.
>
> [1]
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replace
> +realhostip.com+with+Your+Own+Domain+Name
>
>
> Thanks,
> -Nitin
>
> On 05/06/14 4:23 PM, "Animesh Chaturvedi" 
> wrote:
>
> >
> >Folks
> >
> >I wanted to provide an update on shutting down of RealhostIp service.
> >Citrix has decided to move the date by one quarter and the new date will
> >be Sep 30th.  The new dates allow users of CloudStack additional time for
> >updating their infrastructure. While testing Realhostip fixes additional
> >product issues were found which are tracked in following three defects
> >CLOUDSTACK-6499 [1], CLOUDSTACK-6599 [2], CLOUDSTACK-6824 [3] . These
> >defects are fixed in 4.4 and master branch. Please verify in your setups.
> >If you are on older version of CloudStack do plan on upgrading to 4.4 .
> >Additional details on fixes are available in JIRA.  Updated wiki with
> >steps will be published shortly.
> >
> >[1] https://issues.apache.org/jira/browse/CLOUDSTACK-6499
> >[2] https://issues.apache.org/jira/browse/CLOUDSTACK-6599
> >[3] https://issues.apache.org/jira/browse/CLOUDSTACK-6824
> >
> >
> >Thanks
> >Animesh
> >
> >
> >
> >
> >> -Original Message-
> >> From: John Kinsella [mailto:j...@stratosec.co]
> >> Sent: Thursday, April 17, 2014 8:35 AM
> >> To: ; users...@cloudstack.apache.org;
> >> 
> >> Subject: REMINDER RealhostIP going away
> >>
> >> Reminder, folks - please migrate off RealhostIP.com or you¹re going to
> >>get a
> >> nasty surprise this summer. More info at link below.
> >>
> >>
> >>
> https://blogs.apache.org/cloudstack/entry/RealhostIP_service_is_being_ret
> >> ired
>
>


-- 

Andrija Panić
--
  http://admintweets.com
--

Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Santhosh Edukulla]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/
---

Review request for cloudstack, daan Hoogland and Kishan Kavala.


Bugs: CLOUDSTACK-6793
https://issues.apache.org/jira/browse/CLOUDSTACK-6793


Repository: cloudstack-git


Description
---

1. Added fix to retrieve the correct domain id for a given accountid. 
vm_template table and respective vo, does not have a domain id information.
2. Added logic to retrieve correct domainid for a given accountid, only when an 
invalid domainid is retrieved. Throw an exception in case of invalid domain id.
3. Now, creation of tags is getting successful post the fix.


Diffs
-

  engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
  engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
  server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 

Diff: https://reviews.apache.org/r/19874/diff/


Testing
---

Test creation, listing and deletion tags on UserVM ... === TestName: 
test_05_vm_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tag on templates ... === TestName: 
test_06_template_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tags on ISO ... === TestName: 
test_07_iso_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tagson volume ... === TestName: 
test_08_volume_tag | Status : SUCCESS ===
ok


Thanks,

Santhosh Edukulla

Re: Review Request 20316: CLOUDSTACK-1466: Adding automation test cases for Primary Storage Limits

[Gaurav Aradhye]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/20316/
---

(Updated June 6, 2014, 3:10 p.m.)


Review request for cloudstack, Girish Shilamkar and Santhosh Edukulla.


Bugs: CLOUDSTACK-1466
https://issues.apache.org/jira/browse/CLOUDSTACK-1466


Repository: cloudstack-git


Description
---

Adding test suits in Primary Storage Limits test cases.
1)Root/Domain admin limits
2)Domain Limits
3)Resize volume
4)Project Limits
5)Maximum Limits


Diffs (updated)
-

  test/integration/component/test_ps_domain_limits.py PRE-CREATION 
  test/integration/component/test_ps_limits.py PRE-CREATION 
  test/integration/component/test_ps_max_limits.py PRE-CREATION 
  test/integration/component/test_ps_project_limits.py PRE-CREATION 
  test/integration/component/test_ps_resize_volume.py PRE-CREATION 
  tools/marvin/marvin/codes.py ef49c0c 
  tools/marvin/marvin/lib/base.py 2681724 
  tools/marvin/marvin/lib/common.py 91fe053 

Diff: https://reviews.apache.org/r/20316/diff/


Testing
---

Yes


Thanks,

Gaurav Aradhye

Re: [ACS44] blocker issues?

[Daan Hoogland]

Sudha,

- The point is that there are bugs being marked as blockers that where
never discussed on this list. Someone has decided all by their own
they are blocker and didn't ask for feedback. I don't find that
acceptable.
- My colleague Wilder has done some validation. I didn't memorize his
report but I am sure his testing was more then just the release
validation from the wiki.
- We run an integration test which is base on our own marvin scripting
not on the automation at Schuberg Philis.

 So what remains is we should not release with blockers. no duh.

On Thu, Jun 5, 2014 at 9:39 PM, Sudha Ponnaganti
 wrote:
> Daan,
>
> - 1 on releasing with blockers
>
> - We have to get these fixed. People doesn’t get to blockers doesn’t  make 
> severity any less.
> - Also I did not see any upgrade validation done in community. Would be good 
> if someone confirms that they have done it already.
> - Do you know if anyone is running automation on 4.4 branch in their local 
> environments?  It would be good if someone post their results if they are 
> running automation in their local environments.
>
> Thanks
> /Sudha
>
> -Original Message-
> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> Sent: Thursday, June 05, 2014 8:24 AM
> To: dev
> Subject: [ACS44] blocker issues?
>
> LS,
>
> Here is the resent list with blockers at the moment, sorted by last modified.
>
> CLOUDSTACK-6623;Register template does not work as expected, when deploying 
> simulator and xen zones simultaneously on a single management server.;Bharat 
> Kumar;edison su;22/May/14 CLOUDSTACK-6755;[OVS] Can't create more than 7 GRE 
> tunnel networks in xen cluster;Sanjeev N;Murali Reddy;23/May/14 
> CLOUDSTACK-6779;[OVS] Expunging VM (deleting vif) deletes all the rules from 
> ovs bridge flow table;Sanjeev N;Murali Reddy;27/May/14
>
> CLOUDSTACK-6804;[Automation] Add host fails in KVM;Rayees Namathponnan;Kishan 
> Kavala;02/Jun/14 CLOUDSTACK-6603;[Upgrade]DB Exception while Autoscale 
> monitoring after upgrading from 4.3 to 4.4;manasaveloori;Rajesh 
> Battala;02/Jun/14 CLOUDSTACK-6602;[UI] createNetworkACL API action param 
> value passed incorrectly;Jayapal Reddy;Jessica Wang;03/Jun/14 
> CLOUDSTACK-6710;[Automation] VM snapshot failing with NPE in vmware;Rayees 
> Namathponnan;Amogh Vasekar;04/Jun/14 CLOUDSTACK-6842;[Automation] Detach 
> volume fails with
> LibvirtException: Storage pool not found: no storage pool with matching 
> uuid;Rayees Namathponnan;Unassigned;04/Jun/14 CLOUDSTACK-6791;[Automation] 
> DeleteNetworkCmd fails with NullPointerException;Rayees 
> Namathponnan;Unassigned;04/Jun/14 CLOUDSTACK-6825;[Automation] Create 
> template from snapshot fails with NPE in KVM;Rayees 
> Namathponnan;Unassigned;05/Jun/14
>
> The dates that you see are the last updates of the issues.
>
> Given that we are in release period I can not take an issue that has not been 
> for more then a week serious as blocker. What can we say about the rest. I 
> don't hear to much noise on the list about blockers.
> These are the most serious things we should be talking about.
>
> Unless some serious fuzz arises on list I move to mark all of the issues as 
> known issues and go ahead and release.
>
> --
> Daan



-- 
Daan

Re: [ACS44] blocker issues?

[Daan Hoogland]

in so much words, yes please review them. All of you!

On Thu, Jun 5, 2014 at 9:44 PM, Mike Tutkowski
 wrote:
> What I think we should do is ask people to review the list of blockers and
> provide comments (if you feel knowledgeable enough in that area) as to
> whether or not we still feel these are blockers.
>
> Getting more eyes on this list might shed additional light on if these are,
> in fact, true blockers.
>
>
> On Thu, Jun 5, 2014 at 1:39 PM, Sudha Ponnaganti <
> sudha.ponnaga...@citrix.com> wrote:
>
>> Daan,
>>
>> - 1 on releasing with blockers
>>
>> - We have to get these fixed. People doesn’t get to blockers doesn’t  make
>> severity any less.
>> - Also I did not see any upgrade validation done in community. Would be
>> good if someone confirms that they have done it already.
>> - Do you know if anyone is running automation on 4.4 branch in their local
>> environments?  It would be good if someone post their results if they are
>> running automation in their local environments.
>>
>> Thanks
>> /Sudha
>>
>> -Original Message-
>> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
>> Sent: Thursday, June 05, 2014 8:24 AM
>> To: dev
>> Subject: [ACS44] blocker issues?
>>
>> LS,
>>
>> Here is the resent list with blockers at the moment, sorted by last
>> modified.
>>
>> CLOUDSTACK-6623;Register template does not work as expected, when
>> deploying simulator and xen zones simultaneously on a single management
>> server.;Bharat Kumar;edison su;22/May/14 CLOUDSTACK-6755;[OVS] Can't create
>> more than 7 GRE tunnel networks in xen cluster;Sanjeev N;Murali
>> Reddy;23/May/14 CLOUDSTACK-6779;[OVS] Expunging VM (deleting vif) deletes
>> all the rules from ovs bridge flow table;Sanjeev N;Murali Reddy;27/May/14
>>
>> CLOUDSTACK-6804;[Automation] Add host fails in KVM;Rayees
>> Namathponnan;Kishan Kavala;02/Jun/14 CLOUDSTACK-6603;[Upgrade]DB Exception
>> while Autoscale monitoring after upgrading from 4.3 to
>> 4.4;manasaveloori;Rajesh Battala;02/Jun/14 CLOUDSTACK-6602;[UI]
>> createNetworkACL API action param value passed incorrectly;Jayapal
>> Reddy;Jessica Wang;03/Jun/14 CLOUDSTACK-6710;[Automation] VM snapshot
>> failing with NPE in vmware;Rayees Namathponnan;Amogh Vasekar;04/Jun/14
>> CLOUDSTACK-6842;[Automation] Detach volume fails with
>> LibvirtException: Storage pool not found: no storage pool with matching
>> uuid;Rayees Namathponnan;Unassigned;04/Jun/14 CLOUDSTACK-6791;[Automation]
>> DeleteNetworkCmd fails with NullPointerException;Rayees
>> Namathponnan;Unassigned;04/Jun/14 CLOUDSTACK-6825;[Automation] Create
>> template from snapshot fails with NPE in KVM;Rayees
>> Namathponnan;Unassigned;05/Jun/14
>>
>> The dates that you see are the last updates of the issues.
>>
>> Given that we are in release period I can not take an issue that has not
>> been for more then a week serious as blocker. What can we say about the
>> rest. I don't hear to much noise on the list about blockers.
>> These are the most serious things we should be talking about.
>>
>> Unless some serious fuzz arises on list I move to mark all of the issues
>> as known issues and go ahead and release.
>>
>> --
>> Daan
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> *™*



-- 
Daan

Re: [ACS4.3.1][DISCUSS] Minor release of Apache CloudStack 4.3.1

[Daan Hoogland]

Ilya,

You are addressing non developpers, so I am crossposting.

On Fri, Jun 6, 2014 at 4:25 AM, ilya musayev
 wrote:
> Creating a separate thread to see what community thinks on next minor
> release of CloudStack 4.3.1
>
> There are more patches that need to be cherry-picked/backported - i can
> think of several for vmware-related ones that did not make it into 4.3
> branch.
>
> How do we go about finding the ones that did not make it?
>
> I'd like to help with release of 4.3.1, but i will need help of at least 1
> or 2 folks.
>
> You dont need to be a developer to help with release process.
>
>



-- 
Daan

Re: [ACS44] Cherry pick for CLOUDSTACK-6676 and CLOUDSTACK-6853

[Daan Hoogland]

three pulled,
i noticed windows line endings and trailing space in
server/test/resources/createNetworkOffering.xml
please take care.

On Fri, Jun 6, 2014 at 4:57 AM, Alena Prokharchyk
 wrote:
> And one more – fixing the unit test (sorry for that)
>
> commit 96bfa8a73f0054db42beaff6a8ba3af058d3d915
> Author: Alena Prokharchyk 
> Date:   Thu Jun 5 18:20:33 2014 -0700
>
> Fixed unittest
>
> -Alena.
>
> From: Alena Prokharchyk 
> Date: Thursday, June 5, 2014 at 5:45 PM
> To: Daan Hoogland 
> Cc: "dev@cloudstack.apache.org" 
> Subject: [ACS44] Cherry pick for CLOUDSTACK-6676 and CLOUDSTACK-6853
>
> Daan, can you please cherry-pick the commits below to 4.4 branch? Thank you!
>
> commit 66df43572e1a12ad2bd32f8c0aa9acaeb271828e
> Author: Alena Prokharchyk 
> Date:   Thu Jun 5 17:26:01 2014 -0700
>
> CLOUDSTACK-6676: cleanup ip address details on ip disassoc
>
> commit 03623fe57e6597782b628815140283151d066a36
> Author: Alena Prokharchyk 
> Date:   Thu Jun 5 17:07:52 2014 -0700
>
> CLOUDSTACK-6853: Search for non-removed nics only when check if the
> running vm belongs to a certain network
>
>
> -Alena.



-- 
Daan

ssh keypair value overridden by template meta data

[Hiroki Ohashi]

Dear guys

I encountered a problem that a ssh public key of ssh_keypairs for a
newly created instance is overridden by another ssh key in template
meta data. I think this leads to security vulnerability because a
template owner can login to other user's instance created from the
template. So, could you fix this issue?

This behavior is caused by meta data import at commitUserVm method. A
ssh key value specified by an instance owner is set to a UserVmVO
object at line 2986-2988 of
server/src/com/cloud/vm/UserVmManagerImpl.java in 4.3 branch [1], but
this value is overridden at line 3035-3038 by template meta data.

Please note a database contains meta data entries related to a
template which you created from an instance in cloud.template_view
like [2]. 2nd row has detail_name and detail_value about ssh key and
CloudStack override the user specified ssh key value by the owner's
detail value in this situation. It results in delivery of the template
owner's ssh key for the instance created from the template to virtual
router inspite of specification of instance owner's ssh key.

You can reproduce this phenomenon like below.

1. Deploy an instance with a ssh key A by specifying 'keypair'
   value.
2. Create a template from this instance.
3. Deploy an instance with another ssh key B by specifying
   'keypair' value.


[1] server/src/com/cloud/vm/UserVmManagerImpl.java

   2971 private UserVmVO commitUserVm(final DataCenter zone, final
VirtualMachineTemplate template, final String hostName, final String
displayName, final Account owner,
   2972 final Long diskOfferingId, final Long diskSize, final
String userData, final HypervisorType hypervisor, final Account
caller, final Boolean isDisplayVmEnabled,
   2973 final String keyboard, final long accountId, final
ServiceOfferingVO offering, final boolean isIso, final String
sshPublicKey,
   2974 final LinkedHashMap networkNicMap,
final long id, final String instanceName, final String uuidName, final
HypervisorType hypervisorType,
   2975 final Map customParameters) throws
InsufficientCapacityException {
   2976 return Transaction.execute(new
TransactionCallbackWithException() {
   2977 @Override
   2978 public UserVmVO doInTransaction(TransactionStatus
status) throws InsufficientCapacityException {
   2979 UserVmVO vm = new UserVmVO(id, instanceName,
displayName,
   2980 template.getId(), hypervisorType,
template.getGuestOSId(),
   2981 offering.getOfferHA(),
offering.getLimitCpuUse(),
   2982 owner.getDomainId(), owner.getId(),
offering.getId(), userData,
   2983 hostName, diskOfferingId);
   2984 vm.setUuid(uuidName);
   2985
vm.setDynamicallyScalable(template.isDynamicallyScalable());
   2986 if (sshPublicKey != null) {
   2987 vm.setDetail("SSH.PublicKey", sshPublicKey);
   2988 }
   2989
   2990 if (keyboard != null && !keyboard.isEmpty())
   2991 vm.setDetail(VmDetailConstants.KEYBOARD, keyboard);
   2992
   2993 if (isIso) {
   2994 vm.setIsoId(template.getId());
   2995 }
   2996
   2997 if(isDisplayVmEnabled != null){
   2998 if(!_accountMgr.isRootAdmin(caller.getType())){
   2999 throw new PermissionDeniedException(
"Cannot update parameter displayvm, only admin permitted ");
   3000 }
   3001 vm.setDisplayVm(isDisplayVmEnabled);
   3002 }else {
   3003 vm.setDisplayVm(true);
   3004 }
   3005
   3006 // If hypervisor is vSphere, check for clone
type setting.
   3007 if (hypervisorType.equals(HypervisorType.VMware)) {
   3008 // retrieve clone flag.
   3009 UserVmCloneType cloneType = UserVmCloneType.linked;
   3010 String value =
_configDao.getValue(Config.VmwareCreateFullClone.key());
   3011 if (value != null) {
   3012 if (Boolean.parseBoolean(value) == true)
   3013 cloneType = UserVmCloneType.full;
   3014 }
   3015 UserVmCloneSettingVO vmCloneSettingVO =
new UserVmCloneSettingVO(id, cloneType.toString());
   3016 _vmCloneSettingDao.persist(vmCloneSettingVO);
   3017 }
   3018
   3019 long guestOSId = template.getGuestOSId();
   3020 GuestOSVO guestOS = _guestOSDao.findById(guestOSId);
   3021 long guestOSCategoryId = guestOS.getCategoryId();
   3022 GuestOSCategoryVO guestOSCategory =
_guestOSCategoryDao.findById(guestOSCategoryId);
   3023
   3024
   30

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Kishan Kavala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44885
---

Ship it!


Ship It!

- Kishan Kavala


On June 6, 2014, 2:58 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 2:58 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Review Request 17994: CLOUDSTACK-6856 :Fixed Few Data Corruption and retries issues.

[Santhosh Edukulla]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17994/
---

Review request for cloudstack, daan Hoogland and SrikanteswaraRao Talluri.


Bugs: CLOUDSTACK-6856
https://issues.apache.org/jira/browse/CLOUDSTACK-6856


Repository: cloudstack-git


Description
---

Fixed Few cases failing because of data corruption and retries issues.


Diffs
-

  test/integration/smoke/test_deploy_vm.py 473b020 
  test/integration/smoke/test_guest_vlan_range.py bfef4a8 
  test/integration/smoke/test_hosts.py 52827aa 
  test/integration/smoke/test_vm_life_cycle.py 62a807d 
  tools/marvin/marvin/cloudstackTestClient.py 29a298b 
  tools/marvin/setup.py 4c775ad 

Diff: https://reviews.apache.org/r/17994/diff/


Testing
---

yes. 


Thanks,

Santhosh Edukulla

Fwd: New Defects reported by Coverity Scan for cloudstack

[Hugo Trippaers]

FYI

Begin forwarded message:

> From: scan-ad...@coverity.com
> Subject: New Defects reported by Coverity Scan for cloudstack
> Date: 6 juni 2014 13:40:40 CEST
> 
> 
> Hi,
> 
> 
> Please find the latest report on new defect(s) introduced to cloudstack found 
> with Coverity Scan.
> 
> Defect(s) Reported-by: Coverity Scan
> Showing 2 of 2 defect(s)
> 
> 
> ** CID 1220836:  Dereference after null check  (FORWARD_NULL)
> /server/src/com/cloud/hypervisor/HypervisorGuruBase.java: 116 in 
> com.cloud.hypervisor.HypervisorGuruBase.toNicTO(com.cloud.vm.NicProfile)()
> 
> ** CID 1220837:  Resource leak  (RESOURCE_LEAK)
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 301 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 321 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 301 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 321 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> 
> 
> 
> *** CID 1220836:  Dereference after null check  (FORWARD_NULL)
> /server/src/com/cloud/hypervisor/HypervisorGuruBase.java: 116 in 
> com.cloud.hypervisor.HypervisorGuruBase.toNicTO(com.cloud.vm.NicProfile)()
> 110 }
> 111 
> 112 //check whether the this nic has secondary ip addresses set
> 113 //set nic secondary ip address in NicTO which are used for 
> security group
> 114 // configuration. Use full when vm stop/start
> 115 List secIps = null;
CID 1220836:  Dereference after null check  (FORWARD_NULL)
Calling a method on null object "nicVO".
> 116 if (nicVO.getSecondaryIp()) {
> 117 secIps = 
> _nicSecIpDao.getSecondaryIpAddressesForNic(nicVO.getId());
> 118 }
> 119 to.setNicSecIps(secIps);
> 120 return to;
> 121 }
> 
> 
> *** CID 1220837:  Resource leak  (RESOURCE_LEAK)
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 301 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> 295 long id = rs.getLong(1);
> 296 String vlan = rs.getString(2);
> 297 if (vlan == null || "".equals(vlan)) {
> 298 continue;
> 299 }
> 300 String vlanUri = 
> BroadcastDomainType.Vlan.toUri(vlan).toString();
CID 1220837:  Resource leak  (RESOURCE_LEAK)
Overwriting "pstmt" in "pstmt = conn.prepareStatement("update 
 `cloud`.`vlan` set vlan_id=? where id=?")" leaks the resource that "pstmt" 
 refers to.
> 301 pstmt = conn.prepareStatement("update `cloud`.`vlan` 
> set vlan_id=? where id=?");
> 302 pstmt.setString(1, vlanUri);
> 303 pstmt.setLong(2, id);
> 304 pstmt.executeUpdate();
> 305 }
> 306 } catch (SQLException e) {
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 321 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> 315 pstmt.close();
> 316 }
> 317 } catch (SQLException e) {
> 318 }
> 319 }
> 320 s_logger.debug("Done updateing vlan URIs");
CID 1220837:  Resource leak  (RESOURCE_LEAK)
Variable "pstmt" going out of scope leaks the resource it refers to.
> 321 }
> 322 
> 323 @Override
> 324 public File[] getCleanupScripts() {
> 325 String script = Script.findScript("", 
> "db/schema-430to440-cleanup.sql");
> 326 if (script == null) {
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 301 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> 295 long id = rs.getLong(1);
> 296 String vlan = rs.getString(2);
> 297 if (vlan == null || "".equals(vlan)) {
> 298 continue;
> 299 }
> 300 String vlanUri = 
> BroadcastDomainType.Vlan.toUri(vlan).toString();
CID 1220837:  Resource leak  (RESOURCE_LEAK)
Overwriting "pstmt" in "pstmt = conn.prepareStatement("update 
 `cloud`.`vlan` set vlan_id=? where id=?")" leaks the resource that "pstmt" 
 refers to.
> 301 pstmt = conn.prepareStatement("update `cloud`.`vlan` 
> set vlan_id=? where id=?");
> 302   

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44886
---


patch looks alright but it seems to me you have reused a review board entry for 
a second patch. please don't do that. close this one and start a new one next 
time.

- daan Hoogland


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Santhosh Edukulla]

> On June 6, 2014, 12:09 p.m., daan Hoogland wrote:
> > patch looks alright but it seems to me you have reused a review board entry 
> > for a second patch. please don't do that. close this one and start a new 
> > one next time.

Sure, but reviewboard some times in process throws 5xx error and a given review 
request goes in to draft state. Shall i apply a new patch or this patch be 
considered for current one? Going ahead, i will take care to apply with a new 
one.


- Santhosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44886
---


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: [DISCUSS] Increasing VM IOPS by separating golden image in high IOPS partition in Xen Server ?

[Todd Pigram]

Sorry, thought you were based off the link you provided in this reply.

"In our case, we are using CloudStack integrated in VDI solution to provived
pooled VM type[1]. So may be my approach can bring better UX for user with
lower bootime ...

A short change in design are followings
- VM will be deployed with golden primary storage if primary storage is
marked golden and this VM template is also marked as golden.
- Choosing the best deploy destionation for both golden primary storage and
normal root volume primary storage. Chosen host can also access both
storage pools.
- New Xen Server plug-in for modifying VHD parent id.

Is there some place for me to submit my design and code. Can I write a new
proposal in CS wiki ?

[1]:
http://support.citrix.com/proddocs/topic/xendesktop-rho/cds-choose-scheme-type-rho.html
 "


On Thu, Jun 5, 2014 at 11:55 PM, Hieu LE  wrote:

> Hi Todd,
>
>
> On Fri, Jun 6, 2014 at 9:17 AM, Todd Pigram  wrote:
>
> > Hieu,
> >
> > I assume you are using MCS for you golden image? What version of XD?
> Given
> > you are using pooled desktops, have you thought about using a PVS BDM iso
> > and mount it with in your 1000 VMs? This way you can stagger reboots via
> > PVS console or Studio. This would require a change to your delivery
> group.
> >
> >
> Sorry but I did not use MCS or XenDesktop in my company :-)
>
>
> >
> > On Thu, Jun 5, 2014 at 9:28 PM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com
> > > wrote:
> >
> > > 6) The copy_vhd_from_secondarystorage XenServer plug-in is not used
> when
> > > you're using XenServer + XS62ESP1 + XS62ESP1004. In that case, please
> > refer
> > > to copyTemplateToPrimaryStorage(CopyCommand) method in the
> > > Xenserver625StorageProcessor class.
> > >
> >
>
> Thank Mike, I will take note of that.
>
>
> >  >
> > > On Thu, Jun 5, 2014 at 1:56 PM, Mike Tutkowski <
> > > mike.tutkow...@solidfire.com
> > > > wrote:
> > >
> > > > Other than going through a "for" loop and deploying VM after VM, I
> > don't
> > > > think CloudStack currently supports a bulk-VM-deploy operation.
> > > >
> > > > It would be nice if CS did so at some point in the future; however,
> > that
> > > > is probably a separate proposal from Hieu's.
> > > >
> > > >
> > > > On Thu, Jun 5, 2014 at 12:13 AM, Amit Das 
> > > wrote:
> > > >
> > > >> Hi Hieu,
> > > >>
> > > >> Will it be good to include bulk operation of this feature? In
> > addition,
> > > >> does Xen support parallel execution of these operations ?
> > > >>
> > > >> Regards,
> > > >> Amit
> > > >> *CloudByte Inc.* 
> > > >>
> > > >>
> > > >> On Thu, Jun 5, 2014 at 8:59 AM, Hieu LE  wrote:
> > > >>
> > > >> > Mike, Punith,
> > > >> >
> > > >> > Please review "Golden Primary Storage" proposal. [1]
> > > >> >
> > > >> > Thank you.
> > > >> >
> > > >> > [1]:
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Golden+Primary+Storage
> > > >> >
> > > >> >
> > > >> > On Wed, Jun 4, 2014 at 10:32 PM, Mike Tutkowski <
> > > >> > mike.tutkow...@solidfire.com> wrote:
> > > >> >
> > > >> >> Daan helped out with this. You should be good to go now.
> > > >> >>
> > > >> >>
> > > >> >> On Tue, Jun 3, 2014 at 8:50 PM, Hieu LE 
> > wrote:
> > > >> >>
> > > >> >> > Hi Mike,
> > > >> >> >
> > > >> >> > Could you please give edit/create permission on ASF Jira/Wiki
> > > >> >> confluence ?
> > > >> >> > I can not add a new Wiki page.
> > > >> >> >
> > > >> >> > My Jira ID: hieulq
> > > >> >> > Wiki: hieulq89
> > > >> >> > Review Board: hieulq
> > > >> >> >
> > > >> >> > Thanks !
> > > >> >> >
> > > >> >> >
> > > >> >> > On Wed, Jun 4, 2014 at 9:17 AM, Mike Tutkowski <
> > > >> >> > mike.tutkow...@solidfire.com
> > > >> >> > > wrote:
> > > >> >> >
> > > >> >> > > Hi,
> > > >> >> > >
> > > >> >> > > Yes, please feel free to add a new Wiki page for your design.
> > > >> >> > >
> > > >> >> > > Here is a link to applicable design info:
> > > >> >> > >
> > > >> >> > >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Design
> > > >> >> > >
> > > >> >> > > Also, feel free to ask more questions and have me review your
> > > >> design.
> > > >> >> > >
> > > >> >> > > Thanks!
> > > >> >> > > Mike
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > On Tue, Jun 3, 2014 at 7:29 PM, Hieu LE 
> > > >> wrote:
> > > >> >> > >
> > > >> >> > > > Hi Mike,
> > > >> >> > > >
> > > >> >> > > > You are right, performance will be decreased over time
> > because
> > > >> >> writes
> > > >> >> > > IOPS
> > > >> >> > > > will always end up on slower storage pool.
> > > >> >> > > >
> > > >> >> > > > In our case, we are using CloudStack integrated in VDI
> > solution
> > > >> to
> > > >> >> > > provived
> > > >> >> > > > pooled VM type[1]. So may be my approach can bring better
> UX
> > > for
> > > >> >> user
> > > >> >> > > with
> > > >> >> > > > lower bootime ...
> > > >> >> > > >
> > > >> >> > > > A short change in design are followings
> > > >> >> > > > - VM will be deployed with golden primary storage i

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[daan Hoogland]

> On June 6, 2014, 12:09 p.m., daan Hoogland wrote:
> > patch looks alright but it seems to me you have reused a review board entry 
> > for a second patch. please don't do that. close this one and start a new 
> > one next time.
> 
> Santhosh Edukulla wrote:
> Sure, but reviewboard some times in process throws 5xx error and a given 
> review request goes in to draft state. Shall i apply a new patch or this 
> patch be considered for current one? Going ahead, i will take care to apply 
> with a new one.

I have applied locally and am running a vlidation build. will push when passes.


- daan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44886
---


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Anybody addressing this bug ? Overlaping IP subnets across different vlans

[Andrija Panic]

Hi,
aftger upgrade to 4.3, I reported a bug where CS will not let me add
additional IP ranges when there are 2 vlans using same IP range - I
don't see point comparing IP ranges across two separate broadcast domains...

https://issues.apache.org/jira/browse/CLOUDSTACK-6814

Thanks,
-- 

Andrija Panić

Re: New Defects reported by Coverity Scan for cloudstack

[Rajani Karuturi]

The defect density came down to 3.92 from around 6.5 :)

Thanks for fixing the pattern Animesh.


~Rajani



On 05-Jun-2014, at 11:58 am, Hugo Trippaers  wrote:

> Heya,
> 
> Thanks for helping out with the coverity configuration.
> 
> The coverity builds are run automagically from Jenkins every monday and 
> friday. Based on our project size we can submit a maximum of two builds per 
> week. The choices for monday and friday are arbitraty so if other days suit 
> our workflow better any Jenkins admin can adapt the configuration. 
> 
> 
> Cheers,
> 
> Hugo
> 
> On 5 jun. 2014, at 02:01, Animesh Chaturvedi  
> wrote:
> 
>> 
>> While we should wait for the next run for actual results but I am expecting 
>> the defect density as reported by Coverity to drop down from 6.4 to about 
>> 3.4 after excluding AWSAPI generated code.
>> 
>> Animesh
>> 
>>> -Original Message-
>>> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
>>> Sent: Wednesday, June 04, 2014 3:56 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: New Defects reported by Coverity Scan for cloudstack
>>> 
>>> Rajani
>>> 
>>> Thanks for bringing this up, I looked at the analysis settings and I think 
>>> the
>>> pattern to exclude aws code should have a leading /. I have fixed it and 
>>> lets
>>> check it on the next run. We should see a significant bump up in defect 
>>> density.
>>> Good job in noticing this.
>>> 
>>> Thanks
>>> Animesh
>>> 
 -Original Message-
 From: Rajani Karuturi [mailto:rajani.karut...@citrix.com]
 Sent: Monday, June 02, 2014 11:48 PM
 To: dev
 Subject: Re: New Defects reported by Coverity Scan for cloudstack
 
 These are the approximate number of defects reported by coverity from
 the generated code.
 
> /awsapi/src/com/amazon/ec2 -> around 4300
 
> /awsapi/src/com/amazon/s3 -> around 350
 
 
 total defects -> around 6500
 
 ~Rajani
 
 
 
 On 02-Jun-2014, at 4:57 pm, Rajani Karuturi 
>>> wrote:
 
> Hi Hugo,
> 
> awsapi-generated-code is excluded  for the project but I still see 
> issues
 reported in them.
> For example for file src/com/amazon/ec2/DeleteTagsResponseType.java
> 
> 
> Can you check the file exclude pattern? I think .* is missing
> (awsapi/src/com/amazon/.*)
> 
> Fixing this might give us a better report as I see lot of them listed in 
> these
>>> files.
> 
> 
> ~Rajani
> 
> 
> 
> On 29-Nov-2013, at 9:28 pm, Hugo Trippaers  wrote:
> 
>> FYI
>> 
>> Sent from my iPhone
>> 
>> Begin forwarded message:
>> 
>>> From: scan-ad...@coverity.com
>>> Date: 29 november 2013 14:39:56 CET
>>> Subject: New Defects reported by Coverity Scan for cloudstack
>>> 
>>> 
>>> Hi,
>>> 
>>> 
>>> Please find the latest report on new defect(s) introduced to
>>> cloudstack
 found with Coverity Scan.
>>> 
>>> Defect(s) Reported-by: Coverity Scan Showing 6 of 6 defect(s)
>>> 
>>> 
>>> ** CID 1116269:  Nesting level does not match indentation
>>> (NESTING_INDENT_MISMATCH)
>>> /awsapi/src/com/cloud/bridge/service/controller/s3/ServiceProvider
>>> .j
>>> ava: 124 in
>>> com.cloud.bridge.service.controller.s3.ServiceProvider.getManageme
>>> nt
>>> HostId()()
>>> 
>>> ** CID 1133706:  Dereference after null check  (FORWARD_NULL)
>>> /server/src/com/cloud/vm/UserVmManagerImpl.java: 2803 in
>>> 
>>> com.cloud.vm.UserVmManagerImpl$3.doInTransaction(com.cloud.utils.db.
>>> TransactionStatus)()
>>> 
>>> ** CID 1133705:  Resource leak on an exceptional path
>>> (RESOURCE_LEAK)
>>> /server/src/com/cloud/server/ConfigurationServerImpl.java: 638 in
>>> com.cloud.server.ConfigurationServerImpl.updateSSLKeystore()()
>>> 
>>> ** CID 1133704:  SS: Unread field should be static
>>> (FB.SS_SHOULD_BE_STATIC)
>>> /server/src/com/cloud/uuididentity/UUIDManagerImpl.java: 43 in ()
>>> 
>>> ** CID 1133703:  Dm: Dubious method used
>>> (FB.DM_DEFAULT_ENCODING)
>>> /plugins/user-authenticators/ldap/src/org/apache/cloudstack/api/co
>>> mm
>>> and/LdapImportUsersCmd.java: 197 in
>>> 
 org.apache.cloudstack.api.command.LdapImportUsersCmd.generatePasswor
>>> d()()
>>> 
>>> ** CID 1133702:  DLS: Dead local store  (FB.DLS_DEAD_LOCAL_STORE)
>>> /plugins/network-elements/juniper-contrail/src/org/apache/cloudsta
>>> ck
>>> /network/contrail/model/VirtualMachineModel.java: 119 in
>>> org.apache.cloudstack.network.contrail.model.VirtualMachineModel.b
>>> ui
>>> ldServiceInstance(org.apache.cloudstack.network.contrail.model.Mod
>>> el
>>> Controller, java.lang.String)()
>>> 
>>> 
>>> 
>>> 
>>> 
 
>>> _
 ___
>>> _

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44889
---


Commit 91d054cb00f6a5818f2be1c7e4d3f44a2c58bfcc in cloudstack's branch 
refs/heads/4.4-forward from sedukull
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=91d054c ]

CLOUDSTACK-6793: Fixed an invalid Domain Id Issue

Signed-off-by: sedukull 


- ASF Subversion and Git Services


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44890
---


Commit 6c0b43431f7f117c1fcb049c483407b96b56d961 in cloudstack's branch 
refs/heads/4.4 from sedukull
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=6c0b434 ]

CLOUDSTACK-6793: Fixed an invalid Domain Id Issue

Signed-off-by: sedukull 
(cherry picked from commit 91d054cb00f6a5818f2be1c7e4d3f44a2c58bfcc)


- ASF Subversion and Git Services


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

coverity scan: resource leaks in upgrade path

[Rajani Karuturi]

Hi All,
Of the 1697 issues reported by coverity in the latest scan, about 187 are in 
upgrade###to###.java and many of them are due to RESOURCE_LEAKS because of not 
closing the pstmt properly.
For ex:
Resource leak on an exceptional path (RESOURCE_LEAK)

299 }
300 String vlanUri = 
BroadcastDomainType.Vlan.toUri(vlan).toString();
>>> CID 1220837:  Resource leak  (RESOURCE_LEAK)
>>> Overwriting "pstmt" in "pstmt = conn.prepareStatement("update 
>>> `cloud`.`vlan` set vlan_id=? where id=?")" leaks the resource that "pstmt" 
>>> refers to.
301 pstmt = conn.prepareStatement("update `cloud`.`vlan` 
set vlan_id=? where id=?");
302 pstmt.setString(1, vlanUri);
303 pstmt.setLong(2, id);
304 pstmt.executeUpdate();
305 }
306 } catch (SQLException e) {
/engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 321 in 
com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
315 pstmt.close();
316 }
317 } catch (SQLException e) {
318 }
319 }

$ cut -d"," -f 12 < ./Outstanding+Defects.csv | grep 
"engine/schema/src/com/cloud/upgrade/dao/Upgrade" | sort | uniq -c | sort -n | 
awk '{t+=$1}END{print t,"total"}1'
   1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2211to2212.java
   1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade222to224Premium.java
   1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade306to307.java
   1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade30to301.java
   1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade40to41.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to22Premium.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2212to2213.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2213to2214.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade225to226.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade227to228Premium.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade228to229.java
   2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade229to2210.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to224DomainVlans.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade227to228.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade301to302.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade302to303.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade303to304.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade421to430.java
   3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java
   4 /engine/schema/src/com/cloud/upgrade/dao/Upgrade305to306.java
   4 /engine/schema/src/com/cloud/upgrade/dao/Upgrade420to421.java
   7 /engine/schema/src/com/cloud/upgrade/dao/Upgrade224to225.java
   7 /engine/schema/src/com/cloud/upgrade/dao/Upgrade304to305.java
  10 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2214to30.java
  12 /engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java
  13 /engine/schema/src/com/cloud/upgrade/dao/Upgrade222to224.java
  42 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to22.java
  48 /engine/schema/src/com/cloud/upgrade/dao/Upgrade410to420.java
187 total

There are 3 new issues reported Upgrade430to440.java

reviewers/contributors please take care to verify this.

~Rajani

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Daan Hoogland]

Santhosh,

Have you made sure this goes into master as well?


On Fri, Jun 6, 2014 at 3:53 PM, ASF Subversion and Git Services <
asf...@urd.zones.apache.org> wrote:

>This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
>
> Commit 91d054cb00f6a5818f2be1c7e4d3f44a2c58bfcc in cloudstack's branch 
> refs/heads/4.4-forward from sedukull
> [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=91d054c ]
>
> CLOUDSTACK-6793: Fixed an invalid Domain Id Issue
>
> Signed-off-by: sedukull 
>
>
> - ASF Subversion and Git Services
>
> On June 6th, 2014, 9:28 a.m. UTC, Santhosh Edukulla wrote:
>   Review request for cloudstack, daan Hoogland and Kishan Kavala.
> By Santhosh Edukulla.
>
> *Updated June 6, 2014, 9:28 a.m.*
>  *Bugs: * CLOUDSTACK-6793
> 
>  *Repository: * cloudstack-git
> Description
>
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
>
>   Testing
>
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
>
>   Diffs
>
>- engine/schema/src/com/cloud/user/dao/AccountDao.java (2f737cd)
>- engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java (4bc23ed)
>- server/src/com/cloud/tags/TaggedResourceManagerImpl.java (fa7fcb7)
>
> View Diff 
>



-- 
Daan

RE: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Santhosh Edukulla]

Hi Daan,

Yes, for master planning additional approach with few more enhancements, useful 
for future release. It may require modifying schema, which i believe is not 
possible for 4.4 because of code freeze.

Regards
Santhosh

From: Daan Hoogland [daan.hoogl...@gmail.com]
Sent: Friday, June 06, 2014 9:58 AM
To: Santhosh Edukulla
Cc: Kishan Kavala; cloudstack
Subject: Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id 
Issue

Santhosh,

Have you made sure this goes into master as well?


On Fri, Jun 6, 2014 at 3:53 PM, ASF Subversion and Git Services 
mailto:asf...@urd.zones.apache.org>> wrote:
This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/19874/


Commit 91d054cb00f6a5818f2be1c7e4d3f44a2c58bfcc in cloudstack's branch 
refs/heads/4.4-forward from sedukull
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=91d054c ]

CLOUDSTACK-6793: Fixed an invalid Domain Id Issue

Signed-off-by: sedukull 
mailto:santhosh.eduku...@gmail.com>>



- ASF Subversion and Git Services


On June 6th, 2014, 9:28 a.m. UTC, Santhosh Edukulla wrote:

Review request for cloudstack, daan Hoogland and Kishan Kavala.
By Santhosh Edukulla.

Updated June 6, 2014, 9:28 a.m.

Bugs: CLOUDSTACK-6793
Repository: cloudstack-git
Description

1. Added fix to retrieve the correct domain id for a given accountid. 
vm_template table and respective vo, does not have a domain id information.
2. Added logic to retrieve correct domainid for a given accountid, only when an 
invalid domainid is retrieved. Throw an exception in case of invalid domain id.
3. Now, creation of tags is getting successful post the fix.


Testing

Test creation, listing and deletion tags on UserVM ... === TestName: 
test_05_vm_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tag on templates ... === TestName: 
test_06_template_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tags on ISO ... === TestName: 
test_07_iso_tag | Status : SUCCESS ===
ok
Test creation, listing and deletion tagson volume ... === TestName: 
test_08_volume_tag | Status : SUCCESS ===
ok



Diffs

  *   engine/schema/src/com/cloud/user/dao/AccountDao.java (2f737cd)
  *   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java (4bc23ed)
  *   server/src/com/cloud/tags/TaggedResourceManagerImpl.java (fa7fcb7)

View Diff




--
Daan

RE: New Defects reported by Coverity Scan for cloudstack

[Sudha Ponnaganti]

That is better. Hope new ones can be fixed each week while the old ones can be 
prioritized based on high density.

 Original message 
From: Rajani Karuturi
Date:06/06/2014 6:37 AM (GMT-08:00)
To: dev@cloudstack.apache.org
Subject: Re: New Defects reported by Coverity Scan for cloudstack

The defect density came down to 3.92 from around 6.5 :)

Thanks for fixing the pattern Animesh.


~Rajani



On 05-Jun-2014, at 11:58 am, Hugo Trippaers  wrote:

> Heya,
>
> Thanks for helping out with the coverity configuration.
>
> The coverity builds are run automagically from Jenkins every monday and 
> friday. Based on our project size we can submit a maximum of two builds per 
> week. The choices for monday and friday are arbitraty so if other days suit 
> our workflow better any Jenkins admin can adapt the configuration.
>
>
> Cheers,
>
> Hugo
>
> On 5 jun. 2014, at 02:01, Animesh Chaturvedi  
> wrote:
>
>>
>> While we should wait for the next run for actual results but I am expecting 
>> the defect density as reported by Coverity to drop down from 6.4 to about 
>> 3.4 after excluding AWSAPI generated code.
>>
>> Animesh
>>
>>> -Original Message-
>>> From: Animesh Chaturvedi [mailto:animesh.chaturv...@citrix.com]
>>> Sent: Wednesday, June 04, 2014 3:56 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: New Defects reported by Coverity Scan for cloudstack
>>>
>>> Rajani
>>>
>>> Thanks for bringing this up, I looked at the analysis settings and I think 
>>> the
>>> pattern to exclude aws code should have a leading /. I have fixed it and 
>>> lets
>>> check it on the next run. We should see a significant bump up in defect 
>>> density.
>>> Good job in noticing this.
>>>
>>> Thanks
>>> Animesh
>>>
 -Original Message-
 From: Rajani Karuturi [mailto:rajani.karut...@citrix.com]
 Sent: Monday, June 02, 2014 11:48 PM
 To: dev
 Subject: Re: New Defects reported by Coverity Scan for cloudstack

 These are the approximate number of defects reported by coverity from
 the generated code.

> /awsapi/src/com/amazon/ec2 -> around 4300

> /awsapi/src/com/amazon/s3 -> around 350


 total defects -> around 6500

 ~Rajani



 On 02-Jun-2014, at 4:57 pm, Rajani Karuturi 
>>> wrote:

> Hi Hugo,
>
> awsapi-generated-code is excluded  for the project but I still see 
> issues
 reported in them.
> For example for file src/com/amazon/ec2/DeleteTagsResponseType.java
>
>
> Can you check the file exclude pattern? I think .* is missing
> (awsapi/src/com/amazon/.*)
>
> Fixing this might give us a better report as I see lot of them listed in 
> these
>>> files.
>
>
> ~Rajani
>
>
>
> On 29-Nov-2013, at 9:28 pm, Hugo Trippaers  wrote:
>
>> FYI
>>
>> Sent from my iPhone
>>
>> Begin forwarded message:
>>
>>> From: scan-ad...@coverity.com
>>> Date: 29 november 2013 14:39:56 CET
>>> Subject: New Defects reported by Coverity Scan for cloudstack
>>>
>>>
>>> Hi,
>>>
>>>
>>> Please find the latest report on new defect(s) introduced to
>>> cloudstack
 found with Coverity Scan.
>>>
>>> Defect(s) Reported-by: Coverity Scan Showing 6 of 6 defect(s)
>>>
>>>
>>> ** CID 1116269:  Nesting level does not match indentation
>>> (NESTING_INDENT_MISMATCH)
>>> /awsapi/src/com/cloud/bridge/service/controller/s3/ServiceProvider
>>> .j
>>> ava: 124 in
>>> com.cloud.bridge.service.controller.s3.ServiceProvider.getManageme
>>> nt
>>> HostId()()
>>>
>>> ** CID 1133706:  Dereference after null check  (FORWARD_NULL)
>>> /server/src/com/cloud/vm/UserVmManagerImpl.java: 2803 in
>>>
>>> com.cloud.vm.UserVmManagerImpl$3.doInTransaction(com.cloud.utils.db.
>>> TransactionStatus)()
>>>
>>> ** CID 1133705:  Resource leak on an exceptional path
>>> (RESOURCE_LEAK)
>>> /server/src/com/cloud/server/ConfigurationServerImpl.java: 638 in
>>> com.cloud.server.ConfigurationServerImpl.updateSSLKeystore()()
>>>
>>> ** CID 1133704:  SS: Unread field should be static
>>> (FB.SS_SHOULD_BE_STATIC)
>>> /server/src/com/cloud/uuididentity/UUIDManagerImpl.java: 43 in ()
>>>
>>> ** CID 1133703:  Dm: Dubious method used
>>> (FB.DM_DEFAULT_ENCODING)
>>> /plugins/user-authenticators/ldap/src/org/apache/cloudstack/api/co
>>> mm
>>> and/LdapImportUsersCmd.java: 197 in
>>>
 org.apache.cloudstack.api.command.LdapImportUsersCmd.generatePasswor
>>> d()()
>>>
>>> ** CID 1133702:  DLS: Dead local store  (FB.DLS_DEAD_LOCAL_STORE)
>>> /plugins/network-elements/juniper-contrail/src/org/apache/cloudsta
>>> ck
>>> /network/contrail/model/VirtualMachineModel.java: 119 in
>>> org.apache.cloudstack.network.contrail.model.VirtualMachineModel.b
>>> ui
>>> ldServiceInstance(org.apache.cl

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Daan Hoogland]

code freeze does not mean that the schema can't be changed for bug
fixes. Any bug fix can go in if it genuinely serves the purpose of
fixing a bug.

On Fri, Jun 6, 2014 at 4:02 PM, Santhosh Edukulla
 wrote:
> Hi Daan,
>
> Yes, for master planning additional approach with few more enhancements, 
> useful for future release. It may require modifying schema, which i believe 
> is not possible for 4.4 because of code freeze.
>
> Regards
> Santhosh
> 
> From: Daan Hoogland [daan.hoogl...@gmail.com]
> Sent: Friday, June 06, 2014 9:58 AM
> To: Santhosh Edukulla
> Cc: Kishan Kavala; cloudstack
> Subject: Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain 
> Id Issue
>
> Santhosh,
>
> Have you made sure this goes into master as well?
>
>
> On Fri, Jun 6, 2014 at 3:53 PM, ASF Subversion and Git Services 
> mailto:asf...@urd.zones.apache.org>> wrote:
> This is an automatically generated e-mail. To reply, visit: 
> https://reviews.apache.org/r/19874/
>
>
> Commit 91d054cb00f6a5818f2be1c7e4d3f44a2c58bfcc in cloudstack's branch 
> refs/heads/4.4-forward from sedukull
> [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=91d054c ]
>
> CLOUDSTACK-6793: Fixed an invalid Domain Id Issue
>
> Signed-off-by: sedukull 
> mailto:santhosh.eduku...@gmail.com>>
>
>
>
> - ASF Subversion and Git Services
>
>
> On June 6th, 2014, 9:28 a.m. UTC, Santhosh Edukulla wrote:
>
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> By Santhosh Edukulla.
>
> Updated June 6, 2014, 9:28 a.m.
>
> Bugs: CLOUDSTACK-6793
> Repository: cloudstack-git
> Description
>
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
>
>
> Testing
>
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
>
>
>
> Diffs
>
>   *   engine/schema/src/com/cloud/user/dao/AccountDao.java (2f737cd)
>   *   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java (4bc23ed)
>   *   server/src/com/cloud/tags/TaggedResourceManagerImpl.java (fa7fcb7)
>
> View Diff
>
>
>
>
> --
> Daan



-- 
Daan

Re: coverity scan: resource leaks in upgrade path

[Daan Hoogland]

great Rajani, I am having a look at Upgrade43to44 and will backport
the fix to all others that it applies to:

try(PreparedStatement selectstatement =
conn.prepareStatement("SELECT * FROM 1+1");
ResultSet results = selectstatement.executeQuery()) {

}without finally

On Fri, Jun 6, 2014 at 3:55 PM, Rajani Karuturi
 wrote:
> Hi All,
> Of the 1697 issues reported by coverity in the latest scan, about 187 are in 
> upgrade###to###.java and many of them are due to RESOURCE_LEAKS because of 
> not closing the pstmt properly.
> For ex:
> Resource leak on an exceptional path (RESOURCE_LEAK)
>
> 299 }
> 300 String vlanUri = 
> BroadcastDomainType.Vlan.toUri(vlan).toString();
 CID 1220837:  Resource leak  (RESOURCE_LEAK)
 Overwriting "pstmt" in "pstmt = conn.prepareStatement("update 
 `cloud`.`vlan` set vlan_id=? where id=?")" leaks the resource that "pstmt" 
 refers to.
> 301 pstmt = conn.prepareStatement("update `cloud`.`vlan` 
> set vlan_id=? where id=?");
> 302 pstmt.setString(1, vlanUri);
> 303 pstmt.setLong(2, id);
> 304 pstmt.executeUpdate();
> 305 }
> 306 } catch (SQLException e) {
> /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java: 321 in 
> com.cloud.upgrade.dao.Upgrade430to440.updateVlanUris(java.sql.Connection)()
> 315 pstmt.close();
> 316 }
> 317 } catch (SQLException e) {
> 318 }
> 319 }
>
> $ cut -d"," -f 12 < ./Outstanding+Defects.csv | grep 
> "engine/schema/src/com/cloud/upgrade/dao/Upgrade" | sort | uniq -c | sort -n 
> | awk '{t+=$1}END{print t,"total"}1'
>1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2211to2212.java
>1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade222to224Premium.java
>1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade306to307.java
>1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade30to301.java
>1 /engine/schema/src/com/cloud/upgrade/dao/Upgrade40to41.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to22Premium.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2212to2213.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2213to2214.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade225to226.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade227to228Premium.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade228to229.java
>2 /engine/schema/src/com/cloud/upgrade/dao/Upgrade229to2210.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to224DomainVlans.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade227to228.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade301to302.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade302to303.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade303to304.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade421to430.java
>3 /engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java
>4 /engine/schema/src/com/cloud/upgrade/dao/Upgrade305to306.java
>4 /engine/schema/src/com/cloud/upgrade/dao/Upgrade420to421.java
>7 /engine/schema/src/com/cloud/upgrade/dao/Upgrade224to225.java
>7 /engine/schema/src/com/cloud/upgrade/dao/Upgrade304to305.java
>   10 /engine/schema/src/com/cloud/upgrade/dao/Upgrade2214to30.java
>   12 /engine/schema/src/com/cloud/upgrade/dao/Upgrade302to40.java
>   13 /engine/schema/src/com/cloud/upgrade/dao/Upgrade222to224.java
>   42 /engine/schema/src/com/cloud/upgrade/dao/Upgrade218to22.java
>   48 /engine/schema/src/com/cloud/upgrade/dao/Upgrade410to420.java
> 187 total
>
> There are 3 new issues reported Upgrade430to440.java
>
> reviewers/contributors please take care to verify this.
>
> ~Rajani
>
>
>



-- 
Daan

Review Request 22305: Fixed Coverity Issue : 1220837

[Santhosh Edukulla]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22305/
---

Review request for cloudstack, daan Hoogland and Hugo Trippaers.


Repository: cloudstack-git


Description
---

Fixed coverity issue for memory leak reported. 


Diffs
-

  engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java a3ac447 

Diff: https://reviews.apache.org/r/22305/diff/


Testing
---

Ran the build with developer profile, and deployed a zone.


Thanks,

Santhosh Edukulla

Re: KVM + LXC on the same host

[Nguyen Anh Tu]

That should be a good idea, Ilya. At that moment, i'm working on Docker
support. When it's done, we can run only Docker hosts, no need to use KVM
for hosting system vms.

Cheers,
--Tuna

Sent from my GT-N7000
On Jun 5, 2014 7:58 AM, "ilya musayev"  wrote:

> We are considering running KVM and LXC on the same host and hopefully
> control both through cloudstack.
>
> I know there are agents involved for each component, i dont know if we can
> have a hybrid of LXC+KVM.
>
> The use case is simple, we would like the end user to pick LXC/Docker for
> performance, or KVM instance if he really needed all bells and whistles of
> dedicated kernel in fully virtualized environment.
>
> Is anyone aware why we should not mix 2 workloads on the same host? Is it
> possible at this point in time to mix LXC, KVM and CloudStack, i assume the
> answer is no, but perhaps there is a hack i can try.
>
> Thanks
> ilya
>

Build failed in Jenkins: build-systemvm64-4.2 #302

[jenkins]

See 

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on devcloud-continuous-tests (devcloud) in workspace 

Fetching changes from the remote Git repository
Fetching upstream changes from 
https://git-wip-us.apache.org/repos/asf/cloudstack.git
FATAL: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/cloudstack.git
hudson.plugins.git.GitException: Failed to fetch from 
https://git-wip-us.apache.org/repos/asf/cloudstack.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:625)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:847)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:872)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1320)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:609)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:88)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518)
at hudson.model.Run.execute(Run.java:1688)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:231)
Caused by: hudson.plugins.git.GitException: Failed to connect to 
https://git-wip-us.apache.org/repos/asf/cloudstack.git
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.checkCredentials(CliGitAPIImpl.java:1608)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:957)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$200(CliGitAPIImpl.java:71)
at 
org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:197)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152)
at 
org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145)
at hudson.remoting.UserRequest.perform(UserRequest.java:118)
at hudson.remoting.UserRequest.perform(UserRequest.java:48)
at hudson.remoting.Request$2.run(Request.java:326)
at 
hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at hudson.remoting.Engine$1$1.run(Engine.java:63)
at java.lang.Thread.run(Thread.java:679)

Re: Review Request 22305: Fixed Coverity Issue : 1220837

[daan Hoogland]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22305/#review44897
---


I was thinking of

private void updateVlanUris(Connection conn) {
s_logger.debug("updating vlan URIs");
try(PreparedStatement selectstatement = conn.prepareStatement("SELECT 
id, vlan_id FROM `cloud`.`vlan` where vlan_id not like '%:%'");
ResultSet results = selectstatement.executeQuery()) {

while (results.next()) {
long id = results.getLong(1);
String vlan = results.getString(2);
if (vlan == null || "".equals(vlan)) {
continue;
}
String vlanUri = 
BroadcastDomainType.Vlan.toUri(vlan).toString();
try(PreparedStatement updatestatement = 
conn.prepareStatement("update `cloud`.`vlan` set vlan_id=? where id=?");)
{
updatestatement.setString(1, vlanUri);
updatestatement.setLong(2, id);
updatestatement.executeUpdate();
updatestatement.close();
} catch (SQLException e) {
throw new CloudRuntimeException("Unable to update vlan URI 
" + vlanUri + " for vlan record " + id, e);
}
}
} catch (SQLException e) {
throw new CloudRuntimeException("Unable to update vlan URIs ", e);
}
s_logger.debug("Done updateing vlan URIs");
}


I am implementing such a thing for all methods in the update file

- daan Hoogland


On June 6, 2014, 2:50 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22305/
> ---
> 
> (Updated June 6, 2014, 2:50 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed coverity issue for memory leak reported. 
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java a3ac447 
> 
> Diff: https://reviews.apache.org/r/22305/diff/
> 
> 
> Testing
> ---
> 
> Ran the build with developer profile, and deployed a zone.
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Review Request 22305: Fixed Coverity Issue : 1220837

[Santhosh Edukulla]

> On June 6, 2014, 2:54 p.m., daan Hoogland wrote:
> > I was thinking of
> > 
> > private void updateVlanUris(Connection conn) {
> > s_logger.debug("updating vlan URIs");
> > try(PreparedStatement selectstatement = 
> > conn.prepareStatement("SELECT id, vlan_id FROM `cloud`.`vlan` where vlan_id 
> > not like '%:%'");
> > ResultSet results = selectstatement.executeQuery()) {
> > 
> > while (results.next()) {
> > long id = results.getLong(1);
> > String vlan = results.getString(2);
> > if (vlan == null || "".equals(vlan)) {
> > continue;
> > }
> > String vlanUri = 
> > BroadcastDomainType.Vlan.toUri(vlan).toString();
> > try(PreparedStatement updatestatement = 
> > conn.prepareStatement("update `cloud`.`vlan` set vlan_id=? where id=?");)
> > {
> > updatestatement.setString(1, vlanUri);
> > updatestatement.setLong(2, id);
> > updatestatement.executeUpdate();
> > updatestatement.close();
> > } catch (SQLException e) {
> > throw new CloudRuntimeException("Unable to update vlan 
> > URI " + vlanUri + " for vlan record " + id, e);
> > }
> > }
> > } catch (SQLException e) {
> > throw new CloudRuntimeException("Unable to update vlan URIs ", 
> > e);
> > }
> > s_logger.debug("Done updateing vlan URIs");
> > }
> > 
> > 
> > I am implementing such a thing for all methods in the update file

Daan,
1. If exception happens before close EX: For updatestatement.close(), then 
updatstatement still not closed.
2. selectstatement is not closed yet. Is it a sort of contextlib of 
transaction, where it takes care of autoclose?


- Santhosh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22305/#review44897
---


On June 6, 2014, 2:50 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22305/
> ---
> 
> (Updated June 6, 2014, 2:50 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed coverity issue for memory leak reported. 
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java a3ac447 
> 
> Diff: https://reviews.apache.org/r/22305/diff/
> 
> 
> Testing
> ---
> 
> Ran the build with developer profile, and deployed a zone.
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Review Request 22305: Fixed Coverity Issue : 1220837

[daan Hoogland]

> On June 6, 2014, 2:54 p.m., daan Hoogland wrote:
> > I was thinking of
> > 
> > private void updateVlanUris(Connection conn) {
> > s_logger.debug("updating vlan URIs");
> > try(PreparedStatement selectstatement = 
> > conn.prepareStatement("SELECT id, vlan_id FROM `cloud`.`vlan` where vlan_id 
> > not like '%:%'");
> > ResultSet results = selectstatement.executeQuery()) {
> > 
> > while (results.next()) {
> > long id = results.getLong(1);
> > String vlan = results.getString(2);
> > if (vlan == null || "".equals(vlan)) {
> > continue;
> > }
> > String vlanUri = 
> > BroadcastDomainType.Vlan.toUri(vlan).toString();
> > try(PreparedStatement updatestatement = 
> > conn.prepareStatement("update `cloud`.`vlan` set vlan_id=? where id=?");)
> > {
> > updatestatement.setString(1, vlanUri);
> > updatestatement.setLong(2, id);
> > updatestatement.executeUpdate();
> > updatestatement.close();
> > } catch (SQLException e) {
> > throw new CloudRuntimeException("Unable to update vlan 
> > URI " + vlanUri + " for vlan record " + id, e);
> > }
> > }
> > } catch (SQLException e) {
> > throw new CloudRuntimeException("Unable to update vlan URIs ", 
> > e);
> > }
> > s_logger.debug("Done updateing vlan URIs");
> > }
> > 
> > 
> > I am implementing such a thing for all methods in the update file
> 
> Santhosh Edukulla wrote:
> Daan,
> 1. If exception happens before close EX: For updatestatement.close(), 
> then updatstatement still not closed.
> 2. selectstatement is not closed yet. Is it a sort of contextlib of 
> transaction, where it takes care of autoclose?
>

please read 
http://docs.oracle.com/javase/tutorial/essential/exceptions/tryResourceClose.html
ad 1. I forgot to remove the close statement for updatestatement it should not 
be there, thanks.
ad 2. yes it is closed see the read ref.


- daan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/22305/#review44897
---


On June 6, 2014, 2:50 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/22305/
> ---
> 
> (Updated June 6, 2014, 2:50 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Hugo Trippaers.
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed coverity issue for memory leak reported. 
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/upgrade/dao/Upgrade430to440.java a3ac447 
> 
> Diff: https://reviews.apache.org/r/22305/diff/
> 
> 
> Testing
> ---
> 
> Ran the build with developer profile, and deployed a zone.
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Support pure Xen as a hypervisor follow-up

[Dave Scott]

Hi,

Here’s a quick status update:

On 16 May 2014, at 15:22, Dave Scott  wrote:

> Hi,
> 
> On 14 May 2014, at 09:53, sebgoa  wrote:
> 
>> 
>> On Apr 9, 2014, at 2:37 PM, Dave Scott  wrote:
>> 
>>> Hi,
>>> 
>>> Following up from Tim's "Support pure Xen as a hypervisor" proposal last 
>>> month[1] I'd like to start working on this and maybe even make a little bit 
>>> of progress while I'm at CCC in Denver.
>>> 
>>> Helpfully James Bulpin managed to get CS + libvirt + xen to start an 
>>> instance in a simple configuration. Although the patches[2] are not 
>>> intended to be production-ready :) they help highlight some of the areas we 
>>> need to change.
>> 
>> Dave, just to let you know that Tim has done some important "refactoring" to 
>> split up XenServer hypervisor in CS between Xen and XenServer. That way we 
>> could keep using xapi for XS but start moving to libvirt for Xen.
>> 
>> Tim worked in the xen2server branch (don't ask about the name, I messed it 
>> up…:) ).
>> 
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/xen2server
>> 
>> Would be nice to see some of the libvirt stuff in that branch to handle a 
>> new driver for Xen.
>> 
>> Since the two hypervisors will be split up, we could still drop in some 
>> early libvirt patches to handle Xen and put this in 4.5 as a wip.
> 
> Thanks for the links.
> 
> I’m slowly building up a set of patches here:
> 
> https://github.com/djs55/cloudstack/tree/virsh-capabilities
> 
> I think once I’ve gotten to a stable-ish point I’ll rebase on top of Tim’s 
> branch.
> 
> So far I’ve
> * changed the hypervisor detection to use ‘virsh capabilities’ in one place 
> and ‘cat /sys/hypervisor/type’ in another. Thinking about it again, I think 
> it’s probably best to standardise on /sys/hypervisor/type since that will 
> succeed irrespective of whether the libvirtd service is chkconfig’d on or not.
> 
> * in the python cloudutils system setup stuff isKvmEnabled() has become 
> isHypervisorEnabled()
> 
> * added a XenLibvirtDiscoverer similar to the LXC one
> 
> * fixed what I believe is a race in sshExecuteCmdOneShotWithExitCode (which 
> seems to hit me every time, I don’t know why other people seem to be immune 
> from it): see CLOUDSTACK-6621 and review board request 21261
> 
> * added the new hypervisor to hypervisor.list and 
> system.vm.default.hypervisor, so it appears in the UI properly
> 
> * registered a system VM template in the database, using the same qcow2 image 
> as KVM
> 
> For my test host I’m using a XenServer nightly snapshot which comes with a 
> nice modern xen and kernel, and is easy to install bleeding-edge libvirt on 
> top. I had to tweak the kernel configuration and the network configuration 
> but I’m hoping to make it work out of the box in future.
> 
> When I deploy my ‘datacenter’ the discovery phase works, the agent connects 
> and looks healthy in the logs and the UI. The next step is to figure out why 
> the system VM template isn’t being copied to primary storage — for some 
> reason the copy isn’t being attempted but I can’t see any obvious reason why.

I’m now at the stage of getting my system VMs to start via libvirt. The main 
missing feature is support for : low-bandwidth private host<->guest 
control channels. These channels are generally useful things and are needed by 
other projects (like oVirt), so I’d like to add them to libxl and libvirt’s 
libxl driver. There’s a thread on xen-devel and libvir-devel if anyone’s 
interested:

http://www.redhat.com/archives/libvir-list/2014-June/msg00180.html

Once the  are sorted, basic VM operations should work. The next step 
would be to rebase my patches on top of Tim’s renaming changes and tidy them up 
for review.

Cheers,
Dave

> 
> Cheers,
> Dave
> 
>> 
>> -Sebastien
>> 
>>> 
>>> Some of the areas are:
>>> 
>>> 1. hypervisor detection
>>> 
>>> Where we currently look for KVM specifically ("lsmod | grep kvm") we could 
>>> switch to either detecting any Linux hypervisor (by reading 
>>> /sys/hypervisor/type) and assuming if a hypervisor is present then we can 
>>> use libvirt on it (is this a fair assumption?) Or we could white-list “kvm” 
>>> or “xen”. Or we could query libvirt directly (perhaps via 'virsh 
>>> capabilities'?)
>>> 
>>> 2. fiddling with the domain.xml
>>> 
>>> When starting a domain via libvirt the XML configuration has 
>>> hypervisor-specific stuff in it. Some of this is easy to change like:
>>> 
>>> 
>>> 
>>> obviously becomes
>>> 
>>> 
>>> 
>>> and
>>> 
>>> /usr/libexec/qemu-kvm
>>> 
>>> should probably be
>>> 
>>> /some/other/path/qemu-dm
>>> 
>>> Some is a bit more invasive (to the VM) such as the virtual hardware type 
>>> should be switched from "virtio" to "xen" (and the block device in Linux 
>>> will change from /dev/vd* to /dev/xvd*) and we'll have to either implement 
>>> or work around the lack of
>>> 
>>>  ...
>>> 
>>> -- I presume this is a control channel into the system VM. Perhaps we could 
>>> implement 

Re: Support pure Xen as a hypervisor follow-up

[Tim Mackey]

Dave,

I've submitted a merge review request
(https://reviews.apache.org/r/22270/) yesterday.  If you want to avoid
having to potentially deal with a bunch of conflicts, you might want
to see if your patches apply cleanly there and let me know.  Happy to
help with any conflict resolution.

btw, I didn't see a design document up on the
wiki(https://cwiki.apache.org/confluence/display/CLOUDSTACK/4.5+Design+Documents).
 Can you get one there and start a DISCUSS thread?  It'll probably
tease out some gotchas you might not be aware of.

-tim

On Fri, Jun 6, 2014 at 11:04 AM, Dave Scott  wrote:
> Hi,
>
> Here’s a quick status update:
>
> On 16 May 2014, at 15:22, Dave Scott  wrote:
>
>> Hi,
>>
>> On 14 May 2014, at 09:53, sebgoa  wrote:
>>
>>>
>>> On Apr 9, 2014, at 2:37 PM, Dave Scott  wrote:
>>>
 Hi,

 Following up from Tim's "Support pure Xen as a hypervisor" proposal last 
 month[1] I'd like to start working on this and maybe even make a little 
 bit of progress while I'm at CCC in Denver.

 Helpfully James Bulpin managed to get CS + libvirt + xen to start an 
 instance in a simple configuration. Although the patches[2] are not 
 intended to be production-ready :) they help highlight some of the areas 
 we need to change.
>>>
>>> Dave, just to let you know that Tim has done some important "refactoring" 
>>> to split up XenServer hypervisor in CS between Xen and XenServer. That way 
>>> we could keep using xapi for XS but start moving to libvirt for Xen.
>>>
>>> Tim worked in the xen2server branch (don't ask about the name, I messed it 
>>> up…:) ).
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/xen2server
>>>
>>> Would be nice to see some of the libvirt stuff in that branch to handle a 
>>> new driver for Xen.
>>>
>>> Since the two hypervisors will be split up, we could still drop in some 
>>> early libvirt patches to handle Xen and put this in 4.5 as a wip.
>>
>> Thanks for the links.
>>
>> I’m slowly building up a set of patches here:
>>
>> https://github.com/djs55/cloudstack/tree/virsh-capabilities
>>
>> I think once I’ve gotten to a stable-ish point I’ll rebase on top of Tim’s 
>> branch.
>>
>> So far I’ve
>> * changed the hypervisor detection to use ‘virsh capabilities’ in one place 
>> and ‘cat /sys/hypervisor/type’ in another. Thinking about it again, I think 
>> it’s probably best to standardise on /sys/hypervisor/type since that will 
>> succeed irrespective of whether the libvirtd service is chkconfig’d on or 
>> not.
>>
>> * in the python cloudutils system setup stuff isKvmEnabled() has become 
>> isHypervisorEnabled()
>>
>> * added a XenLibvirtDiscoverer similar to the LXC one
>>
>> * fixed what I believe is a race in sshExecuteCmdOneShotWithExitCode (which 
>> seems to hit me every time, I don’t know why other people seem to be immune 
>> from it): see CLOUDSTACK-6621 and review board request 21261
>>
>> * added the new hypervisor to hypervisor.list and 
>> system.vm.default.hypervisor, so it appears in the UI properly
>>
>> * registered a system VM template in the database, using the same qcow2 
>> image as KVM
>>
>> For my test host I’m using a XenServer nightly snapshot which comes with a 
>> nice modern xen and kernel, and is easy to install bleeding-edge libvirt on 
>> top. I had to tweak the kernel configuration and the network configuration 
>> but I’m hoping to make it work out of the box in future.
>>
>> When I deploy my ‘datacenter’ the discovery phase works, the agent connects 
>> and looks healthy in the logs and the UI. The next step is to figure out why 
>> the system VM template isn’t being copied to primary storage — for some 
>> reason the copy isn’t being attempted but I can’t see any obvious reason why.
>
> I’m now at the stage of getting my system VMs to start via libvirt. The main 
> missing feature is support for : low-bandwidth private host<->guest 
> control channels. These channels are generally useful things and are needed 
> by other projects (like oVirt), so I’d like to add them to libxl and 
> libvirt’s libxl driver. There’s a thread on xen-devel and libvir-devel if 
> anyone’s interested:
>
> http://www.redhat.com/archives/libvir-list/2014-June/msg00180.html
>
> Once the  are sorted, basic VM operations should work. The next 
> step would be to rebase my patches on top of Tim’s renaming changes and tidy 
> them up for review.
>
> Cheers,
> Dave
>
>>
>> Cheers,
>> Dave
>>
>>>
>>> -Sebastien
>>>

 Some of the areas are:

 1. hypervisor detection

 Where we currently look for KVM specifically ("lsmod | grep kvm") we could 
 switch to either detecting any Linux hypervisor (by reading 
 /sys/hypervisor/type) and assuming if a hypervisor is present then we can 
 use libvirt on it (is this a fair assumption?) Or we could white-list 
 “kvm” or “xen”. Or we could query libvirt directly (perhaps via 'virsh 
 capabilities'?)

 2. fiddling with t

Re: KVM + LXC on the same host

[Pierre-Luc Dion]

ilya,

Let us know how it goes your hybrid of LXC+KVM. I'm interested to know how
it's going,  I might try that too on the side.




Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect
855-OK-CLOUD (855-652-5683) x1101
- - -

*CloudOps*420 rue Guy
Montréal QC  H3J 1S6
www.cloudops.com
@CloudOps_


On Fri, Jun 6, 2014 at 10:53 AM, Nguyen Anh Tu  wrote:

> That should be a good idea, Ilya. At that moment, i'm working on Docker
> support. When it's done, we can run only Docker hosts, no need to use KVM
> for hosting system vms.
>
> Cheers,
> --Tuna
>
> Sent from my GT-N7000
> On Jun 5, 2014 7:58 AM, "ilya musayev" 
> wrote:
>
> > We are considering running KVM and LXC on the same host and hopefully
> > control both through cloudstack.
> >
> > I know there are agents involved for each component, i dont know if we
> can
> > have a hybrid of LXC+KVM.
> >
> > The use case is simple, we would like the end user to pick LXC/Docker for
> > performance, or KVM instance if he really needed all bells and whistles
> of
> > dedicated kernel in fully virtualized environment.
> >
> > Is anyone aware why we should not mix 2 workloads on the same host? Is it
> > possible at this point in time to mix LXC, KVM and CloudStack, i assume
> the
> > answer is no, but perhaps there is a hack i can try.
> >
> > Thanks
> > ilya
> >
>

Re: Support pure Xen as a hypervisor follow-up

[sebgoa]

On Jun 6, 2014, at 5:17 PM, Tim Mackey  wrote:

> Dave,
> 
> I've submitted a merge review request
> (https://reviews.apache.org/r/22270/) yesterday.  

Unfortunately that patch did not seem to apply cleanly. I have not dug deep 
though..

> If you want to avoid
> having to potentially deal with a bunch of conflicts, you might want
> to see if your patches apply cleanly there and let me know.  Happy to
> help with any conflict resolution.
> 
> btw, I didn't see a design document up on the
> wiki(https://cwiki.apache.org/confluence/display/CLOUDSTACK/4.5+Design+Documents).
> Can you get one there and start a DISCUSS thread?  It'll probably
> tease out some gotchas you might not be aware of.
> 
> -tim
> 
> On Fri, Jun 6, 2014 at 11:04 AM, Dave Scott  wrote:
>> Hi,
>> 
>> Here’s a quick status update:
>> 
>> On 16 May 2014, at 15:22, Dave Scott  wrote:
>> 
>>> Hi,
>>> 
>>> On 14 May 2014, at 09:53, sebgoa  wrote:
>>> 
 
 On Apr 9, 2014, at 2:37 PM, Dave Scott  wrote:
 
> Hi,
> 
> Following up from Tim's "Support pure Xen as a hypervisor" proposal last 
> month[1] I'd like to start working on this and maybe even make a little 
> bit of progress while I'm at CCC in Denver.
> 
> Helpfully James Bulpin managed to get CS + libvirt + xen to start an 
> instance in a simple configuration. Although the patches[2] are not 
> intended to be production-ready :) they help highlight some of the areas 
> we need to change.
 
 Dave, just to let you know that Tim has done some important "refactoring" 
 to split up XenServer hypervisor in CS between Xen and XenServer. That way 
 we could keep using xapi for XS but start moving to libvirt for Xen.
 
 Tim worked in the xen2server branch (don't ask about the name, I messed it 
 up…:) ).
 
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/xen2server
 
 Would be nice to see some of the libvirt stuff in that branch to handle a 
 new driver for Xen.
 
 Since the two hypervisors will be split up, we could still drop in some 
 early libvirt patches to handle Xen and put this in 4.5 as a wip.
>>> 
>>> Thanks for the links.
>>> 
>>> I’m slowly building up a set of patches here:
>>> 
>>> https://github.com/djs55/cloudstack/tree/virsh-capabilities
>>> 
>>> I think once I’ve gotten to a stable-ish point I’ll rebase on top of Tim’s 
>>> branch.
>>> 
>>> So far I’ve
>>> * changed the hypervisor detection to use ‘virsh capabilities’ in one place 
>>> and ‘cat /sys/hypervisor/type’ in another. Thinking about it again, I think 
>>> it’s probably best to standardise on /sys/hypervisor/type since that will 
>>> succeed irrespective of whether the libvirtd service is chkconfig’d on or 
>>> not.
>>> 
>>> * in the python cloudutils system setup stuff isKvmEnabled() has become 
>>> isHypervisorEnabled()
>>> 
>>> * added a XenLibvirtDiscoverer similar to the LXC one
>>> 
>>> * fixed what I believe is a race in sshExecuteCmdOneShotWithExitCode (which 
>>> seems to hit me every time, I don’t know why other people seem to be immune 
>>> from it): see CLOUDSTACK-6621 and review board request 21261
>>> 
>>> * added the new hypervisor to hypervisor.list and 
>>> system.vm.default.hypervisor, so it appears in the UI properly
>>> 
>>> * registered a system VM template in the database, using the same qcow2 
>>> image as KVM
>>> 
>>> For my test host I’m using a XenServer nightly snapshot which comes with a 
>>> nice modern xen and kernel, and is easy to install bleeding-edge libvirt on 
>>> top. I had to tweak the kernel configuration and the network configuration 
>>> but I’m hoping to make it work out of the box in future.
>>> 
>>> When I deploy my ‘datacenter’ the discovery phase works, the agent connects 
>>> and looks healthy in the logs and the UI. The next step is to figure out 
>>> why the system VM template isn’t being copied to primary storage — for some 
>>> reason the copy isn’t being attempted but I can’t see any obvious reason 
>>> why.
>> 
>> I’m now at the stage of getting my system VMs to start via libvirt. The main 
>> missing feature is support for : low-bandwidth private 
>> host<->guest control channels. These channels are generally useful things 
>> and are needed by other projects (like oVirt), so I’d like to add them to 
>> libxl and libvirt’s libxl driver. There’s a thread on xen-devel and 
>> libvir-devel if anyone’s interested:
>> 
>> http://www.redhat.com/archives/libvir-list/2014-June/msg00180.html
>> 
>> Once the  are sorted, basic VM operations should work. The next 
>> step would be to rebase my patches on top of Tim’s renaming changes and tidy 
>> them up for review.
>> 
>> Cheers,
>> Dave
>> 
>>> 
>>> Cheers,
>>> Dave
>>> 
 
 -Sebastien
 
> 
> Some of the areas are:
> 
> 1. hypervisor detection
> 
> Where we currently look for KVM specifically ("lsmod | grep kvm") we 
> could switch to either de

Re: Merge Review Request 22270: Xen 2 XenServer refactoring

[sebgoa]

Rohit,

Any chance you have time to check this one review.
My git kungfu is limited and the patch seems to have conflicts.

Tim did quite a bit of work to get there, so a merge could be call quickly, if 
this patch could apply, that would ease another round of rebasing..

thanks,,

-sebastien

On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:

> I've just submitted a review request which is essentially a merge of
> the xen2server feature branch back into master.  Since this is a
> refactoring of the Xen plugin to make it more explicitly a XenServer
> plugin per the feature:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer,
> I wanted to ensure everyone was aware of what is changing. Diff
> details can be found here: https://reviews.apache.org/r/22270/
> 
> The key item in this change is that what was the xen plugin has now
> moved to become the xenserver plugin to make way for a pure xen
> plugin.  If you are working on something which is XenServer specific,
> you'll probably want to take a look at what I've done, sanity check it
> against your plans and ask questions.  Additionally, if you've done
> some work in XenServer code since the start of April, you might want
> to make certain I didn't accidentally revert any of your changes
> during conflict resolution.  I tried to be careful, but accidents do
> happen.
> 
> The merge point was commit 603eab7 (HEAD yesterday), and from there I
> did a bunch of sanity testing using XenServer 6.2.  I also tested and
> validated with the current HEAD (8b5ec64).  If you were looking at or
> testing anything on the xen2server branch, no new functionality was
> introduced from that branch; this is effectively a merge with
> conflicts resolved.
> 
> Thanks
> 
> -tim

Re: Review Request 17994: CLOUDSTACK-6856 :Fixed Few Data Corruption and retries issues.

[ASF Subversion and Git Services]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17994/#review44906
---


Commit b43517c49629d46c2dd222bef0d8b1b2944abb66 in cloudstack's branch 
refs/heads/master from sedukull
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=b43517c ]

CLOUDSTACK-6856 : Fixed few data corruption and retries issues for failing cases

Signed-off-by: SrikanteswaraRao Talluri 


- ASF Subversion and Git Services


On June 6, 2014, 12:06 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17994/
> ---
> 
> (Updated June 6, 2014, 12:06 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and SrikanteswaraRao Talluri.
> 
> 
> Bugs: CLOUDSTACK-6856
> https://issues.apache.org/jira/browse/CLOUDSTACK-6856
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed Few cases failing because of data corruption and retries issues.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_deploy_vm.py 473b020 
>   test/integration/smoke/test_guest_vlan_range.py bfef4a8 
>   test/integration/smoke/test_hosts.py 52827aa 
>   test/integration/smoke/test_vm_life_cycle.py 62a807d 
>   tools/marvin/marvin/cloudstackTestClient.py 29a298b 
>   tools/marvin/setup.py 4c775ad 
> 
> Diff: https://reviews.apache.org/r/17994/diff/
> 
> 
> Testing
> ---
> 
> yes. 
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Review Request 17994: CLOUDSTACK-6856 :Fixed Few Data Corruption and retries issues.

[SrikanteswaraRao Talluri]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17994/#review44907
---

Ship it!


Ship It!

- SrikanteswaraRao Talluri


On June 6, 2014, 12:06 p.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17994/
> ---
> 
> (Updated June 6, 2014, 12:06 p.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and SrikanteswaraRao Talluri.
> 
> 
> Bugs: CLOUDSTACK-6856
> https://issues.apache.org/jira/browse/CLOUDSTACK-6856
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> Fixed Few cases failing because of data corruption and retries issues.
> 
> 
> Diffs
> -
> 
>   test/integration/smoke/test_deploy_vm.py 473b020 
>   test/integration/smoke/test_guest_vlan_range.py bfef4a8 
>   test/integration/smoke/test_hosts.py 52827aa 
>   test/integration/smoke/test_vm_life_cycle.py 62a807d 
>   tools/marvin/marvin/cloudstackTestClient.py 29a298b 
>   tools/marvin/setup.py 4c775ad 
> 
> Diff: https://reviews.apache.org/r/17994/diff/
> 
> 
> Testing
> ---
> 
> yes. 
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: Support pure Xen as a hypervisor follow-up

[Tim Mackey]

If you can give me the commit ID for your HEAD, I'll check and see
what's up.  It could be another conflict needing resolution.

-tim

On Fri, Jun 6, 2014 at 12:16 PM, sebgoa  wrote:
>
> On Jun 6, 2014, at 5:17 PM, Tim Mackey  wrote:
>
>> Dave,
>>
>> I've submitted a merge review request
>> (https://reviews.apache.org/r/22270/) yesterday.
>
> Unfortunately that patch did not seem to apply cleanly. I have not dug deep 
> though..
>
>> If you want to avoid
>> having to potentially deal with a bunch of conflicts, you might want
>> to see if your patches apply cleanly there and let me know.  Happy to
>> help with any conflict resolution.
>>
>> btw, I didn't see a design document up on the
>> wiki(https://cwiki.apache.org/confluence/display/CLOUDSTACK/4.5+Design+Documents).
>> Can you get one there and start a DISCUSS thread?  It'll probably
>> tease out some gotchas you might not be aware of.
>>
>> -tim
>>
>> On Fri, Jun 6, 2014 at 11:04 AM, Dave Scott  wrote:
>>> Hi,
>>>
>>> Here’s a quick status update:
>>>
>>> On 16 May 2014, at 15:22, Dave Scott  wrote:
>>>
 Hi,

 On 14 May 2014, at 09:53, sebgoa  wrote:

>
> On Apr 9, 2014, at 2:37 PM, Dave Scott  wrote:
>
>> Hi,
>>
>> Following up from Tim's "Support pure Xen as a hypervisor" proposal last 
>> month[1] I'd like to start working on this and maybe even make a little 
>> bit of progress while I'm at CCC in Denver.
>>
>> Helpfully James Bulpin managed to get CS + libvirt + xen to start an 
>> instance in a simple configuration. Although the patches[2] are not 
>> intended to be production-ready :) they help highlight some of the areas 
>> we need to change.
>
> Dave, just to let you know that Tim has done some important "refactoring" 
> to split up XenServer hypervisor in CS between Xen and XenServer. That 
> way we could keep using xapi for XS but start moving to libvirt for Xen.
>
> Tim worked in the xen2server branch (don't ask about the name, I messed 
> it up…:) ).
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/xen2server
>
> Would be nice to see some of the libvirt stuff in that branch to handle a 
> new driver for Xen.
>
> Since the two hypervisors will be split up, we could still drop in some 
> early libvirt patches to handle Xen and put this in 4.5 as a wip.

 Thanks for the links.

 I’m slowly building up a set of patches here:

 https://github.com/djs55/cloudstack/tree/virsh-capabilities

 I think once I’ve gotten to a stable-ish point I’ll rebase on top of Tim’s 
 branch.

 So far I’ve
 * changed the hypervisor detection to use ‘virsh capabilities’ in one 
 place and ‘cat /sys/hypervisor/type’ in another. Thinking about it again, 
 I think it’s probably best to standardise on /sys/hypervisor/type since 
 that will succeed irrespective of whether the libvirtd service is 
 chkconfig’d on or not.

 * in the python cloudutils system setup stuff isKvmEnabled() has become 
 isHypervisorEnabled()

 * added a XenLibvirtDiscoverer similar to the LXC one

 * fixed what I believe is a race in sshExecuteCmdOneShotWithExitCode 
 (which seems to hit me every time, I don’t know why other people seem to 
 be immune from it): see CLOUDSTACK-6621 and review board request 21261

 * added the new hypervisor to hypervisor.list and 
 system.vm.default.hypervisor, so it appears in the UI properly

 * registered a system VM template in the database, using the same qcow2 
 image as KVM

 For my test host I’m using a XenServer nightly snapshot which comes with a 
 nice modern xen and kernel, and is easy to install bleeding-edge libvirt 
 on top. I had to tweak the kernel configuration and the network 
 configuration but I’m hoping to make it work out of the box in future.

 When I deploy my ‘datacenter’ the discovery phase works, the agent 
 connects and looks healthy in the logs and the UI. The next step is to 
 figure out why the system VM template isn’t being copied to primary 
 storage — for some reason the copy isn’t being attempted but I can’t see 
 any obvious reason why.
>>>
>>> I’m now at the stage of getting my system VMs to start via libvirt. The 
>>> main missing feature is support for : low-bandwidth private 
>>> host<->guest control channels. These channels are generally useful things 
>>> and are needed by other projects (like oVirt), so I’d like to add them to 
>>> libxl and libvirt’s libxl driver. There’s a thread on xen-devel and 
>>> libvir-devel if anyone’s interested:
>>>
>>> http://www.redhat.com/archives/libvir-list/2014-June/msg00180.html
>>>
>>> Once the  are sorted, basic VM operations should work. The next 
>>> step would be to rebase my patches on top of Tim’s renaming changes and 
>>> tidy them up for review.
>>>
>

[MERGE] Review Request 22270: Xen 2 XenServer refactoring

[sebgoa]

Folks,

Tim has prepared a pretty significant patch in the review it lists below.
There is also a wiki page describing the change.

Basically it splits XenServer and XenProject (pure Xen) in two separate 
hypervisors. Until now we had used XenServer to also handle XenProject.
It will allow to split the way we connect to the two hypervisors.

Now Tim's patch is actually a patch to *master* and not the xen2server branch.

Since it's a ton of work to rebase such a big refactoring, I am wondering if we 
could not allow this patch to be applied in master directly.
And apply quickly…we are 24 hours now since Tim sent his email, another 48 
would make it 72.

Thoughts ?

-sebastien

On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:

> I've just submitted a review request which is essentially a merge of
> the xen2server feature branch back into master.  Since this is a
> refactoring of the Xen plugin to make it more explicitly a XenServer
> plugin per the feature:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer,
> I wanted to ensure everyone was aware of what is changing. Diff
> details can be found here: https://reviews.apache.org/r/22270/
> 
> The key item in this change is that what was the xen plugin has now
> moved to become the xenserver plugin to make way for a pure xen
> plugin.  If you are working on something which is XenServer specific,
> you'll probably want to take a look at what I've done, sanity check it
> against your plans and ask questions.  Additionally, if you've done
> some work in XenServer code since the start of April, you might want
> to make certain I didn't accidentally revert any of your changes
> during conflict resolution.  I tried to be careful, but accidents do
> happen.
> 
> The merge point was commit 603eab7 (HEAD yesterday), and from there I
> did a bunch of sanity testing using XenServer 6.2.  I also tested and
> validated with the current HEAD (8b5ec64).  If you were looking at or
> testing anything on the xen2server branch, no new functionality was
> introduced from that branch; this is effectively a merge with
> conflicts resolved.
> 
> Thanks
> 
> -tim

Re: Anybody addressing this bug ? Overlaping IP subnets across different vlans

[ilya musayev]

Andrija

I dont know if we can qualify this as a bug, this check was placed with 
some purpose in mind - yet its not clear what it is and why would 
someone think its bad.


https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=a3b1a49c303a929c754561ca07fd8a9ed84e0382

https://issues.apache.org/jira/browse/CLOUDSTACK-3911

Chime in on the discussion in thread above,

Regards,
ilya


On 6/6/14, 5:48 AM, Andrija Panic wrote:

Hi,
aftger upgrade to 4.3, I reported a bug where CS will not let me add
additional IP ranges when there are 2 vlans using same IP range - I
don't see point comparing IP ranges across two separate broadcast domains...

https://issues.apache.org/jira/browse/CLOUDSTACK-6814

Thanks,

Re: [ACS4.3.1][DISCUSS] Minor release of Apache CloudStack 4.3.1

[ilya musayev]

Thanks Dan.

This was meant to be sent to dev list but somehow, i've sent it to users.

Folks, please chime in.


On 6/6/14, 4:39 AM, Daan Hoogland wrote:

Ilya,

You are addressing non developpers, so I am crossposting.

On Fri, Jun 6, 2014 at 4:25 AM, ilya musayev
 wrote:

Creating a separate thread to see what community thinks on next minor
release of CloudStack 4.3.1

There are more patches that need to be cherry-picked/backported - i can
think of several for vmware-related ones that did not make it into 4.3
branch.

How do we go about finding the ones that did not make it?

I'd like to help with release of 4.3.1, but i will need help of at least 1
or 2 folks.

You dont need to be a developer to help with release process.

[ACS44] Cherry pick for CLOUDSTACK-6585

[Alena Prokharchyk]

Daan,

Can you cherry-pick the commit below, to 4.4? Somehow missed to include it to 
the original commit for the CLOUDSTACK-6585

commit e4fc5d88427456b9f9b8d37b49a4406db8c886c1
Author: Alena Prokharchyk 
Date:   Fri Jun 6 10:39:24 2014 -0700

CLOUDSTACK-6585: added missing db upgrade statements

-Alena.

Re: [MERGE] Review Request 22270: Xen 2 XenServer refactoring

[Rohit Yadav]

On Fri, Jun 6, 2014 at 10:14 PM, sebgoa  wrote:

> Folks,
>
> Tim has prepared a pretty significant patch in the review it lists below.
> There is also a wiki page describing the change.
>
> Basically it splits XenServer and XenProject (pure Xen) in two separate
> hypervisors. Until now we had used XenServer to also handle XenProject.
> It will allow to split the way we connect to the two hypervisors.
>
> Now Tim's patch is actually a patch to *master* and not the xen2server
> branch.
>
> Since it's a ton of work to rebase such a big refactoring, I am wondering
> if we could not allow this patch to be applied in master directly.
> And apply quickly...we are 24 hours now since Tim sent his email, another 48
> would make it 72.
>
> Thoughts ?
>


+1 I just reviewed the patch, I was able to apply it cleanly on latest
master and successfully do a clean build. I next tested it with DevCloud
and currently in middle of deploying a basic zone. If if this fails, I
guess it looks good to me for merging, if there are some issues we can
always revisit.

Command log:

wget https://reviews.apache.org/r/22270/diff/raw/  -O xen-tim.patch # the
patch was about 1.8M in size
cd cloudstack
git pull --rebase origin master
git am --ignore-whitespace ../xen-tim.patch
mvn clean install -P systemvm,developer # build was successfully, the
plugin is now Hypervisor XenServer
mvn -P developer -pl developer -Ddeploydb # clean db deployed successfully
mvn -pl :cloud-client-ui jetty:run # UI was up, I'm now in middle of
deploying basic zone

Regards.




>
> -sebastien
>
> On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
>
> > I've just submitted a review request which is essentially a merge of
> > the xen2server feature branch back into master.  Since this is a
> > refactoring of the Xen plugin to make it more explicitly a XenServer
> > plugin per the feature:
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
> ,
> > I wanted to ensure everyone was aware of what is changing. Diff
> > details can be found here: https://reviews.apache.org/r/22270/
> >
> > The key item in this change is that what was the xen plugin has now
> > moved to become the xenserver plugin to make way for a pure xen
> > plugin.  If you are working on something which is XenServer specific,
> > you'll probably want to take a look at what I've done, sanity check it
> > against your plans and ask questions.  Additionally, if you've done
> > some work in XenServer code since the start of April, you might want
> > to make certain I didn't accidentally revert any of your changes
> > during conflict resolution.  I tried to be careful, but accidents do
> > happen.
> >
> > The merge point was commit 603eab7 (HEAD yesterday), and from there I
> > did a bunch of sanity testing using XenServer 6.2.  I also tested and
> > validated with the current HEAD (8b5ec64).  If you were looking at or
> > testing anything on the xen2server branch, no new functionality was
> > introduced from that branch; this is effectively a merge with
> > conflicts resolved.
> >
> > Thanks
> >
> > -tim
>
>

Re: Merge Review Request 22270: Xen 2 XenServer refactoring

[Rohit Yadav]

Hi Sebastien,

On Fri, Jun 6, 2014 at 9:51 PM, sebgoa  wrote:

> Rohit,
>
> Any chance you have time to check this one review.
> My git kungfu is limited and the patch seems to have conflicts.
>
> Tim did quite a bit of work to get there, so a merge could be call
> quickly, if this patch could apply, that would ease another round of
> rebasing..
>

First of all great work Tim, kudos for your work!

I'm able to apply the patch from reviewboard on latest master now, I was
able to cleanly build it. I'm testing it right now against DevCloud, will
reply on the other thread where you were asking about merging it to master
as work from others could cause future conflicts.

The trick to merge such patches (and while working with branch) is to
ignore whitespaces etc, so:
git am --ignore-whitespace 

Tim, I see the xen plugin in new form is xenserver, I've a question - I see
there is only one plugin I see now "Plugin - Hypervisor XenServer", will
this work for both xen (xen.org, opensource version) and xenserver
(citrix's opensource release)?

Replied on other thread regarding merging on master, asap.

Regards.


>
> thanks,,
>
> -sebastien
>
> On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
>
> > I've just submitted a review request which is essentially a merge of
> > the xen2server feature branch back into master.  Since this is a
> > refactoring of the Xen plugin to make it more explicitly a XenServer
> > plugin per the feature:
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
> ,
> > I wanted to ensure everyone was aware of what is changing. Diff
> > details can be found here: https://reviews.apache.org/r/22270/
> >
> > The key item in this change is that what was the xen plugin has now
> > moved to become the xenserver plugin to make way for a pure xen
> > plugin.  If you are working on something which is XenServer specific,
> > you'll probably want to take a look at what I've done, sanity check it
> > against your plans and ask questions.  Additionally, if you've done
> > some work in XenServer code since the start of April, you might want
> > to make certain I didn't accidentally revert any of your changes
> > during conflict resolution.  I tried to be careful, but accidents do
> > happen.
> >
> > The merge point was commit 603eab7 (HEAD yesterday), and from there I
> > did a bunch of sanity testing using XenServer 6.2.  I also tested and
> > validated with the current HEAD (8b5ec64).  If you were looking at or
> > testing anything on the xen2server branch, no new functionality was
> > introduced from that branch; this is effectively a merge with
> > conflicts resolved.
> >
> > Thanks
> >
> > -tim
>
>

Re: Merge Review Request 22270: Xen 2 XenServer refactoring

[Rohit Yadav]

I just read the design document on the wiki
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
and got my answers regarding the work. Thanks.

Regards.


On Fri, Jun 6, 2014 at 11:48 PM, Rohit Yadav  wrote:

> Hi Sebastien,
>
> On Fri, Jun 6, 2014 at 9:51 PM, sebgoa  wrote:
>
>> Rohit,
>>
>> Any chance you have time to check this one review.
>> My git kungfu is limited and the patch seems to have conflicts.
>>
>> Tim did quite a bit of work to get there, so a merge could be call
>> quickly, if this patch could apply, that would ease another round of
>> rebasing..
>>
>
> First of all great work Tim, kudos for your work!
>
> I'm able to apply the patch from reviewboard on latest master now, I was
> able to cleanly build it. I'm testing it right now against DevCloud, will
> reply on the other thread where you were asking about merging it to master
> as work from others could cause future conflicts.
>
> The trick to merge such patches (and while working with branch) is to
> ignore whitespaces etc, so:
> git am --ignore-whitespace 
>
> Tim, I see the xen plugin in new form is xenserver, I've a question - I
> see there is only one plugin I see now "Plugin - Hypervisor XenServer",
> will this work for both xen (xen.org, opensource version) and xenserver
> (citrix's opensource release)?
>
> Replied on other thread regarding merging on master, asap.
>
> Regards.
>
>
>>
>> thanks,,
>>
>> -sebastien
>>
>> On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
>>
>> > I've just submitted a review request which is essentially a merge of
>> > the xen2server feature branch back into master.  Since this is a
>> > refactoring of the Xen plugin to make it more explicitly a XenServer
>> > plugin per the feature:
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
>> ,
>> > I wanted to ensure everyone was aware of what is changing. Diff
>> > details can be found here: https://reviews.apache.org/r/22270/
>> >
>> > The key item in this change is that what was the xen plugin has now
>> > moved to become the xenserver plugin to make way for a pure xen
>> > plugin.  If you are working on something which is XenServer specific,
>> > you'll probably want to take a look at what I've done, sanity check it
>> > against your plans and ask questions.  Additionally, if you've done
>> > some work in XenServer code since the start of April, you might want
>> > to make certain I didn't accidentally revert any of your changes
>> > during conflict resolution.  I tried to be careful, but accidents do
>> > happen.
>> >
>> > The merge point was commit 603eab7 (HEAD yesterday), and from there I
>> > did a bunch of sanity testing using XenServer 6.2.  I also tested and
>> > validated with the current HEAD (8b5ec64).  If you were looking at or
>> > testing anything on the xen2server branch, no new functionality was
>> > introduced from that branch; this is effectively a merge with
>> > conflicts resolved.
>> >
>> > Thanks
>> >
>> > -tim
>>
>>
>

Re: Review Request 19874: CLOUDSTACK-6793 : Fixed an Invalid Domain Id Issue

[Nitin Mehta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19874/#review44945
---

Ship it!


Hi Santosh - We should add the domain id column in the vm_template table. IMHO 
this is not the right fix but will mask problems where domain id is coming out 
-1. We need to see why domainid is coming out as -1 and solve it in the right 
way rather than masking it. 

- Nitin Mehta


On June 6, 2014, 9:28 a.m., Santhosh Edukulla wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19874/
> ---
> 
> (Updated June 6, 2014, 9:28 a.m.)
> 
> 
> Review request for cloudstack, daan Hoogland and Kishan Kavala.
> 
> 
> Bugs: CLOUDSTACK-6793
> https://issues.apache.org/jira/browse/CLOUDSTACK-6793
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> 1. Added fix to retrieve the correct domain id for a given accountid. 
> vm_template table and respective vo, does not have a domain id information.
> 2. Added logic to retrieve correct domainid for a given accountid, only when 
> an invalid domainid is retrieved. Throw an exception in case of invalid 
> domain id.
> 3. Now, creation of tags is getting successful post the fix.
> 
> 
> Diffs
> -
> 
>   engine/schema/src/com/cloud/user/dao/AccountDao.java 2f737cd 
>   engine/schema/src/com/cloud/user/dao/AccountDaoImpl.java 4bc23ed 
>   server/src/com/cloud/tags/TaggedResourceManagerImpl.java fa7fcb7 
> 
> Diff: https://reviews.apache.org/r/19874/diff/
> 
> 
> Testing
> ---
> 
> Test creation, listing and deletion tags on UserVM ... === TestName: 
> test_05_vm_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tag on templates ... === TestName: 
> test_06_template_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tags on ISO ... === TestName: 
> test_07_iso_tag | Status : SUCCESS ===
> ok
> Test creation, listing and deletion tagson volume ... === TestName: 
> test_08_volume_tag | Status : SUCCESS ===
> ok
> 
> 
> Thanks,
> 
> Santhosh Edukulla
> 
>

Re: [MERGE] Review Request 22270: Xen 2 XenServer refactoring

[Rohit Yadav]

Hi again,

I was unable to deploy basic zone, I did:

# marvin and python dependencies were pre-installed
cd tools/devcloud;
python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg

The error I got was related to agent communication, if this can help
resolve the issue:

Exception Occurred :['Traceback (most recent call last):\n', '  File
"../marvin/marvin/deployDataCenter.py", line 136, in addHosts\nret =
self.__apiClient.addHost(hostcmd)\n', '  File
"/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackAPI/cloudstackAPIClient.py",
line 1492, in addHost\nresponse =
self.connection.marvinRequest(command, response_type=response,
method=method)\n', '  File
"/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackConnection.py",
line 378, in marvinRequest\nraise e\n', 'CloudstackAPIException:
Execute cmd: addhost failed, due to: errorCode: 530, errorText:Cannot
transit agent status with event AgentDisconnected for host 1, mangement
server id is 4278190080,Unable to transition to a new state from Creating
via AgentDisconnected\n']
But, in the management server log I see that it identified the product
version:
"Found host devcloud ip=192.168.56.10 product version=1.6.0"

In this case I'm suspecting my local env issue, the logs suggested it was
unable to create the directory "/opt/cloud/bin" and some agent exceptions.

Tim, I think it works but failed on my env due to some env specific issue;
what modification etc. do we have to do to make it work against DevCloud,
or install libs/dependencies inside it?

The patch is clean, builds and ACS runs and upon adding host it identified
and triggered the XenServer plugin so I think we can allow it to merge on
master (but we need to fix it to make it work with Xen.org xen server used
inside DevCloud using deployDataCenter script).

If no one objects, may I merge it on master?

Regards.




On Fri, Jun 6, 2014 at 11:46 PM, Rohit Yadav  wrote:

> On Fri, Jun 6, 2014 at 10:14 PM, sebgoa  wrote:
>
>> Folks,
>>
>> Tim has prepared a pretty significant patch in the review it lists below.
>> There is also a wiki page describing the change.
>>
>> Basically it splits XenServer and XenProject (pure Xen) in two separate
>> hypervisors. Until now we had used XenServer to also handle XenProject.
>> It will allow to split the way we connect to the two hypervisors.
>>
>> Now Tim's patch is actually a patch to *master* and not the xen2server
>> branch.
>>
>> Since it's a ton of work to rebase such a big refactoring, I am wondering
>> if we could not allow this patch to be applied in master directly.
>> And apply quickly...we are 24 hours now since Tim sent his email, another
>> 48 would make it 72.
>>
>> Thoughts ?
>>
>
>
> +1 I just reviewed the patch, I was able to apply it cleanly on latest
> master and successfully do a clean build. I next tested it with DevCloud
> and currently in middle of deploying a basic zone. If if this fails, I
> guess it looks good to me for merging, if there are some issues we can
> always revisit.
>
> Command log:
>
> wget https://reviews.apache.org/r/22270/diff/raw/  -O xen-tim.patch # the
> patch was about 1.8M in size
> cd cloudstack
> git pull --rebase origin master
> git am --ignore-whitespace ../xen-tim.patch
> mvn clean install -P systemvm,developer # build was successfully, the
> plugin is now Hypervisor XenServer
> mvn -P developer -pl developer -Ddeploydb # clean db deployed successfully
> mvn -pl :cloud-client-ui jetty:run # UI was up, I'm now in middle of
> deploying basic zone
>
> Regards.
>
>
>
>
>>
>> -sebastien
>>
>> On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
>>
>> > I've just submitted a review request which is essentially a merge of
>> > the xen2server feature branch back into master.  Since this is a
>> > refactoring of the Xen plugin to make it more explicitly a XenServer
>> > plugin per the feature:
>> >
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
>> ,
>> > I wanted to ensure everyone was aware of what is changing. Diff
>> > details can be found here: https://reviews.apache.org/r/22270/
>> >
>> > The key item in this change is that what was the xen plugin has now
>> > moved to become the xenserver plugin to make way for a pure xen
>> > plugin.  If you are working on something which is XenServer specific,
>> > you'll probably want to take a look at what I've done, sanity check it
>> > against your plans and ask questions.  Additionally, if you've done
>> > some work in XenServer code since the start of April, you might want
>> > to make certain I didn't accidentally revert any of your changes
>> > during conflict resolution.  I tried to be careful, but accidents do
>> > happen.
>> >
>> > The merge point was commit 603eab7 (HEAD yesterday), and from there I
>> > did a bunch of sanity testing using XenServer 6.2.  I also tested and
>> > validated with the current HEAD (8b5ec64).  If you were looking at or
>> > testing anything on the xen

Modifications of HTML/CSS/JS files without recompling

[Ian Duffy]

Hi UI Team,

Last time I worked on the UI I found myself recompiling and restarting
jetty every time I modified a HTML/CSS/JS File.

Do we have some method to avoid doing this? I tried doing a symlink of the
source directory with the target directory which did work but it was messy.

Thanks,

Ian

Re: [MERGE] Review Request 22270: Xen 2 XenServer refactoring

[sebgoa]

On Jun 6, 2014, at 8:41 PM, Rohit Yadav  wrote:

> Hi again,
> 
> I was unable to deploy basic zone, I did:
> 
> # marvin and python dependencies were pre-installed
> cd tools/devcloud;
> python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
> 
> The error I got was related to agent communication, if this can help resolve 
> the issue:
> Exception Occurred :['Traceback (most recent call last):\n', '  File 
> "../marvin/marvin/deployDataCenter.py", line 136, in addHosts\nret = 
> self.__apiClient.addHost(hostcmd)\n', '  File 
> "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackAPI/cloudstackAPIClient.py",
>  line 1492, in addHost\nresponse = self.connection.marvinRequest(command, 
> response_type=response, method=method)\n', '  File 
> "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackConnection.py",
>  line 378, in marvinRequest\nraise e\n', 'CloudstackAPIException: Execute 
> cmd: addhost failed, due to: errorCode: 530, errorText:Cannot transit agent 
> status with event AgentDisconnected for host 1, mangement server id is 
> 4278190080,Unable to transition to a new state from Creating via 
> AgentDisconnected\n']
> 

DevCloud is broken with 4.4 and master...

> But, in the management server log I see that it identified the product 
> version:
> "Found host devcloud ip=192.168.56.10 product version=1.6.0"
> 
> In this case I'm suspecting my local env issue, the logs suggested it was 
> unable to create the directory "/opt/cloud/bin" and some agent exceptions.
> 
> Tim, I think it works but failed on my env due to some env specific issue; 
> what modification etc. do we have to do to make it work against DevCloud, or 
> install libs/dependencies inside it?
> 
> The patch is clean, builds and ACS runs and upon adding host it identified 
> and triggered the XenServer plugin so I think we can allow it to merge on 
> master (but we need to fix it to make it work with Xen.org xen server used 
> inside DevCloud using deployDataCenter script).
> 
> If no one objects, may I merge it on master?
> 
> Regards.
> 
> 
> 
> 
> On Fri, Jun 6, 2014 at 11:46 PM, Rohit Yadav  wrote:
> On Fri, Jun 6, 2014 at 10:14 PM, sebgoa  wrote:
> Folks,
> 
> Tim has prepared a pretty significant patch in the review it lists below.
> There is also a wiki page describing the change.
> 
> Basically it splits XenServer and XenProject (pure Xen) in two separate 
> hypervisors. Until now we had used XenServer to also handle XenProject.
> It will allow to split the way we connect to the two hypervisors.
> 
> Now Tim's patch is actually a patch to *master* and not the xen2server branch.
> 
> Since it's a ton of work to rebase such a big refactoring, I am wondering if 
> we could not allow this patch to be applied in master directly.
> And apply quickly…we are 24 hours now since Tim sent his email, another 48 
> would make it 72.
> 
> Thoughts ?
> 
> 
> +1 I just reviewed the patch, I was able to apply it cleanly on latest master 
> and successfully do a clean build. I next tested it with DevCloud and 
> currently in middle of deploying a basic zone. If if this fails, I guess it 
> looks good to me for merging, if there are some issues we can always revisit.
> 
> Command log:
> wget https://reviews.apache.org/r/22270/diff/raw/  -O xen-tim.patch # the 
> patch was about 1.8M in size
> cd cloudstack
> git pull --rebase origin master
> git am --ignore-whitespace ../xen-tim.patch
> mvn clean install -P systemvm,developer # build was successfully, the plugin 
> is now Hypervisor XenServer
> mvn -P developer -pl developer -Ddeploydb # clean db deployed successfully
> mvn -pl :cloud-client-ui jetty:run # UI was up, I'm now in middle of 
> deploying basic zone
> 
> Regards.
> 
> 
> 
>  
> 
> -sebastien
> 
> On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
> 
> > I've just submitted a review request which is essentially a merge of
> > the xen2server feature branch back into master.  Since this is a
> > refactoring of the Xen plugin to make it more explicitly a XenServer
> > plugin per the feature:
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer,
> > I wanted to ensure everyone was aware of what is changing. Diff
> > details can be found here: https://reviews.apache.org/r/22270/
> >
> > The key item in this change is that what was the xen plugin has now
> > moved to become the xenserver plugin to make way for a pure xen
> > plugin.  If you are working on something which is XenServer specific,
> > you'll probably want to take a look at what I've done, sanity check it
> > against your plans and ask questions.  Additionally, if you've done
> > some work in XenServer code since the start of April, you might want
> > to make certain I didn't accidentally revert any of your changes
> > during conflict resolution.  I tried to be careful, but accidents do
> > happen.
> >
> > The merge point was commit 603eab7 (HEAD yesterday), and from there I
> > did a bu

Re: [MERGE] Review Request 22270: Xen 2 XenServer refactoring

[Rohit Yadav]

On Sat, Jun 7, 2014 at 12:51 AM, sebgoa  wrote:

>
> On Jun 6, 2014, at 8:41 PM, Rohit Yadav  wrote:
>
> > Hi again,
> >
> > I was unable to deploy basic zone, I did:
> >
> > # marvin and python dependencies were pre-installed
> > cd tools/devcloud;
> > python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
> >
> > The error I got was related to agent communication, if this can help
> resolve the issue:
> > Exception Occurred :['Traceback (most recent call last):\n', '  File
> "../marvin/marvin/deployDataCenter.py", line 136, in addHosts\nret =
> self.__apiClient.addHost(hostcmd)\n', '  File
> "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackAPI/cloudstackAPIClient.py",
> line 1492, in addHost\nresponse =
> self.connection.marvinRequest(command, response_type=response,
> method=method)\n', '  File
> "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackConnection.py",
> line 378, in marvinRequest\nraise e\n', 'CloudstackAPIException:
> Execute cmd: addhost failed, due to: errorCode: 530, errorText:Cannot
> transit agent status with event AgentDisconnected for host 1, mangement
> server id is 4278190080,Unable to transition to a new state from Creating
> via AgentDisconnected\n']
> >
>
> DevCloud is broken with 4.4 and master...
>

o.O How do people test ACS who lack dedicated infrastructure? In that case,
let's merge the patch?

Regards.


>
> > But, in the management server log I see that it identified the product
> version:
> > "Found host devcloud ip=192.168.56.10 product version=1.6.0"
> >
> > In this case I'm suspecting my local env issue, the logs suggested it
> was unable to create the directory "/opt/cloud/bin" and some agent
> exceptions.
> >
> > Tim, I think it works but failed on my env due to some env specific
> issue; what modification etc. do we have to do to make it work against
> DevCloud, or install libs/dependencies inside it?
> >
> > The patch is clean, builds and ACS runs and upon adding host it
> identified and triggered the XenServer plugin so I think we can allow it to
> merge on master (but we need to fix it to make it work with Xen.org xen
> server used inside DevCloud using deployDataCenter script).
> >
> > If no one objects, may I merge it on master?
> >
> > Regards.
> >
> >
> >
> >
> > On Fri, Jun 6, 2014 at 11:46 PM, Rohit Yadav 
> wrote:
> > On Fri, Jun 6, 2014 at 10:14 PM, sebgoa  wrote:
> > Folks,
> >
> > Tim has prepared a pretty significant patch in the review it lists below.
> > There is also a wiki page describing the change.
> >
> > Basically it splits XenServer and XenProject (pure Xen) in two separate
> hypervisors. Until now we had used XenServer to also handle XenProject.
> > It will allow to split the way we connect to the two hypervisors.
> >
> > Now Tim's patch is actually a patch to *master* and not the xen2server
> branch.
> >
> > Since it's a ton of work to rebase such a big refactoring, I am
> wondering if we could not allow this patch to be applied in master directly.
> > And apply quickly...we are 24 hours now since Tim sent his email, another
> 48 would make it 72.
> >
> > Thoughts ?
> >
> >
> > +1 I just reviewed the patch, I was able to apply it cleanly on latest
> master and successfully do a clean build. I next tested it with DevCloud
> and currently in middle of deploying a basic zone. If if this fails, I
> guess it looks good to me for merging, if there are some issues we can
> always revisit.
> >
> > Command log:
> > wget https://reviews.apache.org/r/22270/diff/raw/  -O xen-tim.patch #
> the patch was about 1.8M in size
> > cd cloudstack
> > git pull --rebase origin master
> > git am --ignore-whitespace ../xen-tim.patch
> > mvn clean install -P systemvm,developer # build was successfully, the
> plugin is now Hypervisor XenServer
> > mvn -P developer -pl developer -Ddeploydb # clean db deployed
> successfully
> > mvn -pl :cloud-client-ui jetty:run # UI was up, I'm now in middle of
> deploying basic zone
> >
> > Regards.
> >
> >
> >
> >
> >
> > -sebastien
> >
> > On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
> >
> > > I've just submitted a review request which is essentially a merge of
> > > the xen2server feature branch back into master.  Since this is a
> > > refactoring of the Xen plugin to make it more explicitly a XenServer
> > > plugin per the feature:
> > >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer
> ,
> > > I wanted to ensure everyone was aware of what is changing. Diff
> > > details can be found here: https://reviews.apache.org/r/22270/
> > >
> > > The key item in this change is that what was the xen plugin has now
> > > moved to become the xenserver plugin to make way for a pure xen
> > > plugin.  If you are working on something which is XenServer specific,
> > > you'll probably want to take a look at what I've done, sanity check it
> > > against your plans and ask questions.  Additionally, if you've done
> > > some work in XenServe

Re: [ACS4.3.1][DISCUSS] Minor release of Apache CloudStack 4.3.1

[Daan Hoogland]

No Ilya,

I did that on purpose as you where asking non-devs to participate in
the release proces.

On Fri, Jun 6, 2014 at 8:09 PM, ilya musayev
 wrote:
> Thanks Dan.
>
> This was meant to be sent to dev list but somehow, i've sent it to users.
>
> Folks, please chime in.
>
>
>
> On 6/6/14, 4:39 AM, Daan Hoogland wrote:
>>
>> Ilya,
>>
>> You are addressing non developpers, so I am crossposting.
>>
>> On Fri, Jun 6, 2014 at 4:25 AM, ilya musayev
>>  wrote:
>>>
>>> Creating a separate thread to see what community thinks on next minor
>>> release of CloudStack 4.3.1
>>>
>>> There are more patches that need to be cherry-picked/backported - i can
>>> think of several for vmware-related ones that did not make it into 4.3
>>> branch.
>>>
>>> How do we go about finding the ones that did not make it?
>>>
>>> I'd like to help with release of 4.3.1, but i will need help of at least
>>> 1
>>> or 2 folks.
>>>
>>> You dont need to be a developer to help with release process.
>>>
>>>
>>
>>
>



-- 
Daan

Re: [MERGE] Review Request 22270: Xen 2 XenServer refactoring

[sebgoa]

On Jun 6, 2014, at 9:35 PM, Rohit Yadav  wrote:

> 
> 
> 
> On Sat, Jun 7, 2014 at 12:51 AM, sebgoa  wrote:
> 
> On Jun 6, 2014, at 8:41 PM, Rohit Yadav  wrote:
> 
> > Hi again,
> >
> > I was unable to deploy basic zone, I did:
> >
> > # marvin and python dependencies were pre-installed
> > cd tools/devcloud;
> > python ../marvin/marvin/deployDataCenter.py -i devcloud.cfg
> >
> > The error I got was related to agent communication, if this can help 
> > resolve the issue:
> > Exception Occurred :['Traceback (most recent call last):\n', '  File 
> > "../marvin/marvin/deployDataCenter.py", line 136, in addHosts\nret = 
> > self.__apiClient.addHost(hostcmd)\n', '  File 
> > "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackAPI/cloudstackAPIClient.py",
> >  line 1492, in addHost\nresponse = 
> > self.connection.marvinRequest(command, response_type=response, 
> > method=method)\n', '  File 
> > "/Library/Python/2.7/site-packages/Marvin-0.1.0-py2.7.egg/marvin/cloudstackConnection.py",
> >  line 378, in marvinRequest\nraise e\n', 'CloudstackAPIException: 
> > Execute cmd: addhost failed, due to: errorCode: 530, errorText:Cannot 
> > transit agent status with event AgentDisconnected for host 1, mangement 
> > server id is 4278190080,Unable to transition to a new state from Creating 
> > via AgentDisconnected\n']
> >
> 
> DevCloud is broken with 4.4 and master...
> 
> o.O How do people test ACS who lack dedicated infrastructure? In that case, 
> let's merge the patch?
> 

I will wait till tomorrow morning, and merge then if no-one complained. This 
will be a fast merge, but if people are not happy later on, I will revert.

Tim went the extra mile to get this done, so I don't want to miss the window.

-sebastien

> Regards.
>  
> 
> > But, in the management server log I see that it identified the product 
> > version:
> > "Found host devcloud ip=192.168.56.10 product version=1.6.0"
> >
> > In this case I'm suspecting my local env issue, the logs suggested it was 
> > unable to create the directory "/opt/cloud/bin" and some agent exceptions.
> >
> > Tim, I think it works but failed on my env due to some env specific issue; 
> > what modification etc. do we have to do to make it work against DevCloud, 
> > or install libs/dependencies inside it?
> >
> > The patch is clean, builds and ACS runs and upon adding host it identified 
> > and triggered the XenServer plugin so I think we can allow it to merge on 
> > master (but we need to fix it to make it work with Xen.org xen server used 
> > inside DevCloud using deployDataCenter script).
> >
> > If no one objects, may I merge it on master?
> >
> > Regards.
> >
> >
> >
> >
> > On Fri, Jun 6, 2014 at 11:46 PM, Rohit Yadav  wrote:
> > On Fri, Jun 6, 2014 at 10:14 PM, sebgoa  wrote:
> > Folks,
> >
> > Tim has prepared a pretty significant patch in the review it lists below.
> > There is also a wiki page describing the change.
> >
> > Basically it splits XenServer and XenProject (pure Xen) in two separate 
> > hypervisors. Until now we had used XenServer to also handle XenProject.
> > It will allow to split the way we connect to the two hypervisors.
> >
> > Now Tim's patch is actually a patch to *master* and not the xen2server 
> > branch.
> >
> > Since it's a ton of work to rebase such a big refactoring, I am wondering 
> > if we could not allow this patch to be applied in master directly.
> > And apply quickly…we are 24 hours now since Tim sent his email, another 48 
> > would make it 72.
> >
> > Thoughts ?
> >
> >
> > +1 I just reviewed the patch, I was able to apply it cleanly on latest 
> > master and successfully do a clean build. I next tested it with DevCloud 
> > and currently in middle of deploying a basic zone. If if this fails, I 
> > guess it looks good to me for merging, if there are some issues we can 
> > always revisit.
> >
> > Command log:
> > wget https://reviews.apache.org/r/22270/diff/raw/  -O xen-tim.patch # the 
> > patch was about 1.8M in size
> > cd cloudstack
> > git pull --rebase origin master
> > git am --ignore-whitespace ../xen-tim.patch
> > mvn clean install -P systemvm,developer # build was successfully, the 
> > plugin is now Hypervisor XenServer
> > mvn -P developer -pl developer -Ddeploydb # clean db deployed successfully
> > mvn -pl :cloud-client-ui jetty:run # UI was up, I'm now in middle of 
> > deploying basic zone
> >
> > Regards.
> >
> >
> >
> >
> >
> > -sebastien
> >
> > On Jun 5, 2014, at 8:02 PM, Tim Mackey  wrote:
> >
> > > I've just submitted a review request which is essentially a merge of
> > > the xen2server feature branch back into master.  Since this is a
> > > refactoring of the Xen plugin to make it more explicitly a XenServer
> > > plugin per the feature:
> > > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Convert+Xen+usage+to+XenServer,
> > > I wanted to ensure everyone was aware of what is changing. Diff
> > > details can be found here: https://reviews.apache.org/r/22270/
> >

Re: [ACS44] Cherry pick for CLOUDSTACK-6585

[Daan Hoogland]

On Fri, Jun 6, 2014 at 8:14 PM, Alena Prokharchyk
 wrote:
> e4fc5d88427456b9f9b8d37b49a4406db8c886c1


the object is not found. Did you push it?

-- 
Daan

Re: [ACS44] Cherry pick for CLOUDSTACK-6585

[Daan Hoogland]

did you mean 9e2db2f2210904752eea2f3abf87b16bb3080384?

On Fri, Jun 6, 2014 at 9:55 PM, Daan Hoogland  wrote:
> On Fri, Jun 6, 2014 at 8:14 PM, Alena Prokharchyk
>  wrote:
>> e4fc5d88427456b9f9b8d37b49a4406db8c886c1
>
>
> the object is not found. Did you push it?
>
> --
> Daan



-- 
Daan

Re: [ACS44] Cherry pick for CLOUDSTACK-6585

[Alena Prokharchyk]

Yes.

On 6/6/14, 12:57 PM, "Daan Hoogland"  wrote:

>did you mean 9e2db2f2210904752eea2f3abf87b16bb3080384?
>
>On Fri, Jun 6, 2014 at 9:55 PM, Daan Hoogland 
>wrote:
>> On Fri, Jun 6, 2014 at 8:14 PM, Alena Prokharchyk
>>  wrote:
>>> e4fc5d88427456b9f9b8d37b49a4406db8c886c1
>>
>>
>> the object is not found. Did you push it?
>>
>> --
>> Daan
>
>
>
>-- 
>Daan

Re: [ACS44] Cherry pick for CLOUDSTACK-6585

[Daan Hoogland]

done

On Fri, Jun 6, 2014 at 10:04 PM, Alena Prokharchyk
 wrote:
> Yes.
>
> On 6/6/14, 12:57 PM, "Daan Hoogland"  wrote:
>
>>did you mean 9e2db2f2210904752eea2f3abf87b16bb3080384?
>>
>>On Fri, Jun 6, 2014 at 9:55 PM, Daan Hoogland 
>>wrote:
>>> On Fri, Jun 6, 2014 at 8:14 PM, Alena Prokharchyk
>>>  wrote:
 e4fc5d88427456b9f9b8d37b49a4406db8c886c1
>>>
>>>
>>> the object is not found. Did you push it?
>>>
>>> --
>>> Daan
>>
>>
>>
>>--
>>Daan
>



-- 
Daan

Re: KVM + LXC on the same host

[ilya musayev]

Tuna

Thanks for the feedback, conceptually, i was not trying to address the 
issue of sysvms not running on LXC.


Here is the use case as i see it.

Assume you roll out a farm of  10 KVM servers and your density with KVM 
is 200 virtual machines. Most of these VMs dont require independent 
kernel and additional layers of virtualization abstraction. As the 
result, you can place 400 LXC machines and 20 fully virtualized Linux or 
even Windows Servers. If you do chargeback, you can offer the LXC 
machines for much lower price point since we can place more LXC containers.


Your density becomes much greater with LXC and yet you still cover the 
corner case when end-user needs KVM instance.


Pierre-Luc

If I will get to try this scenario - i may have to alter the KVM/LXC 
agent somewhat to make it work with CloudStack, i will let you know. 
Alternative solution would be to run separate LXC and KVM clusters, 
which is also a possibility - but usage and distribution will be uneven.


Regards
ilya

The concept of combining both technologies under one hypervisor
On 6/6/14, 8:42 AM, Pierre-Luc Dion wrote:

ilya,

Let us know how it goes your hybrid of LXC+KVM. I'm interested to know how
it's going,  I might try that too on the side.




Pierre-Luc Dion
Architecte de Solution Cloud | Cloud Solutions Architect
855-OK-CLOUD (855-652-5683) x1101
- - -

*CloudOps*420 rue Guy
Montréal QC  H3J 1S6
www.cloudops.com
@CloudOps_


On Fri, Jun 6, 2014 at 10:53 AM, Nguyen Anh Tu  wrote:


That should be a good idea, Ilya. At that moment, i'm working on Docker
support. When it's done, we can run only Docker hosts, no need to use KVM
for hosting system vms.

Cheers,
--Tuna

Sent from my GT-N7000
On Jun 5, 2014 7:58 AM, "ilya musayev" 
wrote:


We are considering running KVM and LXC on the same host and hopefully
control both through cloudstack.

I know there are agents involved for each component, i dont know if we

can

have a hybrid of LXC+KVM.

The use case is simple, we would like the end user to pick LXC/Docker for
performance, or KVM instance if he really needed all bells and whistles

of

dedicated kernel in fully virtualized environment.

Is anyone aware why we should not mix 2 workloads on the same host? Is it
possible at this point in time to mix LXC, KVM and CloudStack, i assume

the

answer is no, but perhaps there is a hack i can try.

Thanks
ilya

Re: DevCloud installation

[ilya musayev]

I'd boot it into single user mode and reset the password.

On 6/5/14, 6:04 PM, Mike Tutkowski wrote:

Hey Edison,

Seif is using a copy of DevCloud named devcloud2.ova (908,991,488 bytes).
It was downloaded back in August of last year.

I don't use DevCloud much these days as I build this all myself using
XenServer, ESX, and/or KVM (as the need dictates), but for Seif's GSoC
project I thought it might be easier for him to use DevCloud, so I uploaded
the OVA I have to a Google Drive folder of his.

This DevCloud OVA works just fine for me. I am able to log in, but Seif is
not.

Thanks!
Mike


On Thu, Jun 5, 2014 at 6:06 PM, Edison Su  wrote:


Which devcloud ova are you using?


-Original Message-
From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com]
Sent: Thursday, June 05, 2014 1:39 PM
To: dev@cloudstack.apache.org
Subject: DevCloud installation

Hi all,

I am trying to install "DevCloud". I donwloaded the OVA, imported it to
VirtualBox then started the VM up. It asked me to login. I used the

login =

root and thr Password= password It strangely refuses to accept this
combination.
What can be the problem ? (I am on Ubuntu 13.04).
thanks

[DISCUSS] try-with-resouce use

[Daan Hoogland]

H,

I have commited a patch to Upgrade430to440.java that removes all
close() calls on PreparedStament en ResultSet objects, replacing it
with the try() construct in java 7. I was tempted for a short
moment to back-port it to older Upgrade classes. As those upgrade
paths are to be supported by lower versions of java. In master (and
branch 4.4) however these are only to be run with java 7. So should we
fix the upgrades by try-with-resources or not?

-- 
Daan

Re: DevCloud installation

[Mike Tutkowski]

Just heard from Seif.

It was a character-encoding issue with his keyboard.

On Friday, June 6, 2014, ilya musayev  wrote:

> I'd boot it into single user mode and reset the password.
>
> On 6/5/14, 6:04 PM, Mike Tutkowski wrote:
>
>> Hey Edison,
>>
>> Seif is using a copy of DevCloud named devcloud2.ova (908,991,488 bytes).
>> It was downloaded back in August of last year.
>>
>> I don't use DevCloud much these days as I build this all myself using
>> XenServer, ESX, and/or KVM (as the need dictates), but for Seif's GSoC
>> project I thought it might be easier for him to use DevCloud, so I
>> uploaded
>> the OVA I have to a Google Drive folder of his.
>>
>> This DevCloud OVA works just fine for me. I am able to log in, but Seif is
>> not.
>>
>> Thanks!
>> Mike
>>
>>
>> On Thu, Jun 5, 2014 at 6:06 PM, Edison Su  wrote:
>>
>>  Which devcloud ova are you using?
>>>
>>>  -Original Message-
 From: Seif Eddine Jemli [mailto:seifeddineje...@gmail.com]
 Sent: Thursday, June 05, 2014 1:39 PM
 To: dev@cloudstack.apache.org
 Subject: DevCloud installation

 Hi all,

 I am trying to install "DevCloud". I donwloaded the OVA, imported it to
 VirtualBox then started the VM up. It asked me to login. I used the

>>> login =
>>>
 root and thr Password= password It strangely refuses to accept this
 combination.
 What can be the problem ? (I am on Ubuntu 13.04).
 thanks

>>>
>>
>>
>

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

RE: [DISCUSS] try-with-resouce use

[Alex Huang]

In general, as long as it works in the new jvm, I don't see us changing the 
code.  However, if you have the time to do it.  The upgrade classes are meant 
to be immutable as far as data goes.  So the idea is we can do upgrades from 
any version to the current version.  Because of that, then all upgrade classes 
could be run in the new jvm.

I don't see why older upgrade classes in master branch be run in older jvms.

--Alex

> -Original Message-
> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> Sent: Friday, June 6, 2014 1:23 PM
> To: dev
> Subject: [DISCUSS] try-with-resouce use
> 
> H,
> 
> I have commited a patch to Upgrade430to440.java that removes all
> close() calls on PreparedStament en ResultSet objects, replacing it with the
> try() construct in java 7. I was tempted for a short moment to
> back-port it to older Upgrade classes. As those upgrade paths are to be
> supported by lower versions of java. In master (and branch 4.4) however
> these are only to be run with java 7. So should we fix the upgrades by try-
> with-resources or not?
> 
> --
> Daan

Re: [DISCUSS] try-with-resouce use

[Daan Hoogland]

I don't want to fix resource leaks in master in a way we can not
backport. That's my point. And  no I am not going to do a lot of it
but want to set a policy.

On Fri, Jun 6, 2014 at 10:46 PM, Alex Huang  wrote:
> In general, as long as it works in the new jvm, I don't see us changing the 
> code.  However, if you have the time to do it.  The upgrade classes are meant 
> to be immutable as far as data goes.  So the idea is we can do upgrades from 
> any version to the current version.  Because of that, then all upgrade 
> classes could be run in the new jvm.
>
> I don't see why older upgrade classes in master branch be run in older jvms.
>
> --Alex
>
>> -Original Message-
>> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
>> Sent: Friday, June 6, 2014 1:23 PM
>> To: dev
>> Subject: [DISCUSS] try-with-resouce use
>>
>> H,
>>
>> I have commited a patch to Upgrade430to440.java that removes all
>> close() calls on PreparedStament en ResultSet objects, replacing it with the
>> try() construct in java 7. I was tempted for a short moment to
>> back-port it to older Upgrade classes. As those upgrade paths are to be
>> supported by lower versions of java. In master (and branch 4.4) however
>> these are only to be run with java 7. So should we fix the upgrades by try-
>> with-resources or not?
>>
>> --
>> Daan



-- 
Daan

Re: SNAT iptable entry on VirtualRouter

[Sheng Yang]

Hi Sachin,

The nicDevId() you see is coming from
prepareNetworkElementCommand(IpAssocCommand cmd) in CitrixResourceBase in
case of Xen.

You would see this:

if (addVif) {
// Add a new VIF to DomR
String vifDeviceNum =
getLowestAvailableVIFDeviceNum(conn, router);

if (vifDeviceNum == null) {
throw new InternalErrorException("There were no
more available slots for a new VIF on router: " +
router.getNameLabel(conn));
}

nic.setDeviceId(Integer.valueOf(vifDeviceNum));

correctVif = createVif(conn, routerName, router, null,
nic);
correctVif.plug(conn);
// Add iptables rule for network usage
networkUsage(conn, routerIp, "addVif", "eth" +
correctVif.getDevice(conn));
}

And nic.setDeviceId() should set the public nic id(which should be 2 in
your case) to it.

And what's the XenServer version you're using? Could you help to debug it
further more? Sadly we cannot reproduce it in our lab...

Thanks!

--Sheng


On Fri, Jun 6, 2014 at 12:29 AM, Sachchidanand Vaidya 
wrote:

> Hi,
>I'm seeing the same issue with 4.4 code.  After further debug, I see
> that CS mgmt server is sending
> following command to XenHost,
> xensource.log:  /opt/cloud/bin/ipassoc.sh -A -s -f -l
> 10.84.59.131/24 -c eth0 -g 10.84.59.254 
> VirtualRouter's public interface is eth2. Also as per dump of VIF list on
> XenHost, deviceid for public interface
> of domainRouter is 2.
> As part of VirtualRoutingResource.java:generateConfig(), CS mgmt server
> generates this command.
> It generates publicNic = "eth" + ip.getNicDevId()?
> Which deviceId does it refer to? Shouldn't it be the deviceid as per the
> XenHost dump?
> Does anyone have input on what could he happening here ?
>
> Thanks,
> Sachin
>
>
>
> >Hi,
> >I have an isolated network (192.168.3.x/24) being served by
> >VirtualRouter, where 10.84.59.131 is SourceNAT address
> >and eth0 is VN interface of VirtualRouter & eth2 is  the public interface
> >of VirtualRouter.
> >
> > When I look at the nat table entries on the VirtualRouter, it shows
> >following :
> >
> >root@r-6-VM:~# iptables -L -t nat -n -v
> >..
> >..
> >Chain POSTROUTING (policy ACCEPT 330 packets, 22113 bytes)
> > pkts bytes target prot opt in out source
> >destination
> >0 0 SNAT   all  --  *  eth00.0.0.0/0
> >0.0.0.0/0to:10.84.59.131
> >
> >--> Why the "out" interface for the SNAT entry is VN interface (eth0)
> >instead of Public interface (eth2) ?
> >
> >I'm using "Cloudstack Release 4.3.0 (64-bit) Thu Apr 10 20:27:11 UTC
> >2014" cloudstack-release template.
> >
> >Thanks,
> >Sachin
> >
> >---
> >root@r-6-VM:~# ifconfig
> >eth0  Link encap:Ethernet  HWaddr 02:13:87:88:e6:dd
> >  inet addr:192.168.3.226  Bcast:192.168.3.255  Mask:255.255.255.0
> >  inet6 addr: fe80::13:87ff:fe88:e6dd/64 Scope:Link
> >  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >  RX packets:350 errors:0 dropped:0 overruns:0 frame:0
> >  TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
> >  collisions:0 txqueuelen:1000
> >  RX bytes:29400 (28.7 KiB)  TX bytes:602 (602.0 B)
> >  Interrupt:25
> >
> >eth1  Link encap:Ethernet  HWaddr 0e:00:a9:fe:02:6b
> >  inet addr:169.254.2.107  Bcast:169.254.255.255  Mask:255.255.0.0
> >  inet6 addr: fe80::c00:a9ff:fefe:26b/64 Scope:Link
> >  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >  RX packets:3293 errors:0 dropped:0 overruns:0 frame:0
> >  TX packets:2934 errors:0 dropped:0 overruns:0 carrier:0
> >  collisions:0 txqueuelen:1000
> >  RX bytes:444768 (434.3 KiB)  TX bytes:539100 (526.4 KiB)
> >  Interrupt:26
> >
> >eth2  Link encap:Ethernet  HWaddr 06:d5:1c:00:00:0b
> >  inet addr:10.84.59.131  Bcast:10.84.59.255  Mask:255.255.255.0
> >  inet6 addr: fe80::4d5:1cff:fe00:b/64 Scope:Link
> >  UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >  RX packets:133 errors:0 dropped:0 overruns:0 frame:0
> >  TX packets:1451 errors:0 dropped:0 overruns:0 carrier:0
> >  collisions:0 txqueuelen:1000
> >  RX bytes:4636 (4.5 KiB)  TX bytes:113282 (110.6 KiB)
> >  Interrupt:27
> >
> >
>
>

RE: Need to revert CLOUDSTACK-5853

[Prachi Damle]

Hi Marcus,

I have reopened this ticket for 4.4 release, fix needed for this is reverting 
the code added as part of this in earlier versions.

Thanks,
Prachi

-Original Message-
From: Prachi Damle [mailto:prachi.da...@citrix.com] 
Sent: Wednesday, June 04, 2014 1:04 PM
To: Marcus
Cc: dev@cloudstack.apache.org
Subject: Need to revert CLOUDSTACK-5853

Hi Marcus,

This is regarding bug fix: https://issues.apache.org/jira/browse/CLOUDSTACK-5853
I see that as a fix, in ClusterScopeStoragePoolAllocator we remove the storage 
pools from avoid set if they happen to match the tag but are found in avoid set.

However this fix is not correct since it breaks the design of avoid set. A 
storage pool might be put in the 'avoid' set by the calling logic for a valid 
reason like capacity exceeded or creation of disks failed on previous try etc.
In such cases the fix will override the avoid set and keep on trying the same 
pool.


So will it be possible for you to revert 
https://issues.apache.org/jira/browse/CLOUDSTACK-5853?


The correct way to fix this is handling of the avoid set in the caller of the 
StoragePoolAllocator - in most of the VM operations the caller is the 
DeploymentPlanningManager.
I had already added the necessary fix in DeploymentPlanningManager  through 
this ticket https://issues.apache.org/jira/browse/CLOUDSTACK-5426


Thanks,
Prachi

[ANNOUNCE] Demetrius Tsitrelis as committer

[John Kinsella]

Folks - this one’s a little belated - we went through the invite process around 
the
time of the mail issues, and somehow we didn’t send the announcement to dev@.
I noticed while doing some housekeeping this week, and wanted to send out the
announcement anyways just to give Demetrius the recognition. :)

The Project Management Committee (PMC) for Apache CloudStack has
asked Demetrius Tsitrelis to become a committer and we are pleased to announce
that he has accepted.

Being a committer allows many contributors to contribute more autonomously. For
developers, it makes it easier to submit changes and eliminates the need to
have contributions reviewed via the patch submission process. Whether
contributions are development-related or otherwise, it is a recognition of a
contributor's participation in the project and commitment to the project and
the Apache Way.

Please join me in congratulating Demetrius!

-John, on behalf of the CloudStack PMC

[DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...

[Sheng Yang]

Hi all,

Seems it's a good timing to bring back the discussion about the gerrit.

We want to do CI, and improve our code quality. One obvious way of doing
and reduce the workload of devs is introduce a tool to enforce the process.

I've checked out quite a few projects using gerrit, which would force you
to ask for review, and validation before the code can be committed to the
repo. Looks it's really a easier way for devs according what I've heard.

Even our competitor laid out a very detail workflow based on the use of
gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can
make a good reference.

Well, gerrit has been brought up a few times before. And now the new
process we want to enforce just fits what gerrit(or other automation
review/test/commit software) is for.

Maybe it's the time for us to review the possibility of using a tool to
enforce our commits and improve our code quality(as well as transfer
knowledge) again?

--Sheng


On Tue, May 27, 2014 at 8:28 PM, David Nalley  wrote:

> On Tue, May 27, 2014 at 12:52 PM, Alex Huang 
> wrote:
> >> Like Chip, I am very concerned with this being dependent on a single
> >> company, even if its the company that employs me. It isn't sustainable,
> it
> >> excludes others from contributing, and makes the project less
> independent
> >> because it depends on a single company's infrastructure.
> >
> > Agreed there.
> >
> >>
> >> I'm also unclear on the answer to the question in the FAQ. The first
> time I
> >> read it, I got the impression that you were happy to bring it up on
> hardware
> >> at the ASF if the ASF wanted to own it. The second time I read it I
> wondered
> >> if you meant that Citrix was going to attempt to donate hardware.
> >>
> > Sorry if I did not make that clear.  I meant the scripts/code that we
> wrote are checked in publicly and we're willing to help set it up if ASF
> provided the hardware.  I have not approach Citrix on donating the actual
> hardware.  Although I can approach them if it speeds up the adoption
> process.
> >
> >> Finally - what do you think you need from ASF infra to make this happen?
> >>
> >
> > It's currently about 10 servers with two networks.  One network is
> static with IPMI to PXE boot the machines.  The other network is the actual
> data network that CloudStack uses.  That's actually just enough for
> XenServer and KVM.  In order to accommodate for HyperV, Bare Metal, LXC,
> (which we do not have any test cases in the automation suits currently) we
> will need even more machines.  We might be able to use nested
> virtualization for the hypervisors to maintain server count at ten or a
> little more than ten but we haven't explore that yet.
> >
> > The CI process is up and running on those machines but because we didn't
> have CI running on master before, automation tests that were passing for
> 4.3 are now broken again on 4.4. and master.  I think Sudha already
> reported on the list that QA is busy trying to fix all the automation tests
> to bring CI on 4.4-forward and master back to 100% pass rate.
>  Unfortunately, it's been delaying our effort to put this out in the public
> and let the community try this themselves.
> >
> > --Alex
> >
>
> So the board just approved a 3 month budget, but the new board will
> have to take up the remainder of the FY budget shortly after coming
> into office. Perhaps worth coming up with an estimate of what this
> will cost/need and getting it to president@ before that new budget is
> taken up.
>
> --David
>

RE: [ANNOUNCE] Demetrius Tsitrelis as committer

[Prachi Damle]

Congratulations Demetrius! 

-Original Message-
From: John Kinsella [mailto:j...@stratosec.co] 
Sent: Friday, June 06, 2014 4:18 PM
To: 
Subject: [ANNOUNCE] Demetrius Tsitrelis as committer

Folks - this one's a little belated - we went through the invite process around 
the time of the mail issues, and somehow we didn't send the announcement to 
dev@.
I noticed while doing some housekeeping this week, and wanted to send out the 
announcement anyways just to give Demetrius the recognition. :)

The Project Management Committee (PMC) for Apache CloudStack has asked 
Demetrius Tsitrelis to become a committer and we are pleased to announce that 
he has accepted.

Being a committer allows many contributors to contribute more autonomously. For 
developers, it makes it easier to submit changes and eliminates the need to 
have contributions reviewed via the patch submission process. Whether 
contributions are development-related or otherwise, it is a recognition of a 
contributor's participation in the project and commitment to the project and 
the Apache Way.

Please join me in congratulating Demetrius!

-John, on behalf of the CloudStack PMC

RE: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...

[Prachi Damle]

+ 1 to gerrit

-Original Message-
From: Sheng Yang [mailto:sh...@yasker.org] 
Sent: Friday, June 06, 2014 4:27 PM
To: 
Subject: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using 
continuous integration to maintain our code quality...

Hi all,

Seems it's a good timing to bring back the discussion about the gerrit.

We want to do CI, and improve our code quality. One obvious way of doing and 
reduce the workload of devs is introduce a tool to enforce the process.

I've checked out quite a few projects using gerrit, which would force you to 
ask for review, and validation before the code can be committed to the repo. 
Looks it's really a easier way for devs according what I've heard.

Even our competitor laid out a very detail workflow based on the use of gerrit( 
https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can make a good 
reference.

Well, gerrit has been brought up a few times before. And now the new process we 
want to enforce just fits what gerrit(or other automation review/test/commit 
software) is for.

Maybe it's the time for us to review the possibility of using a tool to enforce 
our commits and improve our code quality(as well as transfer
knowledge) again?

--Sheng


On Tue, May 27, 2014 at 8:28 PM, David Nalley  wrote:

> On Tue, May 27, 2014 at 12:52 PM, Alex Huang 
> wrote:
> >> Like Chip, I am very concerned with this being dependent on a 
> >> single company, even if its the company that employs me. It isn't 
> >> sustainable,
> it
> >> excludes others from contributing, and makes the project less
> independent
> >> because it depends on a single company's infrastructure.
> >
> > Agreed there.
> >
> >>
> >> I'm also unclear on the answer to the question in the FAQ. The 
> >> first
> time I
> >> read it, I got the impression that you were happy to bring it up on
> hardware
> >> at the ASF if the ASF wanted to own it. The second time I read it I
> wondered
> >> if you meant that Citrix was going to attempt to donate hardware.
> >>
> > Sorry if I did not make that clear.  I meant the scripts/code that 
> > we
> wrote are checked in publicly and we're willing to help set it up if 
> ASF provided the hardware.  I have not approach Citrix on donating the 
> actual hardware.  Although I can approach them if it speeds up the 
> adoption process.
> >
> >> Finally - what do you think you need from ASF infra to make this happen?
> >>
> >
> > It's currently about 10 servers with two networks.  One network is
> static with IPMI to PXE boot the machines.  The other network is the 
> actual data network that CloudStack uses.  That's actually just enough 
> for XenServer and KVM.  In order to accommodate for HyperV, Bare 
> Metal, LXC, (which we do not have any test cases in the automation 
> suits currently) we will need even more machines.  We might be able to 
> use nested virtualization for the hypervisors to maintain server count 
> at ten or a little more than ten but we haven't explore that yet.
> >
> > The CI process is up and running on those machines but because we 
> > didn't
> have CI running on master before, automation tests that were passing 
> for
> 4.3 are now broken again on 4.4. and master.  I think Sudha already 
> reported on the list that QA is busy trying to fix all the automation 
> tests to bring CI on 4.4-forward and master back to 100% pass rate.
>  Unfortunately, it's been delaying our effort to put this out in the 
> public and let the community try this themselves.
> >
> > --Alex
> >
>
> So the board just approved a 3 month budget, but the new board will 
> have to take up the remainder of the FY budget shortly after coming 
> into office. Perhaps worth coming up with an estimate of what this 
> will cost/need and getting it to president@ before that new budget is 
> taken up.
>
> --David
>

Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...

[John Kinsella]

+1 seems like a good idea.

On Jun 6, 2014, at 4:26 PM, Sheng Yang 
mailto:sh...@yasker.org>> wrote:

Hi all,

Seems it's a good timing to bring back the discussion about the gerrit.

We want to do CI, and improve our code quality. One obvious way of doing
and reduce the workload of devs is introduce a tool to enforce the process.

I've checked out quite a few projects using gerrit, which would force you
to ask for review, and validation before the code can be committed to the
repo. Looks it's really a easier way for devs according what I've heard.

Even our competitor laid out a very detail workflow based on the use of
gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can
make a good reference.

Well, gerrit has been brought up a few times before. And now the new
process we want to enforce just fits what gerrit(or other automation
review/test/commit software) is for.

Maybe it's the time for us to review the possibility of using a tool to
enforce our commits and improve our code quality(as well as transfer
knowledge) again?

--Sheng


On Tue, May 27, 2014 at 8:28 PM, David Nalley 
mailto:da...@gnsa.us>> wrote:

On Tue, May 27, 2014 at 12:52 PM, Alex Huang 
mailto:alex.hu...@citrix.com>>
wrote:
Like Chip, I am very concerned with this being dependent on a single
company, even if its the company that employs me. It isn't sustainable,
it
excludes others from contributing, and makes the project less
independent
because it depends on a single company's infrastructure.

Agreed there.


I'm also unclear on the answer to the question in the FAQ. The first
time I
read it, I got the impression that you were happy to bring it up on
hardware
at the ASF if the ASF wanted to own it. The second time I read it I
wondered
if you meant that Citrix was going to attempt to donate hardware.

Sorry if I did not make that clear.  I meant the scripts/code that we
wrote are checked in publicly and we're willing to help set it up if ASF
provided the hardware.  I have not approach Citrix on donating the actual
hardware.  Although I can approach them if it speeds up the adoption
process.

Finally - what do you think you need from ASF infra to make this happen?


It's currently about 10 servers with two networks.  One network is
static with IPMI to PXE boot the machines.  The other network is the actual
data network that CloudStack uses.  That's actually just enough for
XenServer and KVM.  In order to accommodate for HyperV, Bare Metal, LXC,
(which we do not have any test cases in the automation suits currently) we
will need even more machines.  We might be able to use nested
virtualization for the hypervisors to maintain server count at ten or a
little more than ten but we haven't explore that yet.

The CI process is up and running on those machines but because we didn't
have CI running on master before, automation tests that were passing for
4.3 are now broken again on 4.4. and master.  I think Sudha already
reported on the list that QA is busy trying to fix all the automation tests
to bring CI on 4.4-forward and master back to 100% pass rate.
Unfortunately, it's been delaying our effort to put this out in the public
and let the community try this themselves.

--Alex


So the board just approved a 3 month budget, but the new board will
have to take up the remainder of the FY budget shortly after coming
into office. Perhaps worth coming up with an estimate of what this
will cost/need and getting it to president@ before that new budget is
taken up.

--David


Stratosec - Compliance as a Service
o: 415.315.9385
@johnlkinsella

Re: KVM + LXC on the same host

[John Kinsella]

Hey Ilya -

So, for about a month now we’ve had a system running SmartOS[1], which gives a 
combination of containers and KVM, albeit on illumos instead of linux. In 
general I’m not impressed by SmartOS’s story, but we had one customer asking 
for it (and I sorta expect more, we’re not officially supporting it yet). It’s 
a little early for me to even ponder integrating it with ACS, but might but 
worth a glance if you’re looking for that KVM/container mix. 

Downsides:
 * Compared to a Linux hypervisor, much less feature support (missing Ceph is 
personally bugging me)
 * It’s…Solaris (the OS that won’t go away, c0t0d0s0 gives me flashbacks from 
10 years ago). For us our infrastructure/automation is fairly tuned for linux, 
so really embracing this will require…effort.

John
1: http://smartos.org/

On Jun 4, 2014, at 5:58 PM, ilya musayev  wrote:

> We are considering running KVM and LXC on the same host and hopefully control 
> both through cloudstack.
> 
> I know there are agents involved for each component, i dont know if we can 
> have a hybrid of LXC+KVM.
> 
> The use case is simple, we would like the end user to pick LXC/Docker for 
> performance, or KVM instance if he really needed all bells and whistles of 
> dedicated kernel in fully virtualized environment.
> 
> Is anyone aware why we should not mix 2 workloads on the same host? Is it 
> possible at this point in time to mix LXC, KVM and CloudStack, i assume the 
> answer is no, but perhaps there is a hack i can try.
> 
> Thanks
> ilya

Re: Support pure Xen as a hypervisor follow-up

[Amit Das]

It would be great if there's some kind of document in the wiki on this
proposal.

Regards,
Amit
*CloudByte Inc.* 


On Fri, Jun 6, 2014 at 9:52 PM, Tim Mackey  wrote:

> If you can give me the commit ID for your HEAD, I'll check and see
> what's up.  It could be another conflict needing resolution.
>
> -tim
>
> On Fri, Jun 6, 2014 at 12:16 PM, sebgoa  wrote:
> >
> > On Jun 6, 2014, at 5:17 PM, Tim Mackey  wrote:
> >
> >> Dave,
> >>
> >> I've submitted a merge review request
> >> (https://reviews.apache.org/r/22270/) yesterday.
> >
> > Unfortunately that patch did not seem to apply cleanly. I have not dug
> deep though..
> >
> >> If you want to avoid
> >> having to potentially deal with a bunch of conflicts, you might want
> >> to see if your patches apply cleanly there and let me know.  Happy to
> >> help with any conflict resolution.
> >>
> >> btw, I didn't see a design document up on the
> >> wiki(
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/4.5+Design+Documents
> ).
> >> Can you get one there and start a DISCUSS thread?  It'll probably
> >> tease out some gotchas you might not be aware of.
> >>
> >> -tim
> >>
> >> On Fri, Jun 6, 2014 at 11:04 AM, Dave Scott 
> wrote:
> >>> Hi,
> >>>
> >>> Here’s a quick status update:
> >>>
> >>> On 16 May 2014, at 15:22, Dave Scott  wrote:
> >>>
>  Hi,
> 
>  On 14 May 2014, at 09:53, sebgoa  wrote:
> 
> >
> > On Apr 9, 2014, at 2:37 PM, Dave Scott 
> wrote:
> >
> >> Hi,
> >>
> >> Following up from Tim's "Support pure Xen as a hypervisor" proposal
> last month[1] I'd like to start working on this and maybe even make a
> little bit of progress while I'm at CCC in Denver.
> >>
> >> Helpfully James Bulpin managed to get CS + libvirt + xen to start
> an instance in a simple configuration. Although the patches[2] are not
> intended to be production-ready :) they help highlight some of the areas we
> need to change.
> >
> > Dave, just to let you know that Tim has done some important
> "refactoring" to split up XenServer hypervisor in CS between Xen and
> XenServer. That way we could keep using xapi for XS but start moving to
> libvirt for Xen.
> >
> > Tim worked in the xen2server branch (don't ask about the name, I
> messed it up…:) ).
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/xen2server
> >
> > Would be nice to see some of the libvirt stuff in that branch to
> handle a new driver for Xen.
> >
> > Since the two hypervisors will be split up, we could still drop in
> some early libvirt patches to handle Xen and put this in 4.5 as a wip.
> 
>  Thanks for the links.
> 
>  I’m slowly building up a set of patches here:
> 
>  https://github.com/djs55/cloudstack/tree/virsh-capabilities
> 
>  I think once I’ve gotten to a stable-ish point I’ll rebase on top of
> Tim’s branch.
> 
>  So far I’ve
>  * changed the hypervisor detection to use ‘virsh capabilities’ in one
> place and ‘cat /sys/hypervisor/type’ in another. Thinking about it again, I
> think it’s probably best to standardise on /sys/hypervisor/type since that
> will succeed irrespective of whether the libvirtd service is chkconfig’d on
> or not.
> 
>  * in the python cloudutils system setup stuff isKvmEnabled() has
> become isHypervisorEnabled()
> 
>  * added a XenLibvirtDiscoverer similar to the LXC one
> 
>  * fixed what I believe is a race in sshExecuteCmdOneShotWithExitCode
> (which seems to hit me every time, I don’t know why other people seem to be
> immune from it): see CLOUDSTACK-6621 and review board request 21261
> 
>  * added the new hypervisor to hypervisor.list and
> system.vm.default.hypervisor, so it appears in the UI properly
> 
>  * registered a system VM template in the database, using the same
> qcow2 image as KVM
> 
>  For my test host I’m using a XenServer nightly snapshot which comes
> with a nice modern xen and kernel, and is easy to install bleeding-edge
> libvirt on top. I had to tweak the kernel configuration and the network
> configuration but I’m hoping to make it work out of the box in future.
> 
>  When I deploy my ‘datacenter’ the discovery phase works, the agent
> connects and looks healthy in the logs and the UI. The next step is to
> figure out why the system VM template isn’t being copied to primary storage
> — for some reason the copy isn’t being attempted but I can’t see any
> obvious reason why.
> >>>
> >>> I’m now at the stage of getting my system VMs to start via libvirt.
> The main missing feature is support for : low-bandwidth private
> host<->guest control channels. These channels are generally useful things
> and are needed by other projects (like oVirt), so I’d like to add them to
> libxl and libvirt’s libxl driver. There’s a thread on xen-devel and
> libvir-devel if anyone’s interested:
> >>>
> >>> http://www.

Re: [DISCUSS] Introducing Gerrit for quality? was: [PROPOSAL] Using continuous integration to maintain our code quality...

[Rohit Yadav]

Hi,

On Sat, Jun 7, 2014 at 4:56 AM, Sheng Yang  wrote:

> Hi all,
>
> Seems it's a good timing to bring back the discussion about the gerrit.
>
> We want to do CI, and improve our code quality. One obvious way of doing
> and reduce the workload of devs is introduce a tool to enforce the process.
>
> I've checked out quite a few projects using gerrit, which would force you
> to ask for review, and validation before the code can be committed to the
> repo. Looks it's really a easier way for devs according what I've heard.
>
> Even our competitor laid out a very detail workflow based on the use of
> gerrit( https://wiki.openstack.org/wiki/Gerrit_Workflow ). I guess it can
> make a good reference.
>

I've used gerrit before, it's old and has its own pain points. I suppose we
need an on-premise solution that ASF infra folks can help setup for
projects such as CloudStack.

So, can we consider other/better opensource alternatives such as
Phabricator (phabricator.org), I've used it before and it's great. It comes
with a command line tool and a web ui for all tasks and comes with
following stuff;

- a command line tool (called archanist) which allows you to
review/test/merge patches and while committing it hooks up linters and unit
testing
- it allows you to audit patches i.e. review commits already pushed on a
branch
- it has alarms (herald) which can trigger on bunch of rules and alert us
via email, for example if someone changes database files we can put an
alarm on set of files to get alert emails
- people who have used Github reviewing would have less time learning to
use it
- works with git (hg, svn etc.)
- high quality software with many awards and used/maintained by tons of
companies such as Facebook, Dropbox etc.

Before we start with the actionable items, please just explore it here
http://phabricator.org/tour

Regards.


> Well, gerrit has been brought up a few times before. And now the new
> process we want to enforce just fits what gerrit(or other automation
> review/test/commit software) is for.
>
> Maybe it's the time for us to review the possibility of using a tool to
> enforce our commits and improve our code quality(as well as transfer
> knowledge) again?
>
> --Sheng
>
>
> On Tue, May 27, 2014 at 8:28 PM, David Nalley  wrote:
>
> > On Tue, May 27, 2014 at 12:52 PM, Alex Huang 
> > wrote:
> > >> Like Chip, I am very concerned with this being dependent on a single
> > >> company, even if its the company that employs me. It isn't
> sustainable,
> > it
> > >> excludes others from contributing, and makes the project less
> > independent
> > >> because it depends on a single company's infrastructure.
> > >
> > > Agreed there.
> > >
> > >>
> > >> I'm also unclear on the answer to the question in the FAQ. The first
> > time I
> > >> read it, I got the impression that you were happy to bring it up on
> > hardware
> > >> at the ASF if the ASF wanted to own it. The second time I read it I
> > wondered
> > >> if you meant that Citrix was going to attempt to donate hardware.
> > >>
> > > Sorry if I did not make that clear.  I meant the scripts/code that we
> > wrote are checked in publicly and we're willing to help set it up if ASF
> > provided the hardware.  I have not approach Citrix on donating the actual
> > hardware.  Although I can approach them if it speeds up the adoption
> > process.
> > >
> > >> Finally - what do you think you need from ASF infra to make this
> happen?
> > >>
> > >
> > > It's currently about 10 servers with two networks.  One network is
> > static with IPMI to PXE boot the machines.  The other network is the
> actual
> > data network that CloudStack uses.  That's actually just enough for
> > XenServer and KVM.  In order to accommodate for HyperV, Bare Metal, LXC,
> > (which we do not have any test cases in the automation suits currently)
> we
> > will need even more machines.  We might be able to use nested
> > virtualization for the hypervisors to maintain server count at ten or a
> > little more than ten but we haven't explore that yet.
> > >
> > > The CI process is up and running on those machines but because we
> didn't
> > have CI running on master before, automation tests that were passing for
> > 4.3 are now broken again on 4.4. and master.  I think Sudha already
> > reported on the list that QA is busy trying to fix all the automation
> tests
> > to bring CI on 4.4-forward and master back to 100% pass rate.
> >  Unfortunately, it's been delaying our effort to put this out in the
> public
> > and let the community try this themselves.
> > >
> > > --Alex
> > >
> >
> > So the board just approved a 3 month budget, but the new board will
> > have to take up the remainder of the FY budget shortly after coming
> > into office. Perhaps worth coming up with an estimate of what this
> > will cost/need and getting it to president@ before that new budget is
> > taken up.
> >
> > --David
> >
>

RE: [DISCUSS] try-with-resouce use

[Rajani Karuturi]

I think in the branches 4.4 and above we should use try-resource-construct for 
all the upgrade paths.

Otherwise we will continue to see resource leaks due to copy-paste-modified 
code.

~Rajani
Sent from my Windows Phone

From: Daan Hoogland
Sent: ‎07-‎06-‎2014 01:53
To: dev
Subject: [DISCUSS] try-with-resouce use

H,

I have commited a patch to Upgrade430to440.java that removes all
close() calls on PreparedStament en ResultSet objects, replacing it
with the try() construct in java 7. I was tempted for a short
moment to back-port it to older Upgrade classes. As those upgrade
paths are to be supported by lower versions of java. In master (and
branch 4.4) however these are only to be run with java 7. So should we
fix the upgrades by try-with-resources or not?

--
Daan

SSVM I/O error

[Tejas Gadaria]

Hi,

I am getting below error in logs, on new xenserver setup.
SSVM is working fine. I tried to recreate the vm twice but no luck.

I am able to ping to private, public & link-local ip of SSVM & CPVM but
template is not downloading.

2014-06-07 11:40:59,639 DEBUG [c.c.s.s.SecondaryStorageManagerImpl]
(AgentConnectTaskPool-626:ctx-9c74116d) Successfully programmed secondary
storage Secondary in secondary storage VM s-4-VM
2014-06-07 11:40:59,645 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-626:ctx-9c74116d) Seq 3-1626341406: Sending  { Cmd ,
MgmtId: 345049283025, via: 3(s-1-VM), Ver: v1, Flags: 100011,
[{"com.cloud.agent.api.SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.129.152.32","port":"80","intf":"eth2"}],"isAppendAIp":true,"wait":0}}]
}
2014-06-07 11:40:59,688 DEBUG [c.c.a.t.Request]
(AgentManager-Handler-6:null) Seq 3-1626341406: Processing:  { Ans: ,
MgmtId: 345049283025, via: 3, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"java.io.IOException:
Cannot run program \"/usr/local/cloud/systemvm/ipfirewall.sh\":
java.io.IOException: error=5, Input/output error\n\tat
java.lang.ProcessBuilder.start(ProcessBuilder.java:488)\n\tat
com.cloud.utils.script.Script.execute(Script.java:177)\n\tat
com.cloud.utils.script.Script.execute(Script.java:155)\n\tat
org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.configureIpFirewall(NfsSecondaryStorageResource.java:2262)\n\tat
org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.execute(NfsSecondaryStorageResource.java:1684)\n\tat
org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.executeRequest(NfsSecondaryStorageResource.java:222)\n\tat
com.cloud.storage.resource.PremiumSecondaryStorageResource.defaultAction(PremiumSecondaryStorageResource.java:63)\n\tat
com.cloud.storage.resource.PremiumSecondaryStorageResource.executeRequest(PremiumSecondaryStorageResource.java:59)\n\tat
com.cloud.agent.Agent.processRequest(Agent.java:498)\n\tat
com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:806)\n\tat
com.cloud.utils.nio.Task.run(Task.java:83)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat
java.lang.Thread.run(Thread.java:679)\nCaused by: java.io.IOException:
java.io.IOException: error=5, Input/output error\n\tat
java.lang.UNIXProcess.(UNIXProcess.java:164)\n\tat
java.lang.ProcessImpl.start(ProcessImpl.java:81)\n\tat
java.lang.ProcessBuilder.start(ProcessBuilder.java:470)\n\t... 13
more\n","wait":0}}] }
2014-06-07 11:40:59,688 DEBUG [c.c.a.t.Request]
(AgentConnectTaskPool-626:ctx-9c74116d) Seq 3-1626341406: Received:  { Ans:
, MgmtId: 345049283025, via: 3, Ver: v1, Flags: 10, { Answer } }
2014-06-07 11:40:59,688 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentConnectTaskPool-626:ctx-9c74116d) Details from executing class
com.cloud.agent.api.SecStorageFirewallCfgCommand: java.io.IOException:
Cannot run program "/usr/local/cloud/systemvm/ipfirewall.sh":
java.io.IOException: error=5, Input/output error
at java.lang.ProcessBuilder.start(ProcessBuilder.java:488)
at com.cloud.utils.script.Script.execute(Script.java:177)


Regards,
Tejas