MySQL Client dependency not installed with RPMs

[Adrian Lewis]

Afraid I’m not a dev of any sort but can we add the package ‘mysql’ (MySQL
client package) to the cloud.spec file. If you’re installing the
mysql-server package on the same host as the mgmt server it’s not a problem
but for those installing from the repo and using an external db host, it
doesn’t get installed. Just one little thing to help the ease of
installation.



Not sure about Debian/Ubuntu – anyone able to verify?



Is this worthy of a Jira issue?



Adrian

RE: MySQL Client dependency not installed with RPMs

[Adrian Lewis]

Hi Daan,

No - separate issue. Just bringing it up as you mentioned making changes to
the cloud.spec file in a different email. This is independent of anyone
trying to implement DB HA and just anyone installing from packages and using
an external DB - HA or not.

Adrian

-Original Message-
From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
Sent: 18 June 2014 13:32
To: dev
Subject: Re: MySQL Client dependency not installed with RPMs

Adrian, this is about
https://issues.apache.org/jira/browse/CLOUDSTACK-6892 isn't it?

I am afraid we are communicating along paralel lines here. The last mail I
sent you went to dev@ as well.

On Wed, Jun 18, 2014 at 2:26 PM, Adrian Lewis 
wrote:
> Afraid I’m not a dev of any sort but can we add the package ‘mysql’
> (MySQL client package) to the cloud.spec file. If you’re installing
> the mysql-server package on the same host as the mgmt server it’s not
> a problem but for those installing from the repo and using an external
> db host, it doesn’t get installed. Just one little thing to help the
> ease of installation.
>
>
>
> Not sure about Debian/Ubuntu – anyone able to verify?
>
>
>
> Is this worthy of a Jira issue?
>
>
>
> Adrian



--
Daan

RE: MySQL Client dependency not installed with RPMs

[Adrian Lewis]

https://issues.apache.org/jira/browse/CLOUDSTACK-7038

I'd disagree with it being a docs issue though. Docs can provide a
workaround but the root problem should still be addressed IMHO.

-Original Message-
From: sebgoa [mailto:run...@gmail.com]
Sent: 02 July 2014 16:20
To: dev@cloudstack.apache.org
Subject: Re: MySQL Client dependency not installed with RPMs


On Jun 18, 2014, at 2:26 PM, Adrian Lewis 
wrote:

> Afraid I'm not a dev of any sort but can we add the package 'mysql'
> (MySQL client package) to the cloud.spec file. If you're installing
> the mysql-server package on the same host as the mgmt server it's not
> a problem but for those installing from the repo and using an external
> db host, it doesn't get installed. Just one little thing to help the
> ease of installation.
>
>
>
> Not sure about Debian/Ubuntu - anyone able to verify?
>
>
>
> Is this worthy of a Jira issue?
>
>

Yes please, at the very least it should be documented.
I think it's a docs issue really.

But please file a ticket, thanks

>
> Adrian

RE: [SHOW] Authentication refactoring

[Adrian Lewis]

Hi Rohit,

Not a very constructive email I'm afraid but I too would be very
interested in one-time password authentication for CS. Is anyone that you
know of working on RADIUS auth as this would be a relatively easy way to
integrate a wide number of OTP systems that rely on a secondary auth
challenge for the OTP. This secondary auth mechanism is part of the RADIUS
standard and would cover RSA as well as the system that I'm interested in
implementing (Fortinet's FortiAuthenticator) and many other
enterprise-focussed OTP systems.

Not sure if OTP/2FA would be suitable for API access so a second question
is: Would it be feasible to use different auth backends for the GUI vs the
API? As I understand it, the GUI is simply a 'wrapper' for the API so
perhaps not but I'm sure I'm not alone here in wanting OTP/2FA, perhaps
even at the expense of API access. Contrary to popular belief within the
CS community, not everyone uses the API (shock horror!). Maybe OTP/2FA is
not an issue for API access but I assume it would be a problem for the use
of Puppet/Ansible/Salt etc. Perhaps a source IP ACL so that only specified
IPs can use a standard auth method but all other access mandates OTP/2FA?
Not sure how AWS works with their MFA feature - anyone?

I'm afraid I'm just a (ab)user and couldn't program anything myself - just
curious to see if anyone has any thoughts or existing efforts in this
area?

Cheers,

Adrian

-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: 12 August 2014 11:41
To: dev@cloudstack.apache.org
Subject: Re: [SHOW] Authentication refactoring

>From the user end there is no change, not in UI or any change expected in
clients except one:
Since login and logout are now implemented like your regular api, we don't
allow uses to call login and logout and other such AuthenticatorAPIs
directly like via integration port

Stephen, I'm not sure if we natively support RSA and other things at
present we only have our custom login auth mechanism, signature/key based
auth and a simple SSO (pre-shared key) methods. This refactoring will open
doors for saml, oauth and possibly others.

This is merged on master now, even though I did testing at my end please
let me know if something got broke? From the outside world nothing should
break, i.e. refactoring.

Cheers.

On 12-Aug-2014, at 12:32 pm, Stephen Turner 
wrote:

> Are there any UI changes? Some auth mechanisms might need more than just
username and password (RSA token, for example, or even just "give the 1st,
4th and 5th characters").
>
> --
> Stephen Turner
>
>
> -Original Message-
> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
> Sent: 12 August 2014 04:51
> To: dev
> Subject: [SHOW] Authentication refactoring
>
> Hi,
>
> The way we handle login and logout is hardcoded and since there is no
APICommand/BaseCmd implementation the apidoc, apidiscovery and other don't
discover these apis. For supporting SAML as an authentication mechanism,
I've refactored the Auth mechanism as a pluggable service that loads with
api-server artifact and both login and logout are now implemented as a
pseduo BaseCmd classes.
>
> I call them pseudo because their execute() is never called, the
authentication guards in ApiServlet class make sure we call an
authenticate method of such classes. Since, they are tightly coupled with
cloud-server's ApiServlet it only made sense to have the interface
definition and implementation within the same package/artifact as well.
This also solves the apidoc issue for login/logout and saml related auth
apis.
>
> I'll merge them after sometime and continue working on saml stuff. Will
push the code in the branch "auth-refactor" in an hour for review/testing
now. This does not break anything and should not cause any auth related
issues for all existing clients.
>
> Any suggestions, feedback welcome! Refactoring was pretty straight
forward but I'll make sure to write a wiki page on this before merging to
master.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related
services
>
> IaaS Cloud Design &
Build
> CSForge - rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure
Support
> CloudStack Bootcamp Training
Courses
>
> This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is addressed. Any
views or opinions expressed are solely those of the author and do not
necessarily represent those of Shape Blue Ltd or related companies. If you
are not the intended recipient of this email, you must neither take any
action based upon its 

RE: Launch a VM (not from a template)

[Adrian Lewis]

I'm glad to see that there's a potential workaround for this. Perhaps this
should be somehow incorporated in the installation process to allow
smaller setups to be implemented on hypervisors with existing VMs. The
main benefit in my mind though is as the OP suggests - DRaAS.

I think that having a "registerVolume" that would import the VM's metadata
and register it into CS management, including renaming to suit the
conventions would be a great addition. As the OP has already mentioned,
this would also open up the possibility of using a number of backup and DR
tools already on the market with minimal effort to integrate with CS - it
would bring the RTO down significantly if the volumes were already in
primary storage, ready to be fired up at will.

This would certainly gain favour in scenarios where CS operators are
looking to run BDR as a service for clients (such as ourselves, the OP and
a growing number of managed service providers currently just offering
backup as a service but not DR). I'm pretty sure that vCloud Director has
a few commercial BDR software options that currently won’t work easily
with CS due to this, or at least the perception by software vendors that
it can't easily be done. Simplifying the process of importing existing VMs
would definitely be something that would increase takeup of CS in certain
use-cases and at the same time boost CS's public visibility through it's
'ecosystem' of partners. If the 'product' is made as partner friendly as
possible, they in turn help the marketing efforts. Vision's DoubleTake had
a CS integration but from what I can tell it's largely been forgotten as
it is no longer supported with 4.3 or 4.4 and there doesn’t seem to be a
roadmap for this update.

Adrian

-Original Message-
From: Nitin Mehta [mailto:nitin.me...@citrix.com]
Sent: 23 September 2014 07:15
To: dev@cloudstack.apache.org
Subject: Re: Launch a VM (not from a template)

Marcus - I think its possible even now in CS. I don¹t think we need
another api.
1. You can create a vm using the deployvm api with flag startvm=false.
This would create vm and its corresponding resources db records without
actually creating them. You can give a dummy template for now as it still
needs a templateid (we should remove this dependence in future).
2. You can then use updateVolume to update the volume with storage pool id
and marking it ready.
3. Finally you use start vm api to start the vm. It would create all the
resources and start the vm. Since the volume is already ready it won't go
into creating the root volume.

Thanks,
-Nitin

On 22/09/14 10:08 PM, "Marcus"  wrote:

>So, we have thought about this a bit as well. Our solution, which we
>haven't actually implemented yet, was to create a
"registerVirtualMachine"
>api call that would be similar to deployVirtualMachine but accept a
>"rootdiskid" and storage pool id.  This would essentially enter a vm into
>the db but just copy the " rootdiskid" into the root volumes space rather
>than creating a new one. This would allow root disks to be created
outside
>of cloudstack and then made known to cloudstack. It wouldn't be terribly
>difficult to implement (the current deploy only creates a root disk if it
>doesn't already exist so would be easy to short circuit), you would want
>to
>ensure the service offering matches the storage tags though.
>
>A "registerVolume" would also be useful.
>On Sep 22, 2014 9:18 PM, "Will Stevens"  wrote:
>
>> ws: inline...
>>
>> Thanks for the response Mike.  :)
>>
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Mon, Sep 22, 2014 at 7:39 PM, Mike Tutkowski <
>> mike.tutkow...@solidfire.com> wrote:
>>
>> > If you did #1, how would you pick a relevant compute offering? You
>>would
>> > probably need to look first to make sure at least one existed that
>>could
>> > satisfy your requirement(s) and then make sure the resources could be
>> > marked in advance as if they were being consumed.
>> >
>>
>> ws: so in this case the original VM would be managed by CS and the 3rd
>> party software would be backing it up and replicating it to other
>> zones/regions.  technically, we should have all of the information we
>>need
>> about the VM because CS will already know about the original VM which
>>this
>> VM is getting spun up to replace.  essentially this would be used for
>>DR in
>> a different DC.  if one DC goes down for some reason, this would
>>basically
>> behave like a cold standby in a different DC.
>>
>> you did touch on a pipe dream of mine though (which I am still in the
>> process of thinking through).  I want to be able to spin up a fresh CS
>> install and then discover an existing xen pool and then configure CS to
>> step in as an orchestration tool for the existing infra.  I am still
>> thinking through how this would be possible, but this is outside the
>>scope
>> of this specific problem, so I wo

RE: Shellshock

[Adrian Lewis]

@John - Quite agree. It's not just scripts that need checking either. Very
unsettling to have a vulnerable version of bash on every system vm, many
with direct access to both the CS infrastructure as well as client VMs. All
it takes is for someone to find another vector (e.g. DHCP, DNSmasq) other
than a script to inject system variables and there's suddenly a MUCH bigger
problem.

Is there no way to simply update the version of bash included with the
system vm template? At the moment it seems to be version 4.2.37 which is
vulnerable (based on
http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.1-7-vmware.ova).

I'm not too familiar with what happens to the template as it's deployed but
if I log in as root/password to the system vm template running as downloaded
in VMware Workstation and 'echo $SHELL' I get '/bin/bash' even though
'/bin/sh' is a symlink to '/bin/dash'.

Perhaps someone is already working quietly on this but surely this should be
treated as a fairly major priority? I'd far rather not have bombs in every
system vm in the first place regardless of whether people think there aren't
any detonators.

Adrian

-Original Message-
From: John Kinsella [mailto:j...@stratosec.co]
Sent: 30 September 2014 22:57
To: dev@cloudstack.apache.org
Subject: Re: Shellshock

I’m not worried about any specific use-case, but I’d rather not have
vulnerable software running on SSVMs in general.

John

On Sep 30, 2014, at 2:47 PM, Sheng Yang
mailto:sh...@yasker.org>> wrote:

The parameters of system() function have been verified as valid IP/netmask
format by script, so I don't think other parameters would be able to slip in
in this case.

--Sheng

On Tue, Sep 30, 2014 at 8:38 AM, Go Chiba
mailto:go.ch...@gmail.com>> wrote:

Hi folks,

By my digging, ipcalc included system() function call but debian based our
system vm are using dash as system shell. So I think this shellshock concern
are not directly affected to system vm cgi-bin. right?

GO

from my iPhone

2014/09/30 10:13、Demetrius Tsitrelis
mailto:demetrius.tsitre...@citrix.com>>
のメッセージ:

http://systemvm-public-ip/cgi-bin/ipcalc is a perl script.

-Original Message-
From: Sheng Yang [mailto:sh...@yasker.org]
Sent: Monday, September 29, 2014 5:21 PM
To: mailto:dev@cloudstack.apache.org>>
Subject: Re: Shellshock

http://systemvm-public-ip/cgi-bin/ipcalc is NOT a bash script, so it's
normal that it cannot be exploited.

--Sheng

On Fri, Sep 26, 2014 at 1:57 PM, Demetrius Tsitrelis <
demetrius.tsitre...@citrix.com>
wrote:

Do you mean you tried setting the USER_AGENT like in
https://community.qualys.com/blogs/securitylabs/2014/09/25/qualysguard
-remote-detection-for-bash-shellshock
?


-Original Message-
From: Ian Duffy [mailto:i...@ianduffy.ie]
Sent: Friday, September 26, 2014 6:56 AM
To: CloudStack Dev
Subject: Re: Shellshock

Tried this against the latest system vms built on Jenkins.

Didn't get a successful exploited response. Tested against http://systemvm
- public-ip/cgi-bin/ipcalc
On 25 Sep 2014 16:56, "Abhinandan Prateek" 
wrote:


After heart bleed we are Shell shocked
http://www.bbc.com/news/technology-29361794 !
It may not affect cloudstack directly as it is a vulnerability that affects
bash, and allows the attacker to take control of the system running bash
shell.

-abhi



Stratosec - Secure Finance and Heathcare Clouds http://stratosec.co
o: 415.315.9385
@johnlkinsella

RE: Shellshock

[Adrian Lewis]

I still think that patching bash is the best option. There are a load of
scripts on the system vm that explicitly use bash (/opt/cloud/bin for
example) which take input from the management server so you'd need to
audit all of the input validation rules on the API (maybe it's safe
already). Bear in mind that for a public cloud, legitimate users cannot be
assumed to be trusted. There is also a way to exploit DNSMasq simply by
supplying malicious options in a DHCP request on a client machine. This
does rely on a specific setting in the DNSMasq config which is not set by
default as far as I'm aware so that should at least be safe until someone
decides to use the feature. My point is that simply scanning the system VM
with a tool designed for web servers will help but will only test a
relatively small percentage of potential attack vectors.

These are just a couple of potential attack vectors that I've identified
and I wouldn't have the first clue about programming and have very limited
Linux experience. Surely this is a ticking timebomb until bash is patched?
I'm slightly concerned that so few people appear to care about this. Is
anyone running commercial offerings based on CS concerned about this and
the corresponding compliance issues it raises? Both Amazon and Rackspace
took fairly rapid (and disruptive) action over XSA-108 which has an almost
trivial impact compared with shellshock.

The only solution I can think of is to 'apt-get update bash' on every
system VM but clearly these get fired up dynamically. Is it possible to
boot the template, make modifications and then use as a replacement system
VM? Are there processes that happen on boot that only happen once and
therefore need resetting to recreate the template?



-Original Message-
From: Santhosh Edukulla [mailto:santhosh.eduku...@citrix.com]
Sent: 02 October 2014 23:10
To: dev@cloudstack.apache.org
Subject: RE: Shellshock

We may use the below scanner to identify this vulnerability.  One of our
ex-colleague has written it,  its a remote, network scanner and available
as free download.

http://blog.crowdstrike.com/crowdstrike-shellshock-scanner/

Seems configurable  with custom paths. Please check the note at the end.

Regards,
Santhosh

From: Demetrius Tsitrelis [demetrius.tsitre...@citrix.com]
Sent: Wednesday, October 01, 2014 1:59 PM
To: 
Subject: RE: Shellshock

Actually, I am not sure.  Only the env.cgi script is loaded and, while the
other scripts are in perl, there is nothing in the video which shows the
source for the env.cgi script so it may not be perl.

-Original Message-
From: Demetrius Tsitrelis [mailto:demetrius.tsitre...@citrix.com]
Sent: Wednesday, October 01, 2014 10:52 AM
To: 
Subject: RE: Shellshock

Interestingly this video shows attack against a perl script...
https://www.youtube.com/watch?v=ArEOVHQu9nk

-Original Message-
From: Demetrius Tsitrelis [mailto:demetrius.tsitre...@citrix.com]
Sent: Monday, September 29, 2014 6:13 PM
To: 
Subject: RE: Shellshock

http://systemvm-public-ip/cgi-bin/ipcalc is a perl script.

-Original Message-
From: Sheng Yang [mailto:sh...@yasker.org]
Sent: Monday, September 29, 2014 5:21 PM
To: 
Subject: Re: Shellshock

http://systemvm-public-ip/cgi-bin/ipcalc is NOT a bash script, so it's
normal that it cannot be exploited.

--Sheng

On Fri, Sep 26, 2014 at 1:57 PM, Demetrius Tsitrelis <
demetrius.tsitre...@citrix.com> wrote:

> Do you mean you tried setting the USER_AGENT like in
> https://community.qualys.com/blogs/securitylabs/2014/09/25/qualysguard
> -remote-detection-for-bash-shellshock
> ?
>
>
> -Original Message-
> From: Ian Duffy [mailto:i...@ianduffy.ie]
> Sent: Friday, September 26, 2014 6:56 AM
> To: CloudStack Dev
> Subject: Re: Shellshock
>
> Tried this against the latest system vms built on Jenkins.
>
> Didn't get a successful exploited response. Tested against
> http://systemvm
> - public-ip/cgi-bin/ipcalc
> On 25 Sep 2014 16:56, "Abhinandan Prateek"  wrote:
>
> >
> > After heart bleed we are Shell shocked
> > http://www.bbc.com/news/technology-29361794 !
> > It may not affect cloudstack directly as it is a vulnerability that
> > affects bash, and allows the attacker to take control of the system
> > running bash shell.
> >
> > -abhi
>

RE: Unreleased 4.4.1 packages on cloudstack.apt-get.eu, why?

[Adrian Lewis]

Perhaps it's just me being lazy but I reckon that having defined 'beta' and
RC packages available for download (with some form of marketing to inform
people) would likely increase the sample size for testing, including
different use cases. I'm pretty sure that there are a number of people that
fit the sysadmin role rather than the devops or developer role who just
won't go to the effort of trying to work out what to checkout, build and
package themselves even if they did know how to. I realise that the audience
for CS may be different to many other OS software projects out there but
most well-used projects (including many commercial ones) that I work with go
through this process. The only other issue would be that of feedback - I
know it's an Apache thing but an online forum site would really make
providing feedback much easier than the mailing list.

Adrian

-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: 10 October 2014 09:24
To: dev@cloudstack.apache.org
Subject: Re: Unreleased 4.4.1 packages on cloudstack.apt-get.eu, why?

Yes that's not a bad idea. At some point we had a chat with Hugo and David
about having jenkins generate RPMs with incremented version and setup a repo
for them which lead me to believe that at least Jenkins is capable of that.

So, whoever is in charge of Jenkins, how can we achieve this?

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Erik Weber" 
> To: "dev" 
> Sent: Friday, 10 October, 2014 09:10:35
> Subject: Re: Unreleased 4.4.1 packages on cloudstack.apt-get.eu, why?

> Or make a different folder for unreleased versions?
> That way you could upload daily builds as well if you ever wanted to
>
> How about
> - a stable/release folder, where released versions go
> - a rc folder, for official release versions
> - a nightly/dev folder, for either a nightly version or the latest
> master build
>
> Couple it with jenkins and it would be easy for testers to verify that
> a bug is fixed by upgrading their RC or dev version
>
> --
> Erik
>
> On Fri, Oct 10, 2014 at 9:46 AM, Daan Hoogland
> 
> wrote:
>
>> I was thinking about that. I shouldn't have done this upload but had
>> a reason to. I want to make these available for testers. maybe i can
>> upload them without adding them to the rpm index...?
>>
>> On Thu, Oct 9, 2014 at 7:47 PM, Nux!  wrote:
>>
>> > Perhaps we need a testing repo for non-final packages?
>> >
>> > --
>> > Sent from the Delta quadrant using Borg technology!
>> >
>> > Nux!
>> > www.nux.ro
>> >
>> > - Original Message -
>> > > From: "Daan Hoogland" 
>> > > To: "dev" 
>> > > Sent: Thursday, 9 October, 2014 17:42:50
>> > > Subject: Re: Unreleased 4.4.1 packages on cloudstack.apt-get.eu, why?
>> >
>> > > i did
>> > >
>> > > On Thu, Oct 9, 2014 at 3:07 PM, Ian Duffy  wrote:
>> > >
>> > >> Wildo,
>> > >>
>> > >> Do we have no kind of audit on uploads?
>> > >>
>> > >> On 9 October 2014 12:25, Wido den Hollander  wrote:
>> > >>
>> > >> >
>> > >> >
>> > >> > On 10/09/2014 01:21 PM, Nux! wrote:
>> > >> > > Hello,
>> > >> > >
>> > >> > > I've noticed there are 4.4.1 packages on
>> > >> > http://cloudstack.apt-get.eu/rhel/4.4/.
>> > >> > > Since 4.4.0 is latest release, how come? This is bad, people
>> > >> > > will
>> > >> > install broken stuff.
>> > >> > > Can anyone fix this?
>> > >> > >
>> > >> >
>> > >> > I removed the packages, but I'm not sure who uploaded them.
>> > >> >
>> > >> > They were uploaded on September 30th at 11:12 CET.
>> > >> >
>> > >> > Wido
>> > >> >
>> > >> > > Lucian
>> > >> > >
>> > >> > > --
>> > >> > > Sent from the Delta quadrant using Borg technology!
>> > >> > >
>> > >> > > Nux!
>> > >> > > www.nux.ro
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Daan
>> >
>>
>>
>>
>> --
>> Daan

RE: [VOTE] Apache CloudStack 4.5.0 RC3

[Adrian Lewis]

Seem to be hitting CLOUDSTACK-7671 despite it previously being marked as
fixed.
https://issues.apache.org/jira/browse/CLOUDSTACK-7671

Using the ShapeBlue centos7 testing RPMs with Xenserver 6.5. By the looks of
the jira issue, I'm not the only one. This is a bit of a blocker for me at
least as you can’t reboot the management server without manually modifying
the pid file before starting up the cloudstack-management service.

-Original Message-
From: Geoff Higginbottom [mailto:geoff.higginbot...@shapeblue.com]
Sent: 09 February 2015 13:54
To: dev@cloudstack.apache.org
Subject: RE: [VOTE] Apache CloudStack 4.5.0 RC3

Hi Pierre-Luc,

You are right Basic Networking 'With' security Groups requires Linux Bridge,
and will not work with OVS.

This is a Basic Zone 'Without' security groups so OVS should work.  Linux
Bridge is only required if you use Security Groups.

I know of a CloudStack install running a Basic Zone without Security Groups
on 4.3.x and they are using OVS without any problems.

I did a test upgrade to 4.5.0 and the Hosts stay in Alert State.

Regards

Geoff Higginbottom

D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581

geoff.higginbot...@shapeblue.com

-Original Message-
From: Pierre-Luc Dion [mailto:pd...@cloudops.com]
Sent: 09 February 2015 13:48
To: dev@cloudstack.apache.org
Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC3

Geoff, as far as I know Basic Networking with Security group never worked
with XenServer + OVS, only with Bridge, please correct me if I'm wrong.




On Mon, Feb 9, 2015 at 8:31 AM, Geoff Higginbottom <
geoff.higginbot...@shapeblue.com> wrote:

> -1
>
> When trying to add XenServers to a new 'Basic Zone without Security
> Groups" the Hosts remain in 'Alert' state because they are running OVS
> and not Linux Bridge.
>
> There is no requirement to revert to Linux Bridge unless you are using
> Security Groups, so this test is incorrect.
>
> In 4.3.x this was not the case.
>
> If try and upgrade a 4.3.2 system which uses XenServer 6.2 with OVS
> enabled, all Hosts remain in Alert state as 4.5.0 is expecting Linux
> Bridge
>
> Regards
>
> Geoff Higginbottom
>
> D: +44 20 3603 0542 | S: +44 20 3603 0540 | M: +447968161581
>
> geoff.higginbot...@shapeblue.com
>
> -Original Message-
> From: Abhinandan Prateek [mailto:abhinandan.prat...@shapeblue.com]
> Sent: 09 February 2015 13:27
> To: dev@cloudstack.apache.org
> Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC3
>
> Yes, with -1s no point voting further.
>
> Tested local storage in Advance zone.
> Live VM migration is working with local storage (Xen 6.5 cluster) that
> was broken in RC1, and RC2.
>
> -abhi
>
>
> > On 09-Feb-2015, at 6:27 pm, Ian Duffy  wrote:
> >
> > +0 (no point voting with -1s above)
> >
> > Tested this using Devcloud4 on an advanced zone worked without
> > issue, environment came up, template downloaded was able to create a
> > VM and a egress rule to allow outbound connectivity. Great to see
> > the xen server networking label stuff fixed, this caused some issues
> > for me with automating things with marvin (hence why devcloud4
> > advance didn't work with 4.4).
> >
> > If anyone is interested: (assumes you've read the docs for
> > devcloud4, installed berkshelf/chef-dk, setup the vboxnet adapters)
> >
> > git clone g...@github.com:imduffy15/devcloud4.git
> > cd binary-installation-advanced
> > vagrant up
> >
> >
> > On 9 February 2015 at 09:02, Nux!  wrote:
> >> Daan,
> >>
> >> That's fine. There were no critical issues to be addressed really
> >> by a
> 4.4.3 AFAIK.
> >> Let's get 4.5 right. :)
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> - Original Message -
> >>> From: "Daan Hoogland" 
> >>> To: "dev" 
> >>> Sent: Monday, 9 February, 2015 07:29:13
> >>> Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC3
> >>
> >>> Gents,
> >>>
> >>> 4.4.3 vote was never formally closed. Do we need to continue? If
> >>> so I will create a new rc!
> >>>
> >>> On Mon, Feb 9, 2015 at 1:51 AM, Mike Tutkowski
> >>>  wrote:
>  FYI: I have resolved and closed out
>  https://issues.apache.org/jira/browse/CLOUDSTACK-8233.
> 
>  On Sun, Feb 8, 2015 at 9:44 AM, Mike Tutkowski
>   > wrote:
> 
> > Here is the newly created bug:
> >
> > https://issues.apache.org/jira/browse/CLOUDSTACK-8233
> >
> > On Sun, Feb 8, 2015 at 9:37 AM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> >> -1
> >>
> >> Marcus and I are working on a fix for not being able to create
> >> VMs on KVM due to kvmclock not being recognized by older
> >> versions
> of Libvirt.
> >>
> >> I have most of the testing done. I should be done with the rest
> >> of it today and can check this code in.
> >>
> >> I don't think we have a ticket for this, so I'll go ahead and
> create one.
> >>
> >> Thanks (and sorry for the need for a subsequent RC)!
> 

RE: Google Summer of Code 2015 is coming

[Adrian Lewis]

Would it be bad form to suggest ideas if we're neither student, nor a
mentor?

-Original Message-
From: Tilak Raj Singh [mailto:tila...@gmail.com]
Sent: 06 February 2015 19:03
To: dev@cloudstack.apache.org
Subject: Re: Google Summer of Code 2015 is coming

Hi,

I posted my project idea for the same above in this thread. Have also
created a ticket for the same.
Link is https://issues.apache.org/jira/browse/CLOUDSTACK-8227

On Fri, Feb 6, 2015 at 2:01 PM, Rohit Yadav 
wrote:

> Tilak, I think you need to log in to see the filter. Nevertheless,
> here are my proposed ideas:
>
> Integration Infra:
> https://issues.apache.org/jira/browse/CLOUDSTACK-8208
> Tooling: https://issues.apache.org/jira/browse/CLOUDSTACK-8207
> Bhyve: https://issues.apache.org/jira/browse/CLOUDSTACK-8206
> Docker: https://issues.apache.org/jira/browse/CLOUDSTACK-8205
>
> If you're interested in work on them, we can discuss here or if you
> want a mentor you need to propose an idea here and also create a
> ticket with tag 'gsoc2015' under CloudStack project.
>
>
> On Thursday 05 February 2015 09:41 PM, Tilak Raj Singh wrote:
>
>> @Rohit : Hi I just viewed the the link
>> https://issues.apache.org/jira/issues/?filter=12330309 and couldnt
>> find any projects. Did you add them somewhere else?
>>
>> @All : I am a student and want a mentor for my project on Cloudstack
>> Community. Can I also post a project idea there?
>>
>> Regards
>>
>> On Wed, Feb 4, 2015 at 2:08 PM, Andrei Mikhailovsky
>> 
>> wrote:
>>
>>  +1 for support of Ceph in Xenserver
>>>
>>> Andrei
>>> - Original Message -
>>>
>>>  From: "Erik Weber" 
 To: "dev" 
 Sent: Wednesday, 4 February, 2015 7:35:39 AM
 Subject: Re: Google Summer of Code 2015 is coming

>>>
>>>  Pure Xen support would be nice :-)

>>>
>>>  (and Ceph in XenServer, but that's not really a CloudStack issue)

>>>
>>>  --
 Erik

>>>
>>>  On Tue, Feb 3, 2015 at 9:42 AM, Sebastien Goasguen
>>> 
 wrote:

>>>
>>>  GSoC 2015 is back.
> Time to enter your project proposals in jira if you want to mentor.
>
> Begin forwarded message:
>
>  From: Ulrich Stärk 
>> Subject: Google Summer of Code 2015 is coming
>> Date: February 2, 2015 5:44:52 PM EST
>> To: ment...@community.apache.org
>> Reply-To: ment...@community.apache.org
>> Reply-To: ment...@community.apache.org
>>
>> Hello PMCs (incubator Mentors, please forward this email to your
>>
> podlings),
>
>>
>> Google Summer of Code [1] is a program sponsored by Google
>> allowing
>>
> students to spend their summer
>
>> working on open source software. Students will receive stipends
>> for
>>
> developing open source software
>
>> full-time for three months. Projects will provide mentoring and
>> project
>>
> ideas, and in return have
>
>> the chance to get new code developed and - most importantly - to
>>
> identify and bring in new committers.
>
>>
>> The ASF will apply as a participating organization meaning
>> individual
>>
> projects don't have to apply
>
>> separately.
>>
>> If you want to participate with your project we ask you to do the
>>
> following things by no later than
>
>> 2015-02-13 19:00 UTC (applications from organizations close a
>> week later)
>>
>> 1. understand what it means to be a mentor [2].
>>
>> 2. record your project ideas.
>>
>> Just create issues in JIRA, label them with gsoc2015, and they
>> will show
>>
> up at [3]. Please be as
>
>> specific as possible when describing your idea. Include the
>> programming
>>
> language, the tools and
>
>> skills required, but try not to scare potential students away.
>> They are
>>
> supposed to learn what's
>
>> required before the program starts.
>>
>> Use labels, e.g. for the programming language (java, c, c++,
>> erlang,
>>
> python, brainfuck, ...) or
>
>> technology area (cloud, xml, web, foo, bar, ...) and record them
>> at [5].
>>
>> Please use the COMDEV JIRA project for recording your ideas if
>> your
>>
> project doesn't use JIRA (e.g.
>
>> httpd, ooo). Contact d...@community.apache.org if you need
>> assistance.
>>
>> [4] contains some additional information (will be updated for
>> 2015
>>
> shortly).
>
>>
>> 3. subscribe to ment...@community.apache.org; restricted to
>> potential
>>
> mentors, meant to be used as a
>
>> private list - general discussions on the public
>>
> d...@community.apache.org list as much as possible
>
>> please). Use a recognized address when subscribing (@apache.org
>> or one
>>
> of your alias addresses on
>
>> record).
>>
>> Note that the ASF isn't accepted as a participating org

RE: Network QoS (not bandwidth limiting)

[Adrian Lewis]

enterprise level routing and redundancy varies greatly, and
their customers are designing their apps to be resilient to infrastructure
loss (e.g. most AWS customers). That's of course not entirely the whole
truth, as is evidenced by the work we are seeing on redundant routers, but I
do believe that's why we haven't seen these things from the beginning. They
just haven't been all that important to the target customers, even though
infrastructure engineers are used to providing them.

So now comes my philosophy. In the end, I think the great thing about open
source communities is that if there's the right level of interest, it will
happen.  I'm the kind of person who feels a pang of stress at the idea that
something I work on can't be all things to all people, but after building a
hosting business over the last few years I've begun to realize that it's
really only practical to try to be good for a subset of the market and focus
on that. You'll never please everyone, there are limits to what you can
accomplish, and sometimes it's OK to just concede that your product is not
going to work for everyone. If you don't, you'll spread yourself too thin
and fail everyone. In order to make something great you have to have a limit
on your scope. That's not to say you don't listen to your customers, but you
sometimes have to make hard choices on who to listen to and who to upset.

None of this should be taken as a discouragement to the topics at hand, but
again as someone to takes it personally when I don't deliver I wanted to
provide some follow up to address the "rant" and try to provide perspective
on why the things are the way they are.

On Sat, Feb 21, 2015 at 1:58 PM, Somesh Naidu 
wrote:
> Adrian,
>
> Rant or not, I believe you have raised a valid point and reflect certain
> group of peoples requirement.
>
> Based on your requirement, I believe you are looking for something like
> Vyatta.
>
> Regards,
> Somesh
>
> -Original Message-
> From: Adrian Lewis [mailto:adr...@alsiconsulting.co.uk]
> Sent: Friday, February 20, 2015 8:50 PM
> To: us...@cloudstack.apache.org
> Subject: RE: Network QoS (not bandwidth limiting)
>
> Tempted to suggest some sort of special interest group where
> networking people can have some input into the dev process despite not
> necessarily being able to produce any code themselves. As an example,
> Schuberg Philis have recently done some great work on the redundant
> VPC VR but to a network person, this sort of functionality is almost
> taken for granted (please don't take this as a lack of appreciation).
> Similarly, the lack of end-to-end QoS for applications running on ACS
> seems to me at least to be a fairly significant oversight. ACS is
> known as having very flexible networking compared with some of the
> alternatives but there does still appear to be an enterprise focus on most
> elements that a 'typical'
> developer (dare I say it, web developer) faces but more of a home
> network approach to the networking side (aside from some pretty
> impressive niche features).
>
> We shouldn't need to rely on proprietary 3rd party products to provide
> a similar level of versatility for networking in ACS in my opinion. It
> seems bizarre to me that we have load balancing, distributed routing &
> ACLs with the OVS controller, PVLANs for isolation,  etc, but yet
> still don't have what I would consider basic functions such as better
> control over NAT, firewalling, routing (no dynamic routing protocols
> at all), IPsec, having to specify IP related attributes to what should
> simply be L2 constructs (why does a VPC need to be given a CIDR?!?)
> etc. AWS had a similar issue that lead to the VPC being introduced -
> enterprises consistently rejected the weird and illogical way that
> they did networking back in the day that was overly focussed on web/cloudy
> workloads.
>
> This sounds like a rant and to an extent it is but I'd like to turn it
> into a positive. I feel fairly helpless when the typical response to
> feedback like this is that I should just contribute code. There are a
> number of people that embrace the concept that the community should be
> a collective of not just developers, but at the same time it's pretty
> difficult to feel part of a community that's run almost uniquely by
> developers; it's even a bit intimidating at times. I've seen too many
> commercial companies that abandon innovation in favour of satisfying
> the 'large account' RFC/RFPs and in my opinion the same may apply to a
> project driven largely by the needs of those that can contribute code.
>
> To flip the concept on its head, it would be like a network guy
> creating an amazin

RE: Network QoS (not bandwidth limiting)

[Adrian Lewis]

 most common complaint I see from
> people who are infrastructure maintainers is "why can't I just build
> the infrastructure the way I want and then have it orchestrated?".
> Unfortunately, we can't just automate and integrate with anyone's pet
> design. CloudStack supports many novel and custom network designs
> simply by allowing the option of letting you manage the network
> hardware and being hands-off (shared/public networks), while also
> being pluggable to allow vendors to take over whatever features and
> they wish. I've seen some pretty advanced overlay networking provided
> through third party plugins to CloudStack that take over all network
> functionality and provide more.
>
> What's really being asked for here is for CloudStack to provide and
> maintain a fully-fledged and featured router distribution in its
> provided virtual router. It's an admirable project to have if we can
> get support for it. My guess is there's a bit of a disconnect in
> interest though, because many (but not all) enterprises who want
> CloudStack for infrastructure automation are skeptical about a VM as
> software router and prefer to bring in aforementioned enterprise
> vendors who have their own plugins. People who provide cloud hosting
> and other services tend to use the routers, but their interest in
> enterprise level routing and redundancy varies greatly, and their
> customers are designing their apps to be resilient to infrastructure
> loss (e.g. most AWS customers). That's of course not entirely the
> whole truth, as is evidenced by the work we are seeing on redundant
> routers, but I do believe that's why we haven't seen these things from the
> beginning.
> They just haven't been all that important to the target customers,
> even though infrastructure engineers are used to providing them.
>
> So now comes my philosophy. In the end, I think the great thing about
> open source communities is that if there's the right level of
> interest, it will happen.  I'm the kind of person who feels a pang of
> stress at the idea that something I work on can't be all things to all
> people, but after building a hosting business over the last few years
> I've begun to realize that it's really only practical to try to be
> good for a subset of the market and focus on that. You'll never please
> everyone, there are limits to what you can accomplish, and sometimes
> it's OK to just concede that your product is not going to work for
> everyone. If you don't, you'll spread yourself too thin and fail
> everyone. In order to make something great you have to have a limit on
> your scope. That's not to say you don't listen to your customers, but
> you sometimes have to make hard choices on who to listen to and who to
> upset.
>
> None of this should be taken as a discouragement to the topics at
> hand, but again as someone to takes it personally when I don't deliver
> I wanted to provide some follow up to address the "rant" and try to
> provide perspective on why the things are the way they are.
>
> On Sat, Feb 21, 2015 at 1:58 PM, Somesh Naidu
> 
> wrote:
> > Adrian,
> >
> > Rant or not, I believe you have raised a valid point and reflect
> > certain
> group of peoples requirement.
> >
> > Based on your requirement, I believe you are looking for something
> > like
> Vyatta.
> >
> > Regards,
> > Somesh
> >
> > -Original Message-
> > From: Adrian Lewis [mailto:adr...@alsiconsulting.co.uk]
> > Sent: Friday, February 20, 2015 8:50 PM
> > To: us...@cloudstack.apache.org
> > Subject: RE: Network QoS (not bandwidth limiting)
> >
> > Tempted to suggest some sort of special interest group where
> > networking people can have some input into the dev process despite
> > not necessarily being able to produce any code themselves. As an
> > example, Schuberg Philis have recently done some great work on the
> > redundant VPC VR but to a network person, this sort of functionality
> > is almost taken for granted (please don't take this as a lack of
> > appreciation).
> > Similarly, the lack of end-to-end QoS for applications running on
> > ACS seems to me at least to be a fairly significant oversight. ACS
> > is known as having very flexible networking compared with some of
> > the alternatives but there does still appear to be an enterprise
> > focus on
> most elements that a 'typical'
> > developer (dare I say it, web developer) faces but more of a home
> > network approach to the networking side (aside from some pretty
> > impressive niche feature

RE: [PROPOSAL] DHCP/DNS offload and config drive support for adv zone shared network

[Adrian Lewis]

Can't see the wiki at the moment as it's down for maintenance but on a
slightly different but related note, would it be feasible to use DHCP relay
functionality in dnsmasq on a VR and still get the IP address assigned by an
external DHCP server registered into the ACS MS? Not quite sure if under
normal circumstances ACS picks up the IP from dnsmasq or if ACS manages the
pool and sends dnsmasq static leases. If it's picking up what dnsmasq
decides to lease out, what is this mechanism and does/would it also work for
DHCP relay?

This doesn’t solve the issue of a DHCP server on the same network however
and would still require a VR on the network with upstream connectivity to
the DHCP server.

I'm definitely definitely up for the concept of simple networks with no VR
if we can provision some of the essentials without one. Big +1


-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: 20 March 2015 09:34
To: dev@cloudstack.apache.org
Subject: Re: [PROPOSAL] DHCP/DNS offload and config drive support for adv
zone shared network

+1, good idea

One thing though:  let's make the config drive available for all types of
zones, many people use the basic or adsg zones.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Jayapal Reddy Uradi" 
> To: dev@cloudstack.apache.org
> Sent: Friday, 20 March, 2015 09:12:19
> Subject: [PROPOSAL]  DHCP/DNS offload and config drive support for adv
> zone shared network

> In advanced zone shared network if someone wants to use DHCP server
> outside the cloudstack, currently it can be done by not selecting the
> DHCP service But the problem here is that the VM actual ip is
> different from what cloudstack showing.
>
> If there are no services selected for the network offering there is no
> need of the VR.
> In the absense of VR there should be way to provide password,
> userdata/metadata, ssh keys to user vm.
>
> With this feature we can do the following.
> 1. Create network without VR.
> 2. Retrive the IP from the VM and update it in the cloudstack DB.
> 3. Add config drive support for the VMs in this network.
>
> Please provide your comments for the below FS.
>
> ACS ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-8324
> FS:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53740
> 797
>
>
> Thanks,
> Jayapal

RE: Any VxLan Support on Xenserver

[Adrian Lewis]

I think that the issue is with needing a controller. Unfortunately you can't
just replace VLANs with VXLANs unless there's some form of control plane.
You can do this in a distributed manner using broadcast to multicast
mechanisms but this AFAIK is not implemented in Openvswitch. I'm not quite
sure on the commercial aspects but XenServer 6.5 re-introduced the vSwitch
controller appliance (DVSC) so this could be a candidate but AFAIK, it's
only available on licensed versions of XenServer so any development would
likely come from Citrix. There was some work done by Hugo on ACS with
OpenDaylight but I think this ran into some issues with the Openvswitch
version on XS at the time. That's about all I know but hopefully this helps.
I'd be very interested to hear if anyone is actively doing any work on this
though.

-Original Message-
From: Keerthiraja SJ [mailto:sjkeer...@gmail.com]
Sent: 31 March 2015 08:39
To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: Any VxLan Support on Xenserver

Hi All,

Is there any plan to bring up VxLAN support for xenserver on future release
version.

Thanks,
Keerthi

RE: CentOS Cloud SIG effort

[Adrian Lewis]

Pretty sure that if getting the management server up and running was as
simple as...
1. Install CentOS
2. yum install cloudstack
3. setup-cloudstack-all-in-one.sh
...we'd see many more people at least trying it out.

Some might see the current installation options as easy enough but if
someone could get it up and running without even looking at the docs, I
reckon they'd be more likely to do it and then consult the docs when they
got stuck.

If the packages could be put into the centos-extras repo that would do the
trick. I'm sure there's more to it than my simplistic idea but could we
discuss the viability of this? We could do with a one-stop script that does
everything for the user including installing the mysql/mariadb server aspect
& even setting up NFS shares on the same box (leaving the other more
granular setup scripts for 'advanced' users. If centos-extras is not
feasible, how about EPEL? Might even get some Fedora people interested as
well (if it works on Fedora).

Pretty sure that this particular CentOS SIG is about running the
management/infrastructure side of things as opposed to running centos as a
guest in case anyone is unsure. Although clearly related, earlier comments
about cloud-init are surely more related to another CentOS SIG (Cloud
Instance) aren't they?

Adrian

-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: 02 April 2015 09:20
To: sebgoa
Cc: dev@cloudstack.apache.org; Paul Angus
Subject: Re: CentOS Cloud SIG effort

BTW, if anyone has some concrete ideas/questions re Cloudstack and CentOS,
please come forth so we can mention them during the meeting.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
> From: "Nux!" 
> To: "sebgoa" 
> Cc: dev@cloudstack.apache.org, "Paul Angus" 
> Sent: Thursday, 2 April, 2015 09:08:57
> Subject: Re: CentOS Cloud SIG effort

> Sure, I'll do my best to attend.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
>> From: "sebgoa" 
>> To: dev@cloudstack.apache.org, "Paul Angus" ,
>> "Nux!"
>> 
>> Sent: Thursday, 2 April, 2015 07:54:13
>> Subject: Re: CentOS Cloud SIG effort
>
>> All, and specially Paul and Nux,
>>
>> Any chance you can join this meeting, and see what we need to do on
>> the CentOS front ?
>>
>> On Apr 1, 2015, at 5:03 PM, Rich Bowen  wrote:
>>
>>>
>>>
>>> On 03/27/2015 04:41 AM, Sebastien Goasguen wrote:
> >On Mar 26, 2015, at 4:28 PM, Rich Bowen  wrote:
> >
> >A while back I mentioned to some folks (I think it was this list,
> >but it may have been a subset) that the CentOS community is
> >working on a Cloud SIG (Special Interest Group) effort. You can
> >read a little about it
> >athttp://wiki.centos.org/SpecialInterestGroup/Cloud
> >
> >The idea is to ensure that cloud infrastructure software, like
> >CloudStack, OpenStack, Open Nebula, and Eucalyptus, works solidly
> >on CentOS, has all of the prerequisite packages available, gets CI on
> >the CentOS platform, and so on.
> >
> >At the moment, this is*only*  OpenStack, with the other projects
> >unrepresented.
> >
> >If you are interested in adoption of CloudStack on CentOS (and,
> >by side effect, on Red Hat Enterprise Linux), we'd love to have
> >your participation in this effort.
> >
 Hi Rich, thanks for the ping again.

 We have been in touch with KB (Nux! and I mostly) and submitted our
 scripts for building a cloudstack centOS templates upstream. It
 works and was merged at some point, but it got pulled back because we
 stick some scripts in there.

 Bottom line is that I feel we need to work further upstream in
 cloud-init to improve cloudstack support there, once that’s done,
 we can come back to the CentOS builds for cloudstack.

 fwiw, our install base is probably ~70% centOS and we already have
 centOS7 support.
>>>
>>> In the OpenStack world, we see CentOS as a great way to get the
>>> message out about OpenStack. Wearing my ASF hat, I'd really like to
>>> see the same vehicle be used to get the word out about CloudStack.
>>> CentOS goes to a lot of events, and many of them are ones that
>>> CloudStack isn't at. I'd love to see the Cloud SIG be a way to get
>>> the word about CloudStack into audiences that typically only ever
>>> hear about OpenStack. (Yes, I have split loyalties here, and that's
>>> fine.)
>>>
>>> Anyways, a reminder that we will be having this meeting on
>>> #centos-devel at
>>> 15:00 UTC *tomorrow*, and it would be awesome to at least have some
>>> representation from the CloudStack community there to ask the right
>>> questions and see what we can do, on the CentOS side, to fix these
>>> cloud-init problems and bring CloudStack some more of the CentOS
>>> spotlight. Or even just show up so that folks can meet you and we
>>> can figure out if there's anything we can do to help one a

RE: CentOS Cloud SIG effort

[Adrian Lewis]

Hi Lucian,

This is still a very devops/developer centric approach which in my opinion
is rife within the ACS community (understandably) and which is inadvertently
hostile/elitist to many who might otherwise be interested. I think that many
regular sys admins who perhaps don't want to get involved with docker,
github, compiling stuff, running 3rd party provisioning scripts etc and just
want to run up a quick POC would end up being alienated by this - they don’t
have the time to learn these sorts of technologies if they would never use
them otherwise. Obviously to most of the audience in this list that's not
the case but I really think that there are a lot of potential (albeit likely
small) deployments out there where the admins run a mile if getting
something to work involves the word 'git' or even 'mailing-list'. These
people just go out and buy vCloud Director instead or do without. Citrix
would probably get more customers for CloudPlatform as well if it were very
simple to try out ACS.

ACS needs hobbyists and sys admins in SMBs as well in my opinion, not just
devops people in large corporations or service providers. More people
playing with it and in turn talking/blogging about it and raising its
profile will help immensely. These people need to be convinced that
#Cloudstackworks.

Get packages into Debian/Ubuntu and there's an even greater audience. Grab
the long tail and the rest of the beast comes with it.

Just my opinion btw - perhaps I'm too old-fashioned and need to learn more.

Adrian

-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: 02 April 2015 13:01
To: dev@cloudstack.apache.org
Cc: sebgoa; Paul Angus
Subject: Re: CentOS Cloud SIG effort

Hi Adrian

> Pretty sure that if getting the management server up and running was
> as simple as...
> 1. Install CentOS
> 2. yum install cloudstack
> 3. setup-cloudstack-all-in-one.sh
> ...we'd see many more people at least trying it out.
>
> Some might see the current installation options as easy enough but if
> someone could get it up and running without even looking at the docs,
> I reckon they'd be more likely to do it and then consult the docs when
> they got stuck.

This reminds me of https://github.com/thehyperadvisor/cldstk-deploy , though
there could certainly more/better ideas on how to ease things up, at least
for a PoC.
I'm sure Sebastien will quickly propose docker. :)

>
> If the packages could be put into the centos-extras repo that would do
> the trick. I'm sure there's more to it than my simplistic idea but
> could we discuss the viability of this?

I think this will be difficult to achieve, though I am short on proper
details, I believe the way we (in cloudstack) ship some stuff - particularly
java stuff - is not exactly kosher from a RedHat packaging point of view.
They have their own routine, practices and so on. Furthermore, let's say we
get that right, keeping it up to date in their repo will also be quite an
effort.
I think the idea is good and in an ideal world it's how we'd do it, but
right now with the release cadence of Cloudstack and our few resources, it's
something that - simply - it's not worth doing.

> We could do with a one-stop script that does everything for the user
> including installing the mysql/mariadb server aspect & even setting up
> NFS shares on the same box (leaving the other more granular setup
> scripts for 'advanced' users. If centos-extras is not feasible, how
> about EPEL? Might even get some Fedora people interested as well (if
> it works on Fedora).

See the ansible link I gave above.
Re EPEL and Fedora, they're having trouble maintaining their own stuff, i.e.
they removed openstack from there and are maintaining separate repositories
at https://repos.fedorapeople.org/repos/openstack/

>
> Pretty sure that this particular CentOS SIG is about running the
> management/infrastructure side of things as opposed to running centos
> as a guest in case anyone is unsure. Although clearly related, earlier
> comments about cloud-init are surely more related to another CentOS
> SIG (Cloud
> Instance) aren't they?

Yep, CentOS Cloud Instance SIG is a different project meant to get CentOS
running on all major cloud platforms. This one is somehow successful in that
their official image will boot in Cloudstack and get a ssh key if one is
set, even execute user data, but there are many bugs and other problems. Far
from ideal; it would be great if someone with python skills would take up
polishing the cloud-init Cloudstack source a bit, perhaps as part of GSoC.

My stance on all this is, bother less with packaging or inclusion in CentOS
official repos and focus more on getting it to work as smoothly as possible.

Also, attending the CentOS events with presentations on Cloudstack is a
great idea to raise some awareness.

/imho

Lucian

RE: xenserver 6.5

[Adrian Lewis]

Out of interest, on the assumption that there are no issues with using 6.5
when it's released and there are no backwards-compatibility problems, will
it then work with 4.4.1 or does CS need to be *explicitly* told that newer,
effectively unknown versions are 'acceptable' as a valid hypervisor?
Basically, If we deploy CS 4.4.1 and we like the look of XS 6.5 when it
comes out, will we need to make any changes to CS to start using it? If so,
are these simple edits to the contents of a file or would it require
rebuilding?

-Original Message-
From: Stephen Turner [mailto:stephen.tur...@citrix.com]
Sent: 20 October 2014 15:28
To: dev@cloudstack.apache.org
Subject: RE: xenserver 6.5

I think it should be minimal, because although there are large internal
changes (e.g., 3.x kernel, 64-bit dom0, new Xen, new storage datapath, PVHVM
mode for RHEL/CentOS 7), the interface is essentially unchanged.

-- 
Stephen Turner


-Original Message-
From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
Sent: 20 October 2014 14:32
To: dev
Subject: xenserver 6.5

Does anybody (know of) work on supporting xenserver 6.5 or has an idea of
how much effort that is going to be?

--
Daan

RE: xenserver 6.5

[Adrian Lewis]

With XS 6.5 released, is anyone able to comment on:

1. Does the 4.5 branch need updating to support it?
2. If the changes are so minor, will we see support in 4.3.x or 4.4.x as
well?

Do we consider this to be a feature or bug? If the code for the resource
class stays exactly the same and the only thing blocking the use of XS 6.5
is the checks that CS does when adding a new host, would this not be
considered as a bug? Technically the validation is broken as its intent is
to determine whether or not the current resource class can handle the
hypervisor. If the current resource class can in fact handle XS6.5 but the
validation code says it can't, isn’t this is a bug?

Cheers,

Adrian

-Original Message-
From: Tim Mackey [mailto:tmac...@gmail.com]
Sent: 20 October 2014 20:10
To: dev@cloudstack.apache.org
Subject: Re: xenserver 6.5

Correct on both counts

On Mon, Oct 20, 2014 at 2:55 PM, Daan Hoogland 
wrote:

> thanks Tim, from this I take that hypervisor versions are hardcoded
> still, and xenserver 6.5 is supported since 4.5. correct?
>
> On Mon, Oct 20, 2014 at 8:36 PM, Tim Mackey  wrote:
>
> > Daan,
> >
> > Here are the relevant commits:
> >
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=2b
> e02d1f515d8d089b6596127614fe6b8030d723
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=b7
> f5e95c8f17cf42d35705872b4210db8c2def72
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=67
> 4af6e47313fa18c18536a2daed90d13b9a9a59
> >
> > Mike,
> >
> > Here's an example of the type of DB changes:
> >
> >
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blobdiff;f=
> setup/db/db/schema-441to450.sql;h=e6aae8e3d624744af9f19b132fa8f53b5a4c
> ddb5;hp=34d5f8842005f8a2da4df8a9a838d919cc648831;hb=2be02d1f515d8d089b
> 6596127614fe6b8030d723;hpb=f212aa57c32eb05d6a69730e37ac50bdb1f0a268
> >
> > On Mon, Oct 20, 2014 at 1:51 PM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> > > Yeah, Tim, I'm a little unclear of what you mean by requiring a DB
> > update.
> > >
> > > Can you clarify that?
> > >
> > > Thanks!
> > >
> > > On Mon, Oct 20, 2014 at 11:29 AM, Daan Hoogland <
> daan.hoogl...@gmail.com
> > >
> > > wrote:
> > >
> > > > Tim, these changes are needed? so 4.4.1 will not work with db
> > changes...
> > > Do
> > > > you have a commit id?
> > > >
> > > > On Mon, Oct 20, 2014 at 6:54 PM, Tim Mackey 
> wrote:
> > > >
> > > > > I know that master had a bunch of cleanup work to make things
> > > > > work
> > > better
> > > > > (commits were a month ago), but baring any significant issues,
> being
> > > able
> > > > > to support a newer XenServer should be as simple as a database
> > update.
> > > > So
> > > > > net of this master *today* should work fine with 6.5 (and the
> various
> > > > > pre-release builds since beta.2).
> > > > >
> > > > > On Mon, Oct 20, 2014 at 12:45 PM, Mike Tutkowski <
> > > > > mike.tutkow...@solidfire.com> wrote:
> > > > >
> > > > > > Someone correct me if I'm wrong but, if a previous XenServer
> > resource
> > > > > class
> > > > > > can handle the newer version of XenServer, then I don't
> > > > > > think you
> > > need
> > > > to
> > > > > > make any changes to CloudStack files to use that newer version.
> > > > > >
> > > > > > If you do see some incompatibility with that version of
> XenServer,
> > > then
> > > > > > someone would need to create a new resource class to handle
> > > > > > the discrepancies.
> > > > > >
> > > > > > On Monday, October 20, 2014, Adrian Lewis <
> > > adr...@alsiconsulting.co.uk
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Out of interest, on the assumption that there are no
> > > > > > > issues
> with
> > > > using
> > > > > > 6.5
> > > > > > > when it's released and there are no
> > > > > > > backwards-compatibility
> > > problems,
> > > > > > will
> > > > > > > it then work with 4.4.1 or does CS need to be *explicit

RE: [VOTE] Apache CloudStack 4.5.0 RC1

[Adrian Lewis]

Rohit,

I don’t suppose you will be creating the el7 packages as well will you?

Adrian

-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: 13 January 2015 07:37
To: dev@cloudstack.apache.org
Cc: us...@cloudstack.apache.org
Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC1

(+ users)

Hi everyone,

David has started the voting process for 4.5.0 candidate, please help test
this candidate.
In case you’re unable to build from source, you may use following repository
built from SHA 8db3cbd4ff62b17a8b496026b68cf60ee0c76740:

DEB: http://packages.bhaisaab.org/cloudstack/testing/debian/4.5/
RPM: http://packages.bhaisaab.org/cloudstack/testing/centos/4.5/
SystemVM Templates: http://packages.shapeblue.com/systemvmtemplate/4.5/4.5.0

> On 13-Jan-2015, at 4:46 am, David Nalley  wrote:
>
> Hi folks,
>
> I've created a 4.5.0 release candidate, with the following artifacts
> up for a vote:
>
> Git Branch and Commit SH:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=refs
> /heads/4.5-RC20150112T2256;hb=4.5-RC20150112T2256
> Commit: 8db3cbd4ff62b17a8b496026b68cf60ee0c76740
>
> Source release (checksums and signatures are available at the same
> location):
> https://dist.apache.org/repos/dist/dev/cloudstack/4.5.0-rc1/
>
> PGP release keys (signed using 6FE50F1C):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
> Vote will be open for at least 72 hours.
>
> For sanity in tallying the vote, can PMC members please be sure to
> indicate "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 88 262 30892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design &
Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Software
Engineering
CloudStack Infrastructure
Support
CloudStack Bootcamp Training
Courses

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a
company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.

RE: [VOTE] Apache CloudStack 4.5.0 RC1

[Adrian Lewis]

Thanks!

-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: 15 January 2015 09:33
To: dev@cloudstack.apache.org
Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC1

Hi Adrian,

I'll be creating CentOS7 packages soon, for that I need to procure and setup
a RHEL/CentOS7 machine. I'll try to do that this weekend and will share them
next week.

On Wednesday 14 January 2015 10:52 PM, Adrian Lewis wrote:
> Rohit,
>
> I don’t suppose you will be creating the el7 packages as well will you?
>
> Adrian
>
> -Original Message-
> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
> Sent: 13 January 2015 07:37
> To: dev@cloudstack.apache.org
> Cc: us...@cloudstack.apache.org
> Subject: Re: [VOTE] Apache CloudStack 4.5.0 RC1
>
> (+ users)
>
> Hi everyone,
>
> David has started the voting process for 4.5.0 candidate, please help
> test this candidate.
> In case you’re unable to build from source, you may use following
> repository built from SHA 8db3cbd4ff62b17a8b496026b68cf60ee0c76740:
>
> DEB: http://packages.bhaisaab.org/cloudstack/testing/debian/4.5/
> RPM: http://packages.bhaisaab.org/cloudstack/testing/centos/4.5/
> SystemVM Templates:
> http://packages.shapeblue.com/systemvmtemplate/4.5/4.5.0
>
>> On 13-Jan-2015, at 4:46 am, David Nalley  wrote:
>>
>> Hi folks,
>>
>> I've created a 4.5.0 release candidate, with the following artifacts
>> up for a vote:
>>
>> Git Branch and Commit SH:
>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=tree;h=ref
>> s
>> /heads/4.5-RC20150112T2256;hb=4.5-RC20150112T2256
>> Commit: 8db3cbd4ff62b17a8b496026b68cf60ee0c76740
>>
>> Source release (checksums and signatures are available at the same
>> location):
>> https://dist.apache.org/repos/dist/dev/cloudstack/4.5.0-rc1/
>>
>> PGP release keys (signed using 6FE50F1C):
>> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>>
>> Vote will be open for at least 72 hours.
>>
>> For sanity in tallying the vote, can PMC members please be sure to
>> indicate "(binding)" with their vote?
>>
>> [ ] +1  approve
>> [ ] +0  no opinion
>> [ ] -1  disapprove (and reason why)
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +91 88 262 30892 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related
> services
>
> IaaS Cloud Design &
> Build<http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment
> framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Software
> Engineering<http://shapeblue.com/cloudstack-software-engineering/>
> CloudStack Infrastructure
> Support<http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training
> Courses<http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed.
> Any views or opinions expressed are solely those of the author and do
> not necessarily represent those of Shape Blue Ltd or related
> companies. If you are not the intended recipient of this email, you
> must neither take any action based upon its contents, nor copy or show
> it to anyone. Please contact the sender if you believe you have
> received this email in error. Shape Blue Ltd is a company incorporated
> in England & Wales. ShapeBlue Services India LLP is a company
> incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in
> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> Pty Ltd is a company registered by The Republic of South Africa and is
> traded under license from Shape Blue Ltd. ShapeBlue is a registered
> trademark.
>

--
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 8826230892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab PS. If you see any footer below, I
did not add it :) Find out more about ShapeBlue and our range of CloudStack
related services

IaaS Cloud Design &
Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software
Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure
Support<http://shapeblue.com/cloudstack-infrastructure-support/&

RE: [DISCUSS] we need a better SSVM solution

[Adrian Lewis]

>From a non-dev user's perspective I think Paul's pretty much nailed the key
issues I'd like to see improve with the system VMs. The big one for us is
the ability to customise the VR template to add things like netflow export
and other value-add services through additional software packages without
having to do this individually on each VR deployed.

-Original Message-
From: Ahmad Emneina [mailto:aemne...@gmail.com]
Sent: 29 January 2015 22:17
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] we need a better SSVM solution

Pauls suggestion reminds me of some awesome functionality I see in the
aftermarket android ROM community. That is 'Kitchens'[1].

A utility/site that provides functionality that allows for admins to create
customized system templates...

Giving choices of:
- OS
- kernel
- VPN server
- various other services...

Of course this is fantasy at the moment, I see the lowest barrier to entry
would be a cloud-init style utility where we can pass in commands or
scripts, like the steps to mitigate the GHOST vuln (which seems to be a few
apt commands). That would easily resolve issues where a vulnerable service
could easily be updated post boot, and propagated to all new/restarted
system vm's.

[1] http://forum.xda-developers.com/showthread.php?t=633246

On Thu, Jan 29, 2015 at 1:55 PM, John Kinsella  wrote:

> Decent points. You think the difference between the VR/CP is different
> enough to have a second image?
>
> > On Jan 29, 2015, at 1:41 PM, Paul Angus 
> wrote:
> >
> > Hi All,
> >
> > I think that there are 3 things people would like to see:
> >
> > 1. clear versioning of system vm templates, with some kind of
> compatibility matrix so they know which one(s) they can use with
> different versions of CloudStack
> > 2. an easy way to update the system vm template 3. an easy(ish) way
> > to customise system vm templates
> >
> > It might be worth considering have two types of template a. the
> > console proxy and secondary storage template b. the virtual router/
> > VPC template.
> >
> >
> >
> > Regards
> >
> > Paul Angus
> > Cloud Architect
> > S: +44 20 3603 0540 | M: +447711418784 | T: CloudyAngus
> > paul.an...@shapeblue.com
> >
> > -Original Message-
> > From: John Kinsella [mailto:j...@stratosec.co]
> > Sent: 29 January 2015 18:06
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] we need a better SSVM solution
> >
> > Interesting…
> >
> > Concur on having an open/standardized protocol. Something clustered
> > like
> Serf/Consul could be attractive, but the overhead/requirements of
> those type of things usually scares me away.
> >
> > Having ACS act as a CA would be quite interesting for some things.
> > It’s
> one of the reasons I’ve pondered a “hook” in the past to notify 3rd
> party upon VM creation/deletion/etc. Wonder if we could take advantage
> of dogtag or similar. All that said - setup/management of a CA is a
> PIA and probably outside scope of ACS, unless you did a “light” one
> similar to Puppet by default...
> >
> > An aside on that “hook” idea - something scriptable similar to (I
> > said
> “similar to," no flames!) systemd for this could be interesting.
> >
> > A good portion of users would resist having an agent installed on
> > the
> user VM, but I guess we’re in that position already, and they just
> wouldn’t get the added functionality.
> >
> > One user experience point: Almost every time Parallels comes out
> > with a
> new version, I have to update their agent on my VMs, which on the
> Windows side means a reboot. That gets old, and I’ve only got a
> handful of win VMs there...
> >
> > Going to see if I can puppet-ize one of the SSVMs over the weekend
> > to
> see what other thoughts come up.
> >
> > John
> >
> >> On Jan 29, 2015, at 2:06 AM, Rohit Yadav
> >> 
> wrote:
> >>
> >> Good ideas John.
> >>
> >> I’m in fact already discussing a design I’m calling it "agents
> framework” (suggestions for better name are welcome!), I will try to
> share and update the spec soon that aims for this feature and
> refactoring work for ACS 4.6/master. For now, I’ve shared an
> architecture diagram here and some high level goals:
> >>
> >> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Agents+Frame
> >> work
> >>
> >> Along with this, I’ve strong opinions and interests in just getting
> >> rid
> of Java based agents in systemvms (to reduce memory footprint) and
> replace the current agent-management server protocol (TCP based, which
> connects to only one management server on prt 8250 even if there are
> multiple management servers) with some interoperable protocol such as
> json/http, thrift etc that allows us to build better/scalable console
> proxy services (for example). People don’t discuss much, but virtual
> routers and systemvms are not well tested at all, we should also need
> efforts/infra to test these components with less human QA.
> >>
> >> Regards.
> >>
> >>> On 29-Jan-2015, at 2:14 am, John Kinsella  wrote:
> >>>
> >>> Every time t