Jenkins build is still unstable: simulator-singlerun #191

[jenkins]

See 

Jenkins build is still unstable: simulator-singlerun #192

[jenkins]

See 

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Rohit Yadav]

+1 from my side

Hi Lucian,

About the issue you’re having, please comment on the issue and share 
logs/exceptions. I’m willing to fix any issue so we can release cloudmonkey as 
soon as possible, we’re already on take #3.

Cheers.

On 23-Aug-2014, at 2:07 pm, Nux!  wrote:
> -1 for now (do I have binding "powers"?)
>
> I've started with fresh config (ie. rm -rf ~/.cloudmonkey), but I keep 
> getting this:
>
> 🐵 > sync
> Error processing json: No JSON object could be decoded
> Failed to sync apis, please check your config?
> Note: `sync` requires api discovery service enabled on the CloudStack 
> management server
>
> The log is extensive and with loads of sensitive information, will need 
> clearing which I can't do right now.
>
> All my settings are AFAIK correct.. Previous version was working fine with 
> the same credentials.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
>
> - Original Message -
>> From: "Sebastien Goasguen" 
>> To: dev@cloudstack.apache.org
>> Sent: Friday, 22 August, 2014 7:53:13 PM
>> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
>>
>>
>> On Aug 22, 2014, at 5:16 AM, Rohit Yadav  wrote:
>>
>>> Hi,
>>>
>>> On 22-Aug-2014, at 10:22 am, Sebastien Goasguen  wrote:
>>>
 Rohit, I am not sure what you tried to do with the url setting.

 This is not entirely backward compatible because I need to create a
 profile anyway (add profile under core then move the user section under
 the profile name..etc).
 If I keep host, path etc…it dos create a url key automatically but it
 seems to be hard coded to http://localhost:8080/client/api
>>>
>>> When cloudmonkey starts, it assumes the defaults in case any key is missing
>>> in the config. This issue you’re seeing is because cloudmonkey does not
>>> know which config to pick the new one of old one and what to do if both
>>> old and new ones exist.
>>>
 I was under the impression that it would read existing
 host/path/port/protocol keys and form a a new url value…
>>>
>>> If we had host, port, path, protocol in the config file and the server
>>> profile section as well there won’t be any way to know which one to pick.
>>> To fix that, we don’t write or read these values from config file, but we
>>> still support the set command on them which internally sets the url
>>> parameter.
>>>
>>> This was also done so that any of your automation/script that are still
>>> using set commands on these options won’t fail. And now cloudmonkey will
>>> not fail and exit if there are any missing keys but it would add them in
>>> the config and let you know.
>>>
 It does work with a properly set config file…so still +1 though in a
 perfect world this new version would re-format my config file properly
 including forming the profile section out of the user and server
 sections.
>>>
>>> I think backward compatibility is best maintained at interface level so we
>>> are not taking away any (set) options which was the fix.
>>>
>>> If you’ve any solution on how to handle the case where both old config and
>>> new config are present in the config, let’s fix that? There were couple of
>>> other minor corner cases, supporting them would have added some technical
>>> debt to the code so I avoided that and I think the impact to users would
>>> be minimum. Please advise?
>>
>> Since I don't have time to help with this, I am +1 as is.
>>
>>
>>
>>>
>>> Regards,
>>> Rohit Yadav
>>> Software Architect, ShapeBlue
>>> M. +41 779015219 | rohit.ya...@shapeblue.com
>>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>>
>>>
>>>
>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>
>>> IaaS Cloud Design &
>>> Build
>>> CSForge – rapid IaaS deployment framework
>>> CloudStack Consulting
>>> CloudStack Infrastructure
>>> Support
>>> CloudStack Bootcamp Training
>>> Courses
>>>
>>> This email and any attachments to it may be confidential and are intended
>>> solely for the use of the individual to whom it is addressed. Any views or
>>> opinions expressed are solely those of the author and do not necessarily
>>> represent those of Shape Blue Ltd or related companies. If you are not the
>>> intended recipient of this email, you must neither take any action based
>>> upon its contents, nor copy or show it to anyone. Please contact the
>>> sender if you believe you have received this email in error. Shape Blue
>>> Ltd is a company incorporated in England & Wales. ShapeBlue Services India
>>> LLP is a company incorporated in India and is operated under license from
>>> Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
>>> incorporated in Brasil and is operated under license from Shape Blue Ltd.
>>> ShapeBlue SA Pt

CI: simulator tests and jenkins

[Rajani Karuturi]

We are discussing on the CI infra and the tests/hardware required. It is a
much needed discussion. Its something which everybody agrees that we
require.

Can we take a step back and look at out existing jenkins setup?
No CI system would help us if we dont look at it.

We have [simulator-singlerun] job there which I believe runs after every
commit and executes all the simulator based tests. (I am not sure on this.
Can someone confirm?)
This job has been failing for the past 100+ builds with 1 consistent
failure and a few no ones here and there.
Can we give enough attention to this please and fix any failures
immediately?

We also have a [hotfix-trigger] which is run on any branch with name
matching hotfix*

Anybody aware of any such job which runs test cases/does sanity checking?

[simulator-singlerun]
http://jenkins.buildacloud.org/job/simulator-singlerun/
[hotfix-trigger]
http://jenkins.buildacloud.org/view/simulator/job/simulator-hotfix-trigger/

~Rajani

Unable to add host in private zone

[Giri Prasad]

Hi All,

 I have configured cs 4.1 on ubuntu 12.04 LTS. The usual normal config, 1 zone, 
1 primary/secondary storage, one host (on same server/ip address as cs 
management server - XXX.XXX.X.5) etc. I created a couple of domains and then 
some accounts.  I am using IP address XXX.XXX.X.100 to XXX.XXX.X.109 for pod 
addresses and XXX.XXX.X.110 to XXX.XXX.X.119 for guest traffic.The IP 
XXX.XXX.X.5 is being used for this host in the default zone. Everything works 
fine upto this point.


 Now, I am trying to create a private zone. I am using the same setup as in the 
default zone creation, when the cs is first configured. The only differenece is 
that I am using IP address XXX.XXX.X.120 to XXX.XXX.X.129 for pod addresses and 
XXX.XXX.X.130 to XXX.XXX.X.139 for guest traffic. When pressing the create 
(private) zone button in cs 4.1 gui, it is coming out with the following 
errors.  The same IP XXX.XXX.X.5 is being used for this host in the private 
zone too. Any insights as to why?

Thanks & Regards,
D.Giriprasad

2014-08-25 15:03:30,971 DEBUG [cloud.api.ApiServlet] (catalina-exec-6:null) 
===START===  XXX.XXX.XXX.5 -- POST  
command=addHost&response=json&sessionkey=tUbyrF3OEZWlMQMRCR0JbQJS5Xk%3D
2014-08-25 15:03:30,984 INFO  [cloud.resource.ResourceManagerImpl] 
(catalina-exec-6:null) Trying to add a new host at http://XXX.XXX.XXX.5 in data 
center 2
2014-08-25 15:03:30,987 DEBUG [kvm.discoverer.KvmServerDiscoverer] 
(catalina-exec-6:null) Skipping XXX.XXX.XXX.5 because 
2e9e9f7c-017e-32a1-a45a-236d182fb28c-LibvirtComputingResource is already in the 
database.
2014-08-25 15:03:30,987 WARN  [cloud.resource.ResourceManagerImpl] 
(catalina-exec-6:null) Unable to find the server resources at 
http://XXX.XXX.XXX.5
2014-08-25 15:03:30,988 INFO  [utils.exception.CSExceptionErrorCode] 
(catalina-exec-6:null) Could not find exception: 
com.cloud.exception.DiscoveryException in error code list for exceptions
2014-08-25 15:03:30,988 WARN  [admin.host.AddHostCmd] (catalina-exec-6:null) 
Exception: 
com.cloud.exception.DiscoveryException: Unable to add the host
    at 
com.cloud.resource.ResourceManagerImpl.discoverHostsFull(ResourceManagerImpl.java:867)
    at 
com.cloud.resource.ResourceManagerImpl.discoverHosts(ResourceManagerImpl.java:619)
    at 
org.apache.cloudstack.api.command.admin.host.AddHostCmd.execute(AddHostCmd.java:143)
    at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:162)
    at com.cloud.api.ApiServer.queueCommand(ApiServer.java:505)
    at com.cloud.api.ApiServer.handleRequest(ApiServer.java:355)
    at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:302)
    at com.cloud.api.ApiServlet.doPost(ApiServlet.java:71)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
    at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
    at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
    at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
    at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:615)
    at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
    at 
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
    at 
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:744)
    at 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2282)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:701)
2014-08-25 15:03:30,989 INFO  [cloud.api.ApiServer] (catalina-exec-6:null) 
Unable to add the host
2014-08-25 15:03:30,990 DEBUG [cloud.api.ApiServlet] (catalina-exec-6:null) 
===END===  XXX.XXX.XXX.5 -- POST  
command=addHost&response=json&sessionkey=tUbyrF3OEZWlMQMRCR0JbQJS5Xk%3D
2014-08-25 15:03:31,401 DEBUG [agent.manager.AgentManagerImpl] 
(AgentManager-Handler-4:null) Ping from 1
2014-08-25 15:03:35,059 DEBUG [agent.manager.AgentManagerImpl] 
(AgentManager-Handler-5:null) SeqA 3-87: Processing Seq 3-87:  { Cmd , MgmtId: 
-1, via: 3, Ver: v1, Flags: 11, 
[{"ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n  
\"connections\": []\n}","wait":0}}] }
2014-08-25 15:03:35,144 DEBUG [agent.manager.AgentManagerImpl] 
(AgentManager-Handler-5:null) SeqA 3-87: Sending Seq 3-87:  { Ans: , MgmtId: 
34057587018585, via: 3, Ver: v1, F

cloud-init issue

[Sebastien Goasguen]

Folks,

There is an outstanding issue with cloud-init compatibility.
The VR serves the instance metadata through a web server but there is a 
trailing slash issue:

https://issues.apache.org/jira/browse/CLOUDSTACK-7405

Does anyone know where the settings for this web server are located in the 
cloudstack code ?

-sebastien

Jenkins build is still unstable: simulator-singlerun #193

[jenkins]

See 

Re: S3/Swift Problem around Virtual Size

[Punith S]

thanks for opening this thread mike,

since i only use nfs as my secondary storage provider, i didn't see this
issue till date.

is this issue occurring even using a S3 secondary storage with staging nfs
store ?

if so like edison pointed we need to fetch the virtual size from the nfs
store instead of S3 in the deploymentmanager.

thanks


On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Hey Edison,
>
> Do you know how difficult/easy of a fix this is, who might be available to
> put this fix in, and for what release (hopefully 4.4.1) this fix could find
> its way in?
>
> Thanks!
> Mike
>
>
> On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault <
> fgaudrea...@cloudops.com> wrote:
>
> > Min,
> >
> > Ok, but this is not the behavior I see. Even without requesting a VM
> > create, the template is pushed to the staging NFS at least once. Is it
> > downloaded there or pushed after download, that I am not sure. I was
> > assuming the swift upload bash script was executed after the template is
> on
> > the staging.
> >
> > Anyway... the focus is on the virt size, and you all know the code better
> > than I do :)
> >
> > FG
> > On Aug 22, 2014 5:28 PM, "Min Chen"  wrote:
> >
> >> No. For S3/Swift, register template will directly upload templates to S3
> >> without going through staging NFS. It will only be copied to staging NFS
> >> when we first use that template to provision a VM.
> >>
> >> Thanks
> >> -min
> >>
> >> On 8/22/14 2:25 PM, "Francois Gaudreault" 
> >> wrote:
> >>
> >> >Edison,
> >> >
> >> >Isnt the templates downloaded to the Staging NFS first?
> >> >
> >> >FG
> >> >On Aug 22, 2014 5:20 PM, "Edison Su"  wrote:
> >> >
> >> >> I know the reason why the size of template doesn¹t have correct
> virtual
> >> >> size if it¹s registered in S3/Swift:
> >> >> In case of s3/swift, the template is directly stored into s3/swift
> >> >>through
> >> >> swift/s3 api, there is no place for cloudstack to look into template,
> >> to
> >> >> find out the virtual size during template registration.
> >> >> While, if secondary storage is NFS, the template is first stored on
> >> >> NFS(which has file system), cloudstack can unzip the template(if
> it¹s a
> >> >> zipped file), and read virtual size from the file, then report back
> to
> >> >>mgt
> >> >> server.
> >> >> In order to fix it, we can add some code as: all the templates
> >> >>registered
> >> >> on Swift/S3, need to be downloaded to a NFS intermediate storage
> before
> >> >>it
> >> >> can be consumed by primary storage. After the download finished, then
> >> we
> >> >> check virtual size of template, and report back to mgt server/update
> DB
> >> >>etc.
> >> >>
> >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> >> >> Sent: Friday, August 22, 2014 1:38 PM
> >> >> To: dev@cloudstack.apache.org
> >> >> Cc: Edison Su
> >> >> Subject: S3/Swift Problem around Virtual Size
> >> >>
> >> >> Hi,
> >> >>
> >> >> This was brought up in a different e-mail thread, but I wanted to
> make
> >> >>it
> >> >> more clear that it's related to CloudStack's download code around
> >> >>S3/Swift,
> >> >> so I'm opening up a new thread.
> >> >>
> >> >> Francois (from CloudOps) noticed today that when he downloaded a
> >> >>template
> >> >> (VHD format) to Swift (but it looks like the same applies for S3)
> that
> >> >>the
> >> >> physical and virtual sizes are set to be the same.
> >> >>
> >> >> This appears to have the following consequence:
> >> >>
> >> >> You can download a template with a physical size of, say, 3 GB and a
> >> >>root
> >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual size
> >> >>showing
> >> >> as 20 GB, it shows as 3 GB.
> >> >>
> >> >> This is not an issue with NFS. In that situation, the two sizes are
> >> >> correctly accounted for.
> >> >>
> >> >> What later can happen is the template is downloaded from Swift and
> >> takes
> >> >> up an unexpected amount of space on the XenServer storage repository
> >> >>(SR).
> >> >>
> >> >> If there is enough space on the SR, this isn't too big of a deal.
> >> >>However,
> >> >> for so-called managed storage plug-ins (examples are SolidFire and
> >> >> CloudByte), this will lead to them dynamically creating a SAN volume
> of
> >> >>the
> >> >> wrong size.
> >> >>
> >> >> Francois opened up the following ticket:
> >> >>
> >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406
> >> >>
> >> >> Thanks!
> >> >>
> >> >> --
> >> >> Mike Tutkowski
> >> >> Senior CloudStack Developer, SolidFire Inc.
> >> >> e: mike.tutkow...@solidfire.com
> >> >> o: 303.746.7302
> >> >> Advancing the way the world uses the cloud<
> >> >> http://solidfire.com/solution/overview/?video=play>
> >> >>
> >>
> >>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> *

Jenkins build is still unstable: simulator-singlerun #194

[jenkins]

See 

Re: cloud-init issue

[Wido den Hollander]

On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:

Folks,

There is an outstanding issue with cloud-init compatibility.
The VR serves the instance metadata through a web server but there is a 
trailing slash issue:

https://issues.apache.org/jira/browse/CLOUDSTACK-7405

Does anyone know where the settings for this web server are located in the 
cloudstack code ?



So I think that is done in 
systemvm/patches/debian/config/opt/cloud/bin/vmdata.py


It adds a RewriteRule to the .htaccess like:

entry = "RewriteRule ^meta-data/(.+)$  ../" + folder + 
"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"


I'm not a big mod_rewrite guru however.

Wido


-sebastien

Re: S3/Swift Problem around Virtual Size

[Francois Gaudreault]

Punith,

I highly anticipate the same issue with S3... it shares a lot of code with
swift.

My focus would be swift, but we should fix for both :)

FG
On Aug 25, 2014 6:33 AM, "Punith S"  wrote:

> thanks for opening this thread mike,
>
> since i only use nfs as my secondary storage provider, i didn't see this
> issue till date.
>
> is this issue occurring even using a S3 secondary storage with staging nfs
> store ?
>
> if so like edison pointed we need to fetch the virtual size from the nfs
> store instead of S3 in the deploymentmanager.
>
> thanks
>
>
> On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski <
> mike.tutkow...@solidfire.com> wrote:
>
> > Hey Edison,
> >
> > Do you know how difficult/easy of a fix this is, who might be available
> to
> > put this fix in, and for what release (hopefully 4.4.1) this fix could
> find
> > its way in?
> >
> > Thanks!
> > Mike
> >
> >
> > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault <
> > fgaudrea...@cloudops.com> wrote:
> >
> > > Min,
> > >
> > > Ok, but this is not the behavior I see. Even without requesting a VM
> > > create, the template is pushed to the staging NFS at least once. Is it
> > > downloaded there or pushed after download, that I am not sure. I was
> > > assuming the swift upload bash script was executed after the template
> is
> > on
> > > the staging.
> > >
> > > Anyway... the focus is on the virt size, and you all know the code
> better
> > > than I do :)
> > >
> > > FG
> > > On Aug 22, 2014 5:28 PM, "Min Chen"  wrote:
> > >
> > >> No. For S3/Swift, register template will directly upload templates to
> S3
> > >> without going through staging NFS. It will only be copied to staging
> NFS
> > >> when we first use that template to provision a VM.
> > >>
> > >> Thanks
> > >> -min
> > >>
> > >> On 8/22/14 2:25 PM, "Francois Gaudreault" 
> > >> wrote:
> > >>
> > >> >Edison,
> > >> >
> > >> >Isnt the templates downloaded to the Staging NFS first?
> > >> >
> > >> >FG
> > >> >On Aug 22, 2014 5:20 PM, "Edison Su"  wrote:
> > >> >
> > >> >> I know the reason why the size of template doesn¹t have correct
> > virtual
> > >> >> size if it¹s registered in S3/Swift:
> > >> >> In case of s3/swift, the template is directly stored into s3/swift
> > >> >>through
> > >> >> swift/s3 api, there is no place for cloudstack to look into
> template,
> > >> to
> > >> >> find out the virtual size during template registration.
> > >> >> While, if secondary storage is NFS, the template is first stored on
> > >> >> NFS(which has file system), cloudstack can unzip the template(if
> > it¹s a
> > >> >> zipped file), and read virtual size from the file, then report back
> > to
> > >> >>mgt
> > >> >> server.
> > >> >> In order to fix it, we can add some code as: all the templates
> > >> >>registered
> > >> >> on Swift/S3, need to be downloaded to a NFS intermediate storage
> > before
> > >> >>it
> > >> >> can be consumed by primary storage. After the download finished,
> then
> > >> we
> > >> >> check virtual size of template, and report back to mgt
> server/update
> > DB
> > >> >>etc.
> > >> >>
> > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> > >> >> Sent: Friday, August 22, 2014 1:38 PM
> > >> >> To: dev@cloudstack.apache.org
> > >> >> Cc: Edison Su
> > >> >> Subject: S3/Swift Problem around Virtual Size
> > >> >>
> > >> >> Hi,
> > >> >>
> > >> >> This was brought up in a different e-mail thread, but I wanted to
> > make
> > >> >>it
> > >> >> more clear that it's related to CloudStack's download code around
> > >> >>S3/Swift,
> > >> >> so I'm opening up a new thread.
> > >> >>
> > >> >> Francois (from CloudOps) noticed today that when he downloaded a
> > >> >>template
> > >> >> (VHD format) to Swift (but it looks like the same applies for S3)
> > that
> > >> >>the
> > >> >> physical and virtual sizes are set to be the same.
> > >> >>
> > >> >> This appears to have the following consequence:
> > >> >>
> > >> >> You can download a template with a physical size of, say, 3 GB and
> a
> > >> >>root
> > >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual size
> > >> >>showing
> > >> >> as 20 GB, it shows as 3 GB.
> > >> >>
> > >> >> This is not an issue with NFS. In that situation, the two sizes are
> > >> >> correctly accounted for.
> > >> >>
> > >> >> What later can happen is the template is downloaded from Swift and
> > >> takes
> > >> >> up an unexpected amount of space on the XenServer storage
> repository
> > >> >>(SR).
> > >> >>
> > >> >> If there is enough space on the SR, this isn't too big of a deal.
> > >> >>However,
> > >> >> for so-called managed storage plug-ins (examples are SolidFire and
> > >> >> CloudByte), this will lead to them dynamically creating a SAN
> volume
> > of
> > >> >>the
> > >> >> wrong size.
> > >> >>
> > >> >> Francois opened up the following ticket:
> > >> >>
> > >> >> https://issues.apache.org/jira/browse/CLOUDSTACK-7406
> > >> >>
> > >> >> Thanks!
> > >> >>
> > >> >> --
> > >> >> Mike Tutkowski
> > >> >> Seni

can't associate a Portable IP in an isolated network

[Elbadri Mohamed]

Hello,
after setting up a zone in acs, i added an isolated network and a range of
portable IP, but when i try to associate an IP to the isolated network
(multi-zone = yes ) a message shows up telling me "this entity already
exist" and i couldn't acquire the address

Re: cloud-init issue

[Erik Weber]

On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander  wrote:

> On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
>
>> Folks,
>>
>> There is an outstanding issue with cloud-init compatibility.
>> The VR serves the instance metadata through a web server but there is a
>> trailing slash issue:
>>
>> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
>>
>> Does anyone know where the settings for this web server are located in
>> the cloudstack code ?
>>
>>
> So I think that is done in systemvm/patches/debian/
> config/opt/cloud/bin/vmdata.py
>
> It adds a RewriteRule to the .htaccess like:
>
> entry = "RewriteRule ^meta-data/(.+)$  ../" + folder + "/%{REMOTE_ADDR}/$1
> [L,NC,QSA]"
>
> I'm not a big mod_rewrite guru however.
>
>
Adding a line with "RewriteRule ^meta-data$ ../" + folder
+"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
should suffice.


Tested on a VR with this config:

RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]

RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]


[root@jenkins ~]# curl 10.30.81.1/latest/meta-data

service-offering

availability-zone

local-ipv4

local-hostname

public-ipv4

public-hostname

instance-id

vm-id

public-keys

cloud-identifier


Erik


> Wido
>
>  -sebastien
>>
>>
>

Re: cloud-init issue

[Erik Weber]

On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:

> On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
> wrote:
>
>> On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
>>
>>> Folks,
>>>
>>> There is an outstanding issue with cloud-init compatibility.
>>> The VR serves the instance metadata through a web server but there is a
>>> trailing slash issue:
>>>
>>> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
>>>
>>> Does anyone know where the settings for this web server are located in
>>> the cloudstack code ?
>>>
>>>
>> So I think that is done in systemvm/patches/debian/
>> config/opt/cloud/bin/vmdata.py
>>
>> It adds a RewriteRule to the .htaccess like:
>>
>> entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
>> "/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
>>
>> I'm not a big mod_rewrite guru however.
>>
>>
> Adding a line with "RewriteRule ^meta-data$ ../" + folder
> +"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
> should suffice.
>
> small typo there...



>
> Tested on a VR with this config:
>
> RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]
>
> RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>
> RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>
>
>
A better solution is probably to make the last slash conditional, like this:
RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

tested:
[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP

HTTP/1.1 200 OK

[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP

HTTP/1.1 200 OK

Erik

Re: cloud-init issue

[Nux!]

Thanks Erik!

Hopefully this change will be incorporated soon so we don't need to patch 
cloud-init.



--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -
> From: "Erik Weber" 
> To: "dev" 
> Sent: Monday, 25 August, 2014 12:34:10 PM
> Subject: Re: cloud-init issue
> 
> On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:
> 
> > On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
> > wrote:
> >
> >> On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
> >>
> >>> Folks,
> >>>
> >>> There is an outstanding issue with cloud-init compatibility.
> >>> The VR serves the instance metadata through a web server but there is a
> >>> trailing slash issue:
> >>>
> >>> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
> >>>
> >>> Does anyone know where the settings for this web server are located in
> >>> the cloudstack code ?
> >>>
> >>>
> >> So I think that is done in systemvm/patches/debian/
> >> config/opt/cloud/bin/vmdata.py
> >>
> >> It adds a RewriteRule to the .htaccess like:
> >>
> >> entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
> >> "/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
> >>
> >> I'm not a big mod_rewrite guru however.
> >>
> >>
> > Adding a line with "RewriteRule ^meta-data$ ../" + folder
> > +"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
> > should suffice.
> >
> > small typo there...
> 
> 
> 
> >
> > Tested on a VR with this config:
> >
> > RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]
> >
> > RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
> >
> > RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
> >
> >
> >
> A better solution is probably to make the last slash conditional, like this:
> RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
> 
> tested:
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP
> 
> HTTP/1.1 200 OK
> 
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
> 
> HTTP/1.1 200 OK
> 
> Erik
> 

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Nux!]

Rohit,

Here are the log and the config:

http://tmp.nux.ro/VK9-log
http://tmp.nux.ro/Mt4-config

Let me know if you require more info.

HTH

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -
> From: "Rohit Yadav" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 25 August, 2014 10:22:23 AM
> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
> 
> +1 from my side
> 
> Hi Lucian,
> 
> About the issue you’re having, please comment on the issue and share
> logs/exceptions. I’m willing to fix any issue so we can release cloudmonkey
> as soon as possible, we’re already on take #3.
> 
> Cheers.
> 
> On 23-Aug-2014, at 2:07 pm, Nux!  wrote:
> > -1 for now (do I have binding "powers"?)
> >
> > I've started with fresh config (ie. rm -rf ~/.cloudmonkey), but I keep
> > getting this:
> >
> > 🐵 > sync
> > Error processing json: No JSON object could be decoded
> > Failed to sync apis, please check your config?
> > Note: `sync` requires api discovery service enabled on the CloudStack
> > management server
> >
> > The log is extensive and with loads of sensitive information, will need
> > clearing which I can't do right now.
> >
> > All my settings are AFAIK correct.. Previous version was working fine with
> > the same credentials.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> >
> > - Original Message -
> >> From: "Sebastien Goasguen" 
> >> To: dev@cloudstack.apache.org
> >> Sent: Friday, 22 August, 2014 7:53:13 PM
> >> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
> >>
> >>
> >> On Aug 22, 2014, at 5:16 AM, Rohit Yadav 
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> On 22-Aug-2014, at 10:22 am, Sebastien Goasguen  wrote:
> >>>
>  Rohit, I am not sure what you tried to do with the url setting.
> 
>  This is not entirely backward compatible because I need to create a
>  profile anyway (add profile under core then move the user section under
>  the profile name..etc).
>  If I keep host, path etc…it dos create a url key automatically but it
>  seems to be hard coded to http://localhost:8080/client/api
> >>>
> >>> When cloudmonkey starts, it assumes the defaults in case any key is
> >>> missing
> >>> in the config. This issue you’re seeing is because cloudmonkey does not
> >>> know which config to pick the new one of old one and what to do if both
> >>> old and new ones exist.
> >>>
>  I was under the impression that it would read existing
>  host/path/port/protocol keys and form a a new url value…
> >>>
> >>> If we had host, port, path, protocol in the config file and the server
> >>> profile section as well there won’t be any way to know which one to pick.
> >>> To fix that, we don’t write or read these values from config file, but we
> >>> still support the set command on them which internally sets the url
> >>> parameter.
> >>>
> >>> This was also done so that any of your automation/script that are still
> >>> using set commands on these options won’t fail. And now cloudmonkey will
> >>> not fail and exit if there are any missing keys but it would add them in
> >>> the config and let you know.
> >>>
>  It does work with a properly set config file…so still +1 though in a
>  perfect world this new version would re-format my config file properly
>  including forming the profile section out of the user and server
>  sections.
> >>>
> >>> I think backward compatibility is best maintained at interface level so
> >>> we
> >>> are not taking away any (set) options which was the fix.
> >>>
> >>> If you’ve any solution on how to handle the case where both old config
> >>> and
> >>> new config are present in the config, let’s fix that? There were couple
> >>> of
> >>> other minor corner cases, supporting them would have added some technical
> >>> debt to the code so I avoided that and I think the impact to users would
> >>> be minimum. Please advise?
> >>
> >> Since I don't have time to help with this, I am +1 as is.
> >>
> >>
> >>
> >>>
> >>> Regards,
> >>> Rohit Yadav
> >>> Software Architect, ShapeBlue
> >>> M. +41 779015219 | rohit.ya...@shapeblue.com
> >>> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>>
> >>>
> >>>
> >>> Find out more about ShapeBlue and our range of CloudStack related
> >>> services
> >>>
> >>> IaaS Cloud Design &
> >>> Build
> >>> CSForge – rapid IaaS deployment framework
> >>> CloudStack Consulting
> >>> CloudStack Infrastructure
> >>> Support
> >>> CloudStack Bootcamp Training
> >>> Courses
> >>>
> >>> This email and any attachments to it may be confidential and are intended
> >>> solely for the use of the individual to whom it is addressed. Any views
> >>> or
> >>> opinions e

Review Request 25023: CLOUDSTACK-7405: Allow VR metadata to be accessed without trailing slash

[Erik Weber]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25023/
---

Review request for cloudstack and Marcus Sorensen.


Bugs: CLOUDSTACK-7405
https://issues.apache.org/jira/browse/CLOUDSTACK-7405


Repository: cloudstack-git


Description
---

As per https://issues.apache.org/jira/browse/CLOUDSTACK-7405 cloud-init expects 
to be able to get meta-data directory without using a trailing slash.

Ultimately this should be fixed in cloud-init, but it's an unintrusive fix in 
cloudstack


Diffs
-

  systemvm/patches/debian/config/opt/cloud/bin/vmdata.py f508032 

Diff: https://reviews.apache.org/r/25023/diff/


Testing
---

tested with curl that both new and old url works

[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/vm-id | grep HTTP
HTTP/1.1 200 OK
[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
HTTP/1.1 200 OK


Thanks,

Erik Weber

Jenkins build is still unstable: simulator-singlerun #195

[jenkins]

See 

Re: cloud-init issue

[Wido den Hollander]

On 08/25/2014 01:33 PM, Nux! wrote:

Thanks Erik!



Indeed!


Hopefully this change will be incorporated soon so we don't need to patch 
cloud-init.



I came up with a very small patch: 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=1499972804b6de4fade5c84436e29f9904165e18


Tested it on my local env and worked fine. Both the old and new URL work 
with the same output.


Wido




--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -

From: "Erik Weber" 
To: "dev" 
Sent: Monday, 25 August, 2014 12:34:10 PM
Subject: Re: cloud-init issue

On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:


On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
wrote:


On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:


Folks,

There is an outstanding issue with cloud-init compatibility.
The VR serves the instance metadata through a web server but there is a
trailing slash issue:

https://issues.apache.org/jira/browse/CLOUDSTACK-7405

Does anyone know where the settings for this web server are located in
the cloudstack code ?



So I think that is done in systemvm/patches/debian/
config/opt/cloud/bin/vmdata.py

It adds a RewriteRule to the .htaccess like:

entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"

I'm not a big mod_rewrite guru however.



Adding a line with "RewriteRule ^meta-data$ ../" + folder
+"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
should suffice.

small typo there...






Tested on a VR with this config:

RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]

RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]




A better solution is probably to make the last slash conditional, like this:
RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

tested:
[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP

HTTP/1.1 200 OK

[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP

HTTP/1.1 200 OK

Erik

Re: cloud-init issue

[Sebastien Goasguen]

On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wrote:

> On 08/25/2014 01:33 PM, Nux! wrote:
>> Thanks Erik!
>> 
> 
> Indeed!
> 
>> Hopefully this change will be incorporated soon so we don't need to patch 
>> cloud-init.
>> 
> 
> I came up with a very small patch: 
> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=1499972804b6de4fade5c84436e29f9904165e18
> 
> Tested it on my local env and worked fine. Both the old and new URL work with 
> the same output.
> 

thanks folks, I will pull that in 4.3



> Wido
> 
>> 
>> 
>> --
>> Sent from the Delta quadrant using Borg technology!
>> 
>> Nux!
>> www.nux.ro
>> 
>> 
>> - Original Message -
>>> From: "Erik Weber" 
>>> To: "dev" 
>>> Sent: Monday, 25 August, 2014 12:34:10 PM
>>> Subject: Re: cloud-init issue
>>> 
>>> On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:
>>> 
 On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
 wrote:
 
> On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
> 
>> Folks,
>> 
>> There is an outstanding issue with cloud-init compatibility.
>> The VR serves the instance metadata through a web server but there is a
>> trailing slash issue:
>> 
>> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
>> 
>> Does anyone know where the settings for this web server are located in
>> the cloudstack code ?
>> 
>> 
> So I think that is done in systemvm/patches/debian/
> config/opt/cloud/bin/vmdata.py
> 
> It adds a RewriteRule to the .htaccess like:
> 
> entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
> "/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
> 
> I'm not a big mod_rewrite guru however.
> 
> 
 Adding a line with "RewriteRule ^meta-data$ ../" + folder
 +"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
 should suffice.
 
 small typo there...
>>> 
>>> 
>>> 
 
 Tested on a VR with this config:
 
 RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]
 
 RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
 
 RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
 
 
 
>>> A better solution is probably to make the last slash conditional, like this:
>>> RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>>> 
>>> tested:
>>> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP
>>> 
>>> HTTP/1.1 200 OK
>>> 
>>> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
>>> 
>>> HTTP/1.1 200 OK
>>> 
>>> Erik
>>> 
> 

Re: cloud-init issue

[Wido den Hollander]

On 08/25/2014 02:25 PM, Sebastien Goasguen wrote:


On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wrote:


On 08/25/2014 01:33 PM, Nux! wrote:

Thanks Erik!



Indeed!


Hopefully this change will be incorporated soon so we don't need to patch 
cloud-init.



I came up with a very small patch: 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=1499972804b6de4fade5c84436e29f9904165e18

Tested it on my local env and worked fine. Both the old and new URL work with 
the same output.



thanks folks, I will pull that in 4.3



Hmm. I didn't make a separate branch as a hotfix for 4.4, so how do we 
get this in 4.4.1 as well?


We shouldn't cherry-pick anymore, right?

Wido





Wido




--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -

From: "Erik Weber" 
To: "dev" 
Sent: Monday, 25 August, 2014 12:34:10 PM
Subject: Re: cloud-init issue

On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:


On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
wrote:


On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:


Folks,

There is an outstanding issue with cloud-init compatibility.
The VR serves the instance metadata through a web server but there is a
trailing slash issue:

https://issues.apache.org/jira/browse/CLOUDSTACK-7405

Does anyone know where the settings for this web server are located in
the cloudstack code ?



So I think that is done in systemvm/patches/debian/
config/opt/cloud/bin/vmdata.py

It adds a RewriteRule to the .htaccess like:

entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"

I'm not a big mod_rewrite guru however.



Adding a line with "RewriteRule ^meta-data$ ../" + folder
+"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
should suffice.

small typo there...






Tested on a VR with this config:

RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]

RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]




A better solution is probably to make the last slash conditional, like this:
RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]

tested:
[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP

HTTP/1.1 200 OK

[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP

HTTP/1.1 200 OK

Erik

Re: cloud-init issue

[Sebastien Goasguen]

On Aug 25, 2014, at 8:26 AM, Wido den Hollander  wrote:

> On 08/25/2014 02:25 PM, Sebastien Goasguen wrote:
>> 
>> On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wrote:
>> 
>>> On 08/25/2014 01:33 PM, Nux! wrote:
 Thanks Erik!
 
>>> 
>>> Indeed!
>>> 
 Hopefully this change will be incorporated soon so we don't need to patch 
 cloud-init.
 
>>> 
>>> I came up with a very small patch: 
>>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=1499972804b6de4fade5c84436e29f9904165e18
>>> 
>>> Tested it on my local env and worked fine. Both the old and new URL work 
>>> with the same output.
>>> 
>> 
>> thanks folks, I will pull that in 4.3
>> 
> 
> Hmm. I didn't make a separate branch as a hotfix for 4.4, so how do we get 
> this in 4.4.1 as well?
> 
> We shouldn't cherry-pick anymore, right?

Ideally no, if you make a hotfix/4.4-7405 branch for it you can ask Daan to 
merge it in 4.4 branch

I will also merge it in 4.3



> 
> Wido
> 
>> 
>> 
>>> Wido
>>> 
 
 
 --
 Sent from the Delta quadrant using Borg technology!
 
 Nux!
 www.nux.ro
 
 
 - Original Message -
> From: "Erik Weber" 
> To: "dev" 
> Sent: Monday, 25 August, 2014 12:34:10 PM
> Subject: Re: cloud-init issue
> 
> On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:
> 
>> On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
>> wrote:
>> 
>>> On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
>>> 
 Folks,
 
 There is an outstanding issue with cloud-init compatibility.
 The VR serves the instance metadata through a web server but there is a
 trailing slash issue:
 
 https://issues.apache.org/jira/browse/CLOUDSTACK-7405
 
 Does anyone know where the settings for this web server are located in
 the cloudstack code ?
 
 
>>> So I think that is done in systemvm/patches/debian/
>>> config/opt/cloud/bin/vmdata.py
>>> 
>>> It adds a RewriteRule to the .htaccess like:
>>> 
>>> entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
>>> "/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
>>> 
>>> I'm not a big mod_rewrite guru however.
>>> 
>>> 
>> Adding a line with "RewriteRule ^meta-data$ ../" + folder
>> +"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
>> should suffice.
>> 
>> small typo there...
> 
> 
> 
>> 
>> Tested on a VR with this config:
>> 
>> RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]
>> 
>> RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>> 
>> RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>> 
>> 
>> 
> A better solution is probably to make the last slash conditional, like 
> this:
> RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
> 
> tested:
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP
> 
> HTTP/1.1 200 OK
> 
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
> 
> HTTP/1.1 200 OK
> 
> Erik
> 
>>> 
>> 
> 

Re: Review Request 25023: CLOUDSTACK-7405: Allow VR metadata to be accessed without trailing slash

[Sebastien Goasguen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25023/#review51392
---

Ship it!


even though Wido pushed the same fix to master, I applied your patch to 4.3 :)
ac04e6d97ec9e3afeb16937402efbd306fdccae9

thanks, you can mark the review as submitted

- Sebastien Goasguen


On Aug. 25, 2014, 12:05 p.m., Erik Weber wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25023/
> ---
> 
> (Updated Aug. 25, 2014, 12:05 p.m.)
> 
> 
> Review request for cloudstack and Marcus Sorensen.
> 
> 
> Bugs: CLOUDSTACK-7405
> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> As per https://issues.apache.org/jira/browse/CLOUDSTACK-7405 cloud-init 
> expects to be able to get meta-data directory without using a trailing slash.
> 
> Ultimately this should be fixed in cloud-init, but it's an unintrusive fix in 
> cloudstack
> 
> 
> Diffs
> -
> 
>   systemvm/patches/debian/config/opt/cloud/bin/vmdata.py f508032 
> 
> Diff: https://reviews.apache.org/r/25023/diff/
> 
> 
> Testing
> ---
> 
> tested with curl that both new and old url works
> 
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/vm-id | grep HTTP
> HTTP/1.1 200 OK
> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
> HTTP/1.1 200 OK
> 
> 
> Thanks,
> 
> Erik Weber
> 
>

Re: cloud-init issue

[Sebastien Goasguen]

On Aug 25, 2014, at 8:28 AM, Sebastien Goasguen  wrote:

> 
> On Aug 25, 2014, at 8:26 AM, Wido den Hollander  wrote:
> 
>> On 08/25/2014 02:25 PM, Sebastien Goasguen wrote:
>>> 
>>> On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wrote:
>>> 
 On 08/25/2014 01:33 PM, Nux! wrote:
> Thanks Erik!
> 
 
 Indeed!
 
> Hopefully this change will be incorporated soon so we don't need to patch 
> cloud-init.
> 
 
 I came up with a very small patch: 
 https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=1499972804b6de4fade5c84436e29f9904165e18
 
 Tested it on my local env and worked fine. Both the old and new URL work 
 with the same output.
 
>>> 
>>> thanks folks, I will pull that in 4.3
>>> 
>> 
>> Hmm. I didn't make a separate branch as a hotfix for 4.4, so how do we get 
>> this in 4.4.1 as well?
>> 
>> We shouldn't cherry-pick anymore, right?
> 
> Ideally no, if you make a hotfix/4.4-7405 branch for it you can ask Daan to 
> merge it in 4.4 branch
> 
> I will also merge it in 4.3
> 
> 

Erick, submit the same patch to review board, so I took it from there and patch 
4.3 branch.

Only 4.4 is missing...

> 
>> 
>> Wido
>> 
>>> 
>>> 
 Wido
 
> 
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro
> 
> 
> - Original Message -
>> From: "Erik Weber" 
>> To: "dev" 
>> Sent: Monday, 25 August, 2014 12:34:10 PM
>> Subject: Re: cloud-init issue
>> 
>> On Mon, Aug 25, 2014 at 1:27 PM, Erik Weber  wrote:
>> 
>>> On Mon, Aug 25, 2014 at 1:16 PM, Wido den Hollander 
>>> wrote:
>>> 
 On 08/25/2014 12:12 PM, Sebastien Goasguen wrote:
 
> Folks,
> 
> There is an outstanding issue with cloud-init compatibility.
> The VR serves the instance metadata through a web server but there is 
> a
> trailing slash issue:
> 
> https://issues.apache.org/jira/browse/CLOUDSTACK-7405
> 
> Does anyone know where the settings for this web server are located in
> the cloudstack code ?
> 
> 
 So I think that is done in systemvm/patches/debian/
 config/opt/cloud/bin/vmdata.py
 
 It adds a RewriteRule to the .htaccess like:
 
 entry = "RewriteRule ^meta-data/(.+)$  ../" + folder +
 "/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
 
 I'm not a big mod_rewrite guru however.
 
 
>>> Adding a line with "RewriteRule ^meta-data$ ../" + folder
>>> +"/%{REMOTE_ADDR}/$1 [L,NC,QSA]"
>>> should suffice.
>>> 
>>> small typo there...
>> 
>> 
>> 
>>> 
>>> Tested on a VR with this config:
>>> 
>>> RewriteRule ^meta-data/(.+)$  ../metadata/%{REMOTE_ADDR}/$1 [L,NC,QSA]
>>> 
>>> RewriteRule ^meta-data/$  ../metadata/%{REMOTE_ADDR}/meta-data 
>>> [L,NC,QSA]
>>> 
>>> RewriteRule ^meta-data$  ../metadata/%{REMOTE_ADDR}/meta-data [L,NC,QSA]
>>> 
>>> 
>>> 
>> A better solution is probably to make the last slash conditional, like 
>> this:
>> RewriteRule ^meta-data/?$  ../metadata/%{REMOTE_ADDR}/meta-data 
>> [L,NC,QSA]
>> 
>> tested:
>> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/ | grep HTTP
>> 
>> HTTP/1.1 200 OK
>> 
>> [root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
>> 
>> HTTP/1.1 200 OK
>> 
>> Erik
>> 
 
>>> 
>> 
> 

[ACS441] Hotfix for #7405 (hotfix/4.4-7405)

[Wido den Hollander]

Hi Daan,

Could you merge in hotfix/4.4-7405 for me?

It's a fix for a cloud-init issue.

Sebastien already pulled it in for 4.3.1, but it would be best to also 
have it in 4.4.1.


Wido

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Rohit Yadav]

Hi Lucian,

Your url in the server profile does not look right. You’ve to provide an API 
url and not CloudStack’s UI url.

Something like: https://cloudportal.tld/client/api and not 
https://cloudportal.tld/client/

Can you fix and retry and see if this was the issue?

Cheers.

On 25-Aug-2014, at 1:54 pm, Nux!  wrote:

> Rohit,
>
> Here are the log and the config:
>
> http://tmp.nux.ro/VK9-log
> http://tmp.nux.ro/Mt4-config
>
> Let me know if you require more info.
>
> HTH
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
>
> - Original Message -
>> From: "Rohit Yadav" 
>> To: dev@cloudstack.apache.org
>> Sent: Monday, 25 August, 2014 10:22:23 AM
>> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
>>
>> +1 from my side
>>
>> Hi Lucian,
>>
>> About the issue you’re having, please comment on the issue and share
>> logs/exceptions. I’m willing to fix any issue so we can release cloudmonkey
>> as soon as possible, we’re already on take #3.
>>
>> Cheers.
>>
>> On 23-Aug-2014, at 2:07 pm, Nux!  wrote:
>>> -1 for now (do I have binding "powers"?)
>>>
>>> I've started with fresh config (ie. rm -rf ~/.cloudmonkey), but I keep
>>> getting this:
>>>
>>> 🐵 > sync
>>> Error processing json: No JSON object could be decoded
>>> Failed to sync apis, please check your config?
>>> Note: `sync` requires api discovery service enabled on the CloudStack
>>> management server
>>>
>>> The log is extensive and with loads of sensitive information, will need
>>> clearing which I can't do right now.
>>>
>>> All my settings are AFAIK correct.. Previous version was working fine with
>>> the same credentials.
>>>
>>> --
>>> Sent from the Delta quadrant using Borg technology!
>>>
>>> Nux!
>>> www.nux.ro
>>>
>>>
>>> - Original Message -
 From: "Sebastien Goasguen" 
 To: dev@cloudstack.apache.org
 Sent: Friday, 22 August, 2014 7:53:13 PM
 Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)


 On Aug 22, 2014, at 5:16 AM, Rohit Yadav 
 wrote:

> Hi,
>
> On 22-Aug-2014, at 10:22 am, Sebastien Goasguen  wrote:
>
>> Rohit, I am not sure what you tried to do with the url setting.
>>
>> This is not entirely backward compatible because I need to create a
>> profile anyway (add profile under core then move the user section under
>> the profile name..etc).
>> If I keep host, path etc…it dos create a url key automatically but it
>> seems to be hard coded to http://localhost:8080/client/api
>
> When cloudmonkey starts, it assumes the defaults in case any key is
> missing
> in the config. This issue you’re seeing is because cloudmonkey does not
> know which config to pick the new one of old one and what to do if both
> old and new ones exist.
>
>> I was under the impression that it would read existing
>> host/path/port/protocol keys and form a a new url value…
>
> If we had host, port, path, protocol in the config file and the server
> profile section as well there won’t be any way to know which one to pick.
> To fix that, we don’t write or read these values from config file, but we
> still support the set command on them which internally sets the url
> parameter.
>
> This was also done so that any of your automation/script that are still
> using set commands on these options won’t fail. And now cloudmonkey will
> not fail and exit if there are any missing keys but it would add them in
> the config and let you know.
>
>> It does work with a properly set config file…so still +1 though in a
>> perfect world this new version would re-format my config file properly
>> including forming the profile section out of the user and server
>> sections.
>
> I think backward compatibility is best maintained at interface level so
> we
> are not taking away any (set) options which was the fix.
>
> If you’ve any solution on how to handle the case where both old config
> and
> new config are present in the config, let’s fix that? There were couple
> of
> other minor corner cases, supporting them would have added some technical
> debt to the code so I avoided that and I think the impact to users would
> be minimum. Please advise?

 Since I don't have time to help with this, I am +1 as is.



>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related
> services
>
> IaaS Cloud Design &
> Build
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infr

two bugs to look at

[Sebastien Goasguen]

Anyone up to look at:

https://issues.apache.org/jira/browse/CLOUDSTACK-7327

and 

https://issues.apache.org/jira/browse/CLOUDSTACK-7404

Jenkins build is still unstable: simulator-singlerun #196

[jenkins]

See 

Re: Question about Ceph and Snapshots

[Mike Tutkowski]

Thanks, Wido!

On Monday, August 25, 2014, Wido den Hollander  wrote:

>
>
> On 08/25/2014 04:31 AM, Mike Tutkowski wrote:
>
>> Hey Wido,
>>
>> Can you point me in the right direction to examine the code related to
>> volume snapshots being taken when Ceph is being used for primary storage?
>>
>> I don't see a Ceph storage plug-in that implements the takeSnapshot
>> method.
>>
>
> There is no specific code in the Management Server since it's all done by
> the KVM Agent.
>
> You'll find most code in the LibvirtStorageAdapter in the KVM plugin code.
> That's where it takes the snapshots and copies it over ti Secondary Storage.
>
>
>> In particular, I'm curious how the backup process works.
>>
>>
> In short, it asks Qemu to copy the RBD image to a QCOW2 file on Secondary
> Storage.
>
> Wido
>
>  Thanks!
>>
>>

-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Build failed in Jenkins: cloudstack-4.3-maven-build-noredist #452

[jenkins]

See 


Changes:

[runseb] Updating pom.xml version numbers for release 4.3.1-SNAPSHOT

--
Started by upstream project "cloudstack-4.3-maven-build" build number 560
originally caused by:
 Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on cloudstack-buildslave-centos6-e8b 
(cloudstack-buildslave-centos6) in workspace 

Cloning the remote Git repository
Cloning repository git://git.apache.org/cloudstack.git
Fetching upstream changes from git://git.apache.org/cloudstack.git
Fetching upstream changes from git://git.apache.org/cloudstack.git
Checking out Revision a654894dbd59b9c0674589b988e02b52d8a3dc14 (origin/4.3)
Deleting old artifacts from #449
[copy-to-slave] Copying 'cloudstack-nonoss-deps.tgz', excluding nothing, from 
'file:/var/lib/jenkins/userContent/' on the master to 
' 
on 'cloudstack-buildslave-centos6-e8b'.
[cloudstack-4.3-maven-build-noredist] $ 
/jenkins/tools/hudson.tasks.Maven_MavenInstallation/maven-3.1.1/bin/mvn clean
[INFO] Scanning for projects...
Downloading: 
http://repo.maven.apache.org/maven2/org/apache/cloudstack/cloudstack/4.3.0/cloudstack-4.3.0.pom

[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.cloudstack:xapi:5.6.100-1-SNAPSHOT 
(
 has 1 error
[ERROR] Non-resolvable parent POM: Could not find artifact 
org.apache.cloudstack:cloudstack:pom:4.3.0 in central 
(http://repo.maven.apache.org/maven2) and 'parent.relativePath' points at wrong 
local POM @ line 21, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
Build step 'Invoke top-level Maven targets' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
Archiving artifacts

apidoc error

[Sebastien Goasguen]

Hey,

I am trying to build 4.3.1 and I get this familiar error with apidoc:

[INFO] --- exec-maven-plugin:1.2.1:exec (compile) @ cloud-apidoc ---
Error: Could not find or load main class com.cloud.api.doc.ApiXmlDocWriter

I just don't recall how to fix it...

[ACS441] Hotfix for 7343

[Francois Gaudreault]

Daan,

PLD made a fix for 4.4.1 in the hotfix/4.4-7343 branch.

Can you merge into 4.4? Not sure if he also made the patch for master.

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Jenkins build is still unstable: simulator-singlerun #197

[jenkins]

See 

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Mike Tutkowski]

+1 Nice job, Rohit!

On Thursday, August 21, 2014, Rohit Yadav  wrote:

> Hi All,
>
> Thanks to concerns raised by Sebastien and Erik, I’ve added backward
> compatibility to cloudmonkey to support now deprecated options: host, port,
> protocol and path. Setting them will cause cloudmonkey to warn users but
> set the url option using those values instead. Looking forward to community
> participation and testing.
>
> I've created a new 5.2.0 release candidate of CloudMonkey, with the
> following artifacts up for a vote:
>
> Git Branch and Commit SH:
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/5.2
> Commit: 29695f97494b1046b77d483af2a5f064c168823c
>
> List of changes:
>
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=blob_plain;f=CHANGES;hb=5.2
>
> Source release (checksums and signatures are available at the following
> location as well):
> https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.2.0/
>
> Checksums:
> MD5: 75 33 66 F9 01 FC C5 DD  19 C5 AF 9B AF 73 CB EB
> SHA: B02A488F A60192F5 E74C234A C9B33855 67680142 22F4B4C7 D7F936CC
> 67FDE13C E12FEFD5
>  05371FE7 574A74C8 B0A63DC4 C2FAA064 FD26F07A D3D90319 6774BEB4
>
> PGP release keys (key ID used 0EE3D884):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
> Vote will be open for 72 weekday hours. Since, the voting period overlaps
> an upcoming weekend, we’ll conclude voting on Tuesday, 26th Aug 2014.
>
> For sanity in tallying the vote, can PMC members please be sure to
> indicate "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com 
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>


-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Getting the system template when using Swift as Secondary Storage

[Francois Gaudreault]

Hi,

So far, the only way I found to get SSVM running if using Swift as 
SecStorage is to initially configure the secondary storage as standard 
NFS, get the SSVMs to start, and then switch back to Swift + NFS 
Staging. I have a feeling that this is not the expected behavior. Should 
we be able to start the zone without having to switch from standard NFS 
to Swift once the SSVMs are up?


Thanks!

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Nux!]

Meh, fixed, thanks for the help. :-)

+1 then!

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -
> From: "Rohit Yadav" 
> To: dev@cloudstack.apache.org
> Sent: Monday, 25 August, 2014 1:39:47 PM
> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
> 
> Hi Lucian,
> 
> Your url in the server profile does not look right. You’ve to provide an API
> url and not CloudStack’s UI url.
> 
> Something like: https://cloudportal.tld/client/api and not
> https://cloudportal.tld/client/
> 
> Can you fix and retry and see if this was the issue?
> 
> Cheers.
> 
> On 25-Aug-2014, at 1:54 pm, Nux!  wrote:
> 
> > Rohit,
> >
> > Here are the log and the config:
> >
> > http://tmp.nux.ro/VK9-log
> > http://tmp.nux.ro/Mt4-config
> >
> > Let me know if you require more info.
> >
> > HTH
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> >
> > - Original Message -
> >> From: "Rohit Yadav" 
> >> To: dev@cloudstack.apache.org
> >> Sent: Monday, 25 August, 2014 10:22:23 AM
> >> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)
> >>
> >> +1 from my side
> >>
> >> Hi Lucian,
> >>
> >> About the issue you’re having, please comment on the issue and share
> >> logs/exceptions. I’m willing to fix any issue so we can release
> >> cloudmonkey
> >> as soon as possible, we’re already on take #3.
> >>
> >> Cheers.
> >>
> >> On 23-Aug-2014, at 2:07 pm, Nux!  wrote:
> >>> -1 for now (do I have binding "powers"?)
> >>>
> >>> I've started with fresh config (ie. rm -rf ~/.cloudmonkey), but I keep
> >>> getting this:
> >>>
> >>> 🐵 > sync
> >>> Error processing json: No JSON object could be decoded
> >>> Failed to sync apis, please check your config?
> >>> Note: `sync` requires api discovery service enabled on the CloudStack
> >>> management server
> >>>
> >>> The log is extensive and with loads of sensitive information, will need
> >>> clearing which I can't do right now.
> >>>
> >>> All my settings are AFAIK correct.. Previous version was working fine
> >>> with
> >>> the same credentials.
> >>>
> >>> --
> >>> Sent from the Delta quadrant using Borg technology!
> >>>
> >>> Nux!
> >>> www.nux.ro
> >>>
> >>>
> >>> - Original Message -
>  From: "Sebastien Goasguen" 
>  To: dev@cloudstack.apache.org
>  Sent: Friday, 22 August, 2014 7:53:13 PM
>  Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take
>  #3)
> 
> 
>  On Aug 22, 2014, at 5:16 AM, Rohit Yadav 
>  wrote:
> 
> > Hi,
> >
> > On 22-Aug-2014, at 10:22 am, Sebastien Goasguen 
> > wrote:
> >
> >> Rohit, I am not sure what you tried to do with the url setting.
> >>
> >> This is not entirely backward compatible because I need to create a
> >> profile anyway (add profile under core then move the user section
> >> under
> >> the profile name..etc).
> >> If I keep host, path etc…it dos create a url key automatically but it
> >> seems to be hard coded to http://localhost:8080/client/api
> >
> > When cloudmonkey starts, it assumes the defaults in case any key is
> > missing
> > in the config. This issue you’re seeing is because cloudmonkey does not
> > know which config to pick the new one of old one and what to do if both
> > old and new ones exist.
> >
> >> I was under the impression that it would read existing
> >> host/path/port/protocol keys and form a a new url value…
> >
> > If we had host, port, path, protocol in the config file and the server
> > profile section as well there won’t be any way to know which one to
> > pick.
> > To fix that, we don’t write or read these values from config file, but
> > we
> > still support the set command on them which internally sets the url
> > parameter.
> >
> > This was also done so that any of your automation/script that are still
> > using set commands on these options won’t fail. And now cloudmonkey
> > will
> > not fail and exit if there are any missing keys but it would add them
> > in
> > the config and let you know.
> >
> >> It does work with a properly set config file…so still +1 though in a
> >> perfect world this new version would re-format my config file properly
> >> including forming the profile section out of the user and server
> >> sections.
> >
> > I think backward compatibility is best maintained at interface level so
> > we
> > are not taking away any (set) options which was the fix.
> >
> > If you’ve any solution on how to handle the case where both old config
> > and
> > new config are present in the config, let’s fix that? There were couple
> > of
> > other minor corner cases, supporting them would have added some
> > technical
> > debt to the code so I avoided that and I think the impact to users
>

Jenkins build is unstable: simulator-hotfix-trigger #27

[jenkins]

See 

[MERGE] Merge saml2 branch to master

[Rohit Yadav]

Hi all,

I would like to merge SAML2 SSO/SLO integration with CloudStack.

This auth mechanism is implemented as a plugin with special auth cmds that uses 
an auth framework 
(https://cwiki.apache.org/confluence/display/CLOUDSTACK/Authentication+Refactoring)
 to get the pluggability. The present implementation has addressed several open 
ended questions and was tested to work with feido’s public IdP whose params are 
also set as default config params. Future iterations will try to solve few 
leftover agenda as mentioned on the FS.

Using John Burwell’s recommendation, I’ve not used Spring SAML extension but 
instead used the OpenSAML library and the inbuilt BountyCastle infra for 
auth/X509 stuff.

The major limitation which is by design is that it will work on with HTTP 
redirections bindings (won’t support SOAP and other resolution protocols as per 
SAML2 spec) and x509 signature/usage needs to be improved using either 
CloudStack’s own JKS keystore or create keys in the keystore table when the 
plugin is configured.

More more information please read the proposal, FS and feel free to ask 
questions.

Branch: saml2
Proposal: http://markmail.org/message/4ba4ztmqpud3l4uo
JIRA ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-7083
FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/SAML+2.0+Plugin
Unit tests: Tests for each auth cmd class, SAMLUtils and SAMLAuthenticator, 
fixes unit test for ApiServlet
Build status: clean build works with unit tests, testing using mvn3.0.5 and jdk 
1.7

Compare/diff: https://github.com/apache/cloudstack/compare/master...saml2

As agreed per the branch expectations, I’ll go ahead with the merge after 72 
hours, i.e. on/after Wednesday evening.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Build failed in Jenkins: cloudstack-4.3-maven-build-noredist #453

[jenkins]

See 


Changes:

[runseb] Update CHANGES file with list of bug fixes

--
Started by upstream project "cloudstack-4.3-maven-build" build number 561
originally caused by:
 Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on cloudstack-buildslave-centos6-e8b 
(cloudstack-buildslave-centos6) in workspace 

Fetching changes from the remote Git repository
Fetching upstream changes from git://git.apache.org/cloudstack.git
Checking out Revision b866fe750c4678770a4e9e06bda8ef77586305c7 (origin/4.3)
[copy-to-slave] Copying 'cloudstack-nonoss-deps.tgz', excluding nothing, from 
'file:/var/lib/jenkins/userContent/' on the master to 
' 
on 'cloudstack-buildslave-centos6-e8b'.
[cloudstack-4.3-maven-build-noredist] $ 
/jenkins/tools/hudson.tasks.Maven_MavenInstallation/maven-3.1.1/bin/mvn clean
[INFO] Scanning for projects...
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.cloudstack:xapi:5.6.100-1-SNAPSHOT 
(
 has 1 error
[ERROR] Non-resolvable parent POM: Failure to find 
org.apache.cloudstack:cloudstack:pom:4.3.0 in 
http://repo.maven.apache.org/maven2 was cached in the local repository, 
resolution will not be reattempted until the update interval of central has 
elapsed or updates are forced and 'parent.relativePath' points at wrong local 
POM @ line 21, column 11 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
Build step 'Invoke top-level Maven targets' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
Archiving artifacts

Re: cloud-init issue

[Carlos Reátegui]

Not sure about this ‘fix’.  If the goal is to match AWS here is the way it 
works compared to 4.3.

1) /latest/user-data

On AWS one can query /latest/user-data with or without the trailing slash and 
BOTH return the user-data

On ACS 4.3 /latest/user-data works but with trailing / returns a 404.

2) /latest/meta-data

On AWS /latest/meta-data returns a 301 pointing to trailing / version.  With a 
trailing slash it return the meta-data objects available.

On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it returns the 
meta-data objects available.

3) /latest/meta-data/

On AWS with and without / returns the object value(s)

On ACS 4.3 without a / it returns the value.  With the / it returns a 404.

——

Here are examples of the above (I am also including the headers and in the case 
of user-data only the headers):

ACS 4.3

$ curl -I http://172.30.45.101/latest/user-data/
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:53:18 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1

$ curl -I http://172.30.45.101/latest/user-data
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:53:23 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "10240-0-5006227053140;500622334a140”
Accept-Ranges: bytes

$ curl -i http://172.30.45.101/latest/meta-data
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:54:25 GMT
Server: Apache
Vary: Accept-Encoding
Content-Length: 214
Content-Type: text/html; charset=iso-8859-1



404 Not Found

Not Found
The requested URL /latest/meta-data was not found on this server.


$ curl -i http://172.30.45.101/latest/meta-data/
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:54:32 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "1024a-88-5006227053140;500622334a140"
Accept-Ranges: bytes
Content-Length: 136

service-offering
availability-zone
local-ipv4
local-hostname
public-ipv4
public-hostname
instance-id
vm-id
public-keys
cloud-identifier

$ curl -i http://172.30.45.101/latest/meta-data/local-ipv4; echo
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:55:39 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "10250-d-5006227053140;500622334a140"
Accept-Ranges: bytes
Content-Length: 13

172.30.45.123

$ curl -i http://172.30.45.101/latest/meta-data/local-ipv4/; echo
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:55:45 GMT
Server: Apache
Vary: Accept-Encoding
Content-Length: 232
Content-Type: text/html; charset=iso-8859-1



404 Not Found

Not Found
The requested URL /metadata/172.30.45.123/local-ipv4/ was not found on this 
server.



On AWS EC2

$ curl -I  http://169.254.169.254/latest/user-data
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Accept-Ranges: bytes
ETag: "590463714"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 4790
Connection: close
Date: Mon, 25 Aug 2014 15:57:15 GMT
Server: EC2ws

$ curl -I  http://169.254.169.254/latest/user-data/
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Accept-Ranges: bytes
ETag: "590463714"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 4790
Connection: close
Date: Mon, 25 Aug 2014 15:57:20 GMT
Server: EC2ws

$ curl -i http://169.254.169.254/latest/meta-data
HTTP/1.0 301 Moved Permanently
Location: http://169.254.169.254/latest/meta-data/
Content-Length: 0
Connection: close
Date: Mon, 25 Aug 2014 15:58:00 GMT
Server: EC2ws

$ curl -i http://169.254.169.254/latest/meta-data/; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "1606042795"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 268
Connection: close
Date: Mon, 25 Aug 2014 15:58:50 GMT
Server: EC2ws

ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
iam/
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups

$ curl -i http://169.254.169.254/latest/meta-data/local-ipv4; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "916027669"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 12
Connection: close
Date: Mon, 25 Aug 2014 16:00:32 GMT
Server: EC2ws

10.250.5.124
$ curl -i http://169.254.169.254/latest/meta-data/local-ipv4/; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "916027669"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 12
Connection: close
Date: Mon, 25 Aug 2014 16:00:36 GMT
Server: EC2ws

10.250.5.124




On Aug 25, 2014, at 5:32 AM, Sebastien Goasguen  wrote:

> 
> On Aug 25, 2014, at 8:28 AM, Sebastien Goasguen  wrote:
> 
>> 
>> On Aug 25, 2014, at 8:26 AM, Wido den Hollander  wrote:
>> 
>>> On 08/25/2014 02:25 PM, Sebastien Goasguen wrote:
 
 On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wrote:
 
> On 08/25/2014 01:33 PM, Nux! wrote:
>> Thanks Erik!
>> 
> 
> Indeed!
> 
>> Hopefully this change will be incorporated soon so we don't need to 
>> pat

Re: Review Request 24936: RPM build failing with RHEL7

[Rayees Namathponnan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24936/#review51410
---

Ship it!


Ship It!

- Rayees Namathponnan


On Aug. 22, 2014, 11:03 a.m., Damodar Reddy Talakanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24936/
> ---
> 
> (Updated Aug. 22, 2014, 11:03 a.m.)
> 
> 
> Review request for cloudstack, Rayees Namathponnan and Hugo Trippaers.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> RPM Build was failing on RHEL7 due to the packages changed in RHEL7. 
> Refactored the cloud.spec to make it specific to OS builds.
> 
> 
> Diffs
> -
> 
>   client/tomcatconf/server7-nonssl.xml.in PRE-CREATION 
>   client/tomcatconf/server7-ssl.xml.in PRE-CREATION 
>   packaging/centos63/cloud-management.rc 2f04793 
>   packaging/centos63/cloud.spec 7565f95 
>   packaging/centos63/default/macros.spec PRE-CREATION 
>   packaging/centos63/default/tomcat.sh PRE-CREATION 
>   packaging/centos63/package.sh 07f95fc 
>   packaging/centos63/rhel7/cloud-management.service PRE-CREATION 
>   packaging/centos63/rhel7/macros.spec PRE-CREATION 
>   packaging/centos63/rhel7/tomcat.sh PRE-CREATION 
>   python/lib/cloudutils/serviceConfigServer.py 2c19d7e 
> 
> Diff: https://reviews.apache.org/r/24936/diff/
> 
> 
> Testing
> ---
> 
> Tested on RHEL7 and Centos 6.3
> 
> 
> Thanks,
> 
> Damodar Reddy Talakanti
> 
>

Re: [ACS441] Hotfix for 7343

[Francois Gaudreault]

Anybody backing Daan for 4.4 RM?

We have tested the fix, and so far no issues. This is mainly the same 
fix as S3.


FG

On 2014-08-25, 9:56 AM, Francois Gaudreault wrote:

Daan,

PLD made a fix for 4.4.1 in the hotfix/4.4-7343 branch.

Can you merge into 4.4? Not sure if he also made the patch for master.




--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform & Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: cloud-init issue

[Wido den Hollander]

On 25-08-14 18:02, Carlos Reátegui wrote:

Not sure about this ‘fix’.  If the goal is to match AWS here is the way it 
works compared to 4.3.

1) /latest/user-data

On AWS one can query /latest/user-data with or without the trailing slash and 
BOTH return the user-data

On ACS 4.3 /latest/user-data works but with trailing / returns a 404.



So this fix was just for the meta-data call. Not for the user-data 
query. It could be that there are multiple fixes to be made, but they 
can be addressed separately.



2) /latest/meta-data

On AWS /latest/meta-data returns a 301 pointing to trailing / version.  With a 
trailing slash it return the meta-data objects available.

On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it returns the 
meta-data objects available.

3) /latest/meta-data/

On AWS with and without / returns the object value(s)

On ACS 4.3 without a / it returns the value.  With the / it returns a 404.

——

Here are examples of the above (I am also including the headers and in the case 
of user-data only the headers):

ACS 4.3

$ curl -I http://172.30.45.101/latest/user-data/
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:53:18 GMT
Server: Apache
Vary: Accept-Encoding
Content-Type: text/html; charset=iso-8859-1

$ curl -I http://172.30.45.101/latest/user-data
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:53:23 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "10240-0-5006227053140;500622334a140”
Accept-Ranges: bytes

$ curl -i http://172.30.45.101/latest/meta-data
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:54:25 GMT
Server: Apache
Vary: Accept-Encoding
Content-Length: 214
Content-Type: text/html; charset=iso-8859-1



404 Not Found

Not Found
The requested URL /latest/meta-data was not found on this server.


$ curl -i http://172.30.45.101/latest/meta-data/
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:54:32 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "1024a-88-5006227053140;500622334a140"
Accept-Ranges: bytes
Content-Length: 136

service-offering
availability-zone
local-ipv4
local-hostname
public-ipv4
public-hostname
instance-id
vm-id
public-keys
cloud-identifier

$ curl -i http://172.30.45.101/latest/meta-data/local-ipv4; echo
HTTP/1.1 200 OK
Date: Mon, 25 Aug 2014 15:55:39 GMT
Server: Apache
Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
ETag: "10250-d-5006227053140;500622334a140"
Accept-Ranges: bytes
Content-Length: 13

172.30.45.123

$ curl -i http://172.30.45.101/latest/meta-data/local-ipv4/; echo
HTTP/1.1 404 Not Found
Date: Mon, 25 Aug 2014 15:55:45 GMT
Server: Apache
Vary: Accept-Encoding
Content-Length: 232
Content-Type: text/html; charset=iso-8859-1



404 Not Found

Not Found
The requested URL /metadata/172.30.45.123/local-ipv4/ was not found on this 
server.



On AWS EC2

$ curl -I  http://169.254.169.254/latest/user-data
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Accept-Ranges: bytes
ETag: "590463714"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 4790
Connection: close
Date: Mon, 25 Aug 2014 15:57:15 GMT
Server: EC2ws

$ curl -I  http://169.254.169.254/latest/user-data/
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Accept-Ranges: bytes
ETag: "590463714"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 4790
Connection: close
Date: Mon, 25 Aug 2014 15:57:20 GMT
Server: EC2ws

$ curl -i http://169.254.169.254/latest/meta-data
HTTP/1.0 301 Moved Permanently
Location: http://169.254.169.254/latest/meta-data/
Content-Length: 0
Connection: close
Date: Mon, 25 Aug 2014 15:58:00 GMT
Server: EC2ws

$ curl -i http://169.254.169.254/latest/meta-data/; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "1606042795"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 268
Connection: close
Date: Mon, 25 Aug 2014 15:58:50 GMT
Server: EC2ws

ami-id
ami-launch-index
ami-manifest-path
block-device-mapping/
hostname
iam/
instance-action
instance-id
instance-type
kernel-id
local-hostname
local-ipv4
mac
metrics/
network/
placement/
profile
public-hostname
public-ipv4
public-keys/
reservation-id
security-groups

$ curl -i http://169.254.169.254/latest/meta-data/local-ipv4; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "916027669"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 12
Connection: close
Date: Mon, 25 Aug 2014 16:00:32 GMT
Server: EC2ws

10.250.5.124
$ curl -i http://169.254.169.254/latest/meta-data/local-ipv4/; echo
HTTP/1.0 200 OK
Content-Type: text/plain
Accept-Ranges: bytes
ETag: "916027669"
Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
Content-Length: 12
Connection: close
Date: Mon, 25 Aug 2014 16:00:36 GMT
Server: EC2ws

10.250.5.124




On Aug 25, 2014, at 5:32 AM, Sebastien Goasguen  wrote:



On Aug 25, 2014, at 8:28 AM, Sebastien Goasguen  wrote:



On Aug 25, 2014, at 8:26 AM, Wido den Hollander  wrote:


On 08/25/2014 02:25 PM, Sebastien Goasguen wrote:


On Aug 25, 2014, at 8:16 AM, Wido den Hollander  wro

Jenkins build is still unstable: simulator-singlerun #198

[jenkins]

See 

Re: S3/Swift Problem around Virtual Size

[Mike Tutkowski]

Yes, I expect we'll see the same issue with S3, as well.

Punith - Is this something you might have time to investigate? Perhaps
Edison can point us in the right direction here.


On Mon, Aug 25, 2014 at 5:17 AM, Francois Gaudreault <
fgaudrea...@cloudops.com> wrote:

> Punith,
>
> I highly anticipate the same issue with S3... it shares a lot of code with
> swift.
>
> My focus would be swift, but we should fix for both :)
>
> FG
> On Aug 25, 2014 6:33 AM, "Punith S"  wrote:
>
> > thanks for opening this thread mike,
> >
> > since i only use nfs as my secondary storage provider, i didn't see this
> > issue till date.
> >
> > is this issue occurring even using a S3 secondary storage with staging
> nfs
> > store ?
> >
> > if so like edison pointed we need to fetch the virtual size from the nfs
> > store instead of S3 in the deploymentmanager.
> >
> > thanks
> >
> >
> > On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski <
> > mike.tutkow...@solidfire.com> wrote:
> >
> > > Hey Edison,
> > >
> > > Do you know how difficult/easy of a fix this is, who might be available
> > to
> > > put this fix in, and for what release (hopefully 4.4.1) this fix could
> > find
> > > its way in?
> > >
> > > Thanks!
> > > Mike
> > >
> > >
> > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault <
> > > fgaudrea...@cloudops.com> wrote:
> > >
> > > > Min,
> > > >
> > > > Ok, but this is not the behavior I see. Even without requesting a VM
> > > > create, the template is pushed to the staging NFS at least once. Is
> it
> > > > downloaded there or pushed after download, that I am not sure. I was
> > > > assuming the swift upload bash script was executed after the template
> > is
> > > on
> > > > the staging.
> > > >
> > > > Anyway... the focus is on the virt size, and you all know the code
> > better
> > > > than I do :)
> > > >
> > > > FG
> > > > On Aug 22, 2014 5:28 PM, "Min Chen"  wrote:
> > > >
> > > >> No. For S3/Swift, register template will directly upload templates
> to
> > S3
> > > >> without going through staging NFS. It will only be copied to staging
> > NFS
> > > >> when we first use that template to provision a VM.
> > > >>
> > > >> Thanks
> > > >> -min
> > > >>
> > > >> On 8/22/14 2:25 PM, "Francois Gaudreault"  >
> > > >> wrote:
> > > >>
> > > >> >Edison,
> > > >> >
> > > >> >Isnt the templates downloaded to the Staging NFS first?
> > > >> >
> > > >> >FG
> > > >> >On Aug 22, 2014 5:20 PM, "Edison Su"  wrote:
> > > >> >
> > > >> >> I know the reason why the size of template doesn¹t have correct
> > > virtual
> > > >> >> size if it¹s registered in S3/Swift:
> > > >> >> In case of s3/swift, the template is directly stored into
> s3/swift
> > > >> >>through
> > > >> >> swift/s3 api, there is no place for cloudstack to look into
> > template,
> > > >> to
> > > >> >> find out the virtual size during template registration.
> > > >> >> While, if secondary storage is NFS, the template is first stored
> on
> > > >> >> NFS(which has file system), cloudstack can unzip the template(if
> > > it¹s a
> > > >> >> zipped file), and read virtual size from the file, then report
> back
> > > to
> > > >> >>mgt
> > > >> >> server.
> > > >> >> In order to fix it, we can add some code as: all the templates
> > > >> >>registered
> > > >> >> on Swift/S3, need to be downloaded to a NFS intermediate storage
> > > before
> > > >> >>it
> > > >> >> can be consumed by primary storage. After the download finished,
> > then
> > > >> we
> > > >> >> check virtual size of template, and report back to mgt
> > server/update
> > > DB
> > > >> >>etc.
> > > >> >>
> > > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> > > >> >> Sent: Friday, August 22, 2014 1:38 PM
> > > >> >> To: dev@cloudstack.apache.org
> > > >> >> Cc: Edison Su
> > > >> >> Subject: S3/Swift Problem around Virtual Size
> > > >> >>
> > > >> >> Hi,
> > > >> >>
> > > >> >> This was brought up in a different e-mail thread, but I wanted to
> > > make
> > > >> >>it
> > > >> >> more clear that it's related to CloudStack's download code around
> > > >> >>S3/Swift,
> > > >> >> so I'm opening up a new thread.
> > > >> >>
> > > >> >> Francois (from CloudOps) noticed today that when he downloaded a
> > > >> >>template
> > > >> >> (VHD format) to Swift (but it looks like the same applies for S3)
> > > that
> > > >> >>the
> > > >> >> physical and virtual sizes are set to be the same.
> > > >> >>
> > > >> >> This appears to have the following consequence:
> > > >> >>
> > > >> >> You can download a template with a physical size of, say, 3 GB
> and
> > a
> > > >> >>root
> > > >> >> disk that's supposed to be, say, 20 GB. Instead of the virtual
> size
> > > >> >>showing
> > > >> >> as 20 GB, it shows as 3 GB.
> > > >> >>
> > > >> >> This is not an issue with NFS. In that situation, the two sizes
> are
> > > >> >> correctly accounted for.
> > > >> >>
> > > >> >> What later can happen is the template is downloaded from Swift
> and
> > > >> takes
> > > >> >> up an unexpected amount of space on the 

Re: cloud-init issue

[Sebastien Goasguen]

On Aug 25, 2014, at 1:21 PM, Wido den Hollander  wrote:

> 
> 
> On 25-08-14 18:02, Carlos Reátegui wrote:
>> Not sure about this ‘fix’.  If the goal is to match AWS here is the way it 
>> works compared to 4.3.
>> 
>> 1) /latest/user-data
>> 
>> On AWS one can query /latest/user-data with or without the trailing slash 
>> and BOTH return the user-data
>> 
>> On ACS 4.3 /latest/user-data works but with trailing / returns a 404.
>> 
> 
> So this fix was just for the meta-data call. Not for the user-data query. It 
> could be that there are multiple fixes to be made, but they can be addressed 
> separately.
> 

thanks Carlos for the extensive analysis, Maybe Erik can provide a patch to fix 
all of it :)

>> 2) /latest/meta-data
>> 
>> On AWS /latest/meta-data returns a 301 pointing to trailing / version.  With 
>> a trailing slash it return the meta-data objects available.
>> 
>> On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it returns 
>> the meta-data objects available.
>> 
>> 3) /latest/meta-data/
>> 
>> On AWS with and without / returns the object value(s)
>> 
>> On ACS 4.3 without a / it returns the value.  With the / it returns a 404.
>> 
>> ——
>> 
>> Here are examples of the above (I am also including the headers and in the 
>> case of user-data only the headers):
>> 
>> ACS 4.3
>> 
>> $ curl -I http://172.30.45.101/latest/user-data/
>> HTTP/1.1 404 Not Found
>> Date: Mon, 25 Aug 2014 15:53:18 GMT
>> Server: Apache
>> Vary: Accept-Encoding
>> Content-Type: text/html; charset=iso-8859-1
>> 
>> $ curl -I http://172.30.45.101/latest/user-data
>> HTTP/1.1 200 OK
>> Date: Mon, 25 Aug 2014 15:53:23 GMT
>> Server: Apache
>> Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
>> ETag: "10240-0-5006227053140;500622334a140”
>> Accept-Ranges: bytes
>> 
>> $ curl -i http://172.30.45.101/latest/meta-data
>> HTTP/1.1 404 Not Found
>> Date: Mon, 25 Aug 2014 15:54:25 GMT
>> Server: Apache
>> Vary: Accept-Encoding
>> Content-Length: 214
>> Content-Type: text/html; charset=iso-8859-1
>> 
>> 
>> 
>> 404 Not Found
>> 
>> Not Found
>> The requested URL /latest/meta-data was not found on this server.
>> 
>> 
>> $ curl -i http://172.30.45.101/latest/meta-data/
>> HTTP/1.1 200 OK
>> Date: Mon, 25 Aug 2014 15:54:32 GMT
>> Server: Apache
>> Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
>> ETag: "1024a-88-5006227053140;500622334a140"
>> Accept-Ranges: bytes
>> Content-Length: 136
>> 
>> service-offering
>> availability-zone
>> local-ipv4
>> local-hostname
>> public-ipv4
>> public-hostname
>> instance-id
>> vm-id
>> public-keys
>> cloud-identifier
>> 
>> $ curl -i http://172.30.45.101/latest/meta-data/local-ipv4; echo
>> HTTP/1.1 200 OK
>> Date: Mon, 25 Aug 2014 15:55:39 GMT
>> Server: Apache
>> Last-Modified: Mon, 11 Aug 2014 22:34:05 GMT
>> ETag: "10250-d-5006227053140;500622334a140"
>> Accept-Ranges: bytes
>> Content-Length: 13
>> 
>> 172.30.45.123
>> 
>> $ curl -i http://172.30.45.101/latest/meta-data/local-ipv4/; echo
>> HTTP/1.1 404 Not Found
>> Date: Mon, 25 Aug 2014 15:55:45 GMT
>> Server: Apache
>> Vary: Accept-Encoding
>> Content-Length: 232
>> Content-Type: text/html; charset=iso-8859-1
>> 
>> 
>> 
>> 404 Not Found
>> 
>> Not Found
>> The requested URL /metadata/172.30.45.123/local-ipv4/ was not found on 
>> this server.
>> 
>> 
>> 
>> On AWS EC2
>> 
>> $ curl -I  http://169.254.169.254/latest/user-data
>> HTTP/1.0 200 OK
>> Content-Type: application/octet-stream
>> Accept-Ranges: bytes
>> ETag: "590463714"
>> Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
>> Content-Length: 4790
>> Connection: close
>> Date: Mon, 25 Aug 2014 15:57:15 GMT
>> Server: EC2ws
>> 
>> $ curl -I  http://169.254.169.254/latest/user-data/
>> HTTP/1.0 200 OK
>> Content-Type: application/octet-stream
>> Accept-Ranges: bytes
>> ETag: "590463714"
>> Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
>> Content-Length: 4790
>> Connection: close
>> Date: Mon, 25 Aug 2014 15:57:20 GMT
>> Server: EC2ws
>> 
>> $ curl -i http://169.254.169.254/latest/meta-data
>> HTTP/1.0 301 Moved Permanently
>> Location: http://169.254.169.254/latest/meta-data/
>> Content-Length: 0
>> Connection: close
>> Date: Mon, 25 Aug 2014 15:58:00 GMT
>> Server: EC2ws
>> 
>> $ curl -i http://169.254.169.254/latest/meta-data/; echo
>> HTTP/1.0 200 OK
>> Content-Type: text/plain
>> Accept-Ranges: bytes
>> ETag: "1606042795"
>> Last-Modified: Thu, 03 Apr 2014 23:10:13 GMT
>> Content-Length: 268
>> Connection: close
>> Date: Mon, 25 Aug 2014 15:58:50 GMT
>> Server: EC2ws
>> 
>> ami-id
>> ami-launch-index
>> ami-manifest-path
>> block-device-mapping/
>> hostname
>> iam/
>> instance-action
>> instance-id
>> instance-type
>> kernel-id
>> local-hostname
>> local-ipv4
>> mac
>> metrics/
>> network/
>> placement/
>> profile
>> public-hostname
>> public-ipv4
>> public-keys/
>> reservation-id
>> security-groups
>> 
>> $ curl -i http://169.254.169.254/latest/meta-data/local-ipv4; echo
>> HTTP/1.0 200 OK
>> Content-Type: text/plain
>> Accept-Ranges: bytes
>> ETag: "916027669"
>> Last-Modif

Developing for Project Support

[Will Stevens]

Hey All,
I have had a quick look around at some of the different implementations and
I am seeing that projects in CS seem to be handled as 'special' accounts.
 The API calls have fields like 'projectid' and such, but those details are
not saved into the DB, the only thing that happens in the DB is a new
project account ID is added to the db for that element (think, networks,
instances, etc...)

If I want to extend the functionality of an existing piece of code which
does not support projects to also support projects, do you know of any
specific examples in the code that are good implementations which I should
be using as a good example to follow for adding project support?

Thanks,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

Jenkins build is still unstable: simulator-singlerun #199

[jenkins]

See 

Jenkins build is still unstable: simulator-singlerun #200

[jenkins]

See 

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Chip Childers]

On Thu, Aug 21, 2014 at 02:20:21PM +, Rohit Yadav wrote:
> Hi All,
> 
> Thanks to concerns raised by Sebastien and Erik, I’ve added backward 
> compatibility to cloudmonkey to support now deprecated options: host, port, 
> protocol and path. Setting them will cause cloudmonkey to warn users but set 
> the url option using those values instead. Looking forward to community 
> participation and testing.
> 
> I've created a new 5.2.0 release candidate of CloudMonkey, with the following 
> artifacts up for a vote:
> 
> Git Branch and Commit SH:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/5.2
> Commit: 29695f97494b1046b77d483af2a5f064c168823c
> 
> List of changes:
> https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=blob_plain;f=CHANGES;hb=5.2
> 
> Source release (checksums and signatures are available at the following 
> location as well):
> https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.2.0/
> 
> Checksums:
> MD5: 75 33 66 F9 01 FC C5 DD  19 C5 AF 9B AF 73 CB EB
> SHA: B02A488F A60192F5 E74C234A C9B33855 67680142 22F4B4C7 D7F936CC 67FDE13C 
> E12FEFD5
>  05371FE7 574A74C8 B0A63DC4 C2FAA064 FD26F07A D3D90319 6774BEB4
> 
> PGP release keys (key ID used 0EE3D884):
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> 
> Vote will be open for 72 weekday hours. Since, the voting period overlaps an 
> upcoming weekend, we’ll conclude voting on Tuesday, 26th Aug 2014.
> 
> For sanity in tallying the vote, can PMC members please be sure to indicate 
> "(binding)" with their vote?
> 
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)

+1 (binding)

Re: cloud-init issue

[Erik Weber]

On Mon, Aug 25, 2014 at 6:02 PM, Carlos Reátegui 
wrote:

> Not sure about this ‘fix’.  If the goal is to match AWS here is the way it
> works compared to 4.3.
>
>
The actual fix was just for the reported url, ie. meta-data


> 1) /latest/user-data
>
> On AWS one can query /latest/user-data with or without the trailing slash
> and BOTH return the user-data
>
> On ACS 4.3 /latest/user-data works but with trailing / returns a 404.
>
> 2) /latest/meta-data
>
> On AWS /latest/meta-data returns a 301 pointing to trailing / version.
> With a trailing slash it return the meta-data objects available.
>
>
Is it required to respons with 301, or is 200 with the actual content ok?


> On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it returns
> the meta-data objects available.
>
> 3) /latest/meta-data/
>
> On AWS with and without / returns the object value(s)
>
> On ACS 4.3 without a / it returns the value.  With the / it returns a 404.
>


I'll work on an improved patch that fixes all the URLs.

Erik

Re: Review Request 25023: CLOUDSTACK-7405: Allow VR metadata to be accessed without trailing slash

[Erik Weber]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25023/
---

(Updated Aug. 25, 2014, 7:55 p.m.)


Review request for cloudstack, Marcus Sorensen, Sebastien Goasguen, and Wido 
den Hollander.


Changes
---

This should make all the urls accessible with and without trailing slash.

Testing done:

without trailing slash:
[root@jenkins ~]# for i in "local-hostname" "local-ipv4" "vm-id" "instance-id" 
; do curl -s -I 10.30.81.1/latest/meta-data/$i | grep HTTP; done
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK


with trailing slash:
[root@jenkins ~]# for i in "local-hostname" "local-ipv4" "vm-id" "instance-id" 
; do curl -s -I 10.30.81.1/latest/meta-data/$i/ | grep HTTP; done
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK


Bugs: CLOUDSTACK-7405
https://issues.apache.org/jira/browse/CLOUDSTACK-7405


Repository: cloudstack-git


Description
---

As per https://issues.apache.org/jira/browse/CLOUDSTACK-7405 cloud-init expects 
to be able to get meta-data directory without using a trailing slash.

Ultimately this should be fixed in cloud-init, but it's an unintrusive fix in 
cloudstack


Diffs (updated)
-

  systemvm/patches/debian/config/opt/cloud/bin/vmdata.py f508032 

Diff: https://reviews.apache.org/r/25023/diff/


Testing
---

tested with curl that both new and old url works

[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data/vm-id | grep HTTP
HTTP/1.1 200 OK
[root@jenkins ~]# curl -I -s 10.30.81.1/latest/meta-data | grep HTTP
HTTP/1.1 200 OK


Thanks,

Erik Weber

Re: cloud-init issue

[Erik Weber]

I've uploaded a new patch to reviewboard.

Erik

On Mon, Aug 25, 2014 at 9:32 PM, Erik Weber  wrote:

> On Mon, Aug 25, 2014 at 6:02 PM, Carlos Reátegui 
> wrote:
>
>> Not sure about this ‘fix’.  If the goal is to match AWS here is the way
>> it works compared to 4.3.
>>
>>
> The actual fix was just for the reported url, ie. meta-data
>
>
>> 1) /latest/user-data
>>
>> On AWS one can query /latest/user-data with or without the trailing slash
>> and BOTH return the user-data
>>
>> On ACS 4.3 /latest/user-data works but with trailing / returns a 404.
>>
>> 2) /latest/meta-data
>>
>> On AWS /latest/meta-data returns a 301 pointing to trailing / version.
>> With a trailing slash it return the meta-data objects available.
>>
>>
> Is it required to respons with 301, or is 200 with the actual content ok?
>
>
>> On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it
>> returns the meta-data objects available.
>>
>> 3) /latest/meta-data/
>>
>> On AWS with and without / returns the object value(s)
>>
>> On ACS 4.3 without a / it returns the value.  With the / it returns a 404.
>>
>
>
> I'll work on an improved patch that fixes all the URLs.
>
> Erik
>

Jenkins build is still unstable: simulator-singlerun #201

[jenkins]

See 

Re: Review Request 24892: passwd_server attempts to start but terminates with the exit code 137

[Sheng Yang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24892/#review51437
---

Ship it!


Looks good to me.

- Sheng Yang


On Aug. 20, 2014, 2:25 p.m., bharat kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24892/
> ---
> 
> (Updated Aug. 20, 2014, 2:25 p.m.)
> 
> 
> Review request for cloudstack and Sheng Yang.
> 
> 
> Bugs: CLOUDSTACK-7376
> https://issues.apache.org/jira/browse/CLOUDSTACK-7376
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/CLOUDSTACK-7376
> passwd_server attempts to start but terminates with the exit code 137
> 
> 
> Diffs
> -
> 
>   systemvm/patches/debian/config/opt/cloud/bin/passwd_server 0f4a772 
> 
> Diff: https://reviews.apache.org/r/24892/diff/
> 
> 
> Testing
> ---
> 
> I was able to reproduce this in the older version but not on master.
> but iam putting this patch anyway as this will not effect the regular runs.
> 
> 
> Thanks,
> 
> bharat kumar
> 
>

Review Request 25029: UI: Upgrade jQuery from version 1.6.1 to 1.6.4

[Gabor Apati-Nagy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25029/
---

Review request for cloudstack, Brian Federle and Jessica Wang.


Repository: cloudstack-git


Description
---

Replaced jquery.js file.


Diffs
-

  ui/lib/jquery.js 5d5a1d58ee5148d010b2029c419356440e679941 

Diff: https://reviews.apache.org/r/25029/diff/


Testing
---

I have done smoke test on the UI.


Thanks,

Gabor Apati-Nagy

Review Request for CLOUDSTACK-6254

[David Bierce]

Hello —

I was wondering if someone could take a look at 
https://issues.apache.org/jira/browse/CLOUDSTACK-6254  I’ve submitted a patch 
to work around the issue in 4.2 and 4.3.  It appears fixed in 4.4+ but this is 
a bug that does cause dataloss in the form of templates being deleted.  Unless 
there is a duplicate I didn’t find, this issue seems to be neglected.


Thanks,

David Bierce

Re: Review Request 24779: [CLOUDSTACK-6254] Template disappears when download cleanup

[Nitin Mehta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24779/#review51439
---



plugins/storage/image/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackImageStoreDriverImpl.java


Thanks for working on this patch.
Any reason why you cant take the same approach as in master right now - 
that of adding a param - entityType in the method - can you please look at that 
code and fix it same way - keeping it consistent and more elegant ?


- Nitin Mehta


On Aug. 17, 2014, 3:02 a.m., David Bierce wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24779/
> ---
> 
> (Updated Aug. 17, 2014, 3:02 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-6254
> https://issues.apache.org/jira/browse/CLOUDSTACK-6254
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> PATCH] This is a quick stab at fixing a dataloss bug.  The ultimate
>  solution is to refactor UploadManager to not use any deprecated code. It
>  appears there is still code left over that uses the UploadVO/Dao which no
>  long contains data about URL transfers.  This method was hardcoded to always
>  pass Upload.Type.VOLUME as part of cleanup which was causing templates to be
>  removed entirely from secondary storage not just the symlink on secondary
>  storage.
> 
> Rather than try to refactor all of it out, this puts
> logic for determining if the cleanup task is for a volume or a template
> by doing a lookup on the URL.  It is a duplication of the same logic
> from the calling method but is a very minimal code change until the
> large problem is fixed.
> 
> 
> Diffs
> -
> 
>   
> plugins/storage/image/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackImageStoreDriverImpl.java
>  4796653 
> 
> Diff: https://reviews.apache.org/r/24779/diff/
> 
> 
> Testing
> ---
> 
> On Cloudstack 4.2 4.3
> Set cleanupurl to 30 seconds.  Downloaded a template, cleanup remvoed it from 
> database, didn't remove the template.
> Downloaded Volume, volume was cleaned up from secondary stoage and database.
> 
> 
> Thanks,
> 
> David Bierce
> 
>

Re: Review Request for CLOUDSTACK-6254

[Nitin Mehta]

Hi David - Thanks for working on this. I have sent you review comments.

Thanks,
-Nitin

On 25/08/14 1:47 PM, "David Bierce"  wrote:

>Hello ‹
>
>I was wondering if someone could take a look at
>https://issues.apache.org/jira/browse/CLOUDSTACK-6254  I¹ve submitted a
>patch to work around the issue in 4.2 and 4.3.  It appears fixed in 4.4+
>but this is a bug that does cause dataloss in the form of templates being
>deleted.  Unless there is a duplicate I didn¹t find, this issue seems to
>be neglected.
>
>
>Thanks,
>
>David Bierce

4.4 Logging

[Michael Phillips]

Seems the logging level in 4.4 has been toned down, how do you crank it up to 
verbose levels? 

Re: Review Request 24779: [CLOUDSTACK-6254] Template disappears when download cleanup

[David Bierce]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24779/#review51442
---


- David Bierce


On Aug. 17, 2014, 3:02 a.m., David Bierce wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24779/
> ---
> 
> (Updated Aug. 17, 2014, 3:02 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-6254
> https://issues.apache.org/jira/browse/CLOUDSTACK-6254
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> PATCH] This is a quick stab at fixing a dataloss bug.  The ultimate
>  solution is to refactor UploadManager to not use any deprecated code. It
>  appears there is still code left over that uses the UploadVO/Dao which no
>  long contains data about URL transfers.  This method was hardcoded to always
>  pass Upload.Type.VOLUME as part of cleanup which was causing templates to be
>  removed entirely from secondary storage not just the symlink on secondary
>  storage.
> 
> Rather than try to refactor all of it out, this puts
> logic for determining if the cleanup task is for a volume or a template
> by doing a lookup on the URL.  It is a duplication of the same logic
> from the calling method but is a very minimal code change until the
> large problem is fixed.
> 
> 
> Diffs
> -
> 
>   
> plugins/storage/image/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackImageStoreDriverImpl.java
>  4796653 
> 
> Diff: https://reviews.apache.org/r/24779/diff/
> 
> 
> Testing
> ---
> 
> On Cloudstack 4.2 4.3
> Set cleanupurl to 30 seconds.  Downloaded a template, cleanup remvoed it from 
> database, didn't remove the template.
> Downloaded Volume, volume was cleaned up from secondary stoage and database.
> 
> 
> Thanks,
> 
> David Bierce
> 
>

Jenkins build is still unstable: simulator-singlerun #202

[jenkins]

See 

Re: Review Request 24779: [CLOUDSTACK-6254] Template disappears when download cleanup

[David Bierce]

> On Aug. 25, 2014, 8:56 p.m., Nitin Mehta wrote:
> > plugins/storage/image/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackImageStoreDriverImpl.java,
> >  line 95
> > 
> >
> > Thanks for working on this patch.
> > Any reason why you cant take the same approach as in master right now - 
> > that of adding a param - entityType in the method - can you please look at 
> > that code and fix it same way - keeping it consistent and more elegant ?

Because I'm new to Java and didn't know about the entityType method :)  I'll 
clean it up and resumbit.  While the fix is more elligant, it still feels like 
the correct fix is to remove all references to the UploadVO since the table is 
deprecated and should never have new data in it.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24779/#review51439
---


On Aug. 17, 2014, 3:02 a.m., David Bierce wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24779/
> ---
> 
> (Updated Aug. 17, 2014, 3:02 a.m.)
> 
> 
> Review request for cloudstack.
> 
> 
> Bugs: CLOUDSTACK-6254
> https://issues.apache.org/jira/browse/CLOUDSTACK-6254
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> PATCH] This is a quick stab at fixing a dataloss bug.  The ultimate
>  solution is to refactor UploadManager to not use any deprecated code. It
>  appears there is still code left over that uses the UploadVO/Dao which no
>  long contains data about URL transfers.  This method was hardcoded to always
>  pass Upload.Type.VOLUME as part of cleanup which was causing templates to be
>  removed entirely from secondary storage not just the symlink on secondary
>  storage.
> 
> Rather than try to refactor all of it out, this puts
> logic for determining if the cleanup task is for a volume or a template
> by doing a lookup on the URL.  It is a duplication of the same logic
> from the calling method but is a very minimal code change until the
> large problem is fixed.
> 
> 
> Diffs
> -
> 
>   
> plugins/storage/image/default/src/org/apache/cloudstack/storage/datastore/driver/CloudStackImageStoreDriverImpl.java
>  4796653 
> 
> Diff: https://reviews.apache.org/r/24779/diff/
> 
> 
> Testing
> ---
> 
> On Cloudstack 4.2 4.3
> Set cleanupurl to 30 seconds.  Downloaded a template, cleanup remvoed it from 
> database, didn't remove the template.
> Downloaded Volume, volume was cleaned up from secondary stoage and database.
> 
> 
> Thanks,
> 
> David Bierce
> 
>

Re: cloud-init issue

[Carlos Reátegui]

On Aug 25, 2014, at 12:32 PM, Erik Weber  wrote:

> On Mon, Aug 25, 2014 at 6:02 PM, Carlos Reátegui 
> wrote:
> 
>> Not sure about this ‘fix’.  If the goal is to match AWS here is the way it
>> works compared to 4.3.
>> 
>> 
> The actual fix was just for the reported url, ie. meta-data
> 
> 
>> 1) /latest/user-data
>> 
>> On AWS one can query /latest/user-data with or without the trailing slash
>> and BOTH return the user-data
>> 
>> On ACS 4.3 /latest/user-data works but with trailing / returns a 404.
>> 
>> 2) /latest/meta-data
>> 
>> On AWS /latest/meta-data returns a 301 pointing to trailing / version.
>> With a trailing slash it return the meta-data objects available.
>> 
>> 
> Is it required to respons with 301, or is 200 with the actual content ok?
I would think 200 with actual content is ok.

> 
> 
>> On ACS 4.3 /latest/meta-data returns 404. With a trailing slash it returns
>> the meta-data objects available.
>> 
>> 3) /latest/meta-data/
>> 
>> On AWS with and without / returns the object value(s)
>> 
>> On ACS 4.3 without a / it returns the value.  With the / it returns a 404.
>> 
> 
> 
> I'll work on an improved patch that fixes all the URLs.
> 
> Erik

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Rohit Yadav]

Hi all,

Thanks to everyone for participating.

We started the voting thread on 21st and it’s 26th now (in my tz). The vote for 
CloudStack CloudMonkey 5.2.0 *passes* with

+1 (PMC / binding)
3 person

+1 (non binding)
2 person

0
none

-1
none

I will now prepare the release announcement to go out after 24 hours to give 
the mirrors time to catch up, push the tag and publish the source distribution 
on pypi.

Cheers,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab


Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Mike Tutkowski]

Thanks again, Rohit!


On Mon, Aug 25, 2014 at 5:38 PM, Rohit Yadav 
wrote:

> Hi all,
>
> Thanks to everyone for participating.
>
> We started the voting thread on 21st and it’s 26th now (in my tz). The
> vote for CloudStack CloudMonkey 5.2.0 *passes* with
>
> +1 (PMC / binding)
> 3 person
>
> +1 (non binding)
> 2 person
>
> 0
> none
>
> -1
> none
>
> I will now prepare the release announcement to go out after 24 hours to
> give the mirrors time to catch up, push the tag and publish the source
> distribution on pypi.
>
> Cheers,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.ya...@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework
> CloudStack Consulting
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>



-- 
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[Rohit Yadav]

Hi Mike,

Thanks for participating and you’re welcome!

Will work on an announcement and some documentation; meanwhile can you help 
check svn access as I’m unable to move/publish cloudmonkey on the release 
location using svn?

Everyone, the new version is available via pypi;
pip install —upgrade cloudmonkey

Cheers.

On 26-Aug-2014, at 1:50 am, Mike Tutkowski  wrote:

> Thanks again, Rohit!
>
>
> On Mon, Aug 25, 2014 at 5:38 PM, Rohit Yadav 
> wrote:
>
>> Hi all,
>>
>> Thanks to everyone for participating.
>>
>> We started the voting thread on 21st and it’s 26th now (in my tz). The
>> vote for CloudStack CloudMonkey 5.2.0 *passes* with
>>
>> +1 (PMC / binding)
>> 3 person
>>
>> +1 (non binding)
>> 2 person
>>
>> 0
>> none
>>
>> -1
>> none
>>
>> I will now prepare the release announcement to go out after 24 hours to
>> give the mirrors time to catch up, push the tag and publish the source
>> distribution on pypi.
>>
>> Cheers,
>> Rohit Yadav
>> Software Architect, ShapeBlue
>> M. +41 779015219 | rohit.ya...@shapeblue.com
>> Blog: bhaisaab.org | Twitter: @_bhaisaab
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<
>> http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge – rapid IaaS deployment framework
>> CloudStack Consulting
>> CloudStack Infrastructure Support<
>> http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<
>> http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended
>> solely for the use of the individual to whom it is addressed. Any views or
>> opinions expressed are solely those of the author and do not necessarily
>> represent those of Shape Blue Ltd or related companies. If you are not the
>> intended recipient of this email, you must neither take any action based
>> upon its contents, nor copy or show it to anyone. Please contact the sender
>> if you believe you have received this email in error. Shape Blue Ltd is a
>> company incorporated in England & Wales. ShapeBlue Services India LLP is a
>> company incorporated in India and is operated under license from Shape Blue
>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
>> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
>> a company registered by The Republic of South Africa and is traded under
>> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>>
>
>
>
> --
> *Mike Tutkowski*
> *Senior CloudStack Developer, SolidFire Inc.*
> e: mike.tutkow...@solidfire.com
> o: 303.746.7302
> Advancing the way the world uses the cloud
> *™*

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England & Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Jenkins build is still unstable: simulator-singlerun #203

[jenkins]

See 

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[ilya musayev]

+1 from, i did very basic tests so far - sorry i'm late with response.

On 8/25/14, 5:03 PM, Rohit Yadav wrote:

Hi Mike,

Thanks for participating and you’re welcome!

Will work on an announcement and some documentation; meanwhile can you help 
check svn access as I’m unable to move/publish cloudmonkey on the release 
location using svn?

Everyone, the new version is available via pypi;
pip install —upgrade cloudmonkey

Cheers.

On 26-Aug-2014, at 1:50 am, Mike Tutkowski  wrote:


Thanks again, Rohit!


On Mon, Aug 25, 2014 at 5:38 PM, Rohit Yadav 
wrote:


Hi all,

Thanks to everyone for participating.

We started the voting thread on 21st and it’s 26th now (in my tz). The
vote for CloudStack CloudMonkey 5.2.0 *passes* with

+1 (PMC / binding)
3 person

+1 (non binding)
2 person

0
none

-1
none

I will now prepare the release announcement to go out after 24 hours to
give the mirrors time to catch up, push the tag and publish the source
distribution on pypi.

Cheers,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab


Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<
http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure Support<
http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<
http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England & Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
a company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
*™*

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build
CSForge – rapid IaaS deployment framework
CloudStack Consulting
CloudStack Infrastructure 
Support
CloudStack Bootcamp Training Courses

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England & Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Jenkins build is still unstable: simulator-singlerun #204

[jenkins]

See 

Re: rules for revolutionaries

[David Nalley]

On Mon, Aug 25, 2014 at 2:23 AM, Leo Simons  wrote:
> On Aug 22, 2014, at 6:45 PM, David Nalley  wrote:
>>> This is, by-the-way, why active committers should want to become PMC 
>>> members, to get the binding votes aligned to who is doing the work. The 
>>> ratio PMC member / committer in this project scares me.
>>
>> I am curious why it scares you. It doesn't seem terribly out of the
>> norm.  CloudStack clearly isn't 1:1, or even 2:1 of some projects, but
>> we aren't at the high end of the spectrum either.
>>
>> For comparison here are some other Committer/PMC differences:
>> CloudStack   96 committers, 29 PMC members 3.31 CtP
>> Subversion78 committers, 43 PMC members 1.81 CtP
>> Hadoop 96 committers, 51 PMC members 1.88 CtP
>> Struts 56 committers, 17 PMC members 5.06 CtP
>> Spamassassin  26 Committers, 7 PMC members, 3.71 CtP
>> OpenOffice 140 committers, 27 PMC members, 5.19 CtP
>> httpd110 committers, 55 PMC members 2.00 CtP
>> xalan54 committers, 7 PMC members, 7.71 CtP
>> WS   223 committers, 41 PMC members 5.43 CtP
>> TomEE 28 committers, 8 PMC members, 3.50 CtP
>
> Ha, that’s a nice list. Thanks David!
>
> It was my perception things were way more skewed, I guess because we seem to 
> have no shortage of active committers but there seem to be a much more 
> limited amounts of votes coming through on, say, releases. Good to be set 
> straight.
>
> At some point jakarta was a few hundred committers and 7 PMC members, and 
> that mostly worked out fine, so “scared” would be way too strong a word 
> anyway.
>

Ha - well - Jakarta went the way of the attic (except for the projects
that became independent) so I am not sure that we want to emulate
Jakarta :)

--David

[GitHub] cloudstack-docs-rn pull request: Fixed assorted typos.

[JoshuaRogers]

GitHub user JoshuaRogers opened a pull request:

https://github.com/apache/cloudstack-docs-rn/pull/17

Fixed assorted typos.

Corrected a few typos I found while going through the documentation.

The text "hypervisor" changed case in order to match previous release 
notes, and to match the case of other items on the page.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/JoshuaRogers/cloudstack-docs-rn Typos

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/cloudstack-docs-rn/pull/17.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #17


commit eab6206a43fcbd1a08a866d9b35879319993dd29
Author: Joshua Rogers 
Date:   2014-08-26T04:17:11Z

Fixed assorted typos.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Review Request 24794: Review request for CLOUDSTACK-6169 - assignVirtualMachine leaves associated tags assigned to old account

[Namita Chaudhari]

Gentle reminder..


On Mon, Aug 18, 2014 at 3:24 PM, Namita Chaudhari <
namita.chaudh...@sungard.com> wrote:

>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24794/
> ---
>
> Review request for cloudstack and Marcus Sorensen.
>
>
> Bugs: CLOUDSTACK-6169 and
> https://issues.apache.org/jira/browse/CLOUDSTACK-6169
> https://issues.apache.org/jira/browse/CLOUDSTACK-6169
>
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-6169
>
>
> Repository: cloudstack-git
>
>
> Description
> ---
>
> CLOUDSTACK-6169 - assignVirtualMachine leaves associated tags assigned to
> old account
>
> When a VM is assigned from one account to another, the tags associated to
> the VM points to the new account and not old one.
>
>
> Diffs
> -
>
>   engine/schema/src/com/cloud/tags/ResourceTagVO.java ea1
>   engine/schema/src/com/cloud/tags/dao/ResourceTagDao.java fffe2a47
>   engine/schema/src/com/cloud/tags/dao/ResourceTagsDaoImpl.java 3611999
>   server/src/com/cloud/vm/UserVmManagerImpl.java e5de11f
>
> Diff: https://reviews.apache.org/r/24794/diff/
>
>
> Testing
> ---
>
> Yes.
>
>
> Thanks,
>
> Namita Chaudhari
>
>


-- 

*Namita Chaudhari* ● Engineer - Product Development ● SunGard Availability
Services, India. ● 2nd Floor, Wing 4, Cluster D, MIDC Kharadi Knowledge
Park, Pune - 411 014 ● Email: namita.chaudh...@sungardas.com
 ● www.sungardas.in



[image: Description: cid:image019.png@01CF48EC.6617C7F0]  [image:
Description: cid:image020.png@01CF48EC.6617C7F0]  [image: Description:
cid:image021.png@01CF48EC.6617C7F0]  [image: Description:
cid:image022.png@01CF48EC.6617C7F0]  [image: Description:
cid:image023.png@01CF48EC.6617C7F0]  [image: Description:
cid:image024.png@01CF48EC.6617C7F0]

Re: Review Request 24936: RPM build failing with RHEL7

[Santhosh Edukulla]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24936/#review51500
---


Getting whitespace errors, while applying the patch.

/home/santhosh/softwares/cs_master/cloudstack/.git/rebase-apply/patch:99: 
trailing whitespace.
   port="8080" 
protocol="org.apache.coyote.http11.Http11NioProtocol" 
warning: 1 line adds whitespace errors.

- Santhosh Edukulla


On Aug. 22, 2014, 11:03 a.m., Damodar Reddy Talakanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24936/
> ---
> 
> (Updated Aug. 22, 2014, 11:03 a.m.)
> 
> 
> Review request for cloudstack, Rayees Namathponnan and Hugo Trippaers.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> RPM Build was failing on RHEL7 due to the packages changed in RHEL7. 
> Refactored the cloud.spec to make it specific to OS builds.
> 
> 
> Diffs
> -
> 
>   client/tomcatconf/server7-nonssl.xml.in PRE-CREATION 
>   client/tomcatconf/server7-ssl.xml.in PRE-CREATION 
>   packaging/centos63/cloud-management.rc 2f04793 
>   packaging/centos63/cloud.spec 7565f95 
>   packaging/centos63/default/macros.spec PRE-CREATION 
>   packaging/centos63/default/tomcat.sh PRE-CREATION 
>   packaging/centos63/package.sh 07f95fc 
>   packaging/centos63/rhel7/cloud-management.service PRE-CREATION 
>   packaging/centos63/rhel7/macros.spec PRE-CREATION 
>   packaging/centos63/rhel7/tomcat.sh PRE-CREATION 
>   python/lib/cloudutils/serviceConfigServer.py 2c19d7e 
> 
> Diff: https://reviews.apache.org/r/24936/diff/
> 
> 
> Testing
> ---
> 
> Tested on RHEL7 and Centos 6.3
> 
> 
> Thanks,
> 
> Damodar Reddy Talakanti
> 
>

Re: Review Request 24936: RPM build failing with RHEL7

[Damodar Reddy Talakanti]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24936/
---

(Updated Aug. 26, 2014, 5:31 a.m.)


Review request for cloudstack, Rayees Namathponnan and Hugo Trippaers.


Changes
---

Removed extra white space that was coming in new file.


Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7106

https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7106


Repository: cloudstack-git


Description
---

RPM Build was failing on RHEL7 due to the packages changed in RHEL7. Refactored 
the cloud.spec to make it specific to OS builds.


Diffs (updated)
-

  client/tomcatconf/server7-nonssl.xml.in PRE-CREATION 
  client/tomcatconf/server7-ssl.xml.in PRE-CREATION 
  packaging/centos63/cloud-management.rc 2f04793 
  packaging/centos63/cloud.spec 7565f95 
  packaging/centos63/default/macros.spec PRE-CREATION 
  packaging/centos63/default/tomcat.sh PRE-CREATION 
  packaging/centos63/package.sh 07f95fc 
  packaging/centos63/rhel7/cloud-management.service PRE-CREATION 
  packaging/centos63/rhel7/macros.spec PRE-CREATION 
  packaging/centos63/rhel7/tomcat.sh PRE-CREATION 
  python/lib/cloudutils/serviceConfigServer.py 2c19d7e 

Diff: https://reviews.apache.org/r/24936/diff/


Testing
---

Tested on RHEL7 and Centos 6.3


Thanks,

Damodar Reddy Talakanti

Re: Review Request 24936: RPM build failing with RHEL7

[Damodar Reddy Talakanti]

> On Aug. 26, 2014, 5:03 a.m., Santhosh Edukulla wrote:
> > Getting whitespace errors, while applying the patch.
> > 
> > /home/santhosh/softwares/cs_master/cloudstack/.git/rebase-apply/patch:99: 
> > trailing whitespace.
> >port="8080" 
> > protocol="org.apache.coyote.http11.Http11NioProtocol" 
> > warning: 1 line adds whitespace errors.

uploaded new patch with fixed white space issue.


- Damodar Reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24936/#review51500
---


On Aug. 26, 2014, 5:31 a.m., Damodar Reddy Talakanti wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24936/
> ---
> 
> (Updated Aug. 26, 2014, 5:31 a.m.)
> 
> 
> Review request for cloudstack, Rayees Namathponnan and Hugo Trippaers.
> 
> 
> Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7106
> 
> 
> Repository: cloudstack-git
> 
> 
> Description
> ---
> 
> RPM Build was failing on RHEL7 due to the packages changed in RHEL7. 
> Refactored the cloud.spec to make it specific to OS builds.
> 
> 
> Diffs
> -
> 
>   client/tomcatconf/server7-nonssl.xml.in PRE-CREATION 
>   client/tomcatconf/server7-ssl.xml.in PRE-CREATION 
>   packaging/centos63/cloud-management.rc 2f04793 
>   packaging/centos63/cloud.spec 7565f95 
>   packaging/centos63/default/macros.spec PRE-CREATION 
>   packaging/centos63/default/tomcat.sh PRE-CREATION 
>   packaging/centos63/package.sh 07f95fc 
>   packaging/centos63/rhel7/cloud-management.service PRE-CREATION 
>   packaging/centos63/rhel7/macros.spec PRE-CREATION 
>   packaging/centos63/rhel7/tomcat.sh PRE-CREATION 
>   python/lib/cloudutils/serviceConfigServer.py 2c19d7e 
> 
> Diff: https://reviews.apache.org/r/24936/diff/
> 
> 
> Testing
> ---
> 
> Tested on RHEL7 and Centos 6.3
> 
> 
> Thanks,
> 
> Damodar Reddy Talakanti
> 
>

Review Request 25057: package.sh was failing if we don to pass "-o" option but pass "-p" option

[Damodar Reddy Talakanti]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25057/
---

Review request for cloudstack and Santhosh Edukulla.


Bugs: https://issues.apache.org/jira/browse/CLOUDSTACK-7106

https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/CLOUDSTACK-7106


Repository: cloudstack-git


Description
---

In earlier patch for rhle7 build I missed out to combine another patch which 
was fixing the package.sh failure due to not passing "-o" patch but passing 
"-p" option


Diffs
-

  packaging/centos63/package.sh d239d2e 

Diff: https://reviews.apache.org/r/25057/diff/


Testing
---

Tested on cetnos6.3 and rhel7 with noredist mode and default mode.


Thanks,

Damodar Reddy Talakanti

Jenkins build is still unstable: simulator-singlerun #205

[jenkins]

See 

Re: S3/Swift Problem around Virtual Size

[Punith S]

sure mike,

since i don't have a S3 account, i'm getting one today.

francois, can you brief me how you seeded your templates into S3.

thanks!


On Mon, Aug 25, 2014 at 11:16 PM, Mike Tutkowski <
mike.tutkow...@solidfire.com> wrote:

> Yes, I expect we'll see the same issue with S3, as well.
>
> Punith - Is this something you might have time to investigate? Perhaps
> Edison can point us in the right direction here.
>
>
> On Mon, Aug 25, 2014 at 5:17 AM, Francois Gaudreault <
> fgaudrea...@cloudops.com> wrote:
>
> > Punith,
> >
> > I highly anticipate the same issue with S3... it shares a lot of code
> with
> > swift.
> >
> > My focus would be swift, but we should fix for both :)
> >
> > FG
> > On Aug 25, 2014 6:33 AM, "Punith S"  wrote:
> >
> > > thanks for opening this thread mike,
> > >
> > > since i only use nfs as my secondary storage provider, i didn't see
> this
> > > issue till date.
> > >
> > > is this issue occurring even using a S3 secondary storage with staging
> > nfs
> > > store ?
> > >
> > > if so like edison pointed we need to fetch the virtual size from the
> nfs
> > > store instead of S3 in the deploymentmanager.
> > >
> > > thanks
> > >
> > >
> > > On Sat, Aug 23, 2014 at 3:45 AM, Mike Tutkowski <
> > > mike.tutkow...@solidfire.com> wrote:
> > >
> > > > Hey Edison,
> > > >
> > > > Do you know how difficult/easy of a fix this is, who might be
> available
> > > to
> > > > put this fix in, and for what release (hopefully 4.4.1) this fix
> could
> > > find
> > > > its way in?
> > > >
> > > > Thanks!
> > > > Mike
> > > >
> > > >
> > > > On Fri, Aug 22, 2014 at 3:37 PM, Francois Gaudreault <
> > > > fgaudrea...@cloudops.com> wrote:
> > > >
> > > > > Min,
> > > > >
> > > > > Ok, but this is not the behavior I see. Even without requesting a
> VM
> > > > > create, the template is pushed to the staging NFS at least once. Is
> > it
> > > > > downloaded there or pushed after download, that I am not sure. I
> was
> > > > > assuming the swift upload bash script was executed after the
> template
> > > is
> > > > on
> > > > > the staging.
> > > > >
> > > > > Anyway... the focus is on the virt size, and you all know the code
> > > better
> > > > > than I do :)
> > > > >
> > > > > FG
> > > > > On Aug 22, 2014 5:28 PM, "Min Chen"  wrote:
> > > > >
> > > > >> No. For S3/Swift, register template will directly upload templates
> > to
> > > S3
> > > > >> without going through staging NFS. It will only be copied to
> staging
> > > NFS
> > > > >> when we first use that template to provision a VM.
> > > > >>
> > > > >> Thanks
> > > > >> -min
> > > > >>
> > > > >> On 8/22/14 2:25 PM, "Francois Gaudreault" <
> fgaudrea...@cloudops.com
> > >
> > > > >> wrote:
> > > > >>
> > > > >> >Edison,
> > > > >> >
> > > > >> >Isnt the templates downloaded to the Staging NFS first?
> > > > >> >
> > > > >> >FG
> > > > >> >On Aug 22, 2014 5:20 PM, "Edison Su" 
> wrote:
> > > > >> >
> > > > >> >> I know the reason why the size of template doesn¹t have correct
> > > > virtual
> > > > >> >> size if it¹s registered in S3/Swift:
> > > > >> >> In case of s3/swift, the template is directly stored into
> > s3/swift
> > > > >> >>through
> > > > >> >> swift/s3 api, there is no place for cloudstack to look into
> > > template,
> > > > >> to
> > > > >> >> find out the virtual size during template registration.
> > > > >> >> While, if secondary storage is NFS, the template is first
> stored
> > on
> > > > >> >> NFS(which has file system), cloudstack can unzip the
> template(if
> > > > it¹s a
> > > > >> >> zipped file), and read virtual size from the file, then report
> > back
> > > > to
> > > > >> >>mgt
> > > > >> >> server.
> > > > >> >> In order to fix it, we can add some code as: all the templates
> > > > >> >>registered
> > > > >> >> on Swift/S3, need to be downloaded to a NFS intermediate
> storage
> > > > before
> > > > >> >>it
> > > > >> >> can be consumed by primary storage. After the download
> finished,
> > > then
> > > > >> we
> > > > >> >> check virtual size of template, and report back to mgt
> > > server/update
> > > > DB
> > > > >> >>etc.
> > > > >> >>
> > > > >> >> From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
> > > > >> >> Sent: Friday, August 22, 2014 1:38 PM
> > > > >> >> To: dev@cloudstack.apache.org
> > > > >> >> Cc: Edison Su
> > > > >> >> Subject: S3/Swift Problem around Virtual Size
> > > > >> >>
> > > > >> >> Hi,
> > > > >> >>
> > > > >> >> This was brought up in a different e-mail thread, but I wanted
> to
> > > > make
> > > > >> >>it
> > > > >> >> more clear that it's related to CloudStack's download code
> around
> > > > >> >>S3/Swift,
> > > > >> >> so I'm opening up a new thread.
> > > > >> >>
> > > > >> >> Francois (from CloudOps) noticed today that when he downloaded
> a
> > > > >> >>template
> > > > >> >> (VHD format) to Swift (but it looks like the same applies for
> S3)
> > > > that
> > > > >> >>the
> > > > >> >> physical and virtual sizes are set to be the same.
> > > > >> >>
> > > > >> >> This a