Testing Apache Cloud Stack versions with an (existing) reference environment and CI/CD test automation - how to?

2020-11-19 Thread peter.muryshkin
Hi there,

@Dan & all dealing with testing and test automation in the Apache CloudStack 
context:

the concern driving the motivation for this message is a long term goal to be 
able to quickly validate upcoming Apache Cloud Stack versions as well as other 
changes in the complex heterogeneous cloud environment in terms of 
interoperability with our environment where versions of single components will 
change over time as well:

- Hypervisors: KVM and vmware vCenter
- Storage backend: NetApp

Image the following scenarios:

- a new version of ACS is installed as a test instance
- how to make sure that for example fixed bugs apply also in our environment 
while previous set of features is stable? => run acceptance tests

What I've learnt so far about what is feasible and available in this context:

- we use our custom set of scripts for functional testing of typical user 
scenarios [1]
- there is a mixed testing setup consisting of generators for test environment 
and also an integration testing plan, addressing and enabling new developers to 
build and test local environments [2]

The following concerns appear though to be yet ambiguous:

- a. it seems like there is no CI/CD automation for what is described in [2] 
aka MonkeyBox, additionally there is a recommendation not to use nested 
virtualization but rather bare metal test environment?
- b. instructions in [2] and also [3] refer to a test plan name "Marvin" which 
is as I've understood simultaneously obsolete; if so, what is the current test 
plan to follow?
- c. as it seems, there is no clear instruction/scenario how to install (in a 
CI/CD environment say using a standard Docker image) and run an up-to-date test 
plan against an existing ACS endpoint, that is without the environment 
generator step?

To have some focus and a problem to solve, it would be great to have a 
conversation/exchange with the community towards the question "c" above towards 
elaborating a to "to-go" test plan setup for easy interoperability sanity 
checks upon changes in the software and hardware environment components. Also 
understanding the current situation for the questions "a" and "b" would be 
great.

[1] 
https://gitlab.cc-asp.fraunhofer.de/fcs-public/apache-cs-extensions/tree/master/csextensions
[2] https://github.com/rhtyd/monkeybox 
[3] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Testing

--
Peter Muryshkin
Fraunhofer-Gesellschaft e.V.





[GitHub] [cloudstack-cloudmonkey] synergiator opened a new issue #71: Template "details" does not compare all details

2020-11-19 Thread GitBox


synergiator opened a new issue #71:
URL: https://github.com/apache/cloudstack-cloudmonkey/issues/71


   Compare these two calls: as it seems, template listing does not contain all 
details information.
   
   ```
   list templates id=8c789ebd-c333-4e22-891a-d49d1c07d95a templatefilter=all 
filter=id,name,details,
   
   
+--+---+-+
   |  ID  |   NAME| 
DETAILS |
   
+--+---+-+
   | 408a4ed7-6ce3-4dbb-a937-4c2a1030ddc0 | sles 15 SP1 BUILD-204447b | 
{"rootDiskController":"pvscsi"} |
   
+--+---+-+
   
   list resourcedetails resourceid=408a4ed7-6ce3-4dbb-a937-4c2a1030ddc0 
resourcetype=Template
   
   ++++--+-+
   | FORDISPLAY |KEY | RESOURCEID | RESOURCETYPE |  VALUE  |
   ++++--+-+
   | true   | dataDiskController |   5041 | Template | pvscsi  |
   | true   | nicAdapter |   5041 | Template | Vmxnet3 |
   | true   | rootDiskController |   5041 | Template | pvscsi  |
   
   
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] DaanHoogland opened a new issue #865: [BUG] type : CVS vs CSV

2020-11-19 Thread GitBox


DaanHoogland opened a new issue #865:
URL: https://github.com/apache/cloudstack-primate/issues/865


   **Describe the bug**
   When creating a role, the input asked for is supposed to be CSV (not CVS)
   
   
![image](https://user-images.githubusercontent.com/2486961/99667911-558c6c00-2a6d-11eb-8347-e4f421f52000.png)
   
   **To Reproduce**
   Steps to reproduce the behavior:
   
   1. Log in as a User or an Admin
   2. Go to '...'
   3. Click on ''
   4. Scroll down to ''
   5. See error
   
   **Expected behavior**
   A clear and concise description of what you expected to happen.
   
   **Screenshots**
   If applicable, add screenshots to help explain your problem.
   
   **Desktop (please complete the following information):**
- OS: [e.g. Windows, Mac, iOS, Android with device/screen details if 
applicable]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
   
   **Additional context**
   Add any other context about the problem here.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] ravening opened a new pull request #866: Fix typo in display message

2020-11-19 Thread GitBox


ravening opened a new pull request #866:
URL: https://github.com/apache/cloudstack-primate/pull/866


   Fixes #865 
   
   Small typo
   
   ![Screenshot 2020-11-19 at 15 12 
20](https://user-images.githubusercontent.com/10645273/99677102-a0ac7c00-2a79-11eb-8e2f-ffce5b6a074f.png)
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] ravening commented on pull request #848: Provide option to force delete the project

2020-11-19 Thread GitBox


ravening commented on pull request #848:
URL: 
https://github.com/apache/cloudstack-primate/pull/848#issuecomment-730405551


   > @ravening Could you add a check whether the parameter exists for the API, 
similar to 
https://github.com/apache/cloudstack-primate/blob/master/src/config/section/compute.js#L112
   > It breaks on versions without the cleanup parameter
   
   @davidjumani I can add that but this feature is supported 4.15 onwards only. 
I dont think there will be other version where this is supported



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] ravening commented on pull request #848: Provide option to force delete the project

2020-11-19 Thread GitBox


ravening commented on pull request #848:
URL: 
https://github.com/apache/cloudstack-primate/pull/848#issuecomment-730417308


   > > @ravening Could you add a check whether the parameter exists for the 
API, similar to 
https://github.com/apache/cloudstack-primate/blob/master/src/config/section/compute.js#L112
   > > It breaks on versions without the cleanup parameter
   > 
   > @davidjumani I can add that but this feature is supported 4.15 onwards 
only. I dont think there will be other version where this is supported
   
   @davidjumani made changes



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-cloudmonkey] DaanHoogland commented on issue #71: Template "details" does not list all details

2020-11-19 Thread GitBox


DaanHoogland commented on issue #71:
URL: 
https://github.com/apache/cloudstack-cloudmonkey/issues/71#issuecomment-730434111


   not sure if this is cloudmonkey or the API, have you checked the UI as well 
@synergiator ?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] ravening commented on pull request #845: Display network type in network overview

2020-11-19 Thread GitBox


ravening commented on pull request #845:
URL: 
https://github.com/apache/cloudstack-primate/pull/845#issuecomment-730455816


   @rhtyd @GabrielBrascher I made changes to display vpcname in a new coloumn 
with a router link to it.
   I also made changes to display the domain name as well



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] ravening edited a comment on pull request #845: Display network type in network overview

2020-11-19 Thread GitBox


ravening edited a comment on pull request #845:
URL: 
https://github.com/apache/cloudstack-primate/pull/845#issuecomment-730455816


   @rhtyd @GabrielBrascher I made changes to display vpcname in a new coloumn 
with a router link to it.
   I also made changes to display the domain name as well
   
   **Also note that this depends on backend change for which I have created the 
PR**



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] davidjumani commented on pull request #848: Provide option to force delete the project

2020-11-19 Thread GitBox


davidjumani commented on pull request #848:
URL: 
https://github.com/apache/cloudstack-primate/pull/848#issuecomment-730850546


   @blueorangutan package



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] blueorangutan commented on pull request #848: Provide option to force delete the project

2020-11-19 Thread GitBox


blueorangutan commented on pull request #848:
URL: 
https://github.com/apache/cloudstack-primate/pull/848#issuecomment-730850697


   @davidjumani a Jenkins job has been kicked to build primate packages. I'll 
keep you posted as I make progress.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-primate] blueorangutan commented on pull request #848: Provide option to force delete the project

2020-11-19 Thread GitBox


blueorangutan commented on pull request #848:
URL: 
https://github.com/apache/cloudstack-primate/pull/848#issuecomment-730853611


   Packaging result: :heavy_check_mark:centos :heavy_check_mark:debian 
:heavy_check_mark:archive.
   QA: http://primate-qa.cloudstack.cloud:8080/client/pr/848 (JID-3681)



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-cloudmonkey] slavkap commented on issue #71: Template "details" does not list all details

2020-11-19 Thread GitBox


slavkap commented on issue #71:
URL: 
https://github.com/apache/cloudstack-cloudmonkey/issues/71#issuecomment-730958287


   Hi all, I have tested this on CS 4.15 with cloudmonkey 5.3.3 and 6.0.0 and 
the problem did not appear
   
   cloudmonkey 6.0.0 - one of the details isn't shown when listing resource 
details because its display option is set to false
   ```
   (localcloud) 🐱 > list resourcedetails 
resourceid=4914ee9a-6bee-47c0-baf6-ebe9e718d2cc resourcetype=template
   count = 3
   resourcedetail:
   
++-+--+--+-+
   | FORDISPLAY | KEY |  RESOURCEID  | 
RESOURCETYPE |VALUE|
   
++-+--+--+-+
   | true   | rootDiskController  | 4914ee9a-6bee-47c0-baf6-ebe9e718d2cc | 
Template | osdefault   |
   | true   | TestDisplayDetails3 | 4914ee9a-6bee-47c0-baf6-ebe9e718d2cc | 
Template | TestDisplayDetails3 |
   | true   | TestDisplayDetails1 | 4914ee9a-6bee-47c0-baf6-ebe9e718d2cc | 
Template | TestDisplayDetails1 |
   
++-+--+--+-+
   (localcloud) 🐱 > list templates id=4914ee9a-6bee-47c0-baf6-ebe9e718d2cc 
filter=details,id,name, templatefilter=all
   template:
   count = 1
   
+-+--+-+
   |   DETAILS   |  ID  
|  NAME   |
   
+-+--+-+
   | map[TestDisplayDetails1:TestDisplayDetails1 | 
4914ee9a-6bee-47c0-baf6-ebe9e718d2cc | Centos8 |
   | TestDisplayDetails2:TestDisplayDetails2 |  
| |
   | TestDisplayDetails3:TestDisplayDetails3 |  
| |
   | rootDiskController:osdefault]   |  
| |
   
+-+--+-+
   ```
   
   and Cloudmonkey 5.3.3 result
   
   ```
   (local) 🐵 > list resourcedetails 
resourceid=4914ee9a-6bee-47c0-baf6-ebe9e718d2cc resourcetype=template
   {
 "count": 3,
 "resourcedetail": [
   {
 "fordisplay": true,
 "key": "rootDiskController",
 "resourceid": "4914ee9a-6bee-47c0-baf6-ebe9e718d2cc",
 "resourcetype": "Template",
 "value": "osdefault"
   },
   {
 "fordisplay": true,
 "key": "TestDisplayDetails3",
 "resourceid": "4914ee9a-6bee-47c0-baf6-ebe9e718d2cc",
 "resourcetype": "Template",
 "value": "TestDisplayDetails3"
   },
   {
 "fordisplay": true,
 "key": "TestDisplayDetails1",
 "resourceid": "4914ee9a-6bee-47c0-baf6-ebe9e718d2cc",
 "resourcetype": "Template",
 "value": "TestDisplayDetails1"
   }
 ]
   }
   (local) 🐵 >list templates id=4914ee9a-6bee-47c0-baf6-ebe9e718d2cc 
templatefilter=all filter=details,id,name,
   {
 "count": 1,
 "template": [
   {
 "details": {
   "TestDisplayDetails1": "TestDisplayDetails1",
   "TestDisplayDetails2": "TestDisplayDetails2",
   "TestDisplayDetails3": "TestDisplayDetails3",
   "rootDiskController": "osdefault"
 },
 "id": "4914ee9a-6bee-47c0-baf6-ebe9e718d2cc",
 "name": "Centos8"
   }
 ]
   }
   ```
   
   I have tested it also on CS branch 4.14 again with cloudmonkey 5.3.3/6.1.0 
   
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [cloudstack-cloudmonkey] DaanHoogland commented on issue #71: Template "details" does not list all details

2020-11-19 Thread GitBox


DaanHoogland commented on issue #71:
URL: 
https://github.com/apache/cloudstack-cloudmonkey/issues/71#issuecomment-730983216


   @synergiator can you check as @slavkap has , whether the two missing details 
have for display set?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org