-catching up on email after a break- This is great to see guys, definitely we need to understand our tests better and expand the coverage.
With ASF purchasing more resources on Travis, this is still a very valuable infrastructure to run our CI/BVT until we have a different type of infra. Speeding up the tests is also crucial, so we get feedback on each PR faster. -sebastien On Apr 24, 2015, at 5:14 PM, Abhinandan Prateek <abhinandan.prat...@shapeblue.com> wrote: > Hi Gaurav, > > Your réponse will definitely help us in focusing in the right direction. > > Thanks a lot ! > -abhi > > > >> On 24-Apr-2015, at 7:11 pm, Gaurav Aradhye <gaurav.arad...@clogeny.com> >> wrote: >> >> Abhi, went through the logs. The tests which have FAILED have failed because >> product behavior is not as expected in the test. >> Tests with Exception seem have issue in test code. >> >> Regards, >> Gaurav Aradhye >> >> On Apr 24, 2015, at 4:47 PM, Abhinandan Prateek >> <abhinandan.prat...@shapeblue.com> wrote: >> >>> Attaching the logs. This is with latest 4.5 branch. >>> >>> >>> Find out more about ShapeBlue and our range of CloudStack related services >>> >>> IaaS Cloud Design & >>> Build<http://shapeblue.com/iaas-cloud-design-and-build//> >>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> >>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >>> CloudStack Software >>> Engineering<http://shapeblue.com/cloudstack-software-engineering/> >>> CloudStack Infrastructure >>> Support<http://shapeblue.com/cloudstack-infrastructure-support/> >>> CloudStack Bootcamp Training >>> Courses<http://shapeblue.com/cloudstack-training/> >>> >>> This email and any attachments to it may be confidential and are intended >>> solely for the use of the individual to whom it is addressed. Any views or >>> opinions expressed are solely those of the author and do not necessarily >>> represent those of Shape Blue Ltd or related companies. If you are not the >>> intended recipient of this email, you must neither take any action based >>> upon its contents, nor copy or show it to anyone. Please contact the sender >>> if you believe you have received this email in error. Shape Blue Ltd is a >>> company incorporated in England & Wales. ShapeBlue Services India LLP is a >>> company incorporated in India and is operated under license from Shape Blue >>> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil >>> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is >>> a company registered by The Republic of South Africa and is traded under >>> license from Shape Blue Ltd. ShapeBlue is a registered trademark. >>> <test_acl_isolatednetwork.tar> >>> >>> Regards, >>> -abhi >>> >>> >>>> On 24-Apr-2015, at 4:21 pm, Gaurav Aradhye <gaurav.arad...@clogeny.com> >>>> wrote: >>>> >>>> Hi Abhi, >>>> >>>> FAILED indicates it’s an assertion error. Even if the test case is >>>> expecting a failure, it must have been written in such a way that when an >>>> operation fails, test case passes. >>>> Here it indicates that operation which should have failed has succeeded, >>>> might be a product issue. >>>> >>>> Exception indicates there is an unexpected exception in the code, other >>>> than assertions. >>>> >>>> Can you please share the detailed logs from failed_plus_exceptions.txt >>>> file in the log folder? >>>> >>>> Regards, >>>> Gaurav Aradhye >>>> >>>> On Apr 24, 2015, at 4:04 PM, Abhinandan Prateek >>>> <abhinandan.prat...@shapeblue.com> wrote: >>>> >>>>> >>>>> Have been trying to run some marvin test cases. Need to know how to >>>>> interpret the result. >>>>> >>>>> On running test cases I see several success and failure messages. Lets >>>>> take example of each from output of >>>>> /root/cloudstack/test/integration/component/test_acl_isolatednetwork.py >>>>> on ACS 4.5, taking a sample from output: >>>>> >>>>> === TestName: test_02_createNetwork_admin_foruserinsamedomain | Status : >>>>> SUCCESS === >>>>> >>>>> ***The above says the test case is successful, basically the >>>>> functionality works. >>>>> >>>>> === TestName: test_09_createNetwork_user_foruserinsamedomain | Status : >>>>> FAILED === >>>>> >>>>> ***From the code it appears that the test is expected to fail, this also >>>>> means the test case is successful. >>>>> >>>>> === TestName: test_12_deployvm_admin_foruserinsamedomain | Status : >>>>> EXCEPTION === >>>>> >>>>> ****I assume this means the test case did not execute successfully ? If >>>>> it is on a fresh environment will that mean something is wrong with the >>>>> test itself ? >>>>> >>>>> Can someone share some gyan on above ? >>>>> >>>>> -abhi >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> Find out more about ShapeBlue and our range of CloudStack related services >>>>> >>>>> IaaS Cloud Design & >>>>> Build<http://shapeblue.com/iaas-cloud-design-and-build//> >>>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> >>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> >>>>> CloudStack Software >>>>> Engineering<http://shapeblue.com/cloudstack-software-engineering/> >>>>> CloudStack Infrastructure >>>>> Support<http://shapeblue.com/cloudstack-infrastructure-support/> >>>>> CloudStack Bootcamp Training >>>>> Courses<http://shapeblue.com/cloudstack-training/> >>>>> >>>>> This email and any attachments to it may be confidential and are intended >>>>> solely for the use of the individual to whom it is addressed. Any views >>>>> or opinions expressed are solely those of the author and do not >>>>> necessarily represent those of Shape Blue Ltd or related companies. If >>>>> you are not the intended recipient of this email, you must neither take >>>>> any action based upon its contents, nor copy or show it to anyone. Please >>>>> contact the sender if you believe you have received this email in error. >>>>> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue >>>>> Services India LLP is a company incorporated in India and is operated >>>>> under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is >>>>> a company incorporated in Brasil and is operated under license from Shape >>>>> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of >>>>> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue >>>>> is a registered trademark. >>>> >>> >> > > Find out more about ShapeBlue and our range of CloudStack related services > > IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/> > CloudStack Software > Engineering<http://shapeblue.com/cloudstack-software-engineering/> > CloudStack Infrastructure > Support<http://shapeblue.com/cloudstack-infrastructure-support/> > CloudStack Bootcamp Training > Courses<http://shapeblue.com/cloudstack-training/> > > This email and any attachments to it may be confidential and are intended > solely for the use of the individual to whom it is addressed. Any views or > opinions expressed are solely those of the author and do not necessarily > represent those of Shape Blue Ltd or related companies. If you are not the > intended recipient of this email, you must neither take any action based upon > its contents, nor copy or show it to anyone. Please contact the sender if you > believe you have received this email in error. Shape Blue Ltd is a company > incorporated in England & Wales. ShapeBlue Services India LLP is a company > incorporated in India and is operated under license from Shape Blue Ltd. > Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is > operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company > registered by The Republic of South Africa and is traded under license from > Shape Blue Ltd. ShapeBlue is a registered trademark.
