Re: marvin test cases + travis

2015-05-02 Thread Abhinandan Prateek
n "SMOKE" should be added on a continuous basis. > > -Original Message- > From: Rajani Karuturi [mailto:raj...@apache.org] > Sent: Friday, May 1, 2015 11:09 PM > To: dev@cloudstack.apache.org > Subject: Re: marvin test cases + travis > > -1 on increas

Re: marvin test cases + travis

2015-05-02 Thread Abhinandan Prateek
I think it is a choice between fast response Vs more accurate response. I will rather wait for additional 20 minutes to get a better response. With the commit rate we have that should not overload the system. Yes we should definitely look for quality and coverage of the tests picked then the numb

Re: marvin test cases + travis

2015-05-02 Thread Rohit Yadav
Hi Rajani, > On 01-May-2015, at 7:39 pm, Rajani Karuturi wrote: > > -1 on increasing the time > As a reviewer, I would like an immediate response on a pull request. By the > time I review the code, if CI gives a +1(or -1) I can close the PR then and > there instead of coming back again to merge.

RE: marvin test cases + travis

2015-05-02 Thread Raja Pullela
--Original Message- From: Rajani Karuturi [mailto:raj...@apache.org] Sent: Friday, May 1, 2015 11:09 PM To: dev@cloudstack.apache.org Subject: Re: marvin test cases + travis -1 on increasing the time As a reviewer, I would like an immediate response on a pull request. By the time I review the c

Re: marvin test cases + travis

2015-05-02 Thread Daan Hoogland
If we can lets increase, but if we can't let's not allow for false negatives Op vr 1 mei 2015 om 19:39 schreef Rajani Karuturi : > -1 on increasing the time > As a reviewer, I would like an immediate response on a pull request. By the > time I review the code, if CI gives a +1(or -1) I can close

Re: marvin test cases + travis

2015-05-01 Thread Rajani Karuturi
-1 on increasing the time As a reviewer, I would like an immediate response on a pull request. By the time I review the code, if CI gives a +1(or -1) I can close the PR then and there instead of coming back again to merge. I would suggest running all the jobs under 30 min and increasing the number

Re: marvin test cases + travis

2015-04-30 Thread Rajani Karuturi
At the moment we are creating 10 parallel jobs and in each job we do 1. mvn clean install 2. deploydb, deploy datacenter 3. run tests if we change step 1 to mvn clean install -DskipTests=true and run junit tests only in one job, that would save time. Also, with earlier 5 jobs setup, it used to ru

Re: marvin test cases + travis

2015-04-30 Thread Abhinandan Prateek
+1 for increased timeouts. At the moment we get less than 24 commits per day on master. Only a couple on other branches per day. We can increase the timeout to upto 70 minutes or more. Will try to ensure that in general a single run does not take more than an hour. A timeout window slightly hig

Re: marvin test cases + travis

2015-04-30 Thread Daan Hoogland
Travis is starting to indue the, 'oh, a timeout again' - blindness in me. Let's put a safe time on it. A result should mean something and those time out are like the (decreasing) abundance of Jenkins failures. So +1 for increasing the time On Thu, 30 Apr 2015 18:46 Ian Duffy wrote: > > Do we

Re: marvin test cases + travis

2015-04-30 Thread Ian Duffy
> Do we need to ask for an increase in run time? >From my experience I wouldn't say its required however *it would be very nice*. At the moment what we do is create new parallel jobs and balance the tests across them so each job is under 50mins. Anywho, just my 2c, see what others say/think. On

Re: marvin test cases + travis

2015-04-30 Thread David Nalley
Do we need to ask for an increase in run time? 50 minutes is the OSS default, but the ASF is a paying customer and we can ask for longer timeouts. --David On Thu, Apr 30, 2015 at 10:06 AM, Abhinandan Prateek wrote: > One of the run did timeout. Will balance the runs. > >> On 30-Apr-2015, at 5:59

Re: marvin test cases + travis

2015-04-30 Thread Abhinandan Prateek
One of the run did timeout. Will balance the runs. > On 30-Apr-2015, at 5:59 pm, Abhinandan Prateek > wrote: > > Yes, thanks for checking. I was worried that now some of them may start > timing out. > I have another 10 suites tested, will add them once everything seems fine. > > -abhi > > >> On

Re: marvin test cases + travis

2015-04-30 Thread Abhinandan Prateek
Yes, thanks for checking. I was worried that now some of them may start timing out. I have another 10 suites tested, will add them once everything seems fine. -abhi > On 30-Apr-2015, at 5:32 pm, Rajani Karuturi wrote: > > after correcting the test file path, its working fine > > https://travis

Re: marvin test cases + travis

2015-04-30 Thread Rajani Karuturi
after correcting the test file path, its working fine https://travis-ci.org/karuturi/cloudstack/jobs/60681263 ~Rajani On Thu, Apr 30, 2015 at 4:51 PM, Rajani Karuturi wrote: > I think its due to this commit > > https://github.com/apache/cloudstack/commit/b941480e0df0bfa3377e797126b96cf9b3fbe

Re: marvin test cases + travis

2015-04-30 Thread Rajani Karuturi
I think its due to this commit https://github.com/apache/cloudstack/commit/b941480e0df0bfa3377e797126b96cf9b3fbee77 Here, test file name is changed from testname to [smoke,component]/testname But, while running each test file in the for loop, it prepends the name with smoke and hence, it cant fin

Re: marvin test cases + travis

2015-04-30 Thread Abhinandan Prateek
It is possible, I will check. > On 30-Apr-2015, at 3:43 pm, Rajani Karuturi wrote: > > Hey Abhi, > > In the recent travis runs, its not printing the test case in output ( > https://travis-ci.org/apache/cloudstack/jobs/60634577#L5443) > > It used to print the test case name and status ( > https://

Re: marvin test cases + travis

2015-04-30 Thread Rajani Karuturi
Hey Abhi, In the recent travis runs, its not printing the test case in output ( https://travis-ci.org/apache/cloudstack/jobs/60634577#L5443) It used to print the test case name and status ( https://travis-ci.org/apache/cloudstack/jobs/59958134#L4205) Do you think it can be due to any recent chan

Re: marvin test cases + travis

2015-04-30 Thread Abhinandan Prateek
> On 30-Apr-2015, at 12:05 pm, Rajani Karuturi wrote: > > Hi Abhi, > Can we also add findbugs check in travis? > Let me check the status of findbug, if all the issues are resolved then we can make it part of CI. If not we can make it part of build, and generate and publish the findbug reports

Re: marvin test cases + travis

2015-04-30 Thread Sebastien Goasguen
> On Apr 30, 2015, at 8:35 AM, Rajani Karuturi wrote: > > Hi Abhi, > Can we also add findbugs check in travis? > note that there are other SaaS services like Travis that we could use…like coveralls, tools to create packages etc… literally any CI/CD can be outsourced... > ~Rajani > > On Thu

Re: marvin test cases + travis

2015-04-29 Thread Rajani Karuturi
Hi Abhi, Can we also add findbugs check in travis? ~Rajani On Thu, Apr 30, 2015 at 10:32 AM, Abhinandan Prateek < abhinandan.prat...@shapeblue.com> wrote: > > There are more than 10 additional suites that can still be included to > the travis-CI. > Since the current set runs comfortably I wi