Hi Becket, The tests would be run each time a PR is created/updated this will look similar to https://github.com/apache/storm/pulls. Ducktape tests take about 7-8 hours to run on my laptop. For travis-ci we can split them in groups and run them in parallel. This was done in the POC run which took 1.5 hrs. It had 10 splits with 5 jobs running in parallel. https://travis-ci.org/raghavgautam/kafka/builds/171502069 For apache projects the limit is 30 jobs in parallel and across all projects, so I expect it to take less time but it also depends on the workload at the time. https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci
Thanks, Raghav. On Thu, Nov 3, 2016 at 9:41 AM, Becket Qin <becket....@gmail.com> wrote: > Thanks Raghav, > > +1 for the idea in general. > > One thing I am wondering is when the tests would be run? Would it be run > when we merge a PR or it would be run every time a PR is created/updated? > I am not sure how long do the tests in other projects take. For Kafka it > may take a few hours to run all the ducktape tests, will that be an issue > if we run the tests for each updates of the PR? > > Thanks, > > Jiangjie (Becket) Qin > > On Thu, Nov 3, 2016 at 8:16 AM, Harsha Chintalapani <ka...@harsha.io> > wrote: > > > Thanks, Raghav . I am +1 for having this in Kafka. It will help identify > > any potential issues, especially with big patches. Given that we've some > > tests failing due to timing issues > > can we disable the failing tests for now so that we don't get any false > > negatives? > > > > Thanks, > > Harsha > > > > On Tue, Nov 1, 2016 at 11:47 AM Raghav Kumar Gautam <rag...@apache.org> > > wrote: > > > > > Hi, > > > > > > I want to start a discussion about running ducktape tests for each pull > > > request. I have been working on KAFKA-4345 > > > <https://issues.apache.org/jira/browse/KAFKA-4345> to enable this > using > > > docker on travis-ci. > > > Pull request: https://github.com/apache/kafka/pull/2064 > > > Working POC: https://travis-ci.org/raghavgautam/kafka/builds/171502069 > > > > > > In the POC I am able to run 124/149 tests out of which 88 pass. The > > failure > > > are mostly timing issues. We can run the same scripts on the laptop > with > > > which I am able to run 138/149 tests successfully. > > > > > > For this to work we need to enable travis-ci for Kafka. I can open a > > infra > > > bug to request travis-ci for this. Travis-ci is already running tests > for > > > many apache projects like Storm, Hive, Flume, Thrift etc. see: > > > https://travis-ci.org/apache/. > > > > > > Does this sound interesting ? Please comment. > > > > > > Thanks, > > > Raghav. > > > > > >
