Re: Splitting tests in PTest2

2015-09-28 Thread Prasanth Jayachandran
Szehon, I committed a different patch now to llap branch, HIVE-11923. This will let llap and tez test to run much faster hopefully before the 2h timeout. Once all tests passes, committed then we can split up the tests to run in parallel. Even after HIVE-11923 if the patch didn’t complete before

Re: Splitting tests in PTest2

2015-09-28 Thread Szehon Ho
OK so there is a bit of a chicken-egg problem, I cannot change it until 11642 is in, otherwise it will break the other builds. Once its in I can update the properties file to reference the new LlapCliDriver. Maybe you can check it in and I will try enabling it tonight. On Mon, Sep 28, 2015 at 5:

Re: Splitting tests in PTest2

2015-09-28 Thread Szehon Ho
OK it may be possible, I will take a look! Thanks Szehon On Mon, Sep 28, 2015 at 4:56 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > @Szehon.. MiniLlapCliDriver uses the same set of files from minitez. Can > that be reused as such without duplicating? > > > > On Sep 28, 201

Re: Splitting tests in PTest2

2015-09-28 Thread Prasanth Jayachandran
@Szehon.. MiniLlapCliDriver uses the same set of files from minitez. Can that be reused as such without duplicating? > On Sep 28, 2015, at 6:49 PM, Prasanth Jayachandran > wrote: > > Thanks Szehon! I will look at it and update you once done. > >> On Sep 28, 2015, at 6:37 PM, Szehon Ho wrot

Re: Splitting tests in PTest2

2015-09-28 Thread Prasanth Jayachandran
Thanks Szehon! I will look at it and update you once done. > On Sep 28, 2015, at 6:37 PM, Szehon Ho wrote: > > Yes thanks for looking into it. Can you edit testconfiguration.properties > to add a section for LlapCliDriver tests, see HIVE-7254 for details. That > way the framework can identify

Re: Splitting tests in PTest2

2015-09-28 Thread Szehon Ho
Yes thanks for looking into it. Can you edit testconfiguration.properties to add a section for LlapCliDriver tests, see HIVE-7254 for details. That way the framework can identify which tests it needs to run and then split it into parallel batches across test nodes. Otherwise it will just run all

Splitting tests in PTest2

2015-09-28 Thread Prasanth Jayachandran
Hello everyone Regarding HIVE-11642, the tests for MiniLlapCliDriver are not split across multiple nodes. I can see from the logs that MiniTezCliDriver are broken down into many tests and ran on different machines. http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Buil