+1 On 16/10/13, 23:10, "Siddharth Seth" <ss...@apache.org> wrote:
>There's been a lot of work to make the test runs faster, as well as more >reliable via HIVE-14547, HIVE-13503, and several other jiras. Test >runtimes >are around the 1 hour mark, and going down. There were a few green >pre-commit runs (after years?). At the same time, there's still some flaky >tests. > >We really should try to keep the test runtimes down, as well as the number >of failures - so that the pre-commit runs can provide useful information. > >I'm not sure what the current approach w.r.t precommit runs before a >commit. What I've seen in other projects is that the pre-commit needs to >run, and come back clean (mostly) before a commit goes in. Between what >used to be 5 day wait times, and inconsistent runs - I don't think this is >always followed in Hive. > >It'll be useful to start relying on pre-commit test results again. Given >the flaky tests, I'd suggest the following >1. Pre-commit must be run on a patch before committing (with very few >exceptions) >2. A green test run is ideal >3. In case there are failures - keep track of these as sub-jiras under a >flaky test umbrella jira (Some under HIVE-14547 already) - to be >eventually >fixed. >4. Before committing - cite relevant jiras for a flaky test (create and >cite if it doesn't already exist). > >This should help us build up a list of flaky tests over various runs, >which >will hopefully get fixed at some point. > >Thoughts? > >Thanks, >Sid
