Thanks Sean busbey for starting the discussion.This was one of pain point even I notified to Allen couple of times.IIUC his point is committers and contributors should monitor qbt.
IMHO,instead of post-commit try to fix in pre-commit only?? May be we can get dependcy module(dependency:list) and execute pre-commit. Even auto revert also good option. Following previous discussions which I came across ,just for reference. Suggestions: 1) let qbt give alarm on broken jira. http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201708.mbox/%3c3b4c977e-5920-4f55-8ebe-cc7e10bcf...@effectivemachines.com%3e 2) Run on parent project,as of now it's run on the current module ( where changes happened),this might not completely ignore? 3) Update dummy class,when contributors/committers knows that it can impact other modules such that pre-commit can run. --Brahma Reddy Battula On Thu, 14 Sep 2017 at 11:31 PM, Sean Busbey <bus...@cloudera.com> wrote: > > Committers MUST check the qbt output after a commit. They MUST make sure > their commit didn’t break something new. > > How do we make this easier / more likely to happen? > > For example, I don't see any notice on HADOOP-14654 that the qbt > post-commit failed. Is this a timing thing? Did Steve L just notice the > break before we could finish the 10 hours it takes to get qbt done? > > How solid would qbt have to be for us to do something drastic like > auto-revert changes after a failure? > > > On Thu, Sep 14, 2017 at 11:05 AM, Allen Wittenauer < > a...@effectivemachines.com > > wrote: > > > > > > On Sep 14, 2017, at 8:03 AM, Sean Busbey <bus...@cloudera.com> wrote: > > > > > > * HADOOP-14654 updated commons-httplient to a new patch release in > > > hadoop-project > > > * Precommit checked the modules that changed (i.e. not many) > > > * nightly had Azure support break due to a change in behavior. > > > > OK, so it worked as coded/designed. > > > > > Is this just the cost of our approach to precommit vs post commit > > testing? > > > > Yes. It’s a classic speed vs. size computing problem. > > > > test-patch: quick but only runs a subset of tests > > qbt: comprehensive but takes a very long time > > > > Committers MUST check the qbt output after a commit. They MUST > > make sure their commit didn’t break something new. > > > > > One approach: do a dependency:list of each module and for those that > > show a > > > change with the patch we run tests there. > > > > As soon as you change something like junit, you’re running over > > everything … > > > > Plus, let’s get real: there is a large contingent of committers > > that barely take the time to read or even comprehend the current Yetus > > output. Adding *more* output is the last thing we want to do. > > > > > This will cause a slew of tests to run when dependencies change. For > the > > > change in HADOOP-14654 probably we'd just have to run at the top level. > > > > … e.g., exactly what qbt does for 10+ hours every night. > > > > It’s important to also recognize that we need to be “good > > citizens” in the ASF. If we can do dependency checking in one 10 hour > > streak vs. several, that reduces the load on the ASF build > infrastructure. > > > > > > > > > -- > busbey > -- --Brahma Reddy Battula
