On Wed, Sep 9, 2015 at 8:13 PM, Robert Kowalski <r...@kowalski.gd> wrote: >> So, what is the immediate next step that you propose to fix or improve? >> What is the most squeaky wheel, or the most modest gain that we can make? > > I think one of the biggest problems is that we don't have a working CI and > that PRs are getting merged without running it. The other problem is that > we have a lot of untested changes going into master. > > For a step by step plan I suggest: > > - Setup a simple CI that makes it easy to test before a merge, e.g. like > this [1]. The CI can be really simple, just 1 Erlang version and one OS - > but let's get started somewhere. Do you have other ideas how to solve CI, > PRs and the multi-repo-approach?
Who will setup and host this CI machine? I guess, we don't speak about CI within ASF infra here. Solution of multirepo PRs is simple: use GitHub API to subscript to PR comments and changes. With special comment author sets references to other PR's. Subscription to changes is required to catch PR updates. Special comment is required to prevent false start when CI will try to build PR without related ones. Also it should be able to catch new references over time e.g. because author add missed tests to another repository after CI green build. > - With a working CI system for the multi-repo setup merging should just > happen with a green CI. Flaky tests should get fixed or deleted. Need to define policy what to do if CI is red by a reason not related to PR. > - We would require that every change needs tests if not covered elsewhere > already. This might include that untestable code has to be refactored > first, a problem many legacy applications face (btw a good book on this is > "Working Effectively with Legacy Code". This would automatically make our > code more testable and increase test coverage over time in a > very efficient way. This would also allow us to add features while slowly > refactoring the code. What tools will be used to make this requirement mandatory? And the final note: what is due to date (approximately) for all of this? While CI is a dream, what we can do to tolerate this problem *now*? For instance we can: - Reviewer who said +1 must first run full tests locally and ensure they are ok; - Reviewer must ensure that tests to the related changes are included; and so on. Anything else? -- ,,,^..^,,,
