it is not true that Travis-CI is limited to Linux/Ubuntu, at least there's Mac OS X. and we can set up (later) cross builds for MIPS/ARM/Windows/whatever (not sure about "make check")
cross build would be good starting point, if there was such thing already, we could notice that mingw build got broken 2016-06-07 12:30 GMT+05:00 Samuli Seppänen <sam...@openvpn.net>: > > > … > > > >> IMO, the unit testing patches shouldn't have been merged into the > >> release branch > > > > I agree. This patch was in retrospective clearly not ready for a release > > branch. A lot of people spend time to hot fix a broken build. > > > > My root cause analysis boils down to: > > > > Developers cannot detect multi-platform (build/run) issues with an > > appropriate effort (or at all). > > This is probably correct: the codebase is complex enough to cause > breakage on many types of changes, no matter how carefully the code is > reviewed. This is often because of the sheer number of options and their > invisible interplay. A recent example is the recent persist-tun / WFP > filtering issue, which slipped through all testing and review. > > Regression testing using unit tests could possibly help us prevent this > type of breakages. > > > We already have discussed solutions to this: > > > > 1) Enable developers to run the “authoritative” buildslave tests before > > submitting patches > > The buildmaster can trigger manual builds from a different branch (e.g. > "staging/somebody"). Access to the buildmaster can be granted > selectively to trusted core developers. Non-trivial patches from other > people should probably be tested by one of these core developers before > pushing them to "master". Or we can just accept the fact that "master" > might be broken occasionally and fix the problems promptly. > > > 2) Provide developers tooling to quickly (preferred: locally) run > > iterations while developing > > The Vagrant approach is nice because it will eventually allow developers > to run a fairly extensive smoketests on several various operating > systems with minimal effort: > > <https://github.com/OpenVPN/openvpn/pull/45> > > Right now the OS coverage is fairly minimal, but more variants can be > added easily, and we're not limited to Linux/Ubuntu like with Travis. > > > #1 will need some processes & tooling by the current maintainers. > > #2 will be taken care of with the Vagrant based integration tests. > > > > Opinions? > > In addition to #1 and #2 we have Travis-CI smoke-testing in the pipeline: > > <https://github.com/OpenVPN/openvpn/pull/52> > > Travis-CI gives us some confidence on the quality of GitHub PRs, but > it's test matrix is much narrower than that of Buildbot, so it's > definitely not a panacea. > > The only real way to test the more tricky corner-cases is to make it as > easy as possible to use Git "master" -based builds. For Windows this is > already covered by the Windows "buildslave" and the snapshots it > generates. I can setup apt repositories with Debian/Ubuntu builds based > on Git "master", but right now I don't have enough time to commit to that. > > -- > Samuli Seppänen > Community Manager > OpenVPN Technologies, Inc > > irc freenode net: mattock > > > ------------------------------------------------------------------------------ > What NetFlow Analyzer can do for you? Monitors network bandwidth and > traffic > patterns at an interface-level. Reveals which users, apps, and protocols > are > consuming the most bandwidth. Provides multi-vendor support for NetFlow, > J-Flow, sFlow and other flows. Make informed decisions using capacity > planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e > _______________________________________________ > Openvpn-devel mailing list > Openvpn-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/openvpn-devel >
