Hi all, It seems to me that a number of problems are caused by ATH tests being in a separate repository from the code under test. When regressions occur, they are not discovered until much later. This causes pain for those who investigate ATH test failures. If the ATH tests were moved to the same repository as the code under test (i.e., core or plugins), then these issues would be caught prior to the problematic change being merged and released. This would be a win for those who investigate ATH failures.
Related is the fact that ATH provides a more realistic execution environment than JTH. ATH supports modern JavaScript frameworks, but JTH does not. Being able to easily write ATH tests in core and plugins would be a win for those who make frontend changes. New features could be delivered with associated test automation at the time of feature delivery. I think moving the ATH tests to the same repository as the code under test would be a win on both fronts. ATH and JTH would then both be test _harnesses_ (as their names imply) but not _collections_ of tests: those would live in the same repository as the code under test, and passing them would be a prerequisite for merge and release. Integrating such a system into the toolchain would be a significant project. Providing first-class support for ATH tests in core and plugins in the context of local builds, plugin CI builds, and BOM/PCT builds requires a significant amount of work and likely carries infrastructure prerequisites. Nevertheless, I think such a project would be of large benefit. It would increase our confidence when making UI changes, and it would increase our confidence that BOM bumps do not contain regressions. These benefits would apply to multiple parties across the Jenkins project. Note that this is a proposal for a project and a request for discussion, but I am explicitly not volunteering to implement the solution (at least not by myself). Basil -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAFwNDjoGK5XdfP38zE1WPH-pcVefRrOc7ndvWqApFMzEGEE0zg%40mail.gmail.com.
