On 17 May 2016 at 19:37, Pallav Agarwal <pallavagarwa...@gmail.com> wrote: > For normal users we wouldn't. But currently, arch-testers need to make a > judgement call on what to test when a stable-req bug is filed. Tests run in > src_test are provided by upstream, and does not guarantee that a package > that has been merged will actually run on the system. > If the maintainer could add a couple small scripts to check basic > functionality > of the merged package, it would make testing for arch testers much easier > and reliable. > Let me give an example. Let's say > app-misc/screenfetch-2.7.7 is the current stable package for screenfetch in > the portage tree. > However, on running, it produces an error on the top, along with the proper > output. > If screenfetch-3.0.0 happens to fix that error, maintainer can add a simple > script > > if [ "$(screenfetch 2>&1 1>/dev/null)" != "" ] then > eerror "Still producing error" > fi > > To make sure the build is properly updating the screenfetch version, and > that > the bug has in fact been fixed. This is the only way I can see to reliabily > and automatically test packages that have been merged successfully
I don't think this needs to be an EAPI change. And if we can acheive the goal without one, the better. For instance, repoman is progressing in a way that repoman is becomming pluggable, and that will hopefully be more useful. Mostly, because it will allow us to define checks based on "inherit" lines, and based on project/maintainer values in metadata.xml, in theory. You *could* theoretically augment this on an ebuild-by-ebuild basis, but I would encourage doing as much as we can in this sphere outside that workflow, because it will burden ebuilds with unhelpful stuff, and it will increase commit churn on gentoo simply for QA reasons. But the reality is, you have to think not on ebuild-by-ebuild basis, you have to think how this scales in volume, both against large repositories of ebuilds, and how it scales in regards to 1-maintainer-thousands-of-ebuilds and 1-project-thousands-of-ebuilds and 1-eclass-thousands-of-ebuilds scales. For instance, there's very obvious applications for sets of gentoo defined quality tests in dev-perl/, but anything we devise is going to apply to all of them. _And_ to achieve our goal, we will likely need external dependencies satisfied. And I'd hate to be burdening those hundreds of ebuilds with that overhead, its much better implemented out-of-band first, and then only supplemented on an ebuild-by-ebuild basis. And under this scheme, individual projects can define their custom hooks and tricks in the ebuild themselves as metadata ( like we currently to via eclass variables ), and be handled, not by portage, or PMS, not by EAPI, but by the QA tool in conjunction with eclass designers. Then, no need for formal EAPI is required. The QA tool just sources the ebuild in bash, hands the metadata to the plugin, and the plugin can then execute whatever functions it wanted. And the other benefit of doing this outside EAPI: It doesn't have to be part of PMS, and maintainers of other packaging tools are not compelled to implement a feature that is not relevant to end users. And this also means our velocity with regards to changing what we do here is much quicker, as we're no longer bound by the tight QA controls we have to make sure we don't break users systems, because we are *only* targeting devs. -- Kent KENTNL - https://metacpan.org/author/KENTNL
