Hello Technical Board,
The X team would like to request for your consideration an MRE for X
server stable point releases.
Thank you for approving MRE for mesa last year; we've had good results
so far with several mesa point releases. There have been no major
regressions and all stakeholders seem pleased with the results. Based
on this experience, we'd next like to explore doing similarly with the X
server.
Like mesa, xserver also has a suite of unit tests for doing basic
regression checks; these tests have not been quite as comprehensive as
the mesa Piglit tests, but they are growing in coverage and regressions
in them have been well respected by upstream.
X.org upstream has in recent years also adopted a conservative
bug-fix-only approach to stable updates, which the Ubuntu X team feels
is consistent and compatible with Ubuntu SRU requirements. Indeed,
we've packaged and updated to these point releases in the development
version of Ubuntu entirely without incident or disruption over the past
few cycles, and feel it would be a similar non-event updating the stable
release.
The main reason we want the xserver point releases included in stable
versions of Ubuntu is because they bring numerous proven or well-studied
fixes. In cases that we, or our users, can reproduce the failures, we
have SRU'd the patches directly; we have a long track record of
successful SRUs on xserver - in the few cases we've seen problems these
have been due to one-off fixes, not backports from the point release.
Yet in many cases there are legitimate fixes upstream we simply haven't
reproduced, or that users haven't reported in a way that lets us easily
link to an upstream patch; we feel even these fixes may well help our
users even though we can't pinpoint to specific bug reports.
A secondary, but no less important reason for including the xserver
point releases, is that in some cases in the past we've ended up with
stacked up SRU's of backported patches awaiting review. In practice
these all end up proven safe and go out to users, but by putting them
through SRU serially it saps SRU admin resources and also can delay
getting some fixes out to users in a timely fashion. By shipping the
upstream point releases, we can review and test all the upstream fixes
in one go.
Some of the X server tests may be runnable on the buildd's, however
since X touches hardware I'd like to suggest initially we focus on
testing X as we test mesa with a provisional exception, with manual test
runs against real hardware that are reviewed by a domain expert before
we OK a point update. If we gain confidence that X can be adequately
tested in an entirely synthetic environment, we can request a standing
exception from TB.
There are a number of different test suites available for X, however
some of the tests are old and don't have good coverage of relevant
portions of X. Of the available tests, we think the following is a good
subset to run:
0. in-tree tests: Xserver includes some basic tests run via make
check. These have very small coverage but are simple to run.
1. xorg-gtest / XIT: Actively updated, with coverage of current code
changes. Particularly suited for XInput integration testing.
See http://cgit.freedesktop.org/~whot/xorg-integration-tests/
2. piglit: More of a mesa test, but at least the GLX tests within it
will have some relevance. As this is a long running testsuite, we
would just run that subset of it.
--
technical-board mailing list
technical-board@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/technical-board
