Hi m2-, On Thu, Feb 01, 2001 at 11:50:35PM +0100, Marcelo E. Magallon wrote: > I was just watching a discussion on #debian-devel and the subject of QA > has come up yet again.
Seems like that happens from time to time :)
> The questions that pop up are basically the same as always, namely, how
> can we have some sort of QA procedure that *actually* helps in speeding
> up the release process? The answer is at least partially evident: we
> need people actually doing QA. It looks like Raphael, whom, AFAIK, was
> playing QA coodinator, got caught up in RL, and other people are in a
> similar situation. Can we have an informal heads count here?
Not a lot of answers as of yet... Of course I don't know if somebody
answered in private mail but on the list only aj and shorty have
answered.
> Once we have the people, the other part of the answer is not that
> clear. IMO, a possible solution would be to have a sort of test suite.
> Since the size of Debian together with the size of the QA team makes
> the very idea of a generalized test suite laughable at the moment.
Stop! We don't currently even need a test suite. There is a lot of
known problems in the dist. Just have a look at the BTS. The QA team
should fight against the bug count. As others will probably confirm
I was very committed to QA work back when I joined Debian. It was fun
to go and fix some bugs - download a package, look at the BTS entry
for the package, fix some bugs, submit the patch to the BTS.
Problem is that it is quite inefficient to do that. Often it will
happen that the maintainer does not even apply the patch. In case
he is active I made the experience that the maintainer preferred to
fix the problem himself so that my work went down the drain.
The other side was that I made some patches and nothing happened for
months even though I submitted the patch to the BTS. And you can't
even do an NMU without getting flamed. That's not generally true
of course but I got the impression not uploading the fixed package
is better for my reputation in Debian circles.
> Phil did develop such a beast, but if you look in /usr/lib/debian-test,
> unless you have a particular set of packages, you'll note it's empty.
> Woody's release is hopefully too close to attempt getting these scripts
> written for a significant part of Debian.[1]
Writing tests for significant parts of Debian would be a nice thing
but I thing we have to concentrate on other things first.
> Last year a group of developers demed the idea of using a task manager
> as a good one, in order to help introduce some order into the apparent
> chaos that QA is right now. One example of such software is
> SourceForce (SF is more than a task manager, but the task manager
> contained there in is a good one, IMO). Roland Mas has packaged
> SourceForce for Debian, and as far as I understand it's installed on
> some machine (SPI's, not Debian's). The idea here would be to create
> tasks that can be taken by different people, which would gain us two
> things: splitting the work into small realistic chunks and giving us a
> scoreboard where we can come and say "yes, this was checked". This can
> also serve as a system where people can simply report "I tested this
> particular task on such a system". Of course there's at least one
> flaw: we need to push other developers arround in order to get the task
> list populated.
You know there is some rudimentary task manager on qa.debian.org. From
my experience I would say that it does not help at all. When I have
some time at my hands I would like to say: "Okay, I have 30 spare minutes".
In that case I would need to get something what I can do - fast.
Even if it is an hour the time to check the BTS, ask on irc, if somebody
else is working on it etc. is just too long.
With the changes to the BTS it has become even harder to check if
somebody is working on a bug since I have to wait minutes for every
page of the bts. Looks like only master is generating the BTS pages
right now and that machine is quite overloaded if you ask me. It was
better when there were static pages of the logs around. I think we
need to fix this problem but I have no idea how. A quick look at
some of the BTS script shows that there is a lot one can optimize.
For example the bugreport.cgi script seems to load the full list
of maintainers on each invocation which looks unneeded to me.
> For now, it seems the most urgent task in this yet non-existant list is
> to test the boot-floppies. For *woody*. No, not debian-installer.
> The woody boot-floppies. The people doing QA should forget about
> debian-installer at the moment. AFAIK, only Adam di Carlo is really
> working on them (I'll be glad if I get corrected). The first thing to
> do to get this going is *building* the boot floppies out of CVS. I'm
> not sure if that's doable atm. Adam and the b-f team have done a great
> job documenting all this stuff, so it's not that hard to get up and
> going. Probably the best way to get this started is having the boot
> floppies built on a regular basis, with a list of changes wrt to the
> last built (trivial with cvs2cl) with, perhaps, an addiotional list of
cvs2cl? What is that?
> stuff to actually test.
If you ask me the most urgent thing for woody would be to replace the
boot-floppies. Better release late then release something as antique as
our potato installation system. The core has not changed much since
hamm if I am not mistaken and there were so many people working on
it, I am not sure if anybody does really know what is happening in
all the parts of it.
> The other question that popped up is "what's holding up packages in
> unstable and what can QA do to help that?"
>
> It's not hard to tell that I've made a lot of this mail on the spot,
> mostly triggered by ideas thrown on #d-d. I'm not trying to get things
> going on *right now*, but to get a discussion started about how to go
> from here. The general consensus is woody won't get out of the door
> unless it starts to get (systematically) tested soon.
Sorry, but I don't know a better test than real world usage of the
distribution. Most developers are running unstable so that problems
are often found very early. I installed testing on a few machines in
university without any problems. If you ask me our distribution has still
the highest quality. Still, I see some problems. For example we should
try to find all packages which are unchanged since, say, slink and try
to recompile them or at least check if they are still working. Those
that do not work anymore should be moved to orphaned.
cu
Torsten
pgpBzzlc3F0Hj.pgp
Description: PGP signature
