Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Gergely Madarasz 
On Sun, 24 Oct 1999, Lalo Martins wrote:

> Implementation:
> 
> When the current Unstable (potato) is frozen, instead of
> creating a new Unstable area, we will create the Pool and
> populate it with a copy of potato; plus, create an empty Working
> area and wait for maintainers to start populating it; plus,

Hm... this is an oversight, I believe. The new working area should be
populated by symlinks to the latest stable/frozen...

> delete the "project/experimental" area. Of course the promotion
> automating software must be working and tested by then.

I'm not quite sure about the need to remove the project/experimental
area... The working area wouldn't be updated quite as often as unstable is
currently, so I'd probably use apt on the pool... and I wouldn't want to
use apt on some software from the current project/experimental... 
(For example I'd gladly follow the current NMU series of dpkg, while I'm
not so sure about the dpkg in project/experimental yet)

Greg

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 01:39:23AM +0200, Gergely Madarasz wrote:
> On Sun, 24 Oct 1999, Lalo Martins wrote:
> 
> > Implementation:
> > 
> > When the current Unstable (potato) is frozen, instead of
> > creating a new Unstable area, we will create the Pool and
> > populate it with a copy of potato; plus, create an empty Working
> > area and wait for maintainers to start populating it; plus,
> 
> Hm... this is an oversight, I believe. The new working area should be
> populated by symlinks to the latest stable/frozen...

I don't think so. Let the maintainers decide what is
``working''. If they think that's the version in ``stable'',
then they can easily make things that way.

> > delete the "project/experimental" area. Of course the promotion
> > automating software must be working and tested by then.
> 
> I'm not quite sure about the need to remove the project/experimental
> area... The working area wouldn't be updated quite as often as unstable is
> currently, so I'd probably use apt on the pool... and I wouldn't want to
> use apt on some software from the current project/experimental... 
> (For example I'd gladly follow the current NMU series of dpkg, while I'm
> not so sure about the dpkg in project/experimental yet)

Read the proposal again. What you said just defeats the whole
point. You're not _supposed_ to run apt on pool unless you're
willing to live on the edge. It would be equivalent to running
apt on experimental.

Perhaps we can implement the long-promised feature of apt to
let the user choose one of many installable versions, so you
_can_ run apt on pool and choose the version of your liking.
But that's an entirely different point.

In simple words: if you want safe stuff, ``working''. If you're
willing to run risks, ``pool''. Period.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Gergely Madarasz 
On Sun, 24 Oct 1999, Lalo Martins wrote:

> On Mon, Oct 25, 1999 at 01:39:23AM +0200, Gergely Madarasz wrote:
> > On Sun, 24 Oct 1999, Lalo Martins wrote:
> > 
> > > Implementation:
> > > 
> > > When the current Unstable (potato) is frozen, instead of
> > > creating a new Unstable area, we will create the Pool and
> > > populate it with a copy of potato; plus, create an empty Working
> > > area and wait for maintainers to start populating it; plus,
> > 
> > Hm... this is an oversight, I believe. The new working area should be
> > populated by symlinks to the latest stable/frozen...
> 
> I don't think so. Let the maintainers decide what is
> ``working''. If they think that's the version in ``stable'',
> then they can easily make things that way.

If something is in stable, then it is ``working'' by current definition.
If nothing else is declared ``working'' then the stable version should be
there.

> > > delete the "project/experimental" area. Of course the promotion
> > > automating software must be working and tested by then.
> > 
> > I'm not quite sure about the need to remove the project/experimental
> > area... The working area wouldn't be updated quite as often as unstable is
> > currently, so I'd probably use apt on the pool... and I wouldn't want to
> > use apt on some software from the current project/experimental... 
> > (For example I'd gladly follow the current NMU series of dpkg, while I'm
> > not so sure about the dpkg in project/experimental yet)
> 
> Read the proposal again. What you said just defeats the whole
> point. 

I thought the point of the proposal was to have better release
management, with better tested packages, etc... read later...

> You're not _supposed_ to run apt on pool unless you're
> willing to live on the edge. It would be equivalent to running
> apt on experimental.
>
> Perhaps we can implement the long-promised feature of apt to
> let the user choose one of many installable versions, so you
> _can_ run apt on pool and choose the version of your liking.
> But that's an entirely different point.
> 
> In simple words: if you want safe stuff, ``working''. If you're
> willing to run risks, ``pool''. Period.

There is a big difference between the risks of current unstable and
current project/experimental. As I said, I'm willing to take the risks of
the first, and I'm not willik to take the risks of the second. I expect
that the ``working'' set will be much more stable than current unstable.
Now lets take this: how does the maintainer decide if a package in the
``pool'' can be promoted ? If there are lots of people who tested it...
I (and probably lots of others) wouldn't test it because the risks here
are higher then in current unstable. So how does it get tested well ? 

Please explain the situation with the current dpkg packages in unstable
and project experimental. Remember, I want the latest from the NMU series,
and I don't want dpkg 1.5.x yet.

Greg

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Gergely Madarasz 
In short:

I would like to see the difference between 
1) "Test this please, it'll probably work for you, I just want to know
there are no serious problems before declaring it ``working''" packages
and
2) "This is what I've done so far, you might be interested to check it,
but there is still a lot to do on them" type packages

I might want to auto-upgrade to the first type, and leave the second type
alone.

Greg

Re: new release process (package pool) being proposed

1999-10-25 Thread Anthony Towns 
On Sun, Oct 24, 1999 at 07:34:26PM -0200, Lalo Martins wrote:
> I'm formally proposing the release process that we have been
> discussing for over a year, known as ``package pool'', for
> discussion and voting. The discussion will take place on
> debian-project. Anyone interested should follow this.

I'd like to object on three grounds.

First, proposals without code are pointless. They're fun and all to
discuss and such, but they don't get results.

Second, this isn't enough information to vote on this. We don't know how
well we can do all this stuff until we have code, and thus any vote would
be shockingly underinformed. Major questions include whether any of this
would actually work, how much extra load this will put on our mirrors,
and whether it would disgust the ftp masters so much that no one would
bother maintaining the archive anymore.

Third, voting on `this is what these people will spend their time on in
future' is completely inappropriate. If it's really a better way, they'll
spend their time on it because they want to. If it's a bit ambiguous,
you can spend your time on it if you want to.

Cheers,
aj, who realises he's being a touch hypocritical here

-- 
Anthony Towns <[EMAIL PROTECTED]> 
I don't speak for anyone save myself. PGP encrypted mail preferred.

 ``The thing is: trying to be too generic is EVIL. It's stupid, it 
results in slower code, and it results in more bugs.''
-- Linus Torvalds


pgpMGYgSBuEwG.pgp
Description: PGP signature

Re: new release process (package pool) being proposed

1999-10-25 Thread Jason Gunthorpe 

[Lame cross post to -announce removed, gah]

> The ftpmasters do their work for the project.  They exist
> on behalf of the project.  The project does not exist as result
> of the ftpmasters, it's vice versa.

However, the FTP masters are the resident experts in field of 'ftp archive
mainti', ignoring their advice, or simply not soliciting it is probably
not a wise idea in the long run.

That said, this proposal has no meaning without an actual implementation
of 'Package Pools', and none exists yet. However I know of at least 2
efforts to make one, so maybe it should be shelved until one gets
finished? [It is not a trivial problem and it is a huge amount of effort
to implement!]

Jason

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Jason Gunthorpe 

On Sun, 24 Oct 1999, Lalo Martins wrote:

> This proposal includes erradication of the "experimental" area,
> because very few maintaiers use it, because it's "out of the
> way" for people to download from it, and because it will be
> redundant with the "pool" layer.

Like Gregory said, experimental serves a purpose that is not covered by
your 4 pools - software in there literally does not work..
 
> dependencies resolvable withing "working". This may be checked
> automatically, the necessary code is already in apt-get. If this

Actually it isn't.. APT has algorithms to check a subset of the possible
conditions, but does not check certain conditions involving conflicts,
which prove to be extremely difficult. [Note this only applies to the
distribution as a whole, ie the apt-cache unmet command]

> 3: To be nicer on mirrors, Working should at all times be simply
> a forest of symlinks into Pool, because mirrors can't handle the
> movement of a file (they just delete it from the old location
> and download it again for the new one).

Actually our mirror network is being upgraded to handle hard links which
obviosly solve the migration problem.
 
> 4: dpkg-scanpackages, or whatever is used to generate Packages
> files for apt, must be fixed so that when multiple versions are
> found, the newest one is used (currently it uses the first one
> found, which will give filesystem-dependent results).

dpkg-scanpackages should just include all available versions, the APT
GUI's people are writing can make use of that. [dselect can get upset if
not used through apt, but it could be patched]

> delete the "project/experimental" area. Of course the promotion
> automating software must be working and tested by then.

In 2 months? Not likely..

Jason

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Anthony Towns 
On Sun, Oct 24, 1999 at 07:32:40PM -0200, Lalo Martins wrote:
> PROPOSAL: NEW (`Incremental') RELEASE PROCESS --- DRAFT

Here's my preferred alternative. I don't want to advocate it too strongly,
since it doesn't have working code yet. Consider it advocacy for `further
discussion' if this proposal gets the requisite number of seconds.

Basically, what I'm thinking would work would be three distributions:

stable, testing, unstable (note the sorting order! I'm so proud.)

Stable and unstable would remain more or less exactly as they are now. There
aren't any changes to dinstall, or how/where you upload to, etc.

Testing is a distribution that's completely automatic --- a program
(with minimal human assistance, I'm not entirely clear on how to manage
this) selects packages from unstable that satisfy a number of criteria
and replaces the existing versions in testing with versions from unstable.

The criteria I think would be best are:

* binaries for all appropriate architectures have been built
* they are installable using just packages from testing
* they don't make any other package in testing uninstallable
* the package doesn't have any outstanding release-critical bugs
* this version of the package has been in unstable for a fortnight
  or more

This has a number of nice properties:

* no extra work if you don't really care about the release cycle.
  If your packages are bug free, they get released, if they're not,
  they don't.

* we have a `fairly up to date, and not too broken' distribution
  to point people at, rather than `way old, but not broken' and
  `up to the second, but flakey'.

* presumably this means `testing' gets a much wider audience than
  unstable. as such, the only way to really get a package out
  to `tha people' is to fix your RC bugs, which theoretically
  encourages people to do so promptly, rather than just around
  freeze.

* if you want to upload something to unstable, but don't want it
  released, this is as simple as filing a bug report like:
Subject: foo isn't suitable for release
Package: foo
Version: 1.1
Severity: important
  and the script will ignore the package.

* people are given a reasonable length of time to find any horrible
  bugs in a package before the last working copy of the package
  is removed from the archive forever.

There are a couple of major drawbacks, at the moment.

One, is that the code to do all this isn't written. (Actually some of it
is, and I'm hoping to start testing it around the time potato's released,
but no promises)

The other is that it implies doubling the bandwidth required to mirror
Debian, once to mirror `unstable/foo' then a fortnight later, to mirror
`testing/foo'. A package pool is one way of solving this, but it makes it
difficult to mirror a single architecture.

H.

http://www.debian.org/~ajt/testing-19991025.tar.gz for what code I've
done, fwiw.

Cheers,
aj

-- 
Anthony Towns <[EMAIL PROTECTED]> <http://azure.humbug.org.au/~aj/>
I don't speak for anyone save myself. PGP encrypted mail preferred.

 ``The thing is: trying to be too generic is EVIL. It's stupid, it 
results in slower code, and it results in more bugs.''
-- Linus Torvalds


pgpzbRTEgi0I0.pgp
Description: PGP signature

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 03:19:47AM +0200, Gergely Madarasz wrote:

[initial populating of "working"]
> If something is in stable, then it is ``working'' by current definition.
> If nothing else is declared ``working'' then the stable version should be
> there.

That's a point, touché. If the maintainer doesn't like the
vesion in stable, upload/promote another one, or remove it.

[experimental]
> I thought the point of the proposal was to have better release
> management, with better tested packages, etc... read later...

Yes. That won't happen if "just about anyone" downloads from
pool :-)

> > You're not _supposed_ to run apt on pool unless you're
> > willing to live on the edge. It would be equivalent to running
> > apt on experimental.
> >
> > Perhaps we can implement the long-promised feature of apt to
> > let the user choose one of many installable versions, so you
> > _can_ run apt on pool and choose the version of your liking.
> > But that's an entirely different point.
> > 
> > In simple words: if you want safe stuff, ``working''. If you're
> > willing to run risks, ``pool''. Period.
> 
> There is a big difference between the risks of current unstable and
> current project/experimental. As I said, I'm willing to take the risks of
> the first, and I'm not willik to take the risks of the second. I expect
> that the ``working'' set will be much more stable than current unstable.

Point against experimental: The pool will support multiple
versions of a same package. So you can choose wich one to use.
So experimental becomes redundant.

> Now lets take this: how does the maintainer decide if a package in the
> ``pool'' can be promoted ? If there are lots of people who tested it...
> I (and probably lots of others) wouldn't test it because the risks here
> are higher then in current unstable. So how does it get tested well ? 

Yes of course _someone_ has to run stuff from pool. If I seemed
to say otherwise, I expressed myself poorly.

I just don't want to over-incentivate people to do that. It's
like, it should be there, but with a big red flashing sign.

> Please explain the situation with the current dpkg packages in unstable
> and project experimental. Remember, I want the latest from the NMU series,
> and I don't want dpkg 1.5.x yet.

This would require modification to apt. Then "apt-get install
dpkg" would get the one in "working", (let's assume it's 1.4.1)
and "apt-get install dpkg/1.4.1.16" would get 1.4.1.16 from
pool. If you wanted one that's currently in experimental, you
could do "apt-get install dpkg/1.5.4". If you wanted the latest
on the NMU series, "apt-get install dpkg/1.4.1." or something.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 03:34:41AM +0200, Gergely Madarasz wrote:
> In short:
> 
> I would like to see the difference between 
> 1) "Test this please, it'll probably work for you, I just want to know
> there are no serious problems before declaring it ``working''" packages
> and
> 2) "This is what I've done so far, you might be interested to check it,
> but there is still a lot to do on them" type packages

This might be cool. Yet, keeping experimental as a yet fourth
layer looks like overkill. But if enougt people would like to
keep experimental, by all means, propose an ammendment.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Data does NOT belong in Debian (was: Stop Archive bloat)

1999-10-25 Thread Fabien Ninoles 
On Tue, Oct 19, 1999 at 09:29:53PM +, Alexander Koch wrote:
> [f'up]
> 
> On Tue, 19 October 1999 21:43:57 +0200, Goswin Brederlow wrote:
> > Why not allow Source only packages ?
> 
> Something like that is the only workable thing, methinks.
> Having a source where a source is 99+ % the same data is waste.
> 
> Before that is agreed on (and there is a need, I read it
> here) I would not touch anything. I am strongly against
> the data section and especially the bigger packages.

Why are you against a data section (which is just a separate archives
like contrib or non-us)? I would like to know what's wrong with it
since who doesn't have much opposition in debian-policy.

Please, read the proposal carefully before answering.

> 
> Alexander
> 
> -- 
> "Video killed the radio star ... then the Spice Girls killed the rest."
> -- Alan Braverman
> Alexander Koch - <>< - WWJD - aka Efraim - PGP 0xE7694969 - ARGH-RIPE
> 

-- 

Fabien NinolesChevalier servant de la Dame Catherine des Rosiers
aka Corbeau aka le Veneur Gris   Debian GNU/Linux maintainer
E-mail:[EMAIL PROTECTED]
WebPage:http://www.tzone.org/~fabien
RSA PGP KEY [E3723845]: 1C C1 4F A6 EE E5 4D 99  4F 80 2D 2D 1F 85 C1 70

Re: Data does NOT belong in Debian (was: Stop Archive bloat)

1999-10-25 Thread Fabien Ninoles 
On Wed, Oct 20, 1999 at 10:57:51PM +0200, Goswin Brederlow wrote:
> Torsten Landschoff <[EMAIL PROTECTED]> writes:
> 
> > On Tue, Oct 19, 1999 at 09:43:57PM +0200, Goswin Brederlow wrote:
> >  
> > > Why not allow Source only packages ?
> > 
> > That will win nothing. You can't use apt-get on them, have to rebuilt 
> > and have them twice locally. 
> 
> You could set the arch flag in the packages file to "source-only". Apt 
> could then run "apt-get source" and the source be dowloaded. Then it
> can be compiled or directly installed.

No current way to directly installed it. Also no way to add a depends
against those packages since the package doesn't exist.

> 
> Its only data, there is actually no need to compile anything and dpkg
> could be told to install from a source and debian dir directly. Even
> if it has to be compiled, so what. People without a few MB space won´t 
> download 47 MB packages.

dpkg currently can't install from a debian dir directly. Also think about
pristine source. Having a few mb of space is already a trouble since
you must have at least twice the size of a binary package to install it.
With source only package and the current situations, you need at least
three time (if done correctly and for straight copy package) the size as
free space. That's just can be too much especially if you upgrade from
an old stable to a new stable! We already need an apt able to take into
account how much space is left so it can download a bunch, install it,
clean up packages, download another bunch, etc.

> 
> After installation the source and deb file can be deleted and
> everything is fine again.
> 
> May the Source be with you.
>   Goswin
> 
> PS: Its just a dream to only have to mirror source to install and
> maintain a debian system at the moment, but we´r getting nearer with
> the autobuild demons.
> 

That's a good thing too, but do I really need to build all those xserver
just to install my fbdev server?

-- 

Fabien NinolesChevalier servant de la Dame Catherine des Rosiers
aka Corbeau aka le Veneur Gris   Debian GNU/Linux maintainer
E-mail:[EMAIL PROTECTED]
WebPage:http://www.tzone.org/~fabien
RSA PGP KEY [E3723845]: 1C C1 4F A6 EE E5 4D 99  4F 80 2D 2D 1F 85 C1 70

Re: Data does NOT belong in Debian (was: Stop Archive bloat)

1999-10-25 Thread Fabien Ninoles 
On Tue, Oct 19, 1999 at 09:43:57PM +0200, Goswin Brederlow wrote:
> Philippe Troin <[EMAIL PROTECTED]> writes:
> 
> >   1) The way the Debian archive works requires the data to be stored
> >  twice (source package and .deb).
> 
> Why not allow Source only packages ?

Because installing such package requires you to have three times the
necessary space and there no way to make it installable from dselect
or apt-get install. For sure, if we can make those options works
correctly, I'm not opppsed to it. See also my suggestion about a deb package
who used .orig.tar.gz as an "external" data.tar.gz.

> 
> May the Source be with you.
>   Goswin
> 

-- 

Fabien NinolesChevalier servant de la Dame Catherine des Rosiers
aka Corbeau aka le Veneur Gris   Debian GNU/Linux maintainer
E-mail:[EMAIL PROTECTED]
WebPage:http://www.tzone.org/~fabien
RSA PGP KEY [E3723845]: 1C C1 4F A6 EE E5 4D 99  4F 80 2D 2D 1F 85 C1 70

Re: new release process (package pool) being proposed

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 12:29:23PM +1000, Anthony Towns wrote:
> On Sun, Oct 24, 1999 at 07:34:26PM -0200, Lalo Martins wrote:
> > I'm formally proposing the release process that we have been
> > discussing for over a year, known as ``package pool'', for
> > discussion and voting. The discussion will take place on
> > debian-project. Anyone interested should follow this.
> 
> I'd like to object on three grounds.
> 
> First, proposals without code are pointless. They're fun and all to
> discuss and such, but they don't get results.

Please don't stand on the way. Nobody will code it just for the
fun. That's not "fun"; we're having fun with package pools ideas
since 98. I think it's time this gets implemented. If this is
approved, I can quit my job and implement the whole stuff myself
if necessary, because I'd rather quit my job than quit Debian.
And quitting Debian is what lies on the future if we don't do
something about this sorry excuse for a release process we have.

Also, could you people please stop for a moment and really evaluate
the ammount of code needed? Get real: this is _trivial_. We
already have mail-responding code in the BTS. We already have
archive handling code in dinstall. The ugliest part is modifying
apt, and that isn't really essential.

Anyone willing to buy me a beer if I have this code ready by
Friday?

And assuming I accept the bet, how am I supposed to test it?

> Second, this isn't enough information to vote on this. We don't know how
> well we can do all this stuff until we have code, and thus any vote would
> be shockingly underinformed. Major questions include whether any of this
> would actually work, how much extra load this will put on our mirrors,
> and whether it would disgust the ftp masters so much that no one would
> bother maintaining the archive anymore.

These are not rational arguments. If you want to attack my
objectivity, please be objective.

Please make a list of the code we would need.

Please make a list of the added burdens on the ftpmasters. As
far as I calculated, they work would stay more or less the same.
Perhaps less, if we consider the difference in the freeze
processes. The real problem is the change.

The load on mirrors is the load caused by multiple versions of
each package in pool. The "working" area should be a forest of
symlinks into pool.

> Third, voting on `this is what these people will spend their time on in
> future' is completely inappropriate. If it's really a better way, they'll
> spend their time on it because they want to. If it's a bit ambiguous,
> you can spend your time on it if you want to.

No, nobody could just go and implement this radical change in
the way we work without consulting the other developers. This
would be an astounding demonstration of disrespect.


[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: new release process (package pool) being proposed

1999-10-25 Thread Lalo Martins 
On Sun, Oct 24, 1999 at 08:32:07PM -0600, Jason Gunthorpe wrote:
> 
> That said, this proposal has no meaning without an actual implementation
> of 'Package Pools', and none exists yet. However I know of at least 2
> efforts to make one, so maybe it should be shelved until one gets
> finished? [It is not a trivial problem and it is a huge amount of effort
> to implement!]

No. Please don't dillute the matter. This isn't a technical-only
proposal, is very political - it will affect the way all
developers handle packages. They deserve to have their say.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Sun, Oct 24, 1999 at 08:48:21PM -0600, Jason Gunthorpe wrote:
> 
> Like Gregory said, experimental serves a purpose that is not covered by
> your 4 pools - software in there literally does not work..

Yes, but pool can have multiple versions of a same package.

> > dependencies resolvable withing "working". This may be checked
> > automatically, the necessary code is already in apt-get. If this
> 
> Actually it isn't.. APT has algorithms to check a subset of the possible
> conditions, but does not check certain conditions involving conflicts,
> which prove to be extremely difficult. [Note this only applies to the
> distribution as a whole, ie the apt-cache unmet command]

Hmm. I actually meant to use apt's install-time dependency
check. It's smart enought to know when something is
"installable".

> > 3: To be nicer on mirrors, Working should at all times be simply
> > a forest of symlinks into Pool, because mirrors can't handle the
> > movement of a file (they just delete it from the old location
> > and download it again for the new one).
> 
> Actually our mirror network is being upgraded to handle hard links which
> obviosly solve the migration problem.

You mean, use something similar to inodes to know the real
"identity" of a file? That would be great.

> > 4: dpkg-scanpackages, or whatever is used to generate Packages
> > files for apt, must be fixed so that when multiple versions are
> > found, the newest one is used (currently it uses the first one
> > found, which will give filesystem-dependent results).
> 
> dpkg-scanpackages should just include all available versions, the APT
> GUI's people are writing can make use of that. [dselect can get upset if
> not used through apt, but it could be patched]

Yes, but this means we would require the apt people to fix their
front-ends by the time of the change. I didn't want to add this
extra requirement. If it's ready by then, so much better.

> > delete the "project/experimental" area. Of course the promotion
> > automating software must be working and tested by then.
> 
> In 2 months? Not likely..

Why not? An email responding bot (borrow from BTS), an
archive-handling script (borrow from dinstall) and a
dependency-checker (borrow from apt)? Looks like work for a
week. 2 months give time enought for testing and debugging.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: new release process (package pool) being proposed

1999-10-25 Thread Anthony Towns 
On Mon, Oct 25, 1999 at 03:18:47AM -0200, Lalo Martins wrote:
> Also, could you people please stop for a moment and really evaluate
> the ammount of code needed? Get real: this is _trivial_.

We'd need code to:

* make life easy for the mirrors (either a working package pool,
  or the hard links thing). I've tried writing code to create a
  package pool, it's not that tricky, but it's a pain. I haven't
  looked at dinstall at all ever, so I've no idea how hard it'd
  be to modify dinstall to cope with it. Dealing with hard links
  sounds much more reasonable, but it gets rid of all the 

* do `installability checking'. I've done some code (see my
  previous mail) that checks depends/conflicts within main.
  It's about 2000 LOC, bits of which are incredibly horrible
  and complicated.

* work out how to choose subsets of packages that don't make
  hundreds of others uninstallable --- so that you don't
  install the new lesstif unless you also update all the other
  programs that the new lesstif will make uninstallable. This
  alone is non-trivial.

* interface with the BTS. Note that working out which bugs belong
  to a package has been something of a hack until today, even.
  Really, having the BTS be able to tell you which bugs apply to
  which versions of a package would be desirable to implement this
  properly.

> Please make a list of the code we would need.
> Please make a list of the added burdens on the ftpmasters.

Erm. As proposer, this is really /your/ job.

> > Third, voting on `this is what these people will spend their time on in
> > future' is completely inappropriate. If it's really a better way, they'll
> > spend their time on it because they want to. If it's a bit ambiguous,
> > you can spend your time on it if you want to.
> No, nobody could just go and implement this radical change in
> the way we work without consulting the other developers.

Of course they could. In exactly the same way Debconf appeared. People
discussed it, someone went off and implemented it, and now that we've
got an implementation, we're ready to start using it, and consider adding
it to policy.

Here, we ought to be discussing what we need (which we've done for over
a year now), implementing it, and /then/ working out whether we want to
actually use it or not.

Cheers,
aj

-- 
Anthony Towns <[EMAIL PROTECTED]> 
I don't speak for anyone save myself. PGP encrypted mail preferred.

 ``The thing is: trying to be too generic is EVIL. It's stupid, it 
results in slower code, and it results in more bugs.''
-- Linus Torvalds


pgpOqQqPi1lsW.pgp
Description: PGP signature

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Jason Gunthorpe 

On Mon, 25 Oct 1999, Lalo Martins wrote:

> Yes, but pool can have multiple versions of a same package.

But how on earth is anyone supposed to know which version is the one they
want?
 
> Hmm. I actually meant to use apt's install-time dependency
> check. It's smart enought to know when something is
> "installable".

To apply that individually to each and every package [there are 4000 of
them!] is too slow. It would be better to solve the conflicts problem
which is at least a weeks work, maybe more.
 
> You mean, use something similar to inodes to know the real
> "identity" of a file? That would be great.

rsync uses inodes to map hardlinks, so yes.
 
> Yes, but this means we would require the apt people to fix their
> front-ends by the time of the change. I didn't want to add this
> extra requirement. If it's ready by then, so much better.

If you emit all available versions then APT select automatically the
highest, but internally all are available for any future use. It is not a
good idea to put selecting the highest of many in dpkg-scanpackages.
 
> > In 2 months? Not likely..
 
> Why not? An email responding bot (borrow from BTS), an
> archive-handling script (borrow from dinstall) and a
> dependency-checker (borrow from apt)? Looks like work for a
> week. 2 months give time enought for testing and debugging.

The depenency anaylsis and consideration code alone from APT is nearly
5kloc, code able to effectively replace dinstall (~1kloc of perl) using
this new scheme may weigh in at least 2kloc of C, and the new code
required to ensure consistency I would estimate at around another 3kloc.

Given that the long term average code generation of most people is about
100-200 lines/day that would take you about one month to write, and at
least 3 months to sufficiently test. That is assuming you fully understood
all of the subjects at hand and did not have to learn anything substantial
new. 

Then you would have to tack on another month at least to convert and train
the FTP masters, and begin preparing to convert the archive. 

Jason

Re: new release process (package pool) being proposed

1999-10-25 Thread Robert Jones 
Quoth Anthony Towns on 25 Oct, 1999:

[ Disclaimer:  I am not a Debian developer yet, due to the new-maintainer
  freeze.  I have been following the project for a while, however.  Please
  forgive if this is out of order. ]


> First, proposals without code are pointless. They're fun and all to
> discuss and such, but they don't get results.

There is some code that has been offered for a similar proposal now.  And your
yourself offered it. :)  Besides, this is such a far-reaching policy decision
(effective changing the entire way Debian is versioned and released), I would
be disturbed to see code put into place without some planning done beforehand.

> Second, this isn't enough information to vote on this. We don't know how
> well we can do all this stuff until we have code, and thus any vote would
> be shockingly underinformed. Major questions include whether any of this

So then propose to table the discussion pending further, specific research.
Once we know what we need to know, discussion can be taken up again.  But
don't dismiss things offhand, or table the issue indefinitely, saying "we need
to know more".  Nothing will ever get done.

> Third, voting on `this is what these people will spend their time on in
> future' is completely inappropriate. If it's really a better way, they'll
> spend their time on it because they want to. If it's a bit ambiguous,
> you can spend your time on it if you want to.

As I said, I am relatively new to the Debian project, and I'm not yet a
maintainer, but it strikes me that if we follow this line of thinking, there
is no need for any directing body.  If everyone did what they wanted to, or
didn't do it at all, there would be absolutely no reason for this discussion,
or developer voting -- the whole point of which is to allow the point of view
which shares most, but not all, of the popular opinion to become the accepted
course of action.

So that I may do something other than criticize, here are my constructive
suggestions:

Step 1) Develop a list of specific information that you need to have available
before calling a vote on this, or even discussing the merits further.
Step 2) Call for a vote on tabling pending further investigation [is there
room for this in the Debian Rules of Order? :) ]
Step 3) Assuming vote passes, do the legwork of feasability researh.

That being said, I do like the proposal.  I think perhaps the proposed
modification for an automated promotion might be a nice addition, as would the
maintenance of an experimental branch for stuff that is really and truly
completely nonfunctional.  But I think, with some ammendations and
considerations for real-world use, its a very good idea whose time has long
past come.

Sorry if I stepped on anyone's toes here.

-- 
Robert C. Jones | rjones-at-devzero-dot-org | http://www.devzero.org
  Linux junkie  |   professional sysadmin   | raving lunatic
Please use PGP: http://www.devzero.org/public.asc


pgpQCsYUVmRWj.pgp
Description: PGP signature

Re: new release process (package pool) being proposed

1999-10-25 Thread Wichert Akkerman 
Previously Martin Schulze wrote:
> Apparently I wasn't clear enough.

I had already posted by then...

> The ftpmasters do their work for the project.  They exist
> on behalf of the project.  The project does not exist as result
> of the ftpmasters, it's vice versa.

True. However that doesn't always seem to work that way. A good example
is that we have a consensus to create a science-section in the archive,
but that still hasn't happened yet.

> Thus if the project (or the project leader) wants things to be
> done with the archive, the ftpmasters have to get it implemented
> (with or without help from others) or they will have to be replaced.

True, however I feel that any proposal like this that is made without
consulting them or the release manager is less likely to succeed.

Wichert.

-- 
   
 / Generally uninteresting signature - ignore at your convenience  \
| [EMAIL PROTECTED]http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0  2805 3CB8 9250 2FA3 BC2D |


pgpxwPP749ctO.pgp
Description: PGP signature

Re: new release process (package pool) being proposed

1999-10-25 Thread Martin Schulze 
Wichert Akkerman wrote:
> > The ftpmasters do their work for the project.  They exist
> > on behalf of the project.  The project does not exist as result
> > of the ftpmasters, it's vice versa.
> 
> True. However that doesn't always seem to work that way. A good example
> is that we have a consensus to create a science-section in the archive,
> but that still hasn't happened yet.

Then why?  Does a proper bug report exist?  Is it just slowly processing
bug report?  Or is it something else?

> > Thus if the project (or the project leader) wants things to be
> > done with the archive, the ftpmasters have to get it implemented
> > (with or without help from others) or they will have to be replaced.
> 
> True, however I feel that any proposal like this that is made without
> consulting them or the release manager is less likely to succeed.

True, consulting them is recommended, though it looked like you want
them to decide - which would be the wrong way.

Regards,

Joey

-- 
Debian GNU/Linux   The choice of a GNU generation.

Re: new release process (package pool) being proposed

1999-10-25 Thread Wichert Akkerman 
Previously Martin Schulze wrote:
> Then why?  Does a proper bug report exist?  Is it just slowly processing
> bug report?  Or is it something else?

There is indeed a bugreport, and it's old. Months at least. Last I heard
the only reason was that it was a lot of work...

Wichert.

-- 
   
 / Generally uninteresting signature - ignore at your convenience  \
| [EMAIL PROTECTED]http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0  2805 3CB8 9250 2FA3 BC2D |


pgp81tFZGFlcn.pgp
Description: PGP signature

Re: new release process (package pool) being proposed

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 01:30:44PM +0200, Wichert Akkerman wrote:
> Previously Martin Schulze wrote:
> 
> > Thus if the project (or the project leader) wants things to be
> > done with the archive, the ftpmasters have to get it implemented
> > (with or without help from others) or they will have to be replaced.
> 
> True, however I feel that any proposal like this that is made without
> consulting them or the release manager is less likely to succeed.

Now on these kinds of practical grounds I can understand what
you mean. Thanks a lot.

Anyway, my feeling is that it's a hard decision, involving all
maintainers, and something that has been brought up again and
again for over an year, so it's time we discuss it seriously
and by all means _vote_. If we find out that this is what we
want, then we do whatever is necessary. IMVHO.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 12:12:39AM -0600, Jason Gunthorpe wrote:
> 
> On Mon, 25 Oct 1999, Lalo Martins wrote:
> 
> > Yes, but pool can have multiple versions of a same package.
> 
> But how on earth is anyone supposed to know which version is the one they
> want?

Please elaborate. What are you talking about? Promotion? Apt?
Or just manual downloading?

> > Hmm. I actually meant to use apt's install-time dependency
> > check. It's smart enought to know when something is
> > "installable".
> 
> To apply that individually to each and every package [there are 4000 of
> them!] is too slow. It would be better to solve the conflicts problem
> which is at least a weeks work, maybe more.

Yes, slow. I thought of that. I'll trust you here, IIRC you
know more about apt internals than I do.


> > Yes, but this means we would require the apt people to fix their
> > front-ends by the time of the change. I didn't want to add this
> > extra requirement. If it's ready by then, so much better.
> 
> If you emit all available versions then APT select automatically the
> highest, but internally all are available for any future use. It is not a
> good idea to put selecting the highest of many in dpkg-scanpackages.

I said it would require modifying the _front-ends_. Not that
this is hard to do, just that I don't want to pressure anyone
into doing that. But then again, if you feel like this could be
done I'll trust you.


> > > In 2 months? Not likely..
>  
> > Why not? An email responding bot (borrow from BTS), an
> > archive-handling script (borrow from dinstall) and a
> > dependency-checker (borrow from apt)? Looks like work for a
> > week. 2 months give time enought for testing and debugging.
> 
> The depenency anaylsis and consideration code alone from APT is nearly
> 5kloc, code able to effectively replace dinstall (~1kloc of perl) using
> this new scheme may weigh in at least 2kloc of C, and the new code
> required to ensure consistency I would estimate at around another 3kloc.

Hmm no. Let's think again.

For getting packages into pool, we would still use dinstall.

For promoting packages, we don't need dinstall's features; just
check if the package is fine, and if it is, update the
(sym?)link. So forget ``2kloc''; more like 20loc of a shell
script or 50loc C.

What new code?

> Given that the long term average code generation of most people is about
> 100-200 lines/day that would take you about one month to write, and at
> least 3 months to sufficiently test. That is assuming you fully understood
> all of the subjects at hand and did not have to learn anything substantial
> new. 
> 
> Then you would have to tack on another month at least to convert and train
> the FTP masters, and begin preparing to convert the archive. 

We have to seriously evaluate this. The only way is trying. If
it's really as hard as you think, then we (I) need to find some
other way of migrating to the incremental release system.
Incrementally ;-)

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: new release process (package pool) being proposed

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 04:06:27PM +1000, Anthony Towns wrote:
> On Mon, Oct 25, 1999 at 03:18:47AM -0200, Lalo Martins wrote:
> > Also, could you people please stop for a moment and really evaluate
> > the ammount of code needed? Get real: this is _trivial_.
> 
> We'd need code to:
> 
>   * make life easy for the mirrors (either a working package pool,
> or the hard links thing). I've tried writing code to create a
> package pool, it's not that tricky, but it's a pain. I haven't
> looked at dinstall at all ever, so I've no idea how hard it'd
> be to modify dinstall to cope with it. Dealing with hard links
> sounds much more reasonable, but it gets rid of all the 

Jason informs me this is already done by rsync, and we're
moving our mirrors to rsync, so, this is already solved
regardless of whether we want to use the Incremental process or
not.

>   * do `installability checking'. I've done some code (see my
> previous mail) that checks depends/conflicts within main.
> It's about 2000 LOC, bits of which are incredibly horrible
> and complicated.

Have you tried using apt code?

>   * work out how to choose subsets of packages that don't make
> hundreds of others uninstallable --- so that you don't
> install the new lesstif unless you also update all the other
> programs that the new lesstif will make uninstallable. This
> alone is non-trivial.

This is the hardest part.

As a temporary measure, I imagined the promotion checker like
that initially:

1: if there is a ``batch'' command pending, test if promoting
the whole batch would work, and if it would, do it

2: if all candidate packages can be promoted as a whole, do it

3: otherwise check each candidate individually

So yes, more complicated things like the recent perl problem
would require manual intervention; this intervention would have
the form of someone issuing a ``batch'' command, like:

batch perl-5005 perl libfoo-perl etc etc


>   * interface with the BTS. Note that working out which bugs belong
> to a package has been something of a hack until today, even.
> Really, having the BTS be able to tell you which bugs apply to
> which versions of a package would be desirable to implement this
> properly.

Desirable, but not necessary. Making the BTS more version-aware
is a longstanding dream. But if this isn't _necessary_, let's
not make it a requirement, or we will never get things done. If
we go for the incremental process, then we can tackle the BTS
_once_ the incremental process is working.


> > Please make a list of the code we would need.
> > Please make a list of the added burdens on the ftpmasters.
> 
> Erm. As proposer, this is really /your/ job.

No, because I made a ``political'' proposal - if we _want_ this
new system. If this was a ``technical'' proposal it would have
gone privately to the ftp team.

Anyway I made a (very incomplete) list somewhere in the proposal.

What I meant is: I'm telling you this is not a lot of work.
You (plural)'re telling me otherwise. So please show me facts.
Which you just did, thanks.


> > > Third, voting on `this is what these people will spend their time on in
> > > future' is completely inappropriate. If it's really a better way, they'll
> > > spend their time on it because they want to. If it's a bit ambiguous,
> > > you can spend your time on it if you want to.
>
> > No, nobody could just go and implement this radical change in
> > the way we work without consulting the other developers.
> 
> Of course they could. In exactly the same way Debconf appeared. People
> discussed it, someone went off and implemented it, and now that we've
> got an implementation, we're ready to start using it, and consider adding
> it to policy.

No, it's very different. Debconf is a much less touchy subject
politically. Variations on the ``package pool'' theme have been
around for more than one year, and every time it's brought up
there are strong objections. So I felt we should vote this once
and for all and shut up, then if we know it's ``official'' then
the techs who want it done will implement it just like happened
with debconf.

Or in other words: When debconf was brought up, there was
consensus. If there hadn't been, it would have to be voted on
first. Even if it wasn't voted on, anyone could just have gone
and coded it, because testing debconf wouldn't mean _screw up
the whole archive_. But we can't just ``test'' a new release
process, can we? Either we do it, or we don't.


> Here, we ought to be discussing what we need (which we've done for over
> a year now), implementing it, and /then/ working out whether we want to
> actually use it or not.

Think of it as ``motivation''. A long ago we decided we wanted
apt (no vote was necessary IIRC), then people went off to
implement it. They could work with a lot of motivation because
they knew that the project wanted apt.

If I si

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Joey Hess 
Anthony Towns wrote:
> stable, testing, unstable (note the sorting order! I'm so proud.)
> 
> Stable and unstable would remain more or less exactly as they are now. There
> aren't any changes to dinstall, or how/where you upload to, etc.
> 
> Testing is a distribution that's completely automatic --- a program
> (with minimal human assistance, I'm not entirely clear on how to manage
> this) selects packages from unstable that satisfy a number of criteria
> and replaces the existing versions in testing with versions from unstable.
> 
> The criteria I think would be best are:
> 
>   * binaries for all appropriate architectures have been built
>   * they are installable using just packages from testing
>   * they don't make any other package in testing uninstallable
>   * the package doesn't have any outstanding release-critical bugs
>   * this version of the package has been in unstable for a fortnight
> or more

Nice. :-)

Can people who favor package pools come up with a list of things package
pools give us that this much simpler approach doesn't?

-- 
see shy jo

Re: Proposal: incremental release process (the package pool)

1999-10-25 Thread Lalo Martins 
On Mon, Oct 25, 1999 at 01:40:16PM -0700, Joey Hess wrote:
> 
> Can people who favor package pools come up with a list of things package
> pools give us that this much simpler approach doesn't?

I like this proposal. But I'd list the difference as:

this is more automated, mine is more maintainer-driven.

More automated usually means less control, but this case leaves
enought control.

The problem with this ``much simpler'' approach is that it is
in fact ``much more complex'' implementation-wise; it requires:

- the promotion automation program

- fixing the BTS to tie bugs to versions (so that the program
can know that the bug is on the ``unstable'' version, not on
the one in ``testing'')

It requires more code. If this code was around, _and_ if we can
still have multiple versions of a same package in unstable
which I think is a good idea anyway, then I'd be glad to vote
for this one.

[]s,
   |alo
   +
--
  I am Lalo of deB-org. You will be freed.
 Resistance is futile.

http://www.webcom.com/lalo  mailto:[EMAIL PROTECTED]
 pgp key in the web page

Debian GNU/Linux   ---   http://www.debian.org
Brazil of Darkness - http://www.webcom.com/lalo/BroDar

Re: Data does NOT belong in Debian (was: Stop Archive bloat)

1999-10-25 Thread Goswin Brederlow 
Fabien Ninoles <[EMAIL PROTECTED]> writes:

> On Tue, Oct 19, 1999 at 09:43:57PM +0200, Goswin Brederlow wrote:
> > Philippe Troin <[EMAIL PROTECTED]> writes:
> > 
> > >   1) The way the Debian archive works requires the data to be stored
> > >  twice (source package and .deb).
> > 
> > Why not allow Source only packages ?
> 
> Because installing such package requires you to have three times the
> necessary space and there no way to make it installable from dselect

Actually its only two times. You unpack the source and compile
it. Then you can delete the source and install the deb. Oh, shit, most 
package will still need 3 times the size, becaue they copy stuff into
debin/tmp, but that could be changed to hardlinks or moving to
preserve space. Well its all a dream at the moment, but it can be made 
to work.

> or apt-get install. For sure, if we can make those options works
> correctly, I'm not opppsed to it. See also my suggestion about a deb package
> who used .orig.tar.gz as an "external" data.tar.gz.

At the moment. But that could be changed. Just because its not useable 
now doesn´t mean we shouldn´t think about.

I would realy like a source only distribution, where you only have to
mirror the source directory and "apt-get source-update" will fetch
that source, compile it and install it without problems and without
leaving any garbage behind.

May the Source be with you.
Goswin