Re: [gentoo-dev] Viability of other SCM/version control systems for big repo's

[Ryan Phillips]

* Bret Towe <[EMAIL PROTECTED]> [2005-12-21 23:16]:
> On 12/21/05, Donnie Berkholz <[EMAIL PROTECTED]> wrote:
> > Donnie Berkholz wrote:
> > > I know some of you have done research on how gentoo-x86 converts over to
> > > other systems besides CVS such as SVN, arch, etc. But I can't find the
> > > info anywhere in my archives.
> > >
> > > Could whoever's got it, post it?
> > >
> > > I'm particularly interested in hearing about CVS, SVN, mercurial,
> > > bazaar, darcs.
> >
> > I've downloaded a copy of the gentoo-x86 repo and will run tests myself.
> > Please advise me as to exactly which tests you would like to see, beyond
> > whatever I feel like doing.
> 
> might i also suggest testing out git along with the above listed?
> since i dont know git well enuf or what exactly are requirements of
> a gentoo dev ill just point to some documents
> tutorial can be found here:
> http://www.kernel.org/pub/software/scm/git/docs/tutorial.html
> and documention here:
> http://www.kernel.org/pub/software/scm/git/docs/

I know that some people have big reservations on distributed SCMs, but
why not switch to a distributed format, and cherry pick from other
developer's (and users) repositories?

Git allows for pushing to a centralized server, so it still 'works' in
a similar sense of committing changes to CVS.

cg-branch-add gentoo-main git+ssh://[EMAIL PROTECTED]/
cg-push gentoo-main

-Ryan


pgpHNai8r5qPL.pgp
Description: PGP signature

Re: [gentoo-dev] overlay support current proposal?

[Ryan Phillips]

Stuart Herbert <[EMAIL PROTECTED]> said:
> Thanks for the summary.  I think that's a fair assessment of where we are at.
> 
> The offered software will be trac, svn, and moinmoin.  I'm going to
> look at darcs, and with the help of the haskell team and infra
> determine if we can support it or not.  No-one has expressed a
> preference for a different distributed VCS instead of darcs.
> 
> Just one more thing ...

It sounds to me like the overlays would benefit of using git/cogito.
The Linux Kernel uses this DVCS to full affect. Pulling changes from
other repositories, and even receiving email patches pushed from
people not having their own official repository (or repository http or
ssh accessible).  Any git checkout is a branch, so its easy to stay up
to date with the mainline tree and still work on personal branches.

We need to pick one VCS and only one.  Having multiple systems
requires users to install multiple applications and learn each one.
Not all of them are easy to pick up.  Plus, it would be nice to be
able to merge from the overlays to the Portage trunk.

I think git/cogito might be the solution.  It works for a highly
distributed kernel development, which would be similar to the way the
overlays would work.  Gentoo User A would checkout the kde overlay,
make some changes, cg-commit them to their own overlay, and submit the
patches upstream via an email requesting a pull, or emailing them
patches directly with a git-mkmail command.

An alternative to git would be using subversion.  

*** The main portage tree should be switched away from CVS. ***
There are much better alternatives (svn or git) to use.

CVS is our bottleneck when it comes to development and our users too.
What I see is that the overlays are trying to create branches, when
they should not need to.  Making a PHP or Gnome v2000 overlay is
ridiculous, since a branch is almost free using subversion.  There are
more advantages, like making sure the rest of the tree doesn't break,
and when the branch is stable for package.mask or arch masking then
merge the branch to trunk.  The main tree could live within subversion
(or whatever VCS we choose) as a branch.  It would be easy to keep the
branch up to date with trunk, and then merge the changes to the live
branch.  Major changes to the tree need to be done in a branch where
it should be done.

Overlays should be used only for small additions/changes/or tests.  It
feels like the overlays are already trying to create branches, when in
fact, they would not have to if the main tree was _not_ in CVS.

There are advantages to subversion and advantages to git.  I propose
picking one (I vote for subversion) to use for the overlays.  I also
believe that CVS is now hindering us from reaching our goals as a
project.

Comments?

-Ryan


pgphIgIdx4NkR.pgp
Description: PGP signature

Re: [gentoo-dev] overlay support current proposal?

[Ryan Phillips]

Aron Griffis <[EMAIL PROTECTED]> said:
> Have you followed the threads in the past regarding using other
> version control systems for portage?  Some devs have done benchmarks
> and found that there are blocking issues with subversion, particularly
> because of its repo-wide revisions that prevent multiple commits from
> happening simultaneously.

In actuality, Subversion does 98% of the commit in an initial
transaction, and the blocking only occurs in the last 2% with the FSFS
filesystem.  It really isn't an issue and shouldn't prevent us from
adopting it.

-ryan


pgpInUXlofEMF.pgp
Description: PGP signature

[gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

This is a follow up to Mark's (halcy0n's) thread regarding QA Policies and
seemant's letter on herds, teams, and projects.

I believe the way Gentoo is doing things is broken.  There I have said it.  The
entire project has reached a level of being too political and trying to solve
certain problems in the wrong way.

Some of these problems are intermixed.  Please consider them starting points
for discussion.

__Problem: Developer Growth__

I find that developer growth as being a problem.  Adding a developer to gentoo
should be as easy as 1. has the user contributed numerous (~5+) patches that
helps the project move forward.  If yes, then commit access should be given.
Adding a developer is usually quite a chore.  There are numerous reasons why
this is a problem: having a live tree, taking a test, and not defining within
policy when a person could possibly get commit access. 

All these reasons leave the project stagnant and lacking developers.

Why do people have to take a test?  Is it to make sure they won't break the
tree?  If it is, then the solution of a test is wrong.  We do want to make sure
our developers understand gentoo, but I argue that the bugtracker is all we
need.  As long as a person is adding value to gentoo and they have "proven"
themselves, then they *should* have commit access. 

Perhaps its because of a live tree...

__Problem: Live Tree__

Having a live tree requires people to be perfect.  People are not perfect and
requiring it is ridiculous.  I love having commits in my local tree within the
hour, but having a stable and unstable branch makes a lot of sense.  

CVS doesn't do branching nor tags very well... 

__Problem: CVS__

CVS is one of the worst application ever created.  The portage tree needs to
move to subversion.  A lot of the problems within the project would be solved
by using a better SCM system.  The previous problems regarding the Live Tree
and Developer Growth would be solved, IMHO, by just switching.  Branches Work.
Tags Work.  Reverts work.  Moves work.  I don't see any reason not to use it.
It just plain works.

Projects (gentoo/bsd, embedded, hardened) could choose to keep their own
branches of the portage tree and merge with trunk as needed.  Projects could
stick to traditional solutions like profiles if they so wished. 

Some will probably ask who will merge between branches.  We can do that easily
ourselves.  If I think a package is good to go, then svn merge -r1123:1124 to
the branch. 

Huge projects like Apache, GCC, and KDE already use SVN.

__Problem: QA Policies__ 

http://article.gmane.org/gmane.linux.gentoo.devel/37544

It seems that the QA Policies are a product of a Live Tree, and going partially
non-live would solve the problems listed. 

Everyone here is on the same team.  There will be some breakages in the tree
and those can be dealt with.  Like Seemant [1] said, herds are just groups of
like *packages*.  The QA Policy is wrong when it says cross-team assistance; we
are all on the *same* team.  The tree should naturally work.  If it doesn't
then that is a bug for all of us.

Conflict resolution should not be a subproject.  It should *not* exist at all.
Rules need to be in place to avoid conflict.  Having some sort of voting
structure for all the developers (this doesn't mean requiring everyone to vote)
and not just the council or devrel makes a lot of sense for most things.  If I
don't like how someone is acting within the project there should be a vote and
then see if that person is kicked out.  No trial, no anything besides a vote.
And if I lose I have to deal with it.  Either stay with the project, or find
something else.  This solution just works.

Gentoo should be a fun environment.  The previous paragraph should be taken as
a last resort.

__Problem: GLEPs__

I dislike GLEPs.  Usually they sit on the website for a long long time not
doing anything.  My vote (+1) is get rid of gleps and do everything by email
and a vote by the developers.  AFAIK, the board votes on the GLEPs.  Bad Idea.
It stifles things from getting done, and there is no ownership of who is going
to implement the idea.

A new idea proposal should be mailed to a mailinglist (-innovation?) with
details of timeline to completion, impact, and who is doing the implementation.
If it sounds like a good one, then there is a vote and things proceed.  I like
progress.

__Problem: Voting__

Gentoo has over 200 developers.  People are generally against the voting idea,
but I'm not sure why.  I think voting should work like this:  if 30 developers
(or someother specified number) vote yes to an idea then that idea passes.  It
doesn't require everyone to vote, be at home, be on the computer, and not be on
vacation.

The Apache Foundation already has a decent page regarding this:
http://www.apache.org/foundation/voting.html

The Apache Foundation has over 1300 developers; they must be doing something
right.

If someone misses a vote, too bad.  You weren't there and progress has been
made.  I e

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Jon Portnoy <[EMAIL PROTECTED]> said:
> On Fri, Apr 28, 2006 at 10:14:53AM -0700, Ryan Phillips wrote:
> > 
> > I find that developer growth as being a problem.  Adding a developer to 
> > gentoo
> > should be as easy as 1. has the user contributed numerous (~5+) patches that
> > helps the project move forward.  If yes, then commit access should be given.
> > Adding a developer is usually quite a chore.  There are numerous reasons why
> > this is a problem: having a live tree, taking a test, and not defining 
> > within
> > policy when a person could possibly get commit access. 
> > 
> > All these reasons leave the project stagnant and lacking developers.
> > 
> 
> Maybe certain projects are (and maybe there are a lot of undermaintained 
> packages) but overall I would say we are not really lacking developers; 
> what areas would you say we're lacking devs in exactly?
> 
> The recruitment process should be tightened further to ensure we have a 
> solid, educated dev base. This isn't about shutting people out, it's 
> about ensuring that anyone with commit access is well-versed in how we 
> do things.

I believe we have a problem enticing new devlopers to join.  It
shouldn't be difficult in learning how to commit changes to a tree.

What is "well versed"?  Understanding the ways on how to break the tree?   If 
that 
is the case, then we are doing something wrong. 

> > Why do people have to take a test?  Is it to make sure they won't break the
> > tree?  If it is, then the solution of a test is wrong.  We do want to make 
> > sure
> > our developers understand gentoo, but I argue that the bugtracker is all we
> > need.  As long as a person is adding value to gentoo and they have "proven"
> > themselves, then they *should* have commit access. 
> > 
> 
> Many people with useful contributions can commit garbage due to not 
> quite knowing what they're doing.
> The quiz process is an attempt to address that. We used to recruit the 
> way you suggest and it worked for years; we've since outgrown that. 
> "Testing" recruits provides further education.
> 
> Admittedly the quiz as it stands is archaic and needs reworking. I 
> believe the recruiters team is working on addressing that.

I am arguing that we don't need testing of potential developers.  It
is bad for the community.  It is saying that we don't have any faith
with our recruiting process.  If we only only worried about tree breakage,
then this is the wrong solution.

> > Everyone here is on the same team.  There will be some breakages in the tree
> > and those can be dealt with.  Like Seemant [1] said, herds are just groups 
> > of
> > like *packages*.  The QA Policy is wrong when it says cross-team 
> > assistance; we
> > are all on the *same* team.  The tree should naturally work.  If it doesn't
> > then that is a bug for all of us.
> > 
> 
> OK, well, realistically we are composed of projects working on various 
> areas of Gentoo that must work together with one another to form a 
> whole. Gentoo is not and should not be one big amorphous blob.

I agree.  The mentality should be one project, even if the herds are
split into more project.  I do not like when people say that someone
has stepped on their toes when committing a change to another herd..
Typically people are trying to help.  If there is a breakage then it
is a problem for Gentoo, not just a herd.  Having a live tree just
adds to this problem.

> > Conflict resolution should not be a subproject.  It should *not* exist at 
> > all.
> > Rules need to be in place to avoid conflict.  Having some sort of voting
> > structure for all the developers (this doesn't mean requiring everyone to 
> > vote)
> > and not just the council or devrel makes a lot of sense for most things.
> >  If I
> > don't like how someone is acting within the project there should be a vote 
> > and
> > then see if that person is kicked out.  No trial, no anything besides a 
> > vote.
> > And if I lose I have to deal with it.  Either stay with the project, or find
> > something else.  This solution just works.
> 
> Why should conflict resolution be a popularity contest?

It isn't.  It is how a job works.  If someone isn't getting along with
the team, they are fired.  Same principle.

> > 
> > Gentoo should be a fun environment.  The previous paragraph should be taken 
> > as
> > a last resort.
> > 
> > __Problem: GLEPs__
> > 
> > I dislike GLEPs.  Usually they sit on the website for a long long time not
> > doing anything.  My vote (+1) is get rid of gleps 

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Alec Warner <[EMAIL PROTECTED]> said:
> Ryan Phillips wrote:
> > This is a follow up to Mark's (halcy0n's) thread regarding QA Policies and
> > seemant's letter on herds, teams, and projects.
> > 
> > I believe the way Gentoo is doing things is broken.  There I have said it.  
> > The
> > entire project has reached a level of being too political and trying to 
> > solve
> > certain problems in the wrong way.
> > 
> > Some of these problems are intermixed.  Please consider them starting points
> > for discussion.
> > 
> > __Problem: Developer Growth__
> > 
> > I find that developer growth as being a problem.  Adding a developer to 
> > gentoo
> > should be as easy as 1. has the user contributed numerous (~5+) patches that
> > helps the project move forward.  If yes, then commit access should be given.
> > Adding a developer is usually quite a chore.  There are numerous reasons why
> > this is a problem: having a live tree, taking a test, and not defining 
> > within
> > policy when a person could possibly get commit access. 
> > 
> > All these reasons leave the project stagnant and lacking developers.
> > 
> > Why do people have to take a test?  Is it to make sure they won't break the
> > tree?  If it is, then the solution of a test is wrong.  We do want to make 
> > sure
> > our developers understand gentoo, but I argue that the bugtracker is all we
> > need.  As long as a person is adding value to gentoo and they have "proven"
> > themselves, then they *should* have commit access. 
> > 
> > Perhaps its because of a live tree...
> > 
> 
> I am relatively new, I lurked for quite some time on IRC ( a yearish )
> before finally becoming a dev, and the quiz was not particularly
> difficult, and the questions I didn't know, I asked my Mentor about.  I
> think Mentors in general don't do a very good job ( not complaining
> about mine, mind, just in general ).  I think in some cases, people are
> afraid to ask questions.
> 
> We have the madly successful AT project, and a new Herd Tester project
> is in the works.  I find both of these to be very good ideas and have
> aided in developer growth.
> 
> As for your suggestion, with a "Live Tree" you cannot give random users
> who contribute "5 patches" commit access.  Commiting comes with it an
> inherit responsibility.  The following is an example only:
> 

Ok, so maybe not 5 patches commit access.. How about an active
contributor for 6 months.  I am throwing out ideas. 

> I can go in right now and commit something that destroys anyone's box
> not running SElinux, just bump portage and then watch anyone that uses
> the new version destroy their machine.  Part of this involves having a
> reputation based system.  IMHO this is part of our own tree security.
> I have worked hard in the community to become a developer, and throwing
> that all away to ruin some boxes is silly.  Sure once my changes are
> found they can be revert and a new portage thrown into the tree, but how
> many boxes were ruined first?  What if my commit was unintentional?

So this is a problem with having a live tree.

> > __Problem: Live Tree__
> > 
> > Having a live tree requires people to be perfect.  People are not perfect 
> > and
> > requiring it is ridiculous.  I love having commits in my local tree within 
> > the
> > hour, but having a stable and unstable branch makes a lot of sense.  
> > 
> > CVS doesn't do branching nor tags very well... 
> 
> More details on how Branches and Tags solve the Live Tree problem would
> be good.

We could have a trunk/ and stable/ branch. The stable branch gets
exported to the rsync mirrors.  Trunk/ is where we do the changes,
then merge to stable/ the changes we want.  Should be pretty simple.

> > 
> > __Problem: QA Policies__ 
> > 
> > http://article.gmane.org/gmane.linux.gentoo.devel/37544
> > 
> > It seems that the QA Policies are a product of a Live Tree, and going 
> > partially
> > non-live would solve the problems listed. 
> > 
> > Everyone here is on the same team.  There will be some breakages in the tree
> > and those can be dealt with.  Like Seemant [1] said, herds are just groups 
> > of
> > like *packages*.  The QA Policy is wrong when it says cross-team 
> > assistance; we
> > are all on the *same* team.  The tree should naturally work.  If it doesn't
> > then that is a bug for all of us.
> > 
> > Conflict resolution should not be a subproject.  It should *not* exist at 
> > all.
> > Rules need to be in pla

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Chris White <[EMAIL PROTECTED]> said:
> 
> Sure, then you get this:
> 
> "Hey can I join?"
> "OK"
> "*adds user*"
> -- 2 weeks later --
> "Anyone heard from user?"
> "No"
> 

The solution is to have them been an active contributor for say 6
months.

-ryan


pgpR35ZcUmZet.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Alin Nastac <[EMAIL PROTECTED]> said:
> Ryan Phillips wrote:
> 
> >
> >The council should not vote on gleps are provide policy.  They should
> >be there to handle the money and world-wide problems.
> >
> >The developers should drive innovation; not the council.
> >
> >As in all democracies things get done slowly.  We don't need a
> >democracy within Gentoo, just a clear way of creating progress.
> >
> >  
> >
> Just because we have some elections in our process don't make Gentoo a
> democracy.
> Since we don't have a leader to make the important decisions, we need
> some other form of authority to do the job. A council is the best
> solution to the decisional problem.
> Obviously it has nothing to do with innovation. As always, this is the
> realm of developers.
> 
> In the rest, I basically agree with avenj. No point in repeating what
> Jon already said...

I disagree.  The developers should make *all* the decisions.

Bypass the council.  The council should be there only for when we get
sued, and manage the money we make.

Does anyone agree that having a council is too political?  I strongly
believe it stifles gentoo.

-Ryan


pgplEy9kSx55S.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Grant Goodyear <[EMAIL PROTECTED]> said:
> Ryan Phillips wrote: [Fri Apr 28 2006, 12:14:53PM CDT]
> > __Problem: Developer Growth__
> 
> I've seen suggestions before that one of the things limiting Gentoo's
> growth right now is the hurdles involved in becoming a dev.  I don't
> really think the dev quiz is all that onerous, but I'm willing to listen
> to arguments there.
> 
> > __Problem: Live Tree__
> > 
> > Having a live tree requires people to be perfect.  People are not
> > perfect and requiring it is ridiculous.  I love having commits in my
> > local tree within the hour, but having a stable and unstable branch
> > makes a lot of sense.  
> 
> Does it?  How does having a stable and unstable branch differ from
> having stable and unstable keywords?
> 
> On older idea was peer review.  Any dev could commit to the
> not-yet-ready-for-primetime branch, but those commits would have to be
> peer-reviewed before being added to the user tree.  It's a great idea,
> except (a) nobody wants to do the reviewing job and (b) it would be a
> full time job.
> 
> > 
> > CVS doesn't do branching nor tags very well... 
> > 
> > __Problem: CVS__
> > 
> > CVS is one of the worst application ever created.  The portage tree
> > needs to move to subversion.  A lot of the problems within the project
> > would be solved by using a better SCM system.  The previous problems
> > regarding the Live Tree and Developer Growth would be solved, IMHO, by
> > just switching.  Branches Work.  Tags Work.  Reverts work.  Moves
> > work.  I don't see any reason not to use it.  It just plain works.
> 
> Have you tried using SVN for the portage tree?  I don't know if anybody
> has recently, but in the past when people tried there were two
> significant problems: SVN requires at least 2x the tree size for storage
> on the local machine, and checkouts take something akin to an order of
> magnitude longer than CVS.  The former is annoying, but liveable, but
> the latter is a deal-breaker.
> 
> > __Problem: QA Policies__ 
> > 
> > http://article.gmane.org/gmane.linux.gentoo.devel/37544
> > 
> > It seems that the QA Policies are a product of a Live Tree, and going
> > partially non-live would solve the problems listed. 
> 
> QA does help with fixing broken packages, but in my view their most
> important mandate is to help devs fix bad habits in writing ebuilds.
> Repoman and lists of best-practices help in this regard, but the QA team
> tends to be much better at explaining why something is a bad idea.
> 
> > Conflict resolution should not be a subproject.  It should *not* exist
> > at all.  Rules need to be in place to avoid conflict.  Having some
> > sort of voting structure for all the developers (this doesn't mean
> > requiring everyone to vote) and not just the council or devrel makes a
> > lot of sense for most things.  If I don't like how someone is acting
> > within the project there should be a vote and then see if that person
> > is kicked out.  No trial, no anything besides a vote.  And if I lose I
> > have to deal with it.  Either stay with the project, or find something
> > else.  This solution just works.
> 
> It's worth noting that with the exception of kicking people out of
> Gentoo, much of our practices do, in fact, work just this way, although
> without the formality of a vote.  That's what happens when somebody
> posts to -dev with an idea, it gets kicked around, and some sort of
> consensus is either reached, or it isn't.  In the latter case it's not
> ready for prime time, most likely.
> 
> > __Problem: GLEPs__
> > 
> > I dislike GLEPs.  Usually they sit on the website for a long long time
> > not doing anything.  My vote (+1) is get rid of gleps and do
> > everything by email and a vote by the developers.  AFAIK, the board
> > votes on the GLEPs.  Bad Idea.  It stifles things from getting done,
> > and there is no ownership of who is going to implement the idea.
> 
> > A new idea proposal should be mailed to a mailinglist (-innovation?)
> > with details of timeline to completion, impact, and who is doing the
> > implementation.  If it sounds like a good one, then there is a vote
> > and things proceed.  I like progress.
> 
> It's not quite true that the Council votes on GLEPs, but that's not
> really germane to your overall point.  Despite being the person who
> created GLEPs in the first place, I'm quite willing to admit that the
> concept doesn't seem to work as well as I had hoped.  I'm not sure that
> your idea would work be

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Donnie Berkholz <[EMAIL PROTECTED]> said:
> Tim Yamin wrote:
> >Speaking of which, has anybody done any tests with svk? 
> >(http://svk.elixus.org)
> >And: http://svk.elixus.org/?WhySVK -- it would be interesting to compare
> >checkout performance on it as well.
> 
> I've been planning to do a more detailed comparison of all the popular 
> SCM's out there for probably 6 months, but I just don't have the time 
> right now. If someone wants to pick this up, please let us know.
> 
> Recommended reading: http://www.keltia.net/EuroBSDCon/slides.pdf and 
> www.keltia.net/EuroBSDCon/paper.pdf
> 
> SCMs to test:

cogito
 - Not practical
* the lots of little files doesn't scale well with the size
  of the portage tree
* In addition, git only allows checkins from the project parent.
  A deal breaker in my opinion
cvs
 - Branching sucks
 - Merging is terrible
 - File deletes are bad
 - Atomic Commits
svn
 + Atomic Commits
 + Merging/tagging/brancing is a simple "copy" operation
   http://svnbook.red-bean.com/en/1.1/ch04.html
 + lots of benefits
   http://svnbook.red-bean.com/nightly/en/svn.intro.features.html
   there is more I'm sure people can come up with
 - 2x Drive space
darcs
 - haskell dependency
 - doesn't work on some architectures
   - IMHO, deal breaker
svk
 - not a contender, it is subversion.
   if someone wanted to use svk with the subversion tree they could;
   it is transparent to any other.

-ryan


pgpTsXgV9yu6w.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Donnie Berkholz <[EMAIL PROTECTED]> said:
> Ryan Phillips wrote:
> >git - terrible with lots of tiny little files
> 
> Can you provide some evidence to support this?
> 
> I posted in more detail on SCMs elsewhere today.

Sure.

git only allows commits from the project parent.  Meaning that if
there was a project laid out like:

  portage-tree/
 some-package/
some-package-1.0.0.ebuild
 xorg/
xorg-1.ebuild
 
If I am in the portage-tree/xorg directory, I would be unable to do
a cg-commit.  Git only commits from the parent project directory, so I
would have to change back to the portage-tree and do a commit on the
entire tree.  We should not required that from an SCM.

Subversion versions each directory.  Tha is why one can change into
portage-tree/some-package and do svn svn commit.  This is different
that git, where the entire tree is versioned as one.  Make sense?

Second issue with git, is that with lots of tiny little files things
don't work so well.  I tried converting our portage tree into a git
tree, and it ran for around 2 days until I finally killed it.  If we
didn't want to preserve history, then maybe it would work out.  But
with the problem I outlined above I still don't see it as a contender.
There are lots of times when one would want to do a commit in one
directory.

-ryan


pgpMfm2NTwBbL.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

"Fernando J. Pereda" <[EMAIL PROTECTED]> said:
> On Fri, Apr 28, 2006 at 01:42:40PM -0700, Ryan Phillips wrote:
> > cogito
> >  - Not practical
> > * the lots of little files doesn't scale well with the size
> >   of the portage tree
> 
> Sure, that's why they invented git repack.
> 
> > * In addition, git only allows checkins from the project parent.
> >   A deal breaker in my opinion
> 
> That's not true at all. Not in any sane Git version.

Ferdy:

What I meant is, if you have a change within one directory pending
a commit, and you have a commit pending in a current directory, both
files will be picked up for the commit.  I think that is bad. That
means you can't have pending changes not ready for commit and commit
something.  

yes. git-commit will allow the commit, it will walk the directories
backwards, but it will find all the pending changes and want to commit
them.

I don't think that is beneficial.  I'm open to comments though.

-ryan


pgpFFG53shvAt.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

"Fernando J. Pereda" <[EMAIL PROTECTED]> said:
> Ryan:
> 
> I think you are talking about very old versions of Git:
> 
> On Fri, Apr 28, 2006 at 02:20:43PM -0700, Ryan Phillips wrote:
> > What I meant is, if you have a change within one directory pending
> > a commit, and you have a commit pending in a current directory, both
> > files will be picked up for the commit.  I think that is bad. That
> > means you can't have pending changes not ready for commit and commit
> > something.
> 
> Of course you can have pending commits. You can even have uncommited
> changes in your index since git-commit uses a temporary index when doing
> this kind of checkins.
> 
> > yes. git-commit will allow the commit, it will walk the directories
> > backwards, but it will find all the pending changes and want to commit
> > them.
> 
> It will if you don't use git-commit correctly :)
> 
> > I don't think that is beneficial.  I'm open to comments though.
> 
> 'git commit' semantics are a bit different from 'cvs commit' and 'svn
> commit' semantics. That's probably the reason you faced that problem :)
> 

the only option I saw was git-commit -o and you had to specify the
files that you wanted to commit.

I tried doing a git-commit paths/ and still everything wants to be
committed.

It isn't pretty.

-Ryan


pgpYAOsjE8CQ3.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

"Fernando J. Pereda" <[EMAIL PROTECTED]> said:
> On Fri, Apr 28, 2006 at 02:06:36PM -0700, Ryan Phillips wrote:
> > 
> > Second issue with git, is that with lots of tiny little files things
> > don't work so well.  I tried converting our portage tree into a git
> > tree, and it ran for around 2 days until I finally killed it.  If we
> > didn't want to preserve history, then maybe it would work out.  But
> > with the problem I outlined above I still don't see it as a contender.
> > There are lots of times when one would want to do a commit in one
> > directory.
> > 
> 
> Ryan,
> 
> What tool did you use for that? AFAIK parsecvs[1] has converted some
> very big and broken CVS repositories successfully.
> 
> - ferdy
> 
> [1] git://git.freedesktop.org/~keithp/parsecvs

I was using cvs2git.

I sorta like git in certain aspects. If git would  work better than
CVS or anything other SCM I'm for it.  Right now, _anything_ would be
better than CVS.

-ryan


pgpDqAkxf42aD.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

"Fernando J. Pereda" <[EMAIL PROTECTED]> said:
> On Fri, Apr 28, 2006 at 02:49:18PM -0700, Ryan Phillips wrote:
> > the only option I saw was git-commit -o and you had to specify the
> > files that you wanted to commit.
> > 
> > I tried doing a git-commit paths/ and still everything wants to be
> > committed.
> > 
> > It isn't pretty.
> > 
> 
> Uh, no... thats certainly not true for git-1.3 series, and I belive the
> behavior has been consistent since early february this year when the new
> commit semantics where introduced.
> 
> See this:
> 
> --- 8< ---
> [ $ ~/testy/gitty ] git init-db
> defaulting to local storage area
> [ $ ~/testy/gitty(master) ] echo something > a
> [ $ ~/testy/gitty(master) ] mkdir dir
> [ $ ~/testy/gitty(master) ] echo other thing > dir/b
> [ $ ~/testy/gitty(master) ] git add .
> [ $ ~/testy/gitty(master) ] git commit -m "initial import"
> Committing initial tree 6dc01ab7eb7f19983ae76e72ccb63e3e60aa2dc3
> [ $ ~/testy/gitty(master) ] git status
> nothing to commit
> [ $ ~/testy/gitty(master) ] echo add something here >> dir/b
> [ $ ~/testy/gitty(master) ] echo something there >> a
> [ $ ~/testy/gitty(master) ] git status
> #
> # Changed but not updated:
> #   (use git-update-index to mark for commit)
> #
> # modified: a
> # modified: dir/b
> #
> nothing to commit
> [ $ ~/testy/gitty(master) ] git commit -m "Only things in dir/?" dir/
> [ $ ~/testy/gitty(master) ] git status
> #
> # Changed but not updated:
> #   (use git-update-index to mark for commit)
> #
> # modified: a
> #
> nothing to commit
> [ $ ~/testy/gitty(master) ] 
> --- 8< ---
> 
> It is the same even if you did 'git update-index a' before 'git commit
> -m ... dir/'. However that's something you won't do unless you know what
> you're doing :)
> 

I'm testing with 1.3.1.  You are correct.  The text the is printed by
git is a bit confusing.  If the portage tree can scale to it, then I'm
for it.

Thanks for the clarification.

-ryan


pgpUxk2oFOLaj.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Marius Mauch <[EMAIL PROTECTED]> said:
> Grant Goodyear schrieb:
> >Some questions that need to be answered:
> >
> >* Can the repo be converted while maintaining the history?
> >* How long does a full checkout take?  
> >* How much disk space does a full checkout require?
> >* Is there a viewcvs equivalent available?
> >* Others that I can't think of right now?  (Please feel free to add.)
> 
> Right now portage and repoman have no support for using anything but cvs 
> or rsync for the tree, e.g. repoman commit wouldn't work. Not hard to 
> fix, but something to keep in mind.

Rsync could still be used to deliver up-to-date tree's to people.

Changing the process would of course require backend changes to our
utilities.

The quesion is what is best for the job, and then implement it.

-Ryan


pgpBLVqQi2OKZ.pgp
Description: PGP signature

Re: [gentoo-dev] Gentoo: State of the Union

[Ryan Phillips]

Simon Stelling <[EMAIL PROTECTED]> said:
> Hi Ryan,
> 
> Ryan Phillips wrote:

> >__Problem: Developer Growth__

> I've seen ebuilds from people who have written quite a bunch of ebuilds 
> and were really interested in understanding how they work, but the work 
> they produced just was awful and hurt my eyes. I don't mean that the 
> mean way, I don't expect anything else, and I was just the same. 
> However, the mentoring process and the quiz have helped me a lot to 
> understand not-so-obvious problems.
> 
> >All these reasons leave the project stagnant and lacking developers.
> 
> I wouldn't say so. Just about two weeks or so ago, the recruiters had to 
> defer new requests, because they couldn't deal with them in a timely 
> fashion. You can now tell me that this makes it even worse, but I just 
> see that as the proove of the fact that people are interested in helping 
> us and ready to take the quiz and all the "hassle" involved with 
> becoming a dev.
> 
> Another good reason to keep the current process is the fact that a lot 
> of people become dev and vanish a week after their mentoring period 
> expired. These people would better contribute to the project in a more 
> distant way IMHO, because it takes up a fair amount of time to clean up 
> these accounts afterwards. If you don't beleave me, ask kloeri. ;)

I revised my earlier statement to say consistent involvement for ~6
months.  That is more reasonable.

> >Perhaps its because of a live tree...
> 
> That's surely a big factor.
> 
> >__Problem: Live Tree__
> >
> >Having a live tree requires people to be perfect.  People are not perfect 
> >and
> >requiring it is ridiculous.  I love having commits in my local tree within 
> >the
> >hour, but having a stable and unstable branch makes a lot of sense.  
> 
> It doesn't require people to be perfect. It requires people to think 
> before commiting. If it really required us to be perfect, we wouldn't 
> exist in the first place, would we?

I disagree.  By committing something to the current tree it has the
ability to effect a lot of people.  What happens when we need to reverse a
commit?  It isn't that easy with CVS.

> 
> Having a stable and unstable branch doesn't have many advantages over 
> stable and unstable keywords IMHO, but requires a HUUUGE effort to keep 
> them in sync. It would make things more complicated than necessary. You 
> say we're lacking developers. Do you really want to spend [insert random 
> big number here] of these much-needed developer hours to merging an old 
> with a new tree?

Stable and unstable keywords are a hack on top of a version control
system.  We wouldn't have them if gentoo used an SCM that supports true
branches.  There would be no need.

It doesn't take much time for a herd to say that this branch is stable
for the production tree and do an svn merge or similar.  It isn't a
full time job.  

> >Conflict resolution should not be a subproject.  It should *not* exist at 
> >all.
> 
> You mean conflicts or conflict solution shouldn't exist at all?
> If the former, that's unrealistic. If the latter, why not?

We need a policy so that we can resolve our conflicts.  There are two
types of conflicts.  Personal and Technical.  Personal conflicts can
be resolved simply; a vote, and be done with it.  Technical conflicts
need to be resolved by a voting process so that we can move forward.

Check out the apache voting page I linked to before.

> >Rules need to be in place to avoid conflict.  Having some sort of voting
> >structure for all the developers (this doesn't mean requiring everyone to 
> >vote)
> >and not just the council or devrel makes a lot of sense for most things.  
> >If I
> >don't like how someone is acting within the project there should be a vote 
> >and
> >then see if that person is kicked out.  No trial, no anything besides a 
> >vote.
> >And if I lose I have to deal with it.  Either stay with the project, or 
> >find
> >something else.  This solution just works.
> 
> Do you really think just because 60% of the voting devs agreed on 
> something the other 40% will like it suddenly? 

yes, a majority rule.

> Conflicts cannot be 
> avoided, other than shooting down all people which don't share your 
> point of view.

We need to minimize the conflict and make it easier to overcome those
conflicts.  Having a voting structure would do that.

> >__Problem: GLEPs__
> >
> >I dislike GLEPs.  Usually they sit on the website for a long long time not
> >doing anything.  My vote (+1) is get rid of gleps and do everything by 
> >email
>

Re: [gentoo-dev] Gentoo: State of the Union + suggestion for global dev conference (at bottom, if you want to skip)

[Ryan Phillips]

Stuart Herbert wrote:
>
> I'm offering to lead the effort to establish a global Gentoo developer
> conference, and to do whatever it takes to get everything we need to
> make this happen.  Now who's up for this? :)
>
> Best regards,
> Stu
>   

That sounds like a great idea.

-Ryan
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] Having fun with compression

[Ryan Phillips]

Patrick Lauer <[EMAIL PROTECTED]> said:
> Hi all,
> 
> I had this random idea that many of our distfiles are .tar.gz while more
> efficient compression methods exist. So I did some testing for fun:
> 
> We have ~15k .tar.gz in distfiles. ~6500 .tar.bz2, ~2000 others.
> A short run over 477 distfiles spanning 833M gave me 586M of .tar.bz2 -
> roughly 30% more efficient!
> A comparison run with 7zip gave me 590M files, so bzip2 seems to be
> quite good.
> 
> I don't think repackaging every .tar.gz as .tar.bz2 is a reasonable
> option (breaks MD5 digests, we lose the fallback download from the
> homepage), but maybe this motivates people to save bandwidth and migrate
> their packaging to bzip2.

Patrick, 

did you benchmark CPU load?  Often bzip2 takes 3x as long to
uncompress a package than bzip.  Often, the space savings doesn't
justify the cost of how long it takes for the cpu to decompress the
archive.

-ryan


pgp4Pn7SbQl0c.pgp
Description: PGP signature

Re: [gentoo-dev] Paludis and Profiles

[Ryan Phillips]

Mike Auty <[EMAIL PROTECTED]> said:
> Forgive me,
>   I'm a little new at this and I really don't want to get involved, but
> since my inbox has seen nothing but this for the past day or two, I'm
> going to ask a few questions I'm interested in the answers to...
>   First and foremost is, will adding this to the tree be used for
> function creep, whereby the next request to add to/alter the portage
> tree is backed up by "Well, the profile change was already added to the
> tree"?  I wouldn't want a precedent like this set without the council
> reviewing it.

I really don't see much of an issue of feature creep.  Gentoo/ALT
already has a profile.  It isn't like there are changes to the actual
ebuilds themselves.

>   Thirdly has anything like this ever happened to Debian or the Sourcery
> group?  If so how did they cope with it, and if not, how have they
> avoided it?

SMGL has voting and things get done.

>   As you may have guessed I'm of the, "You can do the same thing with an
> overlay, so why must it be in the tree".  I am however willing to wait
> and see what the council says, why can't the changes to the tree wait
> until then, what is so urgent?  I'm especially intrigued since all this
> is simply to no longer require portage as a dependency of system.  Can't
> paludis peacefully co-exist with a portage installation for a little
> longer, until it's mature?

The question is when is it mature?  I've tried it and Paludis does
work.  There will always be bugs and feature requests.  Its part of
the development process.

Ryan Phillips



pgphGkWfkoHSr.pgp
Description: PGP signature

[gentoo-dev] Retirement

[Ryan Phillips]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Howdy All,

I've decided that its time for me to move on from Gentoo.  I have no
ill feelings and still enjoy using the distribution.  With work and
everything else going on in life I don't have much time to devote to
the distribution anymore.  I am sure at some point I'll emerge from
retirement.  Happy Gentooing.

Regards,
Ryan Phillips
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.3 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEiGxU6cLeDQrpxL8RAvVgAJ4tFZNEeFQe7vnAqOUXgTMntbjKSwCfU65a
4oXr2dqIXVPjAQ3kec5hAfs=
=/oEx
-END PGP SIGNATURE-
-- 
gentoo-dev@gentoo.org mailing list

[gentoo-dev] Fox Package Changes

[Ryan Phillips]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
I have in my bug list a rewrite of the fox packages written by Yaakov
(http://bugs.gentoo.org/show_bug.cgi?id=88924)
.  I want to spread
the word
and get some testing/comments for these before they go into portage.
New Features:
~  * Slotted install for different Fox versions
~* Multiple versions of Fox libraries can be installed
~  allowing for applications to choose which version they need
~  * Fox eclass
~  * Doxygen generated documentation
Yaakov has done some great work on the package.  Please help us out.
There is
a tarball (please get the latest one) on the bug posting.
Thanks,
Ryan
Gentoo Developer
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.1 (GNU/Linux)
iQEVAwUBQmVumFBxA/+aEdQ1AQIqVwf+P1oF2aoRUkc6bpvM7ZAweYR+zjOm575o
rc46n0ApBWNhUbsRA0T7Io0Gtq+MR+ytblEAHYayWZy9z08G6LV3hidkstV3wk81
A6ktjLkd3xQlybIhMai/5ybYYsNFB3X3GfIpebfQ3bTZKALif8ORzbTGGo76J71D
Drizzgrm+ozkeIQ712JN5A7OH3JnvJAhqgoII3QeATxBy80JKjNVr96ajZ3um/36
cZbBbi70HxT0sutjpLif33VlFGdgePdK9KPX/fyDaFq6qKhdvlzbUq6eECbCykZR
X2bdUn4rzyDwRrqd6X2+6sPKkemezI/Z0c+GLOgKDScobSMP5etZgg==
=4Wst
-END PGP SIGNATURE-
--
gentoo-dev@gentoo.org mailing list

[gentoo-dev] New Fox Ebuilds Committed for testing

[Ryan Phillips]

I have committed (and package.masked) new Fox ebuilds created by
Yaakov Selkowitz for testing.

http://bugs.gentoo.org/show_bug.cgi?id=88924

I am sure there will be bug reports or requests.  Please add them to
the bug list.

Regards,
Ryan Phillips
Gentoo Developer


pgpeaaKwE46TP.pgp
Description: PGP signature