Well, this is all interesting :) After reading Michele's email, I just
had to spend hours looking at Mercurial (& the like) as deeply as the
day would let me (I'm on PTO, so I can do this). I thought I had a
good idea about "distributed" version control system but, as it turns
out, a few surprises were there waiting for me. I have always been a
believer in "development against the known reference version" (now,
that's me being old fashioned). So, yes... Mercurial must be a painful
thing. Based on what I have read, mercurial seems a little too
chaotic for my taste. I do believe there is one way to manage a source
repository (did I really just say that?) wrt to input/output. Well,
honestly, I am way too big of a "exactly what is going to any of the
products I build" junky, to even accept some of the basic premises of
Mercurial. Like any workingDirectory is a potential branch?
Personally, I tend to think that when it comes to managing those "ins
and outs" of code lines (I'm talking about production code lines - and
not in the sense of "experimental" as Mercurial would say), VCS should
enforce structure, rules and accountability, in the sense of knowing
where code goes (again that's me being old fashioned and yes, perhaps
a little “corporate” minded). But... I do recognize some potential.
The benefit that jumps up @ me (that's just my take) is that Mercurial
has no real sense of “Mercurial best practices” and has no bias for
particular model more than another (which i think is a good thing
here).
That said, I would usually take more time in reaching my own
conclusions about such things, but, I feel compelled to send what I
think at this point. I will keep those 2 key areas needing focus that
Michele mentions (version control and bug tracking).
Here are my 2 cents:
** "if something is not doing it for me , I change it”.
Michele mentions the that feature development is slowed by bug fixing.
Well, I have certainly seen that before, and worse. I really do
encourage the web2py leads to re-think you source control strategies.
When symptoms like above surface, it is often an indication of harder
times ahead. As an example, without mentioning names, I have started
with a company last spring, where things have gotten so out of control
in this matter, that slower feature development is the least of there
problems. The changes required put development/QA/release processes
back on track are severe. Not a good place to be. And this stemmed
from fast growth and not having an infrastructure in place to handle
higher levels of activity. I am not familiar with the details wrt
process in the web2py community. but if both the existing
infrastructure and rising volume are an area of concern, then I would
recommend, looking at things from another angle. I would define your
needs (i.e. Professor Di Pierro has mentioned his requirements, i.e.
the one branch model where he can control commits to the code line –
and if it is what he is comfortable with, then THAT is where I would
start.
** “If I can't change something that's not doing it for me.... I make
it work"
There are opportunities for change while preserving Professor Di
Pierro's comfort zone (wrt what goes in to the production/golden
branch). One, If keeping with Mercurial is a requirement in the
analysis, then I would change what Mercurial looks like and its
“driving experience”. Web2py has all the functionality needed to
create a front end designed to better that experience. Rules and
structure can certainly be backed in to that front end and set as
“guide lines to the contributors”. Remember, Mercurial has been
conceived without direction, in the sense where strategies to suite
you own needs entirely possible.
The distributed nature of this model (contribution vs “corporate”/
proprietary), does impose that that chaotic mesh and hierarchical
structure will continue to be a consideration. However, at the top
level, that is where these efforts can have a positive impact.
This is what I would suggest we look at:
3 branches to manage “going to production” :
1.the golden/production branch (this belongs to Prof. Di Pierro)
2.the silver/staging branch (this is where Prof, Di Pirerro picks up
changes/patches/features,etc.)
3.the bronze/testing branch (this is where the code that made it up
the ladder goes)
Note: test suites can occur at the silver level as well (same
pattern)
So it looks like this (sorry for the bad graphic):
golden ________________________>
^ ^
| pull | pull
silver __|_____________|_________> \
^ ^ |
managed by
| pull | |
web2py leads
bronze __x__|_____x__|____________> /
|___test |___test : test pulls a specific
changeset
that gets pulled into silver,where
prof. Di Pierro pickes and chooses
his updates according to
bug
tracking (change descriptions need
contain bug tracking number – this
provides good build/test
automation)
checkPoint ----------------------------------------------- Filtered
(bug/feature/etc.)
^^^^^^^^^^^^^^^^^^^^
| | | | | | | | | pulled into
X X X X X X X X X X bronze
I don't know if you are currently using a specific bug tracking
system, but they are typically easy to interface with and made part of
build/release & test processes/automation. I.e. As part of a release
process, I would set rules with the source control system where non-
bugTraking releated changes can either be automatically rejected, or
moved to another set of prioritiesArea, etc... the build (or packaged
fileset, or whatever the output is) contains a detailed inventory of
bug fixes/features/etc... as part of an automated delivery system
(these are part of the build notes)...
This 3 branch model does provide the required solution to the initial
asks:
Prof Di Pierro retains the model he likes and is comfortable with (at
the hight level).
Tests can occur at any given time (no code freeze needed) by both
pulling from a changeset (where code change commits can continue) and
by setting rules and guidelines at applied at the checkPoint
bug tracking can and should be used (specially with a good forecast of
growth) as part of all processes (test and releases)
It is possible to “ease” the Mercurial user experience.
If this is of interest, I will be happy to 1) provide a more detailed
account and 2) help provide an implementation plan as well as (sorry,
another detailed account) on how to apply this on an ongoing release-
to-release basis (we haven't yet covered applying this to specific
release plans).
I realize this was lengthy for this forum and do apologies.I do think
that a foundation should be settled before any other plan (again, that
me and my work habits – doesn't necessarily mean the only way to do
things).
Just my 2 cents,
let me know if any questions, comments or if I am being vague or
unclear, I will be happy to discuss and/or go into any detail.
Mart :)
On Aug 24, 6:24 am, Kenneth Lundström <kenneth.t.lundst...@gmail.com>
wrote:
> Hello Paul,
>
> I d be happy helping putting up a Linux box for web2py. I use CentOS as
> my distribution, but if needed it should not be a big problem setting it
> up on Ubuntu either.
>
> Kenneth
>
> > Hi all,
>
> > Sounds like there's a little momentum in this :-)
>
> > Here's what I can contribute to the party:
>
> > 1. Part of the test management tool I am writing is a 'case
> > management' facility. It is basic, but it supports multiple case
> > types. Cases can be issues/incidents/defects, to do/actions, and
> > notifications to review changes to other entities in the system (in my
> > case, requirements, stories, tests etc...). There is a messaging/
> > notifications module, uploads/attachments and a case notes history.
> > This functionality is part of the system we will will make available
> > as a free hosted option to Agile Development teams. I'd be delighted
> > to offer this to "the team" to use. I benefit by getting feedback and
> > ideas for improvement. I also get a little kudos with the testing
> > market :O)
>
> > 2. As a sideline we have (currently 4) servers colocated at a data
> > centre that is a mile away from where I live (Maidenhead UK).
> > Currently, all are using Windows 2008 and I have my dev web2py set up
> > on one of them. I would like to deploy on Linux, I use Mysql at the
> > moment - the only proprietary code I currently use is Windows and I
> > want to be 100% open source. So I'd be happy to provide a Linux box
> > for a Linux expert to set up apache/mail/web2py. I'm familiar with
> > Ubuntu, but if there's a preferred distribution - please advise - I'll
> > use that. I anticipate this would host the tools required of the team,
> > plus my own development environment. I would deploy on another server.
> > I guess there should be a mirror of the web2py team content somewhere
> > else on the planet. (I could also use my Amazon web services account,
> > but this is a tad more expensive for me).
>
> > 3. I'm sure none of us want to be drawn into a bureacratic, committee
> > based group- but a little organisation is required. I also host three
> > community stes using Druapl CMS. One is public (www.uktmf.com) and I
> > have two private ones that are probably a better model for a small
> > group).I also use Drupal for my own company website.I'd be happy to
> > host (initially on Windows, but I'd migrate to the Linux box later) a
> > Drupal site for the group to use. The value of a CMS is we could
> > define a brief terms of reference for the group, assign roles etc and
> > make a start. Mailing lists are a bit cumbersome :O)
>
> > 4. There's also a slight possibility I can corral some professional
> > testers into helping us. There's an interesting group I know who do
> > weekend testing for fun - mainly exploratory, but if we released apps
> > that needed some testing, I can publicise this in my testing network
> > we might get them on board. Just a thought.
>
> > This is what I can contribute.
>
> > Paul.
>
> > On Aug 23, 10:47 am, Michele Comitini<michele.comit...@gmail.com>
> > wrote:
>
> >> Hi all,
>
> >> I do develomplent with many different things, different languages
> >> architectures and looking for tools for managing software projects is
> >> a must for me
>
> >> ;-)
>
> >> I think the "problem" here is that web2py has started to receive the
> >> deserved attention by the user and developer community.
> >> Prof. Massimo is doing his best to keep with the growing volume of
> >> request. The question is: "design of new features is slowed by
> >> bugfixing?"
>
> >> Well I red from Massimo 2 alerts similar to the following (Massimo
> >> feel free to correct me if I am wrong!!):
> >> 1. "Please remind me of patches to be applied"
> >> 2. "I need to keep track of messages, so no IRC"
>
> >> Due to web2py success things will get worse, then I would suggest to
> >> start focusing on 2 main points to see if we can find some
> >> ideas/solutions.
>
> >> 1) version control
>
> >> I perfectly understand Massimo position: dealing with mercurial
> >> (git,bzr) is a pain! Anyhow we must help Massimo
> >> delegate some dirty work to others, since we need Massimo to do the
> >> important stuff.
> >> I think that somehow Massimo's web2py problems resemble those that
> >> Linus' Linux faced at the end of '90s. I just
> >> remind that Linus to end the thing had written the git system!
>
> >> Please take time to read this
> >> chapter:http://hgbook.red-bean.com/read/collaborating-with-other-people.html
>
> >> and
> >> this:http://hgbook.red-bean.com/read/collaborating-with-other-people.html#...
>
> >> The model for Linux development IMHO is too much at this time, but
> >> some ideas should be taken into consideration.
>
> >> 2) issue tracking
>
> >> We *must* setup a ticket system. Discussion on groups,IRC will
> >> eventually lead to a new ticket, but *only* tickets must be
> >> taken into account for bugfixing. Code snippets, error log, must be
> >> tracked there.
>
> >> ciao,
> >> mic
>
> >> 2010/8/23 mart<msenecal...@gmail.com>:
>
> >>> Hi Again,
>
> >>> So, spending the day with my 3 girls certainly does provide
> >>> perspective on requirements and how well attached we can be to them
> >>> sometimes ;). That in mind, I certainly do understand your personal
> >>> requirements on keeping what you know and what works for you (I
> >>> respect old fashion ;) ). We can work with that, while hopefully
> >>> bringing something efficient, scalable and most certainly flexible,
> >>> while remaining respectful of what is important to Mr Di Pierro who
> >>> brought us all here.
>
> >>> Although I haven't spent too much time with Mercurial, most concepts
> >>> don't change, and implementation well... that's all it is really. I
> >>> had look @ your src repository and I find it is very telling of how
> >>> you do things and what is important. As I understand, the goal is to
> >>> meet 2 separate requirements that inevitably impact one another with
> >>> current structure. The desired outcome: no freeze of the code line
> >>> while allowing for planned testing iterations to move forward (while
> >>> enabling Mr Di Pierro to maintain those elements of the current model
> >>> which are important to him). I think it's entirely doable and please
> >>> don't hesitate to stop me if I get carried away... I would like to
> >>> start, if there are no objections, by getting a high level
> >>> understanding of current practices. So, I'll throw a few questions out
> >>> there. (I will try t keep the number of questions short although it
> >>> may not appear that way). Perhaps, this could be taken to another area
> >>> to minimize the ruckus?
>
> >>> I like the idea of getting a group together and collaborate on
> >>> developing a proposal. As the more input we have, the better we can
> >>> serve this type of development model (the concept of contributors) in
> >>> the web2py dev world in particular. I see that Mr Di Pierro commits
> >>> all changes to the single branch (default).
>
> >>> Here's are a few questions with that:
>
> >>> Where do developers check-in or commit their changes while in
> >>> development?
> >>> Where does the src going to dev come from and at what frequency does
> >>> it get synced with the reference code line (if at all) ?
> >>> Is the reference code line stable (no changes) or is it in constant
> >>> flux?
>
> >>> Since Massimo, is doing the commits, I assume that everybody keeps a
> >>> sandbox copy of the src? Is there a mechanism in place which makes
> >>> sure that everyone is working off the same starting point? If not, how
> >>> are merge conflicts handled presently?
>
> >>> Does the code get reviewed before making its way to Massimo who will
> >>> be committing the changes (or not committing)?
>
> >>> As the release guy , my first and most important consumer of builds
> >>> is QA - the testers usually get first dibs on my time ;) - as they
> >>> are the ones blessing builds and enabling them to move to the next
> >>> levels. I tend to want to have them in mind when writing automation
> >>> and making sure they can interface as smoothly as possible to my
> >>> automation with there own.
>
> >>> When going to Test, what get's handed off (src or build)?
> >>> Is there any regular automated/manual testing? Or is it the case where
> >>> bigger testing efforts are done later in the release cycle?
> >>> how are builds identified with those test results?
>
> >>> Good release strategies do help, so here are just a few questions on
> >>> that subject:
>
> >>> Have you a defined plan for release strategies? (i.e. moving forward
> >>> between releases from 1.83.x to1.84.x, .... to 1.90.x etc.) - or are
> >>> releases treated as milestones?
> >>> milestone strategies within those releases?
> >>> how do you keep track of previous releases?
>
> >>> So, depending on the extent of adherence to release practices we want
> >>> to look at, there are many elements worthy of attention. I am ready
> >>> and willing to spend time in helping web2py plan and implement release
> >>> methodology (if that is desired) in line with your growth expectation
> >>> (which I can only imagine it being very high). What I can do to start
> >>> is setup a structure on Google (although i have noe clue yet what the
> >>> procedures are yet to get that going... a simple sign up?) and play
> >>> with a few ideas. So, where do we go from here, meaning how do we get
> >>> a group of interested people to join in?
>
> >>> I am open for discussion with anyone interested.
>
> >>> Thanks,
> >>> Mart :)
>
> >>> On Aug 22, 1:11 pm, mart<msenecal...@gmail.com> wrote:
>
> >>>> This is sounding like fun! My experience is mostly with Adobe (10
> >>>> years) working with cross-continent& distributed dev efforts. So,
> >>>> getting the chance to work with the great folks from web2py on a
> >>>> "contribution model" (for the lack of a better term) sounds real
> >>>> exciting to me! :)
>
> ...
>
> read more »
