Re: [Twisted-Python] Twisted on PyPI

[exarkun]

On 02:19 am, gl...@twistedmatrix.com wrote:

On Jun 1, 2013, at 5:00 AM, exar...@twistedmatrix.com wrote:
Well, someone seems to have done so now, though I don't see how to 
find out who or when from the PyPI user interface.


As I said in my message, it was Thomas; if that's ambiguous, I meant 
Thomas Hervé.


Sorry, I didn't see the message where you said that.
Also, Twisted Web2 was left alone (ie, it is still on PyPI) which 
strikes me as an odd decision.


Web2 didn't have the same issues (containing the same files as other 
distributions but with a different name) since it was always 
distributed separately.


It might make sense to remove it, too, but if we do, it's for different 
reasons.  If nothing depends on it, I say we should kill it.


Removing it seems like the right thing to me.  I have no idea if 
anything depends on it, but having it on PyPI only means that things 
could continue to depend on it - or even worse, new things might start 
depending on it.


Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] github, again

[Glyph]

Hi Twisted developers,

This weekend I had a discussion with many Twisted developers, both local to and 
visiting San Francisco.  The topic came up of how to get more long-term 
contributors to participate more regularly in the project - particularly, doing 
code reviews, but also, developing and contributing to complex fixes and 
features that new contributors might not be able to tackle.

One suggestion that almost everybody made immediately was: we should use Github 
for code reviews.

In the past, I've heard this suggestion given mainly as a way to contribute 
more code, which does not appeal to me, since we are already swamped reviewing 
all the code that is currently being contributed.

This time, however, it's been pitched as a way to get people to do more 
reviews, which I'm keenly interested in.  Why would people do more reviews on 
Github?  In a nutshell, it's a lot less work.  Here are some reasons why:

Instead of having to run 'force-builds' on the command line, or load a buildbot 
status page, Github has a way for a build system to report build success 
automatically, so you can see immediately within a pull request if the changes 
that it proposes are "good to merge".  You can see this at work with Travis 
here: .  Originally I thought that 
this was a Travis-CI feature, but have since learned that this is apparently 
easy - trivial even - to hook up to Buildbot, since it's a simple HTTP API to 
invoke when a build completes, and there is even some existing buildbot 
infrastructure (deployed by Django, among others) to automate it.
Instead of having to describe each patch location so that you can comment on it 
in a single message, if you want to put a comment on a particular part of a 
diff in a Github pull request, you can just click on it and start typing.
In addition to the diff, it's reasonably easy to see the code in context on the 
web, which is faster than getting it into one's local development environment.
If a review is successful, instead of having to have a local development 
environment, a committer can just hit the "approve" button and it's landed 
immediately.
Instead of having to read through all history ever to see what's still 
relevant, a pull request will hide comments that address outdated diffs, 
allowing the change author to easily see what remains.

These advantages are not comprehensive, but they're the more significant ones I 
remember from this discussion.

A prerequisite for using Github for code reviews would be using Git rather than 
Subversion.  Luckily there's not much work to do in this area, thanks to Tom's 
excellent work on the Git import and automatic Github mirror.  As a bonus, by 
using Git instead of Subversion, we can start properly recording merge metadata.

In this discussion, Alex Gaynor pointed out that Django has a hybrid workflow 
where they still use Trac for bug tracking, and Github for code review.  We 
would therefore not need to come up with a way to migrate all of our tickets to 
Github issues (which seems, oddly, to be fairly unpopular even among those who 
like github a lot).

What would need to happen in order for this to take place?

We'd need some consensus (hence this message).
We'd need to update the release process 
 and our development 
documentation  
to refer to the relevant Git commands rather than Subversion commands.
We'd need a redirect from  and 
 that would point at 
 and 
 respectively.
We'd need a Github web hook that could poke Buildbot to kick off commits.
We'd need Buildbot integration to update Github pull requests with build 
results when builds complete.
We'd need someone to install git rather than bzr on all the buildbots, and 
update the configuration of the builders to get the code from a git rather than 
Subversion URL.
Someone will need to convert every open ticket in review to a pull request.

I do anticipate some objections.

One objection is that each of the above tasks is going to take some work.

I am fairly confident that some of the people who have educated me here will 
step forward to volunteer to do it.  Please reply to this message if you'd like 
to volunteer, saying what you'd like to volunteer to do.  If not, then I guess 
that objection stands :-).

Another is that this might not be worth that investment of effort.  This is why 
it was nice to have Alex contributing to the discussion: Django did basically 
this very change (right down to the "Trac for tickets / Github for pull 
requests" distinction), at a much higher scale than we have, and as he 
described it the change was *well* worth it.

Another objection is that Github is proprietary software, and an 
externally-maintained 

Re: [Twisted-Python] github, again

[Jonathan Stoppani]

Hi Glyph,

That is great news. I already helped with Braid and would be interested in 
contributing some work in this area.

Cheers,
Jonathan

On Jun 3, 2013, at 16:59, Glyph  wrote:

> Hi Twisted developers,
> 
> This weekend I had a discussion with many Twisted developers, both local to 
> and visiting San Francisco.  The topic came up of how to get more long-term 
> contributors to participate more regularly in the project - particularly, 
> doing code reviews, but also, developing and contributing to complex fixes 
> and features that new contributors might not be able to tackle.
> 
> One suggestion that almost everybody made immediately was: we should use 
> Github for code reviews.
> 
> In the past, I've heard this suggestion given mainly as a way to contribute 
> more code, which does not appeal to me, since we are already swamped 
> reviewing all the code that is currently being contributed.
> 
> This time, however, it's been pitched as a way to get people to do more 
> reviews, which I'm keenly interested in.  Why would people do more reviews on 
> Github?  In a nutshell, it's a lot less work.  Here are some reasons why:
> 
>   • Instead of having to run 'force-builds' on the command line, or load 
> a buildbot status page, Github has a way for a build system to report build 
> success automatically, so you can see immediately within a pull request if 
> the changes that it proposes are "good to merge".  You can see this at work 
> with Travis here: .  Originally I 
> thought that this was a Travis-CI feature, but have since learned that this 
> is apparently easy - trivial even - to hook up to Buildbot, since it's a 
> simple HTTP API to invoke when a build completes, and there is even some 
> existing buildbot infrastructure (deployed by Django, among others) to 
> automate it.
>   • Instead of having to describe each patch location so that you can 
> comment on it in a single message, if you want to put a comment on a 
> particular part of a diff in a Github pull request, you can just click on it 
> and start typing.
>   • In addition to the diff, it's reasonably easy to see the code in 
> context on the web, which is faster than getting it into one's local 
> development environment.
>   • If a review is successful, instead of having to have a local 
> development environment, a committer can just hit the "approve" button and 
> it's landed immediately.
>   • Instead of having to read through all history ever to see what's 
> still relevant, a pull request will hide comments that address outdated 
> diffs, allowing the change author to easily see what remains.
> 
> These advantages are not comprehensive, but they're the more significant ones 
> I remember from this discussion.
> 
> A prerequisite for using Github for code reviews would be using Git rather 
> than Subversion.  Luckily there's not much work to do in this area, thanks to 
> Tom's excellent work on the Git import and automatic Github mirror.  As a 
> bonus, by using Git instead of Subversion, we can start properly recording 
> merge metadata.
> 
> In this discussion, Alex Gaynor pointed out that Django has a hybrid workflow 
> where they still use Trac for bug tracking, and Github for code review.  We 
> would therefore not need to come up with a way to migrate all of our tickets 
> to Github issues (which seems, oddly, to be fairly unpopular even among those 
> who like github a lot).
> 
> What would need to happen in order for this to take place?
> 
>   • We'd need some consensus (hence this message).
>   • We'd need to update the release process 
>  and our development 
> documentation 
>  to refer to 
> the relevant Git commands rather than Subversion commands.
>   • We'd need a redirect from  
> and  that would point at 
>  and 
>  respectively.
>   • We'd need a Github web hook that could poke Buildbot to kick off 
> commits.
>   • We'd need Buildbot integration to update Github pull requests with 
> build results when builds complete.
>   • We'd need someone to install git rather than bzr on all the 
> buildbots, and update the configuration of the builders to get the code from 
> a git rather than Subversion URL.
>   • Someone will need to convert every open ticket in review to a pull 
> request.
> 
> I do anticipate some objections.
> 
> One objection is that each of the above tasks is going to take some work.
> 
> I am fairly confident that some of the people who have educated me here will 
> step forward to volunteer to do it.  Please reply to this message if you'd 
> like to volunteer, saying what you'd like to volunteer to do.  If not, then I 
> guess that

Re: [Twisted-Python] github, again

[Ralph Meijer]

On 2013-06-03 22:59, Glyph wrote:

Hi Twisted developers,

[..]

>

One suggestion that almost everybody made immediately was: we should use
Github for code reviews.


As mentioned on IRC, the only comment I have is about the lack of proper 
e-mail addresses associated with commits. Tom is investigating if this 
can be done with .mailmap, instead of having to make a proper mirror again.


Otherwise: do it.

--
ralphm

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[Christian Kampka]

Am 03.06.2013 22:59, schrieb Glyph:
> Another objection is that Github is proprietary software, and an
> externally-maintained service that we'd be depending upon.
> 
> One solution to the "proprietary software" thing is the availability of
> the MIT-licensed .  It's a largely feature-complete
> clone of Github; if, for some reason, we need to migrate away from
> Github in a hurry, it will be relatively painless to set up Gitlab
> instead, and the fact that Git is a DVCS means every contributor will
> serve as a backup.  The main reason I would not suggest just deploying
> it is that it creates another sticky infrastructure-management problem,
> and while Braid is great, I'd prefer to avoid creating /more/ work in
> that area.  Github also has APIs for literally all of their features, so
> we can create a backup script.
> 
> (Also worth noting: Gitlab is an open-source competitor to Github, but
> they still trust Github enough to  host
> their own development there.)

It may be worth noting that the reason for gitlab being hosted on github
is probably due to the fact that up until very recently, gitlab had no
way for a non-registered user to access gitlab at all. Even though they
have introduced a "public area" for projects this does only include
anonymously cloning of a repository, you still cannot browse code or
look at issues, pull/merge request or wikis without a user account.

Although gitlab is great for internal projects, this lack of a proper
support for public features makes it imo not that suitable for open
projects.

Cheers,
Chris




signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[Christopher Armstrong]

On Mon, Jun 3, 2013 at 3:59 PM, Glyph  wrote:

>
> One suggestion that almost everybody made immediately was: we should use
> Github for code reviews.
>

I'm +1 on the whole proposition as described.

Finally, my own minor concern: Github has no notion of a "code review" as a
> unit of work.  A pull request is just "open" until it is "closed".  Closing
> pull requests to request changes would be jarring to the cultural norms
> associated with Github's UI.  All the github users I've spoken with, even
> those who follow processes which are effectively identical to Twisted's,
> have assured me that this is not really an issue.  A code review is
> "accepted" when you merge it; it's "rejected" if the pull request is still
> open but has some comments on it.  This will make porting over <
> http://twistedmatrix.com/highscores/> a bit challenging, but I think it
> would be worth letting that break for the time being.
>

I don't really like the idea of maintaining review state in Trac,
especially since one of the points of this discussion is to make the life
of the reviewer easier. My feeling is that the slight difference in review
workflow on PRs -- the fact that there is no "responsibility transfer"
mechanism -- will not be a serious problem in practice, and that we should
have a culture of closing abandoned PRs.

-- 
Christopher Armstrong
http://radix.twistedmatrix.com/
http://planet-if.com/
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[Jamu Kakar]

Hi,

On Mon, Jun 3, 2013 at 4:48 PM, Christopher Armstrong
 wrote:
> On Mon, Jun 3, 2013 at 3:59 PM, Glyph  wrote:
>> One suggestion that almost everybody made immediately was: we should use
>> Github for code reviews.
>
> I'm +1 on the whole proposition as described.

Me too.

>> Finally, my own minor concern: Github has no notion of a "code review" as
>> a unit of work.  A pull request is just "open" until it is "closed".
>> Closing pull requests to request changes would be jarring to the cultural
>> norms associated with Github's UI.  All the github users I've spoken with,
>> even those who follow processes which are effectively identical to
>> Twisted's, have assured me that this is not really an issue.  A code review
>> is "accepted" when you merge it; it's "rejected" if the pull request is
>> still open but has some comments on it.  This will make porting over
>>  a bit challenging, but I think it
>> would be worth letting that break for the time being.
>
> I don't really like the idea of maintaining review state in Trac, especially
> since one of the points of this discussion is to make the life of the
> reviewer easier. My feeling is that the slight difference in review workflow
> on PRs -- the fact that there is no "responsibility transfer" mechanism --
> will not be a serious problem in practice, and that we should have a culture
> of closing abandoned PRs.

Something that has worked well for me on other projects is to use
simple conventions.  When you finally approve a branch you leave a
comment like 'jkakar:approve'.  If you expect changes you leave a
comment like 'jkakar:needs-fixing'.  In other words, you don't really
need an app-enforced mechanism if you have a simple convention.  I
propose starting with the simplest convention: the reviewing must add
an 'author:approve' comment when they're finally happy.

Thanks,
J.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[Glyph]

On Jun 3, 2013, at 5:35 PM, Jamu Kakar  wrote:

> Hi,
> 
> On Mon, Jun 3, 2013 at 4:48 PM, Christopher Armstrong
>  wrote:
>> On Mon, Jun 3, 2013 at 3:59 PM, Glyph  wrote:
>>> One suggestion that almost everybody made immediately was: we should use
>>> Github for code reviews.
>> 
>> I'm +1 on the whole proposition as described.
> 
> Me too.
> 
>>> Finally, my own minor concern: Github has no notion of a "code review" as
>>> a unit of work.  A pull request is just "open" until it is "closed".
>>> Closing pull requests to request changes would be jarring to the cultural
>>> norms associated with Github's UI.  All the github users I've spoken with,
>>> even those who follow processes which are effectively identical to
>>> Twisted's, have assured me that this is not really an issue.  A code review
>>> is "accepted" when you merge it; it's "rejected" if the pull request is
>>> still open but has some comments on it.  This will make porting over
>>>  a bit challenging, but I think it
>>> would be worth letting that break for the time being.
>> 
>> I don't really like the idea of maintaining review state in Trac, especially
>> since one of the points of this discussion is to make the life of the
>> reviewer easier. My feeling is that the slight difference in review workflow
>> on PRs -- the fact that there is no "responsibility transfer" mechanism --
>> will not be a serious problem in practice, and that we should have a culture
>> of closing abandoned PRs.
> 
> Something that has worked well for me on other projects is to use
> simple conventions.  When you finally approve a branch you leave a
> comment like 'jkakar:approve'.  If you expect changes you leave a
> comment like 'jkakar:needs-fixing'.  In other words, you don't really
> need an app-enforced mechanism if you have a simple convention.  I
> propose starting with the simplest convention: the reviewing must add
> an 'author:approve' comment when they're finally happy.

Honestly, we don't have much of an enforcement mechanism as it is.  We just 
manually add the 'review' keyword when something goes into review, by typing 
'review' into the keywords box; you also have to manually un-assign the ticket, 
and people often forget that part.

The two things that notice this are our IRC bot and our High Scores page.

The lack of a 'keyword' mechanism somewhat distracted me from the main issue 
though; if we just had high scores and the announcement bot scan the comments 
for the strings "Please fix." for out-of-review and "Please review." for 
back-into-review, we could both maintain the exact same workflow and reward 
politeness ;-).

Thanks for the suggestion, Jamu!

-glyph


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[meejah]

Christian Kampka  writes:

> Although gitlab is great for internal projects, this lack of a proper
> support for public features makes it imo not that suitable for open
> projects.

Sorry to butt in, but to add to this Gitlab doesn't support the "fork
and pull request" model of GitHub -- instead they use branches (in one
repo) and "merge requests". This might work fine for you guys, of
course, but if you're expecting a "drop in" replacement for github,
that's not what you'll get ;)

It sounds like aspects of this are currently in the latest stuff,
however, so it might work like github in this respect "soon":

https://github.com/gitlabhq/gitlabhq/pull/3597

Cheers,
meejah

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[Terry Jones]

I sent most of the below off-list to Glyph earlier, as my comments were a
bit half-assed and I'm not really (or not at all) a Twisted contributor.
Glyph suggested I mail them to the list anyway, and to try adding some more
concrete reasons for being +1 on the suggested change.

---  [ Original mail ]

I'm +1 bigtime on moving towards git/github. I really dislike git, but it
gets the job done, and github is awesome. Github changed the way we work,
it removed a ton of friction and overhead in reviewing (you're right, the
click to comment on a diff is really convenient). I don't have too many
complaints about the ticket system, but sure it could be better. The really
impressive thing about Github is how incredibly quickly they move. It gets
better and better and better all the time.  We migrated away from svn +
trac to bzr + launchpad and finally to git + github. The latter blows the
former away massively, IMO.  I'm not trying to give formal quantitative
feedback, just my subjective opinion.  I've also felt for ages that there's
too much overhead in trying to contribute to Twisted. I'm a lazy/selfish
kind of member of the Twisted community, and when I'm faced with the
thought of having to set up all that old-fashioned (from my POV) machinery
like svn and combinator and what-not to think about really contributing or
running tests or whatever, some part of me just thinks "no, no, no, I'm not
going back to that world".  (see above re lazy, if that wasn't clear).

OK, sorry for a bit of a rambling subjective mail. I'm sure you've heard
all this before. The funny thing about these changes is that before you
make them you always think things are pretty much fine. After you make them
you wonder how you ever lived with the old system.  That may not be the
case here for you or for you & the Twisted community, but it certainly was
for me with the move to github.

---

Glyph> Maybe you could be a bit more specific as to the steps that Github
eliminates, and _concrete_ ways in which it is more efficient.

I'd say there are probably dozens of ways in which Gthub improved things
for us. Many of them are minor, of course, but they all add up and the
usability difference between Github and Trac or Launchpad is extreme.

Ease of use and the anticipated amount of effort something is going to take
are very important aspects of tools, in my opinion. If you know some action
- like starting a review or branching or merging or commenting etc. - is
going to take a certain amount of effort, that correlates (unless you're
being paid or doing this out of some extreme reason, like being a core dev
on a project) with how likely you are to do it. (E.g., making branches - it
was possible in sccs, in rcs, in cvs, in svn, but oh, the pain! When
making branches *and merging them* became so extraordinarily simple and
pain free, it changed the way people worked.)  As I said above, I'm a
lazy/selfish Twisted onlooker. Many has been the time when I'd have been
happy to chip in on something (e.g., last week someone was asking for some
reviewing help) but the thought of getting the required machinery in place,
and using it, even if it's just svn (ugh, ugh, ugh) stops me.  That's a
feeling I've had about Twisted many times over the last years. It's the
feeling I have when someone asks me to go back to helping them on some old
perl code - you just don't wanna go there :-) It's ugly and not fun.  Sorry
if these are wimpy reasons, but to me usability friction (real or
perceived) is very important.

A very nice thing about Github is that you can have conversations about the
diff right on the page, inline, where the diff is shown. You don't need to
download the branch, to use the command-line (and i LOVE the command line
BTW), or to do anything like that. Github is smart enough to hide the
discussion once a subsequent commit comes in that addresses the line being
discussed (sure, that could go wrong but in practice it doesn't seem to).
 One click to merge branches is great - in fact we made a rule that merging
is always done via Github. Adding of milestones and labels to issues is
smooth, as is viewing various simple subsets of issues. The project
activity graphs in Github are very interesting. GFM (Github flavored
markdown) is really easy to use and the results are attractive. Gists are
great. It's nice being able to have a wiki that's also just a repo. I very
much like the "pull requests are a place where discussion takes place"
approach - as opposed to holding that pull requests are just where branches
go when they're "finished" and ready to be merged. BTW, it's very
interesting to read about how the folks at Github use Github themselves [1,
2]. The workflow we developed is very similar, only slightly more complex.
If people want more examples, I can try to go more systematically through
my usage of Github & say more.

As mentioned above, Github moves (improves) incredibly quickly. They put
pretty much every other project I know to shame (includin

Re: [Twisted-Python] github, again

[Christopher Armstrong]

On Mon, Jun 3, 2013 at 8:48 PM, meejah  wrote:

>
> It sounds like aspects of this are currently in the latest stuff,
> however, so it might work like github in this respect "soon":
>
> https://github.com/gitlabhq/gitlabhq/pull/3597
>
>

Whoah. Did anyone else notice that "coveralls" bot? that is pretty sweet.


-- 
Christopher Armstrong
http://radix.twistedmatrix.com/
http://planet-if.com/
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] github, again

[tds...@gmail.com]

Hi,

what about Bitbucket (www.bitbucket.org) and mercurial ?

Don't they provide the same features ?

I'm asking because we are in Python land. ;-)




Regards,

Wolfgang



___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python