Re: Managing zyz.apache.org (was RE: WELCOME to dev@community.apache.org)

2015-01-09 Thread Ulrich Stärk
If I'm not mistaken, SVN does the same (combine and compress all changes prior 
to sending it over
the wire).

Uli

On 2015-01-08 10:28, Robert Metzger wrote:
> I think there is a performance difference between git and svn because with
> git, you are syncing repositories, not files. Git is usually compressing
> the repository before sending it over the network.
> I did a little test with our website directory and pushed it to github:
> 
> git add : 7 seconds (17k files)
> git repo size: 58 MB (du -sh in ".git" dir)
> git push to github : 35 seconds (using the same internet connection as with
> the svn)
> 
> I think we can achieve the same performance at apache using git for the
> websites.
> 
> I agree with you that I should probably take a closer look into the CMS
> external build feature. Using svn or git for 17k files is certainly wrong.
> In particular because we add generated files (javadocs, documentation) to
> the VCS which doesn't have any value.
> 
> I'll get in touch with Infra on a separate thread to resolve our website
> issues.
> 
> 
> 
> On Thu, Jan 8, 2015 at 10:01 AM, Daniel Gruno  wrote:
> 
>>
>> On 2015-01-08 09:55, Robert Metzger wrote:
>>
>>> Hi,
>>>
>>>  If there is sufficient demand, however, that could change - the code that

>>> would make this possible does exist.
>>>
>>> I would like to express demand from the Flink project. svn is a pain to
>>> use
>>> (since we host javadocs and our documentation on our website, the upload
>>> usually runs for 6+ hours. Probably because its so many files).
>>>
>> How is that in any way related to svn? svn or git, if you have that many
>> files, it's going to take ages to upload regardless of which repository
>> system you use.
>>
>> If it makes it easier, you might peruse http://www.apache.org/dev/
>> cmsref.html#external-build and see if you can't use an external build job
>> for you site - that way, you just commit the changes to your template/raw
>> docs, and the CMS system builds the javadoc for you.
>>
>>
>>
>>> Almost all incoming incubator projects are using git nowadays. Many of
>>> them
>>> use GitHub and the "gh-pages" hosting feature. I guess a lot of them would
>>> appreciate a similar feature from the ASF infra.
>>>
>>>
>>>
>>> On Wed, Jan 7, 2015 at 9:42 PM, Daniel Gruno 
>>> wrote:
>>>
>>>  Hi Jay,
 It sounds like you have the CMS setup that is quite common in the ASF.
 If you would rather have straight-up svnpubsub, you can ask for it. That
 way, whatever static content you push to svn will instantly be shown on
 your web site without having to publish through the UI first.

 The main reason we do not support git in this workflow is that git does
 not enable single-file checkouts, and that we haven't properly tested
 gitwcsub (a git version of svnwcsub which is the frontend for svnpubsub)
 for web sites yet. If there is sufficient demand, however, that could
 change - the code that would make this possible does exist.

 With regards,
 Daniel.


 On 2015-01-07 21:29, jay vyas wrote:

  thanks daniel...
>
>here at bigtop we are 100% git based.  so having an svn account ,
> just
> to
> push changes to a site, seems to slow us down alot.
>
> is SVN required  ? or is there another way?
>
> right now we have a system that uses maven, followed by svn and then we
> have to approve the changes in the web ui.
>
> would rather just push static html pages to our git repo , the way we
> push
> everything else.
>
> are all apache projects using SVN or do some folks have an easier
> workflow ?
>
>
>
>
>
>
> On Wed, Jan 7, 2015 at 3:12 PM, Daniel Gruno 
> wrote:
>
>   Essentially, github uses the same method as we do with svnpubsub.
>
>> Files are pushed to a repository and then from there pushed directly to
>> the web site.
>>
>> Is there anything specific about the github model that you think differ
>> from how we do things?
>> Apart from it being git and not subversion, obviously.
>>
>> With regards,
>> Daniel.
>>
>> On 2015-01-07 21:06, jay vyas wrote:
>>
>>   Hi apache !
>>
>>> Whats the simplest way to maintain the xyz.apache.org site?  Right
>>> now
>>> we
>>> push to SVN, but would be great to use something like the github.io
>>> model,
>>> where the static pages are just hosted directly.
>>>
>>>
>>> On Wed, Jan 7, 2015 at 2:56 PM, 
>>> wrote:
>>>
>>>Hi! This is the ezmlm program. I'm managing the
>>>
>>>  dev@community.apache.org mailing list.

 I'm working for my owner, who can be reached
 at dev-ow...@community.apache.org.

 Acknowledgment: I have added the address

   jayunit100.apa...@gmail.com

 to the dev mailing list.

 Welcome to dev@co

Re: A maturity model for Apache projects

2015-01-09 Thread Rich Bowen



On 01/07/2015 04:43 AM, Scott Wilson wrote:

I think we also need to discuss whether we expect projects to undertake 
self-evaluation and reflection, or whether we'd have a process of review 
involving peers, mentors, shepherds etc.


No, I absolutely don't want to create another stack of overhead, or Yet 
Another hoop to jump through, as you said.


I think "directed self-evaluation" is the way I'd want to go - one or 
two persons from ComDev, in conjunction with two or three persons from 
the PMC, doing an evaluation. I imagine this as a function of ComDev, 
not of the board. That is, it's a community/project strengthening 
exercise, not a Big Hammer.


--
Rich Bowen - rbo...@rcbowen.com - @rbowen
http://apachecon.com/ - @apachecon


Re: Managing zyz.apache.org (was RE: WELCOME to dev@community.apache.org)

2015-01-09 Thread Branko Čibej
On 09.01.2015 14:04, Ulrich Stärk wrote:
> If I'm not mistaken, SVN does the same (combine and compress all changes 
> prior to sending it over
> the wire).

Not entirely; it sends deltas and will compress new files IF the server
has mod_deflate configured (but they usually don't because there was a
huge memory leak there for the longest time), but doesn't combine them.
It can't, since it doesn't have the repository available locally.
Subversion working copy state can differ substantially from repository
state, yet the commit can still succeed; this is not the case with Git.

On 2015-01-08 10:28, Robert Metzger wrote:
>> I think there is a performance difference between git and svn because with
>> git, you are syncing repositories, not files. Git is usually compressing
>> the repository before sending it over the network.
>> I did a little test with our website directory and pushed it to github:
>>
>> git add : 7 seconds (17k files)
>> git repo size: 58 MB (du -sh in ".git" dir)
>> git push to github : 35 seconds (using the same internet connection as with
>> the svn)

In this case, Subversion will generate a PUT request for each file,
whereas Git will, IIRC, send a single file across the wire. The problem
here is that Subversion does not (yet) pipeline PUTs, so the difference
you're seeing is the request latency.

We've been talking about pipelining PUTs for ages ... but no-one is
currently working on that, so ... we need more incentive! :)

-- Brane



Re: Managing zyz.apache.org (was RE: WELCOME to dev@community.apache.org)

2015-01-09 Thread Branko Čibej
On 09.01.2015 15:43, Branko Čibej wrote:
> On 09.01.2015 14:04, Ulrich Stärk wrote:
>> If I'm not mistaken, SVN does the same (combine and compress all changes 
>> prior to sending it over
>> the wire).
> Not entirely; it sends deltas and will compress new files IF the server
> has mod_deflate configured (but they usually don't because there was a
> huge memory leak there for the longest time), but doesn't combine them.
> It can't, since it doesn't have the repository available locally.
> Subversion working copy state can differ substantially from repository
> state, yet the commit can still succeed; this is not the case with Git.
>
> On 2015-01-08 10:28, Robert Metzger wrote:
>>> I think there is a performance difference between git and svn because with
>>> git, you are syncing repositories, not files. Git is usually compressing
>>> the repository before sending it over the network.
>>> I did a little test with our website directory and pushed it to github:
>>>
>>> git add : 7 seconds (17k files)
>>> git repo size: 58 MB (du -sh in ".git" dir)
>>> git push to github : 35 seconds (using the same internet connection as with
>>> the svn)
> In this case, Subversion will generate a PUT request for each file,
> whereas Git will, IIRC, send a single file across the wire. The problem
> here is that Subversion does not (yet) pipeline PUTs, so the difference
> you're seeing is the request latency.
>
> We've been talking about pipelining PUTs for ages ... but no-one is
> currently working on that, so ... we need more incentive! :)


FWIW, such commits should be a lot faster via svn:// than via http://.
But we can't use svn:// at all at the ASF because the repo replication
method we currently use only supports http://.

-- Brane



Re: A maturity model for Apache projects

2015-01-09 Thread Bertrand Delacretaz
On Fri, Jan 9, 2015 at 3:23 PM, Rich Bowen  wrote:
> ...I imagine this as a function of ComDev, not of the
> board. That is, it's a community/project strengthening exercise, not a Big
> Hammer...

+1

-Bertrand