Re: [PATCH] verify at each commit

2013-03-26 Thread Daniel Shahaf
Branko Čibej wrote on Wed, Mar 27, 2013 at 05:14:51 +0100: > On 26.03.2013 23:11, Daniel Shahaf wrote: > > [[[ > > Run the per-revision verify code on a transaction just before it becomes > > a revision. The intent is to catch corruption bugs as early as possible. > > > > * subversion/libsvn_fs/fs

Re: [PATCH] #include limts.h for ULONG_MAX

2013-03-26 Thread Joe Swatosh
Okay, thanks for giving me hints on this. As a matter of fact, my apr does NOT have a section like this. It is just doing a straight up copy of apr.hw. I've downloaded a few different versions of apr and none of them have a block like that in apr.hw. Heading over to the apr list now to ask for more

Re: [PATCH] verify at each commit

2013-03-26 Thread Branko Čibej
On 26.03.2013 23:11, Daniel Shahaf wrote: > [[[ > Run the per-revision verify code on a transaction just before it becomes > a revision. The intent is to catch corruption bugs as early as possible. > > * subversion/libsvn_fs/fs-loader.c > (svn_fs_commit_txn): As above. > ]]] > > [[[ > Index: sub

Re: [Patch] Allow creating SWIG build targets without a JDK

2013-03-26 Thread Joe Swatosh
Thank you! r1461395 -- Joe On Tue, Mar 26, 2013 at 5:01 AM, Daniel Shahaf wrote: > +1

Re: [PATCH] use apr macros to make cache-membuffer.c more portable

2013-03-26 Thread Joe Swatosh
Thanks! -- Joe On Tue, Mar 26, 2013 at 2:22 AM, Stefan Fuhrmann wrote: > On Tue, Mar 26, 2013 at 1:34 AM, Joe Swatosh wrote: >> >> Inspired by Bert's comment in another thread about apr macros > > > Hi Joe, > > Thanks for the patch. Committed as 1461030. > > -- Stefan^2. > >> >> [[[ >> Use apr

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Ben Reser wrote on Tue, Mar 26, 2013 at 17:12:33 -0700: > On Tue, Mar 26, 2013 at 4:39 PM, Ben Reser wrote: > > However, this pre-commit script (which requires the Python bindings) > > should catch all the variations. It also checks for other control > > characters, but you can easily change that

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 4:39 PM, Ben Reser wrote: > However, this pre-commit script (which requires the Python bindings) > should catch all the variations. It also checks for other control > characters, but you can easily change that by changing the > control_chars set at the top of the file. Sl

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 12:47 PM, Daniel Shahaf wrote: > We've noted on IRC that this might fail if /foo and /foo\n get modified > in the same revision. Actually at least with FSFS the commit will fail in that case because FSFS will believe you tried to add the same file twice. However, this pre

RE: svn_wc_add_from_disk2 doesn't support skip_some_checks

2013-03-26 Thread Bert Huijben
> -Original Message- > From: MARTIN PHILIP [mailto:codematt...@ntlworld.com] On Behalf Of > Philip Martin > Sent: dinsdag 26 maart 2013 23:15 > To: Daniel Shahaf > Cc: Marshall Schor; dev@subversion.apache.org > Subject: svn_wc_add_from_disk2 doesn't support skip_some_checks > > Daniel S

Re: svn_wc_add_from_disk2 doesn't support skip_some_checks

2013-03-26 Thread Daniel Shahaf
Philip Martin wrote on Tue, Mar 26, 2013 at 22:15:22 +: > Daniel Shahaf writes: > > > Marshall Schor wrote on Tue, Mar 26, 2013 at 15:29:43 -0400: > >> I recently created some new binary files. I happened to pick a name for > >> these > >> that ended in ".data", thinking that would remind m

svn_wc_add_from_disk2 doesn't support skip_some_checks

2013-03-26 Thread Philip Martin
Daniel Shahaf writes: > Marshall Schor wrote on Tue, Mar 26, 2013 at 15:29:43 -0400: >> I recently created some new binary files. I happened to pick a name for >> these >> that ended in ".data", thinking that would remind me and others in the future >> that this was just data. >> > > Thanks, r

[PATCH] verify at each commit

2013-03-26 Thread Daniel Shahaf
[[[ Run the per-revision verify code on a transaction just before it becomes a revision. The intent is to catch corruption bugs as early as possible. * subversion/libsvn_fs/fs-loader.c (svn_fs_commit_txn): As above. ]]] [[[ Index: subversion/libsvn_fs/fs-loader.c ==

Re: Misleading error report

2013-03-26 Thread Daniel Shahaf
Marshall Schor wrote on Tue, Mar 26, 2013 at 15:29:43 -0400: > I recently created some new binary files. I happened to pick a name for these > that ended in ".data", thinking that would remind me and others in the future > that this was just data. > Thanks, r1461328.

Re: [PATCH] Update and switch APIs call conflict resolver at end of operation

2013-03-26 Thread bert
I don’ t think the strict ordening will be a problem for old clients at the client level. We don't promis ordering in the ra layers anyway and we have changed the ordering many times before. As long as we call the callbacks I think clients would be fine. And it will solve the existing problem of

Misleading error report

2013-03-26 Thread Marshall Schor
I recently created some new binary files. I happened to pick a name for these that ended in ".data", thinking that would remind me and others in the future that this was just data. Because I'm an Apache committer, I had followed the svn config setup in http://apache.org/dev/svn-eol-style.txt. I

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Stefan Sperling
On Tue, Mar 26, 2013 at 04:29:19PM -0400, C. Michael Pilato wrote: > On 03/26/2013 04:16 PM, Daniel Shahaf wrote: > > svn_fs.h:1202 > > (I thought I'd given that pointer upthread) > > Yup. That's precisely the comment I was thinking of, too. That comment was written by Jim Blandy many years ago

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread C. Michael Pilato
On 03/26/2013 04:16 PM, Daniel Shahaf wrote: > Ivan Zhakov wrote on Wed, Mar 27, 2013 at 00:13:12 +0400: >> On Tue, Mar 26, 2013 at 11:48 PM, Daniel Shahaf >> wrote: >>> Ivan Zhakov wrote on Tue, Mar 26, 2013 at 21:57:42 +0400: Can we just extend [FS validation] to prohibit all characters wi

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread C. Michael Pilato
On 03/26/2013 04:16 PM, Daniel Shahaf wrote: > Ivan Zhakov wrote on Wed, Mar 27, 2013 at 00:13:12 +0400: >> On Tue, Mar 26, 2013 at 11:48 PM, Daniel Shahaf >> wrote: >>> Ivan Zhakov wrote on Tue, Mar 26, 2013 at 21:57:42 +0400: Can we just extend [FS validation] to prohibit all characters wi

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 1:13 PM, Ivan Zhakov wrote: > On Tue, Mar 26, 2013 at 11:48 PM, Daniel Shahaf > wrote: >> We can but that would break an API promise. (A promise that we might be >> fine with breaking.) > I don't see API breakage here: > 1. We already prohibited some FS paths (with . or

Re: [PATCH] Update and switch APIs call conflict resolver at end of operation

2013-03-26 Thread Julian Foad
Stefan Sperling wrote: > On Tue, Mar 26, 2013 at 04:26:33PM +, Julian Foad wrote: >> With this patch, subversion/svn/update-cmd.c:svn_cl__update() will do this: >> >>   * Call svn_client_update4(...) >> >>     - with ctx->conflict_func2 set to svn_cl__conflict_func_interactive() >>      

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Ivan Zhakov wrote on Wed, Mar 27, 2013 at 00:13:12 +0400: > On Tue, Mar 26, 2013 at 11:48 PM, Daniel Shahaf > wrote: > > Ivan Zhakov wrote on Tue, Mar 26, 2013 at 21:57:42 +0400: > >> Can we just extend [FS validation] to prohibit all characters with ascii > >> code < 31 ? > > > > We can but tha

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ivan Zhakov
On Tue, Mar 26, 2013 at 11:48 PM, Daniel Shahaf wrote: > Ivan Zhakov wrote on Tue, Mar 26, 2013 at 21:57:42 +0400: >> Can we just extend [FS validation] to prohibit all characters with ascii >> code < 31 ? > > We can but that would break an API promise. (A promise that we might be > fine with br

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ivan Zhakov
On Tue, Mar 26, 2013 at 11:07 PM, Ben Reser wrote: > On Tue, Mar 26, 2013 at 10:57 AM, Ivan Zhakov wrote: >> We already have svn_fs__path_valid() and validate all incoming FS >> paths in fs-loader.c. Current svn_fs__path_valid() implementation is: >> [[[ >> svn_error_t * >> svn_fs__path_valid(con

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Ivan Zhakov wrote on Tue, Mar 26, 2013 at 21:57:42 +0400: > Can we just extend [FS validation] to prohibit all characters with ascii code > < 31 ? We can but that would break an API promise. (A promise that we might be fine with breaking.)

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Ben Reser wrote on Tue, Mar 26, 2013 at 12:03:43 -0700: > On Tue, Mar 26, 2013 at 9:29 AM, Daniel Shahaf > wrote: > > Fair enough. Infra would be interested in a pre-commit hook script that > > checks for control characters in filenames and aborts the transaction. > > Use validate-files.py in t

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 10:57 AM, Ivan Zhakov wrote: > We already have svn_fs__path_valid() and validate all incoming FS > paths in fs-loader.c. Current svn_fs__path_valid() implementation is: > [[[ > svn_error_t * > svn_fs__path_valid(const char *path, apr_pool_t *pool) > { > /* UTF-8 encoded s

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 9:29 AM, Daniel Shahaf wrote: > Fair enough. Infra would be interested in a pre-commit hook script that > checks for control characters in filenames and aborts the transaction. Use validate-files.py in the trunk tools/hook-scripts with a conf file like so: [repositories]

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ivan Zhakov
On Tue, Mar 26, 2013 at 8:40 PM, C. Michael Pilato wrote: > On 03/26/2013 12:10 PM, Daniel Shahaf wrote: >> Or we could forbid newlines in pathnames. The only problem with that is >> that the 1.0 API promised they would work... but if no one uses that, >> I'd be fine with calling it an erratum an

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Stefan Sperling
On Tue, Mar 26, 2013 at 12:40:56PM -0400, C. Michael Pilato wrote: > On 03/26/2013 12:10 PM, Daniel Shahaf wrote: > > Or we could forbid newlines in pathnames. The only problem with that is > > that the 1.0 API promised they would work... but if no one uses that, > > I'd be fine with calling it an

Re: [PATCH] Update and switch APIs call conflict resolver at end of operation

2013-03-26 Thread Stefan Sperling
On Tue, Mar 26, 2013 at 04:26:33PM +, Julian Foad wrote: > With this patch, subversion/svn/update-cmd.c:svn_cl__update() will do this: > >   * Call svn_client_update4(...) > >     - with ctx->conflict_func2 set to svn_cl__conflict_func_interactive() >       which does interactive or non-inter

Re: [PATCH] Update and switch APIs call conflict resolver at end of operation

2013-03-26 Thread Julian Foad
  -- Certified & Supported Apache Subversion Downloads: http://www.wandisco.com/subversion/download - Original Message - > From: Julian Foad > To: Subversion Development > Cc: StefanSperling ; Bert Huijben ; Stefan > Küng > Sent: Tuesday, 26 March 2013, 12:26 > Subject: [PATCH] Up

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread C. Michael Pilato
On 03/26/2013 12:10 PM, Daniel Shahaf wrote: > Or we could forbid newlines in pathnames. The only problem with that is > that the 1.0 API promised they would work... but if no one uses that, > I'd be fine with calling it an erratum and disallowing it henceforth. I'm very much in the "let libsvn_f

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Stefan Sperling wrote on Tue, Mar 26, 2013 at 17:27:00 +0100: > On Tue, Mar 26, 2013 at 06:10:35PM +0200, Daniel Shahaf wrote: > > I'd call it a DoS if you can commit such a file and can't later 'svn rm > > URL' it. > > You cannot 'svnsync' the repository anymore, even if you rm the URL. > So you

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Stefan Sperling
On Tue, Mar 26, 2013 at 06:10:35PM +0200, Daniel Shahaf wrote: > I'd call it a DoS if you can commit such a file and can't later 'svn rm > URL' it. You cannot 'svnsync' the repository anymore, even if you rm the URL. So you can DoS a master/slave setup and force the slave to be out-of-date with no

[PATCH] Update and switch APIs call conflict resolver at end of operation

2013-03-26 Thread Julian Foad
Following on from the fix for issue #4316 "Merge errors out after resolving conflicts" r1459012, in which I made the merge APIs call the conflict resolver callback for all conflicts before returning, I mentioned that we should do the same for update and switch [1].  I'll explain a bit more. Cur

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
Ben Reser wrote on Tue, Mar 26, 2013 at 09:06:56 -0700: > On Tue, Mar 26, 2013 at 6:36 AM, Stefan Sperling wrote: > > We should add some C tests as well to verify API behaviour at the > > client layer and at the repos layer. > > > > Given the ripple effects of this problem in FSFS revision files I

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Ben Reser
On Tue, Mar 26, 2013 at 6:36 AM, Stefan Sperling wrote: > We should add some C tests as well to verify API behaviour at the > client layer and at the repos layer. > > Given the ripple effects of this problem in FSFS revision files I think > we should ensure that the Subversion server blocks such f

Re: Filenames with trailing newlines wreak havoc

2013-03-26 Thread Daniel Shahaf
On Tue, Mar 26, 2013 at 02:36:15PM +0100, Stefan Sperling wrote: > When a file with a trailing newline (ASCII 0x0a) has made its way into > the repository, there are various areas of the system that exhibit failure > modes. Failure modes I observed in one particular case are discussed below. The o

Re: [PATCH] Code tidying for subversion/include/config.h

2013-03-26 Thread C. Michael Pilato
On 03/26/2013 07:54 AM, Gabriela Gibson wrote: > I've found that if you use one "*" in the comment like so: > > /* code-only comment */ > > Doxygen doesn't pick this up Yup. > I could not find a way to preserve the comment placement in the > documentation, however, given the layout of the html

Filenames with trailing newlines wreak havoc

2013-03-26 Thread Stefan Sperling
When a file with a trailing newline (ASCII 0x0a) has made its way into the repository, there are various areas of the system that exhibit failure modes. Failure modes I observed in one particular case are discussed below. I intend to file separate issues for each failure mode to keep track of them

Re: [Patch] Allow creating SWIG build targets without a JDK

2013-03-26 Thread Daniel Shahaf
+1

Re: [PATCH] Code tidying for subversion/include/config.h

2013-03-26 Thread Gabriela Gibson
On 25/03/13 13:40, C. Michael Pilato wrote: Note that if there's a way to make the additional comment *not* show up in our doxygen docs, that's preferred -- I don't suspect the indentation-dependent layout of that header will survive the transformation to that output format. After trying out @

Re: [PATCH] #include limts.h for ULONG_MAX

2013-03-26 Thread Philip Martin
Joe Swatosh writes: > Thanks Philip and Bert, perhaps this along with Philip's changes? > > [[[ > Index: subversion/include/svn_types.h > === > --- subversion/include/svn_types.h (revision 1460925) > +++ subversion/include/svn_t

Re: [PATCH] use apr macros to make cache-membuffer.c more portable

2013-03-26 Thread Stefan Fuhrmann
On Tue, Mar 26, 2013 at 1:34 AM, Joe Swatosh wrote: > Inspired by Bert's comment in another thread about apr macros > Hi Joe, Thanks for the patch. Committed as 1461030. -- Stefan^2. > [[[ > Use apr macros to make code more portable. > > * subversion/libsvn_subr/cache-membuffer.c > (mac