Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-11 Thread Nick Piper
On 10/05/11 16:08, Bert Huijben wrote: > The python bindings need it, so I would say: Add it there. The attached patch fixes the FD leak entirely within the Python bindings. > (And maybe make it easier to use from other bindings when it proves useful. > That is why I suggested adding the func

RE: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-10 Thread Bert Huijben
> -Original Message- > From: Julian Foad [mailto:julian.f...@wandisco.com] > Sent: maandag 9 mei 2011 22:51 > To: Nick Piper > Cc: dev@subversion.apache.org > Subject: Re: AW: Python bindings: Use of svn_swig_py_make_stream() - > avoiding leaking fds > >

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-10 Thread Daniel Shahaf
Before we invent another stream API (which isn't a small patch at all, and has a bunch of design decisions involved AFAICT), can we please document the semantics of svn_stream_close()? How it interacts with clearing the stream's pool, whether it's permitted not to call it at all, and whether it's

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-10 Thread Nick Piper
On 09/05/11 21:51, Julian Foad wrote: > Nick Piper wrote: >> Add svn_stream_close_on_cleanup(), which will register a stream to be >> automatically closed when the pool is destroyed. > > First impression: the public name makes it sound like a user-level > thing, but this is for use by a stream's i

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Julian Foad
Nick Piper wrote: > Add svn_stream_close_on_cleanup(), which will register a stream to be > automatically closed when the pool is destroyed. First impression: the public name makes it sound like a user-level thing, but this is for use by a stream's implementation, right? Not by its users. I don'

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Nick Piper
This version: * Makes the new behaviour optional, via stream_close_on_cleanup() (which the Python API then uses.) (Thanks Bert.) * Uses apr_pool_cleanup_register() rather than the 'pre' variant, in order to match the reset of svn and be compatible with older APR (thanks Philip.) [[[ Add svn_stre

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread 'Daniel Shahaf'
Bert Huijben wrote on Mon, May 09, 2011 at 17:49:52 +0200: > > > > -Original Message- > > From: Daniel Shahaf [mailto:danie...@elego.de] > > Sent: maandag 9 mei 2011 17:31 > > To: Nick Piper > > Cc: dev@subversion.apache.org > >

RE: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Bert Huijben
> -Original Message- > From: Daniel Shahaf [mailto:danie...@elego.de] > Sent: maandag 9 mei 2011 17:31 > To: Nick Piper > Cc: dev@subversion.apache.org > Subject: Re: AW: Python bindings: Use of svn_swig_py_make_stream() - > avoiding leaking fds > > Nick Piper

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Philip Martin
Daniel Shahaf writes: >> @@ -73,6 +95,9 @@ svn_stream_create(void *baton, apr_pool_t *pool) >>stream->mark_fn = NULL; >>stream->seek_fn = NULL; >>stream->buffered_fn = NULL; >> + stream->pool = pool; >> + apr_pool_pre_cleanup_register(stream->pool, stream, >> +

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Daniel Shahaf
Nick Piper wrote on Mon, May 09, 2011 at 16:19:23 +0100: > Thanks for reviewing. > > On 09/05/11 15:53, Daniel Shahaf wrote: > > Nick Piper wrote on Mon, May 09, 2011 at 15:31:53 +0100: > > >> + /* This is used to close the stream if the pool is being destroyed. > >> + It was first introduce

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Nick Piper
Thanks for reviewing. On 09/05/11 15:53, Daniel Shahaf wrote: > Nick Piper wrote on Mon, May 09, 2011 at 15:31:53 +0100: >> + /* This is used to close the stream if the pool is being destroyed. >> + It was first introduced for the Python API, to allow implicit >> + closing. >> + */ >>

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Daniel Shahaf
Nick Piper wrote on Mon, May 09, 2011 at 15:31:53 +0100: > On 09/05/11 12:59, Philip Martin wrote: > > > Yes, that would work. An alternative would be to deregister the pool > > cleanup on an explicit close. > > Here is a version that does this. It's tested with the existing tests, > plus my sma

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Nick Piper
On 09/05/11 12:59, Philip Martin wrote: > Yes, that would work. An alternative would be to deregister the pool > cleanup on an explicit close. Here is a version that does this. It's tested with the existing tests, plus my small test case, and it stops the leaking. [[[ Automatically svn_stream_c

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Philip Martin
Daniel Shahaf writes: > Philip Martin wrote on Mon, May 09, 2011 at 12:33:14 +0100: >> Daniel Shahaf writes: >> >> > +static apr_status_t >> > +close_stream_cleanup(void *stream) >> > +{ >> > + apr_status_t apr_err = APR_SUCCESS; >> > + svn_error_t *err; >> > + >> > + err = svn_stream_clos

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Daniel Shahaf
Philip Martin wrote on Mon, May 09, 2011 at 12:33:14 +0100: > Daniel Shahaf writes: > > > +static apr_status_t > > +close_stream_cleanup(void *stream) > > +{ > > + apr_status_t apr_err = APR_SUCCESS; > > + svn_error_t *err; > > + > > + err = svn_stream_close(stream); > > + if (err) > > +

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Philip Martin
Daniel Shahaf writes: > +static apr_status_t > +close_stream_cleanup(void *stream) > +{ > + apr_status_t apr_err = APR_SUCCESS; > + svn_error_t *err; > + > + err = svn_stream_close(stream); > + if (err) > +{ > + apr_err = err->apr_err; > + svn_error_clear(err); > +} > + >

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Daniel Shahaf
Nick Piper wrote on Mon, May 09, 2011 at 11:21:10 +0100: > > > On 09/05/11 09:05, Markus Schaber wrote: > > Hi, Hick, > > > >> Von: Piper, Nick [mailto:nick.pi...@logica.com] > > > >> In Python, f is a "file" instance, but that will be converted to a > >> svn_stream_t by swig (by svn_swig_py_m

Re: AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Nick Piper
On 09/05/11 09:05, Markus Schaber wrote: > Hi, Hick, > >> Von: Piper, Nick [mailto:nick.pi...@logica.com] > >> In Python, f is a "file" instance, but that will be converted to a >> svn_stream_t by swig (by svn_swig_py_make_stream()). >> >> That file then goes out of scope in Python, but the fi

AW: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-09 Thread Markus Schaber
Hi, Hick, > Von: Piper, Nick [mailto:nick.pi...@logica.com] > In Python, f is a "file" instance, but that will be converted to a > svn_stream_t by swig (by svn_swig_py_make_stream()). > > That file then goes out of scope in Python, but the file descriptor is > never closed. I don't see where we

Re: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-07 Thread Nick Piper
On 07/05/11 08:27, Daniel Shahaf wrote: > Piper, Nick wrote on Sat, May 07, 2011 at 05:00:05 +: >> I've attached a script which reproduces the problem in a minimal way. > > The attachment never made it to the list... > > (bad MIME type?) Sorry for that, a copy is now at https://gist.github

Re: Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-07 Thread Daniel Shahaf
Piper, Nick wrote on Sat, May 07, 2011 at 05:00:05 +: > I've attached a script which reproduces the problem in a minimal way. The attachment never made it to the list... (bad MIME type?)

Python bindings: Use of svn_swig_py_make_stream() - avoiding leaking fds

2011-05-06 Thread Piper, Nick
Dear svn-devs, I'm trying to work through some repository corruption problems we're having, and one problem I've noticed during the investigation is that our Python application (running in mod_wsgi in Apache httpd) appears to be keeping (deleted) files open in /tmp. I've narrowed this down to