Re: [Python-Dev] SRE should release the GIL (was: no subject)
Martin v. Löwis wrote: > Formally: no; it access a Python string/Python unicode object all > the time. > > Now, since all the shared objects it accesses are immutable, likely > no harm would be done releasing the GIL. I think SRE was originally > also intended to operate on array.array objects; this would have > caused bigger problems. SRE can operate on anything that implements the buffer interface. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Problems with the Python Memory Manager
Hi Jim, You wrote: > >(2) Is he allocating new _types_, which I think don't get properly > > collected. (Off-topic) For reference, as far as I know new types are properly freed. There has been a number of bugs and lots of corner cases to fix, but I know of no remaining one. This assumes that the new types are heap types allocated in some official way -- either by Python code or by somehow calling type() from C. A bientot, Armin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reference leaks
Neal Norwitz <[EMAIL PROTECTED]> writes: > There are still a few reference leaks I've been able to identify. I > didn't see an obvious solution to these (well, I saw one obvious > solution which crashed, so obviously I was wrong). > > When running regrtest with -R here are the ref leaks reported: > > test_codeccallbacks leaked [2, 2, 2, 2] references > test_compiler leaked [176, 242, 202, 248] references > test_generators leaked [254, 254, 254, 254] references > test_tcl leaked [35, 35, 35, 35] references > test_threading_local leaked [36, 36, 28, 36] references > test_urllib2 leaked [-130, 70, -120, 60] references > > test_compiler and test_urllib2 are probably not real leaks, but data > being cached. I'm not really sure if test_tcl is a leak or not. > Since there's a lot that goes on under the covers. I didn't see > anything obvious in _tkinter.c. > > I have no idea about test_threading_local. It's very odd, but probably not a leak. > I'm pretty certain test_codeccallbacks and test_generators are leaks. Isn't test_codeccallbacks just the extra references you get from registering an error handler? test_generators is new, I think. Cheers, mwh -- Good? Bad? Strap him into the IETF-approved witch-dunking apparatus immediately!-- NTK now, 21/07/2000 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reference leaks
Hi Neal,
On Thu, Nov 24, 2005 at 07:35:06PM -0800, Neal Norwitz wrote:
> The itertools.tee() fix I thought was quite obvious:
>
> +++ Modules/itertoolsmodule.c (working copy)
> @@ -356,7 +356,8 @@
> {
> if (tdo->nextlink == NULL)
> tdo->nextlink = teedataobject_new(tdo->it);
> - Py_INCREF(tdo->nextlink);
> + else
> + Py_INCREF(tdo->nextlink);
> return tdo->nextlink;
> }
No, if this object is saved as a cache on 'tdo' then obviously it needs
to keep a reference on its own. This reference will go away in
teedataobject_dealloc().
After debugging, the problem is a reference cycle: the teedataobject
'head' has a field 'it' pointing to the generator-iterator '_fib()',
which has a reference back to 'head'. So what is missing is making
teedataobject GC-aware, which it current isn't.
I suspect that there are other itertools types in the same situation.
A bientot,
Armin.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reference leaks
Neal Norwitz wrote:
> [...]
> test_codeccallbacks
>
> import codecs
> def test_callbacks():
> def handler(exc):
> l = [u"<%d>" % ord(exc.object[pos]) for pos in xrange(exc.start, exc.end)]
> return (u"[%s]" % u"".join(l), exc.end)
> codecs.register_error("test.handler", handler)
> # the {} is necessary to cause the leak, {} can hold data too
> codecs.charmap_decode("abc", "test.handler", {})
>
> test_callbacks()
> # leak from PyUnicode_DecodeCharmap() each time test_callbacks() is called
Can you move the call to codecs.register_error() out of test_callbacks()
and retry?
Bye,
Walter Dörwald
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Bug bz2.BZ2File(...).seek(0,2) + patch
Hi,
I found a bug in bz2 python module. Example:
import bz2
bz2.BZ2File("test.bz2","r")
bz2.seek(0,2)
assert bz2.tell() != 0
Details and *patch* at:
http://sourceforge.net/tracker/index.php?func=detail&aid=1366000&group_id=5470&atid=105470
Please CC-me for all your answers.
Bye, Victor
--
Victor Stinner - student at the UTBM (Belfort, France)
http://www.haypocalc.com/wiki/Accueil
signature.asc
Description: This is a digitally signed message part
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] SRE should release the GIL (was: no subject)
Hi all, I've implemented a patch, please visit bug 1366311 for details. https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1366311&group_id=5470 This patch only release the GIL when the engine perform a low level search *and* if the object searched is a string or a unicode string. The GIL will not be released for any other kind of objects, as there is no guarantee of immutability of the buffer during the run. I've tested this with a couple of simple tests, and also by running the application Duncan talked about. My testing indicates that everything works as before with the added value that our application is still responsive even when processing some of the more egregious regular expressions. As it is my first foray into python module writing I'll welcome any feedback you may have on the patch. Regards, -- Eric On 11/25/05, Fredrik Lundh <[EMAIL PROTECTED] > wrote: Martin v. Löwis wrote:> Formally: no; it access a Python string/Python unicode object all> the time.>> Now, since all the shared objects it accesses are immutable, likely> no harm would be done releasing the GIL. I think SRE was originally > also intended to operate on array.array objects; this would have> caused bigger problems.SRE can operate on anything that implements the buffer interface. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug bz2.BZ2File(...).seek(0,2) + patch
On Fri, Nov 25, 2005, Victor STINNER wrote: > > I found a bug in bz2 python module. Example: > > Details and *patch* at: > http://sourceforge.net/tracker/index.php?func=detail&aid=1366000&group_id=5470&atid=105470 Thanks! Particularly with the Thanksgiving weekend, you may not get any other responses for a while. Please be patient. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "If you think it's expensive to hire a professional to do the job, wait until you hire an amateur." --Red Adair ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] reference leaks
On 11/25/05, Walter Dörwald <[EMAIL PROTECTED]> wrote:
>
> Can you move the call to codecs.register_error() out of test_callbacks()
> and retry?
It then leaks 3 refs on each call to test_callbacks().
n
--
>>> import codecs
[24540 refs]
>>>
[24541 refs]
>>> def handler(exc):
... l = [u"<%d>" % ord(exc.object[pos]) for pos in xrange(exc.start, exc.end)]
... return (u"[%s]" % u"".join(l), exc.end)
...
[24575 refs]
>>> codecs.register_error("test.handler", handler)
[24579 refs]
>>>
[24579 refs]
>>> def test_callbacks():
... # the {} is necessary to cause the leak
... codecs.charmap_decode("abc", "test.handler", {})
...
[24604 refs]
>>> test_callbacks()
[24608 refs]
>>> test_callbacks()
[24611 refs]
>>> test_callbacks()
[24614 refs]
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
