Re: [Python-Dev] pdb segfaults in 2.5 trunk?

[Neal Norwitz]

On 4/10/06, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 4/10/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> >
> > It appears the problem is an object/mem mismatch: both PyOS_Readline in
> > pgenmain.c, and PyOS_StdioReadline use PyObject_MALLOC, but bltinmodule.c
> > is freeing the pointer with PyMem_FREE.
>
> This (Readline using PyObject) was due to my recent change to fix
> Guido's problem last night.
> I didn't realize anything seeped out.  All calls to PyOS_StdioReadline
> would need to be updated. I can do that tonight.  Hmm, that means this
> will be an API change.  I wonder if I should revert my fix and just
> deal with a second alloc and copy of the data to fix Guido's original
> problem.

I partially reverted my fix from last night.  It appears to work for
both Guido's original problem and Phillip's subsequent problem.  YMMV.
 It would be great if someone could review all the PyMem_* and
PyObject_* allocs and frees to ensure consistency.

I wonder if the code would be clearer if the encoding was changed back
to using PyObject_*.  Then there would only be a few clear cases for
using PyMem_* in Parser/.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Py_Finalize does not release all memory, not even closely

[Martin v. Löwis]

Several places in the documentation claim that Py_Finalize will
release all memory:

http://docs.python.org/api/embedding.html

says that literally, and

http://docs.python.org/api/initialization.html#l2h-778

suggests it is a bug when memory is not freed in Py_Finalize.

This has left people to believe that this is a bug:

https://sourceforge.net/tracker/index.php?func=detail&aid=1445210&group_id=5470&atid=105470

However, I don't see any chance to make this promise even remotely.
Objects allocated in extension modules, and held in global variables
(e.g. socketmodule.c:socket_error, socket_herror, socket_gaierror,
socket_timeout) will never be released, right?

And because of the small objects allocator, their pool will remain
allocated, right? And, then, the arena.

So ISTM that invoking Py_Finalize after importing socket will yield
atleast 256KiB garbage. Of course, that's not real garbage, because
the next Py_Initialize'd interpreter will continue to allocate from
the arenas. But still, the actual objects that the modules hold
on to will not be reclaimed until the process terminates.

Please correct me if I'm wrong.

Regards,
Martin

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] DRAFT: python-dev summary for 2006-03-01 to 2006-03-15

[Thomas Heller]

Steven Bethard wrote:
> Ok, if I summarize any more of python-dev, my brain's going to explode. ;-)
> 
> Here's the summaries for the first half of March.  Let me know what to fix!
> 


> ---
> Maintaining ctypes in SVN trunk
> ---
> 
> Thomas Heller put ctypes into the Python SVN repository, and with the
> help of perky, Neal Norwitz and Thomas Wouters, updated it to take
> advantage of the new ssize_t feature. Thomas Heller has now moved the
> "official" ctypes development to the Python SVN.

The last sentence is not correct, sorry if the discussion led you to
this conclusion.  At least for some time, the "official version" will
stay in sourceforge CVS, because it is easier to test ctypes on the SF
compile farm.

Thanks for the summaries,

Thomas

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_Finalize does not release all memory, not even closely

[Guido van Rossum]

I'm afraid it was wishful thinking on my part.

The best we can try to hope for is to ensure that repeatedly calling
Py_Initialize and Py_Finalize doesn't leak too much memory.

--Guido

On 4/11/06, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> Several places in the documentation claim that Py_Finalize will
> release all memory:
>
> http://docs.python.org/api/embedding.html
>
> says that literally, and
>
> http://docs.python.org/api/initialization.html#l2h-778
>
> suggests it is a bug when memory is not freed in Py_Finalize.
>
> This has left people to believe that this is a bug:
>
> https://sourceforge.net/tracker/index.php?func=detail&aid=1445210&group_id=5470&atid=105470
>
> However, I don't see any chance to make this promise even remotely.
> Objects allocated in extension modules, and held in global variables
> (e.g. socketmodule.c:socket_error, socket_herror, socket_gaierror,
> socket_timeout) will never be released, right?
>
> And because of the small objects allocator, their pool will remain
> allocated, right? And, then, the arena.
>
> So ISTM that invoking Py_Finalize after importing socket will yield
> atleast 256KiB garbage. Of course, that's not real garbage, because
> the next Py_Initialize'd interpreter will continue to allocate from
> the arenas. But still, the actual objects that the modules hold
> on to will not be reclaimed until the process terminates.
>
> Please correct me if I'm wrong.
>
> Regards,
> Martin
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_Finalize does not release all memory, not even closely

[Martin v. Löwis]

Guido van Rossum wrote:
> I'm afraid it was wishful thinking on my part.
> 
> The best we can try to hope for is to ensure that repeatedly calling
> Py_Initialize and Py_Finalize doesn't leak too much memory.

Ok. Unless somebody still claims to the contrary, I will go ahead
and weaken the promises in the documentation. Then it's not a bug
anymore that it leaks :-)

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Py_Finalize does not release all memory, not even closely

[Tim Peters]

[Martin v. Löwis]
> Several places in the documentation claim that Py_Finalize will
> release all memory:
>
> http://docs.python.org/api/embedding.html
>
> says that literally,

It's wrong ;-).

> and
>
> http://docs.python.org/api/initialization.html#l2h-778
>
> suggests it is a bug when memory is not freed in Py_Finalize.
>
> This has left people to believe that this is a bug:
>
> 

Well, there may well be a bug (or multiple bugs) underlying that one
too.  It's one thing for Py_Finalize() not to release all memory (it
doesn't and probably never will), but it's not necessarily the same
thing if running Py_Initialize() ... Py_Finalize() repeatedly keeps
leaking more and more memory.

> However, I don't see any chance to make this promise even remotely.
> Objects allocated in extension modules, and held in global variables
> (e.g. socketmodule.c:socket_error, socket_herror, socket_gaierror,
> socket_timeout) will never be released, right?

Not unless the module has a finalization function called by
Py_Finalize() that frees such things (like PyString_Fini and
PyInt_Fini).

Other globals allocated via a

static PyObject *someglobal = NULL;

...

if (someglobal == NULL)
someglobal = allocate_an_object_somehow();

pattern shouldn't contribute to continuing leaks across
Py_Initialize() ... Py_Finalize() loops.

> And because of the small objects allocator, their pool will remain
> allocated, right?  And, then, the arena.

Before Python 2.5, arenas are never freed, period.  In Python 2.5, an
arena will be freed if and only if it contains no allocated object by
the time Py_Finalize ends.

There may also be trash cycles that aren't collected during
Py_Finalize because I had to comment out Py_Finalize's second call to
PyGC_Collect(); new-style class objects are among the trash thingies
leaked (although if Py_Initialize() is called again, it's possible
that they'll get cleaned up by cyclic gc after all).

See Misc/SpecialBuilds.txt, section Py_TRACE_REFS, entry
PYTHONDUMPREFS, for a way to get a dump of all heap objects
Py_Finalize leaves alive.  I doubt that's been run in years; Guido and
I used it in 2.3b1 to cure some unreasonably high finalization leakage
at the time.

> So ISTM that invoking Py_Finalize after importing socket will yield
> atleast 256KiB garbage. Of course, that's not real garbage, because
> the next Py_Initialize'd interpreter will continue to allocate from
> the arenas.

I'm not clear on whether, e.g., init_socket() may get called more than
once if socket-slinging code appears in a Py_Initialize() ...
Py_Finalize().  If it doesn't, then, e.g., the unconditional

socket_gaierror = PyErr_NewException("socket.gaierror", ...);

won't contribute to ongoing leaks.  But if it does, then we'll
systematically leak the exception object on each loop trip.

> But still, the actual objects that the modules hold on to will not be 
> reclaimed
> until the process terminates.

Without a _Fini() function called by Py_Finalize, that's correct.

> Please correct me if I'm wrong.

If you ever are, I will ;-)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Phillip J. Eby]

I just noticed that two stdlib modules (runpy and test.test_importhooks) 
contain reimplementations of the base PEP 302 algorithm, or loaders 
wrapping the standard (pre-302) import machinery.

Meanwhile, the 'imp' module exports an undocumented IMP_HOOK constant 
(since Python 2.3), that is used internally to Python/import.c but never 
actually returned from the imp API.

Further, the machinery called by imp.find_module() actually does the full 
PEP 302 search dance - but then skips any PEP 302 importers in the process, 
because the wrapper doesn't let it return a loader object.

What I'd like to do is make the necessary modifications to import.c that 
would allow you to access the loaders found by the C version of find_module.

I propose to create a new API, 'imp.find_loader()' and have it return a PEP 
302-compatible loader object, even for cases that would normally be handled 
via 'imp.load_module()'.  In such cases, the loader returned would be an 
instance of one of a loader class similar to those in runpy, 
test_importhooks, and setuptools (which also has similar code).

What I'm not sure of is where to put the loader class.  It seems to me 
there should be a stdlib module, but it doesn't seem worth writing in C, 
especially with so many Python implementations floating around.

I could create a new Python module for them, but we already have so many 
import-related modules floating around.  Would it be reasonable to add them 
to 'pkgutil', which until now has contained only one function?  This would 
help cut down on some of the code duplication, without adding yet another 
module to the stdlib.

An additional issue: "pydoc" needs to be able to determine what submodules, 
if any, exist in a package, but the PEP 302 protocol does not provide for 
this ability.  I'd like to add optional additional methods to the PEP 302 
"importer" protocol (and to any stdlib importer objects) to support the 
type of filesystem-like queries performed by pydoc.  This should allow 
pydoc to be compatible with zip files as well as regular files, and any 
future PEP 302 importers that provide the necessary features.

Having these features would also let me cut some code out of setuptools' 
"pkg_resources" module, that adds some of these features using adapter 
registries.

It may be too late for me to be able to implement all of this in time for 
alpha 2, but at minimum I think the 'find_loader()' addition and the move 
of the import wrapper classes to 'pkgutil' could be accomplished.

Replacing pydoc's myriad stat(), listdir() and other path hacking with 
clean importer protocols might be out of reach, but I think it's worth a 
try.  For one thing, it would mean that shipping the Python stdlib in a 
zipfile would not inhibit pydoc's ability to display help for stdlib 
modules and packages.  (Currently, pydoc does not work correctly with 
packages that are in zipfiles or which are spread across multiple 
directories, and it is unable to "discover" modules that are in zipfiles.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pdb segfaults in 2.5 trunk?

[Delaney, Timothy (Tim)]

Neal Norwitz wrote:

> I partially reverted my fix from last night.  It appears to work for
> both Guido's original problem and Phillip's subsequent problem.  YMMV.
>  It would be great if someone could review all the PyMem_* and
> PyObject_* allocs and frees to ensure consistency.

Definitely seems to me that it would be worthwhile in debug mode adding
a field specifying which memory allocator was used, and checking for
mismatches in the deallocators.

I know this has been suggested before, but with number of mismatches
being found now it seems like it should be put into place. I'm sure it
will cause buildbot to go red ... ;)

I might see if I can work up a patch over the easter long weekend if no
one beats me to it. What files should I be looking at (it would be my
first C-level python patch)?

Tim Delaney
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Nick Coghlan]

Phillip J. Eby wrote:
> I propose to create a new API, 'imp.find_loader()' and have it return a PEP 
> 302-compatible loader object, even for cases that would normally be handled 
> via 'imp.load_module()'.  In such cases, the loader returned would be an 
> instance of one of a loader class similar to those in runpy, 
> test_importhooks, and setuptools (which also has similar code).

runpy tries to retrieve "imp.get_loader()" before falling back to its own 
emulation - switching that to look for "find_loader()" instead would be fine 
(although I'd probably leave the emulation in place, since it allows the 
module to be used on 2.4 as well as 2.5).

Also, 'find' more accurately reflects the search effort that goes on.

> What I'm not sure of is where to put the loader class.  It seems to me 
> there should be a stdlib module, but it doesn't seem worth writing in C, 
> especially with so many Python implementations floating around.
> 
> I could create a new Python module for them, but we already have so many 
> import-related modules floating around.  Would it be reasonable to add them 
> to 'pkgutil', which until now has contained only one function?  This would 
> help cut down on some of the code duplication, without adding yet another 
> module to the stdlib.

Would it be worth the effort to switch 'imp' to being a hybrid module?

> An additional issue: "pydoc" needs to be able to determine what submodules, 
> if any, exist in a package, but the PEP 302 protocol does not provide for 
> this ability.  I'd like to add optional additional methods to the PEP 302 
> "importer" protocol (and to any stdlib importer objects) to support the 
> type of filesystem-like queries performed by pydoc.  This should allow 
> pydoc to be compatible with zip files as well as regular files, and any 
> future PEP 302 importers that provide the necessary features.

runpy needs a get_filename() method, so it knows what to set __file__ too - 
currently its emulation supports that, but it isn't officially part of the PEP 
302 API.

> Having these features would also let me cut some code out of setuptools' 
> "pkg_resources" module, that adds some of these features using adapter 
> registries.
> 
> It may be too late for me to be able to implement all of this in time for 
> alpha 2, but at minimum I think the 'find_loader()' addition and the move 
> of the import wrapper classes to 'pkgutil' could be accomplished.
> 
> Replacing pydoc's myriad stat(), listdir() and other path hacking with 
> clean importer protocols might be out of reach, but I think it's worth a 
> try.  For one thing, it would mean that shipping the Python stdlib in a 
> zipfile would not inhibit pydoc's ability to display help for stdlib 
> modules and packages.  (Currently, pydoc does not work correctly with 
> packages that are in zipfiles or which are spread across multiple 
> directories, and it is unable to "discover" modules that are in zipfiles.)

FWIW, I've already had this experience between the original version of runpy, 
and the PEP 302 based version. As I recall, the "real" code in that module is 
now only a few dozen lines - everything else is there to work around the 
absence of imp.find_loader(). Prior to the rewrite, I don't think the module 
was that much shorter overall, and it only supported file system based 
single-directory packages.

Cheers,
Nick.

-- 
Nick Coghlan   |   [EMAIL PROTECTED]   |   Brisbane, Australia
---
 http://www.boredomandlaziness.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposal: expose PEP 302 facilities via 'imp' and 'pkgutil'

[Phillip J. Eby]

At 07:43 AM 4/12/2006 +1000, Nick Coghlan wrote:
>Phillip J. Eby wrote:
> > I propose to create a new API, 'imp.find_loader()' and have it return a 
> PEP
> > 302-compatible loader object, even for cases that would normally be 
> handled
> > via 'imp.load_module()'.  In such cases, the loader returned would be an
> > instance of one of a loader class similar to those in runpy,
> > test_importhooks, and setuptools (which also has similar code).
>
>runpy tries to retrieve "imp.get_loader()" before falling back to its own
>emulation - switching that to look for "find_loader()" instead would be fine
>(although I'd probably leave the emulation in place, since it allows the
>module to be used on 2.4 as well as 2.5).

I was proposing to move the emulation features (at least the classes) to 
pkgutil, or in the alternative, making them a 
published/documented/supported API, rather than private.


>Would it be worth the effort to switch 'imp' to being a hybrid module?

I have no idea; what's a hybrid module?


>runpy needs a get_filename() method, so it knows what to set __file__ too -
>currently its emulation supports that, but it isn't officially part of the 
>PEP
>302 API.

It sounds like maybe a new PEP is needed to document all the extensions to 
the importer/loader protocols.  :(

Interestingly, these are all more examples of where adaptation or generic 
functions would be handy to have in the stdlib.  Duck-typing protocols 
based on attribute names are hard to extend and have to wait for new 
library releases for upgrades.  :(

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pdb segfaults in 2.5 trunk?

[Tim Peters]

[Delaney, Timothy (Tim)]
> Definitely seems to me that it would be worthwhile in debug mode adding
> a field specifying which memory allocator was used, and checking for
> mismatches in the deallocators.
>
> I know this has been suggested before, but with number of mismatches
> being found now it seems like it should be put into place. I'm sure it
> will cause buildbot to go red ... ;)
>
> I might see if I can work up a patch over the easter long weekend if no
> one beats me to it. What files should I be looking at (it would be my
> first C-level python patch)?

A couple weeks back Adam DePrince said he was going to do this,
although I haven't heard more about it.  See that thread for hints
about a workable approach:

http://mail.python.org/pipermail/python-dev/2006-March/062848.html

The bulk of the work would be in obmalloc.c.  More-or-less
excruciating #if'ery would also be needed in objimpl.h and pymem.h, to
remap the build-type-independent API names to appropriate
build-type-dependent concrete calls.  For example, the current
debug-build:

#define PyMem_MALLOCPyObject_MALLOC

would have to get messier, so that a call to PyMem_MALLOC could be
distinguished from a call to PyObject_MALLOC to begin with.  They'd
probably both have to change, to call a common doesn't-yet-exist entry
point with a "which flavor of malloc is this?" flag argument.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PyObject_REPR()

[Thomas Wouters]

I hadn't noticed this macro defined in Include/object.h before:/* Helper for passing objects to printf and the like */#define PyObject_REPR(obj) PyString_AS_STRING(PyObject_Repr(obj))I may not be right, but I don't see how this can't help but not free the intermediate PyString object. It doesn't seem to be used, except for in situations where Python is not going to continue working much longer anyway (specifically, in 
compile.c and ceval.c.) I could not be right about *that*, too, though ;-) It strikes me that it should not be used, or maybe renamed to _PyObject_REPR. It's wasn't added in 2.5 or 2.4, though, so it's not particularly new and I can't guarantee that it's not used in any third party code. But, then again, anyone using it isn't free of leaks. Should removing or renaming it be done in 
2.5 or in Py3K?Triple-negative'ly y'rs,-- Thomas Wouters <[EMAIL PROTECTED]>Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_REPR()

[Raymond Hettinger]

> I may not be right, but I don't see how this can't help but not free the 
> intermediate PyString object.

Good catch.

> It doesn't seem to be used, except for in situations where Python is not 
> going 
> to continue working
> much longer anyway (specifically, in compile.c and ceval.c.)

Those internal uses ought to be replaced with better code.


> It strikes me that it should not be used, or maybe renamed to _PyObject_REPR.
> It's wasn't added in 2.5 or 2.4, though, so it's not particularly new and I 
> can't guarantee
> that it's not used in any third party code. But, then again, anyone using it 
> isn't free of leaks.
> Should removing or renaming it be done in 2.5 or in Py3K?

Since it is intrinsically buggy, I would support removal in Py2.5


Raymond 

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_REPR()

[Neal Norwitz]

On 4/11/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>
> > It strikes me that it should not be used, or maybe renamed to 
> > _PyObject_REPR.
> > Should removing or renaming it be done in 2.5 or in Py3K?
>
> Since it is intrinsically buggy, I would support removal in Py2.5

+1 on removal.  Google only turned up a handleful of uses that I saw.

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_REPR()

[Jeremy Hylton]

It's intended as an internal debugging API.  I find it very convenient
for adding with a printf() here and there, which is how it got added
long ago.  It should really have a comment mentioning that it leaks
the repr object, and starting with an _ wouldn't be bad either.

Jeremy

On 4/11/06, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> On 4/11/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> >
> > > It strikes me that it should not be used, or maybe renamed to 
> > > _PyObject_REPR.
> > > Should removing or renaming it be done in 2.5 or in Py3K?
> >
> > Since it is intrinsically buggy, I would support removal in Py2.5
>
> +1 on removal.  Google only turned up a handleful of uses that I saw.
>
> n
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] building with C++

[Anthony Baxter]

I've done a lot of the work to get Python to build with g++ - this was 
suggested a while ago as a useful thing to take advantage of C++'s 
more stringent type checking. There's a few more little bits and 
pieces to be done, and it looks like a bunch of extern "C" {}s need 
to be sprinkled through the code as a form of magic pixie dust. I 
can't spend much more time on this now, but if someone wants to take 
over and finish it off, that'd be great. Note that I have only 
addressed errors, not warnings - there's a bunch of odd looking 
warnings around the place, such as the code in 
PyLong_AsUnsignedLongLong that returns a -1 despite the return value 
being declared as unsigned... 

Anthony
-- 
Anthony Baxter <[EMAIL PROTECTED]>
It's never too late to have a happy childhood.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Preserving the blamelist

[Tim Peters]

All the trunk buildbots started failing about 5 hours ago, in
test_parser.  There have been enough checkins since then that the
boundary between passing and failing is about to scroll off forever. 
So, here's the blamelist before it vanishes:

"""
   1.

  Changed by: thomas.wouters
  Changed at: Wed 12 Apr 2006 00:06:36
  Branch: trunk
  Revision: 45285

  Changed files:
  * trunk/Grammar/Grammar
  Comments:

  Fix SF bug #1466641: multiple adjacent 'if's in listcomps and
genexps, as in
  [x for x in it if x if x], were broken for no good reason by the PEP 308
  patch.


   2.

  Changed by: thomas.wouters
  Changed at: Wed 12 Apr 2006 00:08:00
  Branch: trunk
  Revision: 45286

  Changed files:
  * trunk/Lib/test/test_grammar.py
  * trunk/Python/graminit.c
  Comments:

  Part two of the fix for SF bug #1466641: Regenerate graminit.c
and add test
  for the bogus failure.
"""

I'd whine about not checking buildbot health after a code change,
except that it's much more tempting to point out that Thomas couldn't
have run the test suite successfully on his own box in this case :-)

While I'm spreading guilt, a bunch of filepath-related tests started
failing Monday on the "g4 osx.4 trunk" buildbot, but the blamelist for
that one is long gone.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Neal Norwitz]

On 4/11/06, Tim Peters <[EMAIL PROTECTED]> wrote:
>
> I'd whine about not checking buildbot health after a code change,
> except that it's much more tempting to point out that Thomas couldn't
> have run the test suite successfully on his own box in this case :-)

Tsk, tsk, Thomas.  Should be fixed now.  When the grammar changes,
Modules/parsermodule.c also needs to be updated.

> While I'm spreading guilt, a bunch of filepath-related tests started
> failing Monday on the "g4 osx.4 trunk" buildbot, but the blamelist for
> that one is long gone.

I didn't realize that.  Once the tests are passing again, I can try to
take a look.

I'm concerned about the negative ref leak in test_contextlib.  I
wonder if this was a result of PJE's fix for generators?

n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PyObject_REPR()

[Neal Norwitz]

Ok, then how about prefixing with _, adding a comment saying in big,
bold letters: FOR DEBUGGING PURPOSES ONLY, THIS LEAKS, and only
defining in a debug build?

n
--
On 4/11/06, Jeremy Hylton <[EMAIL PROTECTED]> wrote:
> It's intended as an internal debugging API.  I find it very convenient
> for adding with a printf() here and there, which is how it got added
> long ago.  It should really have a comment mentioning that it leaks
> the repr object, and starting with an _ wouldn't be bad either.
>
> Jeremy
>
> On 4/11/06, Neal Norwitz <[EMAIL PROTECTED]> wrote:
> > On 4/11/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> > >
> > > > It strikes me that it should not be used, or maybe renamed to 
> > > > _PyObject_REPR.
> > > > Should removing or renaming it be done in 2.5 or in Py3K?
> > >
> > > Since it is intrinsically buggy, I would support removal in Py2.5
> >
> > +1 on removal.  Google only turned up a handleful of uses that I saw.
> >
> > n
> > ___
> > Python-Dev mailing list
> > Python-Dev@python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: 
> > http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
> >
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Preserving the blamelist

[Tim Peters]

[Neal Norwitz]
> ...
> I'm concerned about the negative ref leak in test_contextlib.  I
> wonder if this was a result of PJE's fix for generators?

I don't know, but if you do

while 1:
test_contextlib.test_main()
gc.collect()

under a debug build it eventually (> 500 iterations) crashes with

Fatal Python error: deallocating None

That's a pretty good clue ;-)  The

Py_XDECREF(gen->gi_frame);

in gen_dealloc() is on the call stack at the time, and cyclic gc is active.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] IMPORTANT 2.5 API changes for C Extension Modules and Embedders

[Neal Norwitz]

If you don't write or otherwise maintain Python Extension Modules
written in C (or C++) or embed Python in your application,
you can stop reading.

Python 2.5 alpha 1 was released April 5, 2006.  The second alpha
should be released in a few weeks.  There are several changes
which can cause C extension modules or embedded applications
to crash the interpreter if not fixed.  Periodically, I will send out
these reminders with updated information until 2.5 is released.

  * support for 64-bit sequences (eg, > 2GB strings)
  * memory allocation modifications

64-bit changes
--
There are important changes that are in 2.5 to support 64-bit systems.
The 64-bit changes can cause Python to crash if your module is not upgraded
to support the changes.  Python was changed internally to use 64-bit
values on 64-bit machines for indices.  If you've got a machine with
more than 16 GB of RAM, it would be great if you can test Python with
large (> 2GB) strings and other sequences.

For more details about the Python 2.5 schedule:
http://www.python.org/dev/peps/pep-0356/
For more details about the 64-bit change:
http://www.python.org/dev/peps/pep-0353/
How to fix your module:
http://www.python.org/dev/peps/pep-0353/#conversion-guidelines

The effbot wrote a program to check your code and find potential
problems with the 64-bit APIs.
http://svn.effbot.python-hosting.com/stuff/sandbox/python/ssizecheck.py

Memory Allocation Modifications
---
In previous versions of Python, it was possible to use different
families of APIs (PyMem_* vs. PyObject_*) to allocate and free
the same block of memory.  APIs in these families include:

  PyMem_*:PyMem_Malloc, PyMem_Realloc, PyMem_Free,
  PyObject_*: PyObject_Malloc, PyObject_Realloc, PyObject_Free

There are a few other APIs with similar names and also the macro variants.

In 2.5, if allocate a block of memory with one family, you must reallocate
or free with the same family.  That means:

  If you allocate with PyMem_Malloc (or MALLOC), you must reallocate
  with PyMem_Realloc (or REALLOC) and free with PyMem_Free (or FREE).
  If you allocate with PyObject_Malloc (or MALLOC), you must reallocate
  with PyObject_Realloc (or REALLOC) and free with PyObject_Free (or FREE).

Using inconsistent APIs can cause double frees or otherwise crash
the interpreter.  It is fine to mix and match functions or macros
within the same family.

Please test and upgrade your extension modules!

Cheers,
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com