Re: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :)
Greg Ewing wrote: > Guido van Rossum wrote: > >> Its maps have methods to >> return keys, values and items, but these return neither new lists nor >> iterators; they return "views" which obey set (or multiset, in the >> case of items) semantics. > >> I'd like to explore this as an alternative to making keys() etc. >> return iterators. > > This sounds like a really really good idea! > > It would solve Jim's problem, because the result of > d.keys() would print out just like a real list, and > then he could backspace over the .keys() and do > something else. If these things returned iterables rather than actual iterators (i.e. similar to what xrange currently does), it would also address the fact that caching references to iterators just doesn't work. For example: keys = d.keys() big_key = max(keys) little_key = max(keys) If keys() returns a list or other iterable, all is good. If it returns an actual iterator though. . . I do find it somewhat interesting that we're considering moving the standard containers to a more numpy-ish view of the world, though (i.e. one where multiple mutable views of a data structure are common in order to avoid unnecessary copying) Would that philosophy be extended to slicing operations, though? Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-3000] Iterators for dict keys, values, and items == annoying :)
On 3/24/06, Nick Coghlan <[EMAIL PROTECTED]> wrote: > I do find it somewhat interesting that we're considering moving the standard > containers to a more numpy-ish view of the world, though (i.e. one where > multiple mutable views of a data structure are common in order to avoid > unnecessary copying) > > Would that philosophy be extended to slicing operations, though? That's a question I hadn't though about yet. I'm not sure I like it, but it sure is consistent with other view APIs; like everywhere, if you need a separate object, you can place a list() around it. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expose the array interface in Python 2.5?
On 3/17/06, Thomas Heller python.net> wrote: > Accessing Python arrays (Numeric arrays, Numeric array, or Numpy > array) as ctypes arrays, and vice versa, without copying any memory, > would be a good thing. This does bring up a point. I was thinking that a really bare-bones nd-array type would be very useful by itself as a way to exchange data. However, if ctypes were to return nd-arrays from C function calls, then we would want at least a rudimentary way of directly accessing the data. I'd sure like to see indexing and slicing, at at least. Also if ctypes were to allow nd-arrays as a way to pass data into a C function, then you'd need a build-in python way to create the data in the first place. If that's too much for now, I'd still like to have the basic data structure defined in the standard library (or built-in) as soon as possible. Greg Ewing wrote: > It might be all right for writers of new > extensions, but there are existing modules (PIL, ctypes, etc.) > that already have their own way of storing data, and it > seems to me it would be easier for the maintainers of > those modules to add a new interface to the existing > data than to rearrange their internal structure to use > this new C-object. Can we have both? A defined interface, that existing code can be adapted to provide, and a new C-Object, that future code can just use. If the goal is to have as many extension types as possible use the same base object, the sooner a standard object is provided the better. There are those of us in the scientific computing community that would love to just have numpy be part of the standard library. It really is big, and maybe too special purpose for that, but at least having the core array in there would be great. For years, I've been dealing with modules like wxPython that don't understand numpy arrays. As a result, passing in a list of points to be drawn is actually faster than passing in a numpy array of points, even though the numpy array of points has the data in the exact binary representation that wxWidgets expects. The problem is that the wrapper code doesn't know that, because Robin (quite reasonably) didn't want to have wxPython depend on Numeric. While a standard interface could support this, it would be great if wxPython could also do things like pass an image buffer back to Python efficiently. Another point is that n-dimensional arrays really are very useful for all sorts of stuff that have nothing to do with high-performance Numeric computing. I find myself using numpy for almost every little script I write, even though most of it is not performance bounded at all. I suspect that if we get a n-dimensional array type into Python, one that allows n-d slicing, it will see a LOT of use by people who now think they have no use for numpy. My guess is that a 2-dimensional object array would get the most use, but why not support n-d while you're at it? Having an easy and natural way to store and manipulate a "table" of objects, for instance, would be handy for many, many users. I'm still a tiny bit confused about the proposed individual pieces involved, but I'd like to see, in order of priority (and difficulty): 1) A standard n-d array interface (numpy already defines this, but outside of the numpy community, who knows about it?) 2) A standard, n-d array C-Object, inheritable for use by various extension packages. 3) A standard n-d array object, with a python interface for indexing and slicing for access and assignment (modeled after, or better yet taken directly from, numpy). 4) A standard n-d array object that supports array-wise arithmetic and functions (ufuncs, in numpy parlance). There is no reason it has to have anything to do with the current array module. For instance, having the arrays be of fixed size is just fine. It's really not that big a deal to make a new one when you need to change the size, after all, we live with that for immutable objects, like strings. just my $0.02 -Chris ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] howto return malloc()ed memory from C -> Python
Hi, I am wrapping libsndfile to Python, using SWIG and don't know howto return a malloc()ed memory area back to Python. I've found CObjects, which would let me pass free() as cleanup function, but I'd rather return a buffer object (PyBuffer_FromReadWriteMemory) so that I could prevent copying of data, but I don't know how to prevent the memory leak then. Can I install some kind of cleanup hook? Please also reply to me personally, as I am not on the python-dev list. Thanks, Uwe -- Conway's Law: In any organization there will always be one person who knows what is going on. This person must be fired. pgplOFkkG4VK3.pgp Description: PGP signature ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Fwd: Re: r43214 - peps/trunk/pep-3000.txt]
On 2006-03-22 at 20:22:58 [+0100], M.-A. Lemburg <[EMAIL PROTECTED]> wrote: > Charlie, > > you might want to reply to this one... > > Original Message > Subject: Re: [Python-Dev] r43214 - peps/trunk/pep-3000.txt > Date: Wed, 22 Mar 2006 19:37:45 +0100 > From: "Martin v. Löwis" <[EMAIL PROTECTED]> > To: M.-A. Lemburg <[EMAIL PROTECTED]> > CC: Michael Hudson <[EMAIL PROTECTED]>, [email protected],Fredrik > Lundh <[EMAIL PROTECTED]> > References: <[EMAIL PROTECTED]> > <[EMAIL PROTECTED]><[EMAIL PROTECTED]> > <[EMAIL PROTECTED]> > > M.-A. Lemburg wrote: > >> And we still have someone actively interested in maintaining the OS2 > >> port, it seems. > > > > Dito for BeOS, now under the name Zeta OS. > > Who is the one interested in maintaining the BeOS port? the last > checkins related to BeOS seem to originate from the 2001/2002 period. yellowTAB includes Python 2.4 as part of their ZETA OS which, as Marc-André notes is a further development of BeOS. Francois Revol has done the work to keep Python up to date and working with the existing maintainer, Donn Cave, on this. It's a much more solid release than all previous versions with much greater support for Python extensions. Even Zope runs with only one slight change as flock() is not yet supported. Charlie ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] r43214 - peps/trunk/pep-3000.txt
[bcc python-dev to move to python-3000] On 3/22/06, Michael Hudson <[EMAIL PROTECTED]> wrote: > "Fredrik Lundh" <[EMAIL PROTECTED]> writes: > > > neal.norwitz wrote: > > > >> +Outstanding Issues > >> +== > >> + > >> +* Require C99, so we can use // comments, named initializers, declare > >> variables > >> + without introducing a new scope, among other benefits. > > > > gcc only, in other words ? > > Heh, I was going to make this point as well: it's not clear that MSVC > will ever support C99. It supports some of the features listed here, > of course, but probably won't support everything. Note this is the outstanding issues section. So everything here is a question. By the time 3k is released, who knows what compilers will support. > >> +* Remove support for old systems, including: OS2, BeOS, RISCOS, (SGI) > >> Irix, Tru64 > > > > what's old with tru64 ? it's not that uncommon in places where Python > > has a strong presence, you can still buy AXP hardware throughout 2006, > > and HP says they'll keep developing and supporting the software platform > > at least through 2011. > > And we still have someone actively interested in maintaining the OS2 > port, it seems. OS2 was a mistake. I created this list a few days ago before Alan said he was interested in maintaining it. Tru64 is difficult (I think there are still some open bugs that go back years) because no developer has access to any of these boxes. It would be good for people interested in these platforms to speak up and offer their time or at least access to the platform so we can test. I haven't researched Tru64 status, but I remember hearing a bunch of times how it wasn't going to be supported. If we support these platforms, we still need to decide what versions. Again remember all of these are questions. n ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] howto return malloc()ed memory from C -> Python
On Wed, Mar 22, 2006, Uwe Mayer wrote: > > I am wrapping libsndfile to Python, using SWIG and don't know howto > return a malloc()ed memory area back to Python. Please post this question to comp.lang.python; it's not appropriate for python-dev. Note that you're unlikely to get many useful responses if you don't subscribe to c.l.py. -- Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expose the array interface in Python 2.5?
Travis E. Oliphant wrote: > Last year, during development of NumPy, the concept of the array > interface was developed as a means to share array_like data. It was > realized that it is this concept that needs to be explored further and > pushed into Python itself, rather than a particular incarnation of an > array. > > It is very important for many people to get access to memory with some > description of how that memory should be interpreted as an array. > Several Python packages could benefit if Python had some notion of an > array interface that was at least supported in a duck-typing fashion. I'm astonished that our BDFL doesn't speak up on this issue. IIRC, it was the (his?) plan to let the numpy community develop an interface, which would then be included into Python. > The description of what we've come up with so far and is implemented in > NumPy (and Numarray and last released Numeric) is at > > http://numeric.scipy.org/#array_interface > > Quite a few of us would love to see this get into Python proper, but > have very little free-time to spare to make sure it happens. > > Would it be possible to add at least the C-struct array interface to the > Python arrayobject in time for Python 2.5? > > Is someone on this list willing to help make it happen? What excatly would you like to see in the core? If I read the url above correctly, it would only be required to attach some attributes to existing array-like objects. Are ctypes and arrayobject the only candidates, or do you want these attributes on other data types as well (strings, unicode)? OTOH, with a 2.5 alpha release next week, it may be too late for anything... Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expose the array interface in Python 2.5?
On 3/24/06, Thomas Heller <[EMAIL PROTECTED]> wrote: > I'm astonished that our BDFL doesn't speak up on this issue. IIRC, > it was the (his?) plan to let the numpy community develop an interface, > which would then be included into Python. Where's the PEP? You don't expect me to argue the point for the numpy community do you? Once there's a PEP I'll read it and comment. Last I saw this thread Travis was asking for volunteers to write the PEP. Personally I think this is too close to the Python 2.5 release schedule to go ahead; but that's really up to the release managers. However, without a PEP there's an automatic "no". -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expose the array interface in Python 2.5?
Guido van Rossum wrote: > On 3/24/06, Thomas Heller <[EMAIL PROTECTED]> wrote: >> I'm astonished that our BDFL doesn't speak up on this issue. IIRC, >> it was the (his?) plan to let the numpy community develop an interface, >> which would then be included into Python. > > Where's the PEP? You don't expect me to argue the point for the numpy > community do you? Once there's a PEP I'll read it and comment. Last I > saw this thread Travis was asking for volunteers to write the PEP. > Personally I think this is too close to the Python 2.5 release > schedule to go ahead; but that's really up to the release managers. > However, without a PEP there's an automatic "no". > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) Ok, sounds fair. Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 343: A messy contextmanager corner case
I was just updating an experimental context library for the latest PEP 343 protocol, and I ran into a rather messy situation with the recently added error-suppression feature, when used in combination with @contextmanager, that caused some of my tests to fail. Basically, the problem is this. The way contextmanager is implemented in the PEP, __exit__ will always either throw an exception or return True in the exception case. It will never return a false value to reraise the original exception. The problem with this is that it requires chained __exit__ handlers to trap errors in any other __exit__ methods they call, unless contextmanager can somehow detect that the reraised exception is the same one, and suppress it. This case doesn't affect nominal use of a single contextmanager in a 'with' statement. It only affects contexts that need to call other contexts' __exit__ methods. But it seems to me that it might be a good idea to make GeneratorContextManager.__exit__() simply return False if the generator reraises an exception. That way, an __exit__() that raises an error can always be considered broken, which simplifies context chaining quite a bit. This would basically consist of adding another "except:" clause to the __exit__() method, to check if the raised value is the same value as the one that was passed in to throw(). The test_contextlib tests all pass with this change, but of course I would add a test to ensure that __exit__() returns False if the same value is reraised. I would also update the PEP to reflect the updated implementation, and to explain that __exit__() methods should not propagate exceptions unless they are unable to fulfill their contract. Any objections? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 342 support for string exceptions in throw()
Should geniter.throw() issue a deprecation warning for string exceptions? My first thought was yes, since that's what raise() does. On the other hand, one of the key motivating uses for throw() is to allow exception propagation on a coroutine stack. In this use case, the original "raise" of the string exception is what should be warned about. Issuing a warning for the line of code calling "throw()" is misleading, since that is not the line that is "wrong". So, here's what I propose to do instead. Rather than support arbitrary string exceptions, which are deprecated anyway, they should only be allowed in the case where a non-None traceback argument is provided. Thus, string exceptions could be *propagated* using throw(), but not *initiated*. Meanwhile, if you throw() a string exception with no traceback argument, you would receive the same error you do now. Any objections? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > Should geniter.throw() issue a deprecation warning for string exceptions? > > My first thought was yes, since that's what raise() does. > > On the other hand, one of the key motivating uses for throw() is to allow > exception propagation on a coroutine stack. In this use case, the original > "raise" of the string exception is what should be warned about. Issuing a > warning for the line of code calling "throw()" is misleading, since that is > not the line that is "wrong". > > So, here's what I propose to do instead. Rather than support arbitrary > string exceptions, which are deprecated anyway, they should only be allowed > in the case where a non-None traceback argument is provided. Thus, string > exceptions could be *propagated* using throw(), but not *initiated*. > > Meanwhile, if you throw() a string exception with no traceback argument, > you would receive the same error you do now. > > Any objections? -0. I think it's overkill to warn for any string exceptions thrown this way. Since the only use case for using throw() is to pass an exception you just caught, I don't see that putting the warning is useful -- it's just more code that in practice is never triggered. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Another PEP 343 contextmanager glitch
It appears that the contextmanager implementation (both in the stdlib and
as described in the PEP) doesn't correctly handle the case where the
generator yields a new value in response to throw(). It responds by
suppressing the original exception, but IIRC earlier versions of the PEP
threw a RuntimeError for this scenario.
Unless there are any objections, I'd like to restore this by replacing the
"return True" that follows the throw(), with:
raise RuntimeError("generator didn't stop after throw()")
and again, update the PEP accordingly.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
At 02:36 PM 3/24/2006 -0800, Guido van Rossum wrote: >I think it's overkill to warn for any string exceptions thrown this >way. Since the only use case for using throw() is to pass an exception >you just caught, I don't see that putting the warning is useful -- >it's just more code that in practice is never triggered. My proposal was that throw() should only succeed or fail, never warn. If you throw() a string exception with a traceback, it Just Works. If you throw() a string exception without a traceback, you get an immediate TypeError, just like in the 2.5 trunk now. Is that acceptable? i.e., was that what you were "-0"-ing? The only change is that throw() would now *accept* string exceptions without warning or error, if and only if you supply a traceback. That is, if you are effectively re-raising an existing exception. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 02:36 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I think it's overkill to warn for any string exceptions thrown this > >way. Since the only use case for using throw() is to pass an exception > >you just caught, I don't see that putting the warning is useful -- > >it's just more code that in practice is never triggered. > > My proposal was that throw() should only succeed or fail, never warn. > > If you throw() a string exception with a traceback, it Just Works. > > If you throw() a string exception without a traceback, you get an immediate > TypeError, just like in the 2.5 trunk now. > > Is that acceptable? i.e., was that what you were "-0"-ing? Actually that's slightly worse; I'm not sure we guarantee that exceptions you catch *always* have a traceback object associated with them. So -1. > The only change is that throw() would now *accept* string exceptions > without warning or error, if and only if you supply a traceback. That is, > if you are effectively re-raising an existing exception. The current state is that it always allows them, right? I don't see a reason to change that, and I don't think you've made a strong argument for changing it -- I don't think the use case matters. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote: >The current state is that it always allows them, right? No. It doesn't allow them. Support for string exceptions was never actually implemented; I'm trying to implement it now. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
Oops, that's definitely a bug. Please go ahead and fix in both places.
(I'm still trying to fathom the other PEP 343 issue you brought up.)
--Guido
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> It appears that the contextmanager implementation (both in the stdlib and
> as described in the PEP) doesn't correctly handle the case where the
> generator yields a new value in response to throw(). It responds by
> suppressing the original exception, but IIRC earlier versions of the PEP
> threw a RuntimeError for this scenario.
>
> Unless there are any objections, I'd like to restore this by replacing the
> "return True" that follows the throw(), with:
>
> raise RuntimeError("generator didn't stop after throw()")
>
> and again, update the PEP accordingly.
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote: > >The current state is that it always allows them, right? > > No. It doesn't allow them. Support for string exceptions was never > actually implemented; I'm trying to implement it now. Oops, I see. I'm +1 for allowing them, -1 for doing any kind of detection of situations that might not allow them in the future. (You're not warning about classic classes either are you?) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
At 03:06 PM 3/24/2006 -0800, Guido van Rossum wrote:
>On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> > At 03:04 PM 3/24/2006 -0800, Guido van Rossum wrote:
> > >The current state is that it always allows them, right?
> >
> > No. It doesn't allow them. Support for string exceptions was never
> > actually implemented; I'm trying to implement it now.
>
>Oops, I see. I'm +1 for allowing them, -1 for doing any kind of
>detection of situations that might not allow them in the future.
Okay - so allow them without warning or error, even if somebody does
'gen.throw("abc")'?
I was proposing to only add support for this if a non-None traceback is
supplied. I guess I'm confused as to how you could end up with
sys.exc_info()[2] not containing a traceback object.
The idea here was that the only valid use case I can think of for
throw()-ing a string exception is when using throw() to propagate a
previously-raised exception.
Since throw() is a new feature, it seems reasonable to me to not provide
string exception support -- *except* for backward compatibility needs, such
as propagating a string exception that was already raised elsewhere.
Thus, it's not a case of removing support for string exceptions, but rather
introducing a new feature that doesn't support creating new code that
raises string exceptions.
But, string exceptions definitely should be supported for propagation,
because you a coroutine could call some old code that uses string
exceptions, so you need to be able to re-throw() them to outer coroutines.
>(You're not warning about classic classes either are you?)
Nope.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
At 03:05 PM 3/24/2006 -0800, Guido van Rossum wrote: >Oops, that's definitely a bug. Please go ahead and fix in both places. Will do. >(I'm still trying to fathom the other PEP 343 issue you brought up.) Here's the paragraph I'm proposing to add to the PEP to explain that issue: NOTE: to facilitate chaining of contexts, __exit__() methods should *not* reraise the error that is passed in, because it is always the responsibility of the *caller* to reraise. If an __exit__() method exits with *any* exception, the caller of __exit__() should interpret this as meaning that the __exit__() method itself failed to perform the desired cleanup or other operations. Currently, the PEP is ambiguous on this point. When "with" calls __exit__() this point is irrelevant because either way, the same exception will be raised/propagated. But if you are writing Python code that explicitly uses the context protocol (e.g. the 'nested' context manager), it should be clear as to whether a given __exit__() invocation failed or succeeded. Thus, __exit__() should only raise an exception if it *failed*. (This then implies that the @contextmanager decorator must catch any re-raising done by the wrapped generator.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote:
> Okay - so allow them without warning or error, even if somebody does
> 'gen.throw("abc")'?
Correct. There's no current code that does this, and I doubt that few
people will write any, so I don't see the value of trying to trap or
warn for this case.
> I was proposing to only add support for this if a non-None traceback is
> supplied. I guess I'm confused as to how you could end up with
> sys.exc_info()[2] not containing a traceback object.
I'm not sure either -- but there's so much convoluted code that I
can't prove to myself it can't happen either.
> The idea here was that the only valid use case I can think of for
> throw()-ing a string exception is when using throw() to propagate a
> previously-raised exception.
>
> Since throw() is a new feature, it seems reasonable to me to not provide
> string exception support -- *except* for backward compatibility needs, such
> as propagating a string exception that was already raised elsewhere.
>
> Thus, it's not a case of removing support for string exceptions, but rather
> introducing a new feature that doesn't support creating new code that
> raises string exceptions.
>
> But, string exceptions definitely should be supported for propagation,
> because you a coroutine could call some old code that uses string
> exceptions, so you need to be able to re-throw() them to outer coroutines.
And I see no reason to complicate the code with an additional check
that doesn't serve a purpose. The purpose of reminding people not to
write g.throw("abc") seems artificial to me. I'd rather see less code,
meaning less maintenance, and no need to remove the check later (or
when I merge your changes into the p3yk branch).
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 03:05 PM 3/24/2006 -0800, Guido van Rossum wrote: > >Oops, that's definitely a bug. Please go ahead and fix in both places. > > Will do. > > > >(I'm still trying to fathom the other PEP 343 issue you brought up.) > > Here's the paragraph I'm proposing to add to the PEP to explain that issue: > > NOTE: to facilitate chaining of contexts, __exit__() methods should > *not* reraise the error that is passed in, because it is always the > responsibility of the *caller* to reraise. If an __exit__() method > exits with *any* exception, the caller of __exit__() should > interpret this as meaning that the __exit__() method itself > failed to perform the desired cleanup or other operations. > > Currently, the PEP is ambiguous on this point. When "with" calls > __exit__() this point is irrelevant because either way, the same exception > will be raised/propagated. > > But if you are writing Python code that explicitly uses the context > protocol (e.g. the 'nested' context manager), it should be clear as to > whether a given __exit__() invocation failed or succeeded. Thus, > __exit__() should only raise an exception if it *failed*. > > (This then implies that the @contextmanager decorator must catch any > re-raising done by the wrapped generator.) I guess I like the ambiguity -- to the outer __exit__, it shouldn't make any difference whether the exception was re-raised by the inner __exit__ or by the finally clause containing it. After all, if there *wasn't* an outer __exit__, there wouldn't be any difference to the user either, whether the re-raise came from __exit__ or from finally. If you still disagree, can you produce a test case that's currently broken? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
At 03:27 PM 3/24/2006 -0800, Guido van Rossum wrote: >I guess I like the ambiguity -- to the outer __exit__, it shouldn't >make any difference whether the exception was re-raised by the inner >__exit__ or by the finally clause containing it. After all, if there >*wasn't* an outer __exit__, there wouldn't be any difference to the >user either, whether the re-raise came from __exit__ or from finally. > >If you still disagree, can you produce a test case that's currently broken? I discovered the issue when I updated to the latest implementation and it broke the tests for a context manager I had written. This context manager is basically a transaction manager that accumulates context managers for resources used in the transaction, then calls all their __exit__() methods from its __exit__. It has to be able to distinguish between an __exit__() that failed (which means a critical failure of the overall transaction!) and an exit that's merely allowing the original exception to propagate (which means that the other handlers should still be invoked, and that everything's working normally). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 03:27 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I guess I like the ambiguity -- to the outer __exit__, it shouldn't > >make any difference whether the exception was re-raised by the inner > >__exit__ or by the finally clause containing it. After all, if there > >*wasn't* an outer __exit__, there wouldn't be any difference to the > >user either, whether the re-raise came from __exit__ or from finally. > > > >If you still disagree, can you produce a test case that's currently broken? > > I discovered the issue when I updated to the latest implementation and it > broke the tests for a context manager I had written. This context manager > is basically a transaction manager that accumulates context managers for > resources used in the transaction, then calls all their __exit__() methods > from its __exit__. It has to be able to distinguish between an __exit__() > that failed (which means a critical failure of the overall transaction!) > and an exit that's merely allowing the original exception to propagate > (which means that the other handlers should still be invoked, and that > everything's working normally). It seems you're proposing something that is *not* equivalent to with A: with B: ... since in that case a failure of the inner __exit__ (whether a re-raise or a real failure) would still invoke the outer __exit__. Is that a good idea? I'm not against recommending in the PEP that __exit__ shouldn't re-raise but instead should return False to signal a re-raise, and fixing any existing code that re-raises in __exit__. But I'm still questioning your use case; why is it important not to call the outer __exit__ methods in your case? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
At 03:23 PM 3/24/2006 -0800, Guido van Rossum wrote:
>And I see no reason to complicate the code with an additional check
>that doesn't serve a purpose. The purpose of reminding people not to
>write g.throw("abc") seems artificial to me. I'd rather see less code,
>meaning less maintenance, and no need to remove the check later
I think you're still thinking as though there was support, and I'm
proposing adding new code to restrict the support. There is *no* support
for string exceptions now, so we are going to be adding code, period. :)
For comparison purposes, here is the patch for doing it my way:
Index: Objects/genobject.c
===
--- Objects/genobject.c (revision 43297)
+++ Objects/genobject.c (working copy)
@@ -249,7 +249,10 @@
Py_INCREF(typ);
}
}
- else {
+
+ /* Allow reraise of builtin string exceptions if traceback was given */
+
+ else if (!PyString_CheckExact(typ) || !tb) {
/* Not something you can raise. throw() fails. */
PyErr_Format(PyExc_TypeError,
"exceptions must be classes, or instances,
not %s",
And now, here's the patch for doing it your way:
Index: Objects/genobject.c
===
--- Objects/genobject.c (revision 43297)
+++ Objects/genobject.c (working copy)
@@ -249,7 +249,10 @@
Py_INCREF(typ);
}
}
- else {
+
+ /* Allow raising builtin string exceptions */
+
+ else if (!PyString_CheckExact(typ)) {
/* Not something you can raise. throw() fails. */
PyErr_Format(PyExc_TypeError,
"exceptions must be classes, or instances,
not %s",
As you can see, there are only a few characters difference between the two,
so I'm not sure there's a significant maintenance overhead involved either
way. Indeed, we've probably used more time discussing it than will ever be
spent maintaining it. :)
> (or
>when I merge your changes into the p3yk branch).
Actually, you won't need to merge them if Python 3000 doesn't support
string exceptions. The code you've already got should be correct for that,
because again, there is *no* support for string exceptions in the trunk
right now.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
OK, I give up. I'm giving you a -1 on the test for a non-empty traceback without explaining it further. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
At 03:50 PM 3/24/2006 -0800, Guido van Rossum wrote: >I'm not against recommending in the PEP that __exit__ shouldn't >re-raise but instead should return False to signal a re-raise, and >fixing any existing code that re-raises in __exit__. But I'm still >questioning your use case; why is it important not to call the outer >__exit__ methods in your case? I didn't say it was important not to call them; I said it was important to be able to *tell* whether any of the __exit__ methods had actually failed, since this indicates a critical failure of transaction rollback. That is, there was some part of the transaction that could not be rolled back. This is a distinct issue from whatever caused the transaction to be aborted. Also, the __exit__ methods that failed should be logged, but not the __exit__ methods that are simply propagating an exception after performing their rollback behavior. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 342 support for string exceptions in throw()
At 03:58 PM 3/24/2006 -0800, Guido van Rossum wrote: >OK, I give up. I'm giving you a -1 on the test for a non-empty >traceback without explaining it further. Not a problem; I just wanted to make sure you were rejecting the same thing that I was proposing. :) I've checked in the "your way" version. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 03:50 PM 3/24/2006 -0800, Guido van Rossum wrote: > >I'm not against recommending in the PEP that __exit__ shouldn't > >re-raise but instead should return False to signal a re-raise, and > >fixing any existing code that re-raises in __exit__. But I'm still > >questioning your use case; why is it important not to call the outer > >__exit__ methods in your case? > > I didn't say it was important not to call them; I said it was important to > be able to *tell* whether any of the __exit__ methods had actually failed, > since this indicates a critical failure of transaction rollback. That is, > there was some part of the transaction that could not be rolled back. This > is a distinct issue from whatever caused the transaction to be > aborted. Also, the __exit__ methods that failed should be logged, but not > the __exit__ methods that are simply propagating an exception after > performing their rollback behavior. You could probably tell the difference by comparing the exception it raises to the exception you pass it. But by all means update the PEP to discourage or disallow re-raising and fix any re-raising code you find. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
At 04:34 PM 3/24/2006 -0800, Guido van Rossum wrote: >You could probably tell the difference by comparing the exception it >raises to the exception you pass it. That's what I changed @contextmanager to do, because it's the only place where you have a good reason to re-raise (i.e., the generator re-raises in order to request propagation). Thus, this inspection code should only have to appear in one place: @contextmanager, rather than it having to be in every other context that needs to tell the difference. >But by all means update the PEP >to discourage or disallow re-raising and fix any re-raising code you >find. Done. By the way, I just got approval earlier this week to work on various Python 2.5 tasks as a "20% project" at OSAF, so today's my day this week to try to burn through as many outstanding Python 2.5 issues as possible. Hence all my activity on PEPs 342 and 343. My current rough plans for wsgiref, setuptools and the rest can be found at: http://peak.telecommunity.com/DevCenter/Python25 if anybody's interested. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Expose the array interface in Python 2.5?
Chris Barker wrote: > I'm still a tiny bit confused about the proposed individual pieces > involved, but I'd like to see, in order of priority (and difficulty): > > 1) A standard n-d array interface (numpy already defines this, but > outside of the numpy community, who knows about it?) This is pretty much a matter of taking Travis's array interface and checking it in as a PEP. Checking it in is the easy part, but I'm not clear on what actually constitutes the array interface. The array interface is apparently here: http://numeric.scipy.org/array_interface.html but it only talks about exposing this info to Python code - it doesn't discuss how to expose or use it at the C level (unless the intent is that it should be accessed via the PyObject abstract API, in which case a PEP would need to say that explicitly). However, the top of that page references the genarray PEP: http://svn.scipy.org/svn/PEP/PEP_genarray.txt Here it gets even *more* confusing, because the dimarray objects in that PEP don't appear to expose the array interface described by the page above. If this PEP is out of date, then the reference needs to be removed until it is brought up to speed (a disclaimer on the draft PEP probably wouldn't hurt either). A useful PEP for a C-level array interface would need to cover the following three things that an extension author would need to know in order to either produce or consume generic arrays: 1. The C-level protocol for exposing and retrieving the array interface (potentially using the PyObject API to access Python-level attributes) 2. How to use the array interface to access an extension type's data 3. How to use the array interface to modify an extension type's data This is doable in whatever time frame we like, since it is documenting a convention, rather than an implementation, and so isn't coupled directly to the release cycle. However, if it's available soon (before the second alpha?), it may be possible to update arrayobject and ctypes to expose this interface for Python 2.5. > 2) A standard, n-d array C-Object, inheritable for use by various > extension packages. > > 3) A standard n-d array object, with a python interface for indexing and > slicing for access and assignment (modeled after, or better yet taken > directly from, numpy). For the sake of test writers' sanity, these would probably need to be done together so that at least some of the tests could be written in Python. Due to the conflicting slice semantics between the standard library (slices are copies) and numpy (slices are mutable views), I'd actually suggest that the Python interface for this simple object *shouldn't* support slicing (at least, not in its first incarnation). Regardless, I can't see either of these steps happening before Python 2.6. If we try to do step 2 without doing step 3, then we're stuck either writing all the test code in C, or else shipping an untested component. Neither seems like a good idea. > 4) A standard n-d array object that supports array-wise arithmetic and > functions (ufuncs, in numpy parlance). Definitely not before 2.6 :) > There is no reason it has to have anything to do with the current array > module. For instance, having the arrays be of fixed size is just fine. > It's really not that big a deal to make a new one when you need to > change the size, after all, we live with that for immutable objects, > like strings. The array module is the easiest place to put the code - then you would have "array.array" as a 1 dimensional resizeable array, and "array.dimarray" as a multi-dimensional array. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://www.boredomandlaziness.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
On 3/24/06, Phillip J. Eby <[EMAIL PROTECTED]> wrote: > At 04:34 PM 3/24/2006 -0800, Guido van Rossum wrote: > >You could probably tell the difference by comparing the exception it > >raises to the exception you pass it. > > That's what I changed @contextmanager to do, because it's the only place > where you have a good reason to re-raise (i.e., the generator re-raises in > order to request propagation). > > Thus, this inspection code should only have to appear in one place: > @contextmanager, rather than it having to be in every other context that > needs to tell the difference. Ah. After reading what you checked in I understand it better. The issue is the slight impedance mismatch between throw() and __exit__(): throw() always has to re-raise the exception, because returning a value already has a different meaning (the generator yielded again). That's what I originally implemented for __exit__() after I realized the need for a way to swallow exceptions, but then after reasonable opposition that this would make writing a "vanilla" __exit__() a bit tricky in a way that encourages buggy code, I changed it to returning True to swallow the exception, and returning False *or* re-raising to re-raise. (returning in any way also meant to continue a non-local goto.) I think that the code you added: +except: +if sys.exc_info()[1] is not value: +raise could use a comment explaining what's going on; the difference between returning None and re-raising the exception is really quite subtle here, and only ever matters if something else besides the with-statement framework is calling __exit__. (I would add this distinction to the PEP too.) > By the way, I just got approval earlier this week to work on various Python > 2.5 tasks as a "20% project" at OSAF, so today's my day this week to try to > burn through as many outstanding Python 2.5 issues as possible. Hence all > my activity on PEPs 342 and 343. My current rough plans for wsgiref, > setuptools and the rest can be found at: > > http://peak.telecommunity.com/DevCenter/Python25 > > if anybody's interested. Great news! I'm glad you're planning to finish up the PEP 302 stuff. I'd like to see some discussion of your proposed change to .pth file processing; this could cause some surprises and will slow down all Python's start-up time due to the necessity to list every directory on sys.path. This is IMO a bad thing; there must be nothing to keep people from writing tiny scripts in Python, and slow startup time is an inhibitor there. (If you don't believe this could be a problem, try it with a long path containing several directories on NFS. This is the standard setup of Google developer workstations, and it gives one a different outlook on life.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Another PEP 343 contextmanager glitch
At 08:13 PM 3/24/2006 -0800, Guido van Rossum wrote: >I think that the code you added: > >+except: >+if sys.exc_info()[1] is not value: >+raise > >could use a comment explaining what's going on; Done. > the difference between >returning None and re-raising the exception is really quite subtle >here, and only ever matters if something else besides the >with-statement framework is calling __exit__. (I would add this >distinction to the PEP too.) Also done. >I'd like to see some discussion of your proposed change to .pth file >processing; this could cause some surprises and will slow down all >Python's start-up time due to the necessity to list every directory on >sys.path. This is IMO a bad thing; there must be nothing to keep >people from writing tiny scripts in Python, and slow startup time is >an inhibitor there. Okay, I'll cross that off then. Setuptools will just have to keep using its "install a special site.py on PYTHONPATH" trick to support PYTHONPATH installs. > (If you don't believe this could be a problem, try >it with a long path containing several directories on NFS. This is the >standard setup of Google developer workstations, and it gives one a >different outlook on life.) Ouch. All that computing power and you can't even copy files to your own machines? Tsk, tsk. ;) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
