Re: [Python-Dev] Adding content to exception messages
Nicolas Fleury wrote:
> I do a lot of exception re-raising at work; I use that
> technique to add content to exception messages while
> keeping the original stack. I even created a reraise
> function that I use that way:
>
> try:
> parser.parseFile(file)
> exeption Exception, exception:
> reraise(exception,
> "Error at line %s in file %s" % (x,y))
>
> (x,y) are details, but you get the idea.
>
With PEP 344, this could simply be:
try:
parser.parseFile(file)
exeption Exception, exception:
raise type(exception)("Error at line %s in file %s" % (x,y))
Introspectively,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Simpler finalization semantics (was Re: PEP 343 - Abstract Block Redux)
Michael Chermside wrote:
> If I write
>
> with foo:
>BLOCK
>
> where I should have written
>
> with locked(foo):
>BLOCK
>
> ...it silently "succeeds" by doing nothing. I CLEARLY intended to
> do the appropriate cleanup (or locking, or whatever), but it doesn't
> happen.
Ah, thanks. Like Guido, I had something in the back of my head saying it didn't
like the idea, but I couldn't figure out the reason. I think you just nailed it.
Plus, there is a nice alternative which is to provide a 'clean it up if it
needs
it' resource in the standard library:
class resource(object):
def __init__(self, obj):
self.obj = obj
self.enter = getattr(obj, "__enter__", None)
self.exit = getattr(obj, "__exit__", None)
def __enter__(self):
if self.enter is not None:
self.enter()
# For consistency, always return the object
return self.obj
def __exit__(self, *exc_info):
if self.exit is not None:
self.exit(*exc_info)
Then 'I don't know if this needs cleaning up or not' can be written:
with resource(foo):
# If foo needs cleaning up, it will be.
A refinement would provide the option to specify the enter/exit methods
directly:
class resource(object):
def __init__(self, obj, *other_args):
self.obj = obj
if other_args:
if len(other_args) != 2:
raise TypeError("need 1 or 3 arguments")
self.enter = args[0]
self.exit = None
self.exit_no_args = args[1]
else:
self.enter = getattr(obj, "__enter__", None)
self.exit = getattr(obj, "__exit__", None)
self.exit_no_args = None
def __enter__(self):
if self.enter is not None:
self.enter()
# For consistency, always return the object
return self.obj
def __exit__(self, *exc_info):
if self.exit is not None:
self.exit(*exc_info)
elif self.exit_no_args is not None:
self.exit()
That would let any object with a standard 'clean me up method' be easily used
in
a with statement:
with resource(bar, None, bar.clear):
# bar will be cleared when we're done
Cheers,
Nick.
--
Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia
---
http://boredomandlaziness.blogspot.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
Guido van Rossum wrote: > I believe that in the discussion about PEP 343 vs. Nick's PEP 3XX > (http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html, still > awaiting PEP moderator approval I believe?) It turns out my submission email took the scenic route, and I wasn't using a proper text editor so the formatting of the raw text version is messed up. Given that I need to clean that formatting up, I figure I might as well update it in light of recent discussions before I resubmit it. > the main difference is > that Nick proposes a way to inject an exception into a generator; and > I've said a few times that I like that idea. If the current generator integration is dropped from PEP 343, I can rewrite my PEP to be *just* about the combination of PEP 288 and PEP 325 you describe, and use PEP 343 integration as a motivating use case. The alternative would be to just submit and immediately withdraw it (since the user defined statement semantics now match PEP 343, and I basically like the generator interface you are proposing here, there wouldn't be much left of my PEP except for the big 'Rejected Options' section giving my understanding of the reasons we didn't take up various options). > - g.close() throws a GeneratorExit exception in the generator, and > catches it (so g.close() itself does not raise an exception). > g.close() is idempotent -- if the generator is already closed, it is a > no-op. If the generator, against the rules, yields another value, it > is nevertheless marked closed. Can't we make the method raise a RuntimeError when the generator breaks the rules? Or should we just print a warning instead (like the deletion code does if __del__ raises an exception)? > - When a generator is GC'ed, its close() method is called (which is a > no-op if it is already closed). This is like giving it a __del__ method though, since it can now resurrect arbitrary objects from a cycle it is involved in. I think you made the right call elsewhere, when you convinced me that generator's shouldn't have a __del__ method any more than files should. Cheers, Nick. -- Nick Coghlan | [EMAIL PROTECTED] | Brisbane, Australia --- http://boredomandlaziness.blogspot.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding content to exception messages
On Thu, 19 May 2005, Nick Coghlan wrote:
> With PEP 344, this could simply be:
>
>try:
>parser.parseFile(file)
>exeption Exception, exception:
>raise type(exception)("Error at line %s in file %s" % (x,y))
Only if we also made all exceptions new-style classes.
That's just a minor nit, though. The more important question to me is:
Do you care about the difference between a secondary exception that was
raised intentionally (as in your example) and a secondary exception due
to a problem in exception handling?
(For me, the answer is yes.)
-- ?!ng
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 344: Exception Chaining and Embedded Tracebacks
On Monday 16 May 2005 22:41, Ka-Ping Yee wrote: > http://www.python.org/peps/pep-0344.html | 2. Whenever an exception is raised, if the exception instance does | not already have a '__context__' attribute, the interpreter sets | it equal to the thread's exception context. Should that be "if the exception instance does not already have a __context__ attribute or the value of that attribute is None" -- Toby Dickenson ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - New kind of yield statement?
On 5/19/05, Greg Ewing <[EMAIL PROTECTED]> wrote: > Michael Hudson wrote: > > > This is, to me, neat and clear. I don't find the idea that iterators > > are tied to exactly 1 for loop an improvement (even though they > > usually will be). > > To fix this in a fully backward-compatible way, we > need some way of distinguishing generators that > expect to be finalized. I don't see anything that needs to be "fixed" here. Sure, generators that expect to be finalised will not be finalised simply by the fact that a for loop exits, but that's fine - it's not part of the spec of a for loop that it does finalise the generator. Adding that guarantee to a for loop is a change in spec, not a fix. Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 343 - New kind of yield statement?
At 04:44 PM 5/19/2005 +1200, Greg Ewing wrote: >Michael Hudson wrote: > > > This is, to me, neat and clear. I don't find the idea that iterators > > are tied to exactly 1 for loop an improvement (even though they > > usually will be). > >To fix this in a fully backward-compatible way, we >need some way of distinguishing generators that >expect to be finalized. No, we don't; Guido's existing proposal is quite sufficient for using yield. We dont' need to create another old/new distinction here. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
At 06:09 PM 5/19/2005 +1200, Greg Ewing wrote: >Guido van Rossum wrote: > > > - When a generator is GC'ed, its close() method is called (which is a > > no-op if it is already closed). > >Does this mean that all generators will be ineligible >for cyclic garbage collection (since they implicitly >have something equivalent to a __del__ method)? No, since it's implemented in C. (The C equivalent to __del__ does not interfere with cyclic GC.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] python-dev Summary for 2005-05-01 through 2005-05-15 [draft]
Here's the first draft of the python-dev summary for the first half of
May. Please send any corrections or suggestions to the summarizers (CC'ed).
==
Summary Announcements
==
--
PEP 340 Episode 2: Revenge of the With (Block)
--
This fortnight's Python-Dev was dominated again by another nearly 400
messages on the topic of anonymous block statements. The discussion
was a little more focused than the last thanks mainly to Guido's
introduction of `PEP 340`_. Discussion of this PEP resulted in a
series of other PEPs, including
* `PEP 342`_: Enhanced Iterators, which broke out into a separate
PEP the parts of `PEP 340`_ that allowed code to pass values into
iterators using ``continue EXPR`` and yield-expressions.
* `PEP 343`_: Anonymous Block Redux, a dramatically simplified
version of `PEP 340`_, which removed the looping nature of the
anonymous blocks and the injection-of-exceptions semantics for
generators.
* `PEP 3XX`_: User Defined ("with") Statements, which proposed
non-looping anonymous blocks accompanied by finalization semantics
for iterators and generators in for loops.
Various details of each of these proposals are discussed below in the
sections:
1. `Enhanced Iterators`_
2. `Separate APIs for Iterators and Anonymous Blocks`_
3. `Looping Anonymous Blocks`_
4. `Loop Finalization`_
At the time of this writing, it looked like the discussion was coming
very close to a final agreement; `PEP 343`_ and `PEP 3XX`_ both agreed
upon the same semantics for the block-statement, the keyword had been
narrowed down to either ``do`` or ``with``, and Guido had agreed to
add back in to `PEP 343`_ some form of exception-injection semantics
for generators.
.. _PEP 340: http://www.python.org/peps/pep-0340.html
.. _PEP 342: http://www.python.org/peps/pep-0342.html
.. _PEP 343: http://www.python.org/peps/pep-0343.html
.. _PEP 3XX: http://members.iinet.net.au/~ncoghlan/public/pep-3XX.html
[SJB]
=
Summaries
=
--
Enhanced Iterators
--
`PEP 340`_ incorporated a variety of orthogonal features into a single
proposal. To make the PEP somewhat less monolithic, the method for
passing values into an iterator was broken off into `PEP 342`_. This
method includes:
* updating the iterator protocol to use .__next__() instead of .next()
* introducing a new builtin next()
* allowing continue-statements to pass values into iterators
* allowing generators to receive values with a yield-_expression_
Though these features had seemed mostly uncontroversial, Guido seemed
inclined to wait for a little more motivation from the co-routiney
people before accepting the proposal.
Contributing threads:
- `Breaking off Enhanced Iterators PEP from PEP 340 `__
[SJB]
Separate APIs for Iterators and Anonymous Blocks
`PEP 340`_ had originally proposed to treat the anonymous block
protocol as an extension of the iterator protocol. Several problems
with this approach were raised, including:
* for-loops could accidentally be used with objects requiring blocks,
meaning that resources would not get cleaned up properly
* blocks could be used instead of for-loops, violating TOOWTDI
As a result, both `PEP 343`_ and `PEP 3XX`_ propose decorators for
generator functions that will wrap the generator object appropriately
to match the anonymous block protocol. Generator objects without the
proposed decorators would not be usable in anonymous block statements.
Contributing threads:
- `PEP 340 -- loose ends `__
- `PEP 340 -- concept clarification `__
[SJB]
Looping Anonymous Blocks
A few issues arose as a result of `PEP 340`_'s formulation of
anonymous blocks as a variation on a loop.
Because the anonymous blocks of `PEP 340`_ were defined in terms of
while-loops, there was some discussion as to whether they should have
an ``else`` clause like Python ``for`` and ``while`` loops do. There
didn't seem to be one obvious interpretation of an ``else`` block
though, so Guido rejected the ``else`` block proposal.
The big issue with looping anonymous blocks, however, was in the
handling of ``break`` and ``continue`` statements. Many use cases for
anonymous blocks did not require loops. However, because `PEP 340`_
anonymous blocks were implemented in terms of loops, ``break`` and
``continue`` acted much like they would in a loop. This meant that in
code like::
for item in items:
with lock:
if handle(item):
break
the ``break`` statement would only break out of the anonymo
Re: [Python-Dev] Adventures with Decimal
On 5/18/05, Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> >>> from decimal import getcontext
> >>> context = getcontext()
> >>> x = context.create_decimal('3.104')
> >>> y = context.create_decimal('2.104')
> >>> z = context.create_decimal('0.000')
> >>> context.prec = 3
> >>> x + y
> Decimal("5.21")
> >>> x + z + y
> Decimal("5.20")
My point here is to always remind everybody that Decimal solves the
problem with binary floating point, but not with representation
issues. If you don't have enough precision (for example to represent
one third), you'll get misterious results.
That's why, IMO, the Spec provides two traps, one for Rounded, and one
for Inexact, to be aware of what exactly is happening.
> As for why the normal Decimal constructor is context free, PEP 327
> indicates discussion on the subject, but who made the decision and why
> is not clear.
There was not decision. Originally the context didn't get applied in
creation time. And then, the situation arised where it would be nice
to be able to apply it in creation time (for situations when it would
be costly to not do it), so a method in the context was born.
.Facundo
Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal construction
On 5/18/05, Aahz <[EMAIL PROTECTED]> wrote: > Not sure what the "right" answer is, but I wanted to stick my oar in to > say that I think that Decimal has not been in the field long enough or > widely-enough used that we should feel that the API has been set in > stone. If there's agreement that a mistake was made, let's fix it! +1. BTW, it's worth noting that for Money (http://sourceforge.net/projects/pymoney) we decided to apply the context at creation time .Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal construction
> Not sure what the "right" answer is, but I wanted to stick my oar in to > say that I think that Decimal has not been in the field long enough or > widely-enough used that we should feel that the API has been set in > stone. If there's agreement that a mistake was made, let's fix it! There is not agreement. I prefer the current behavior and think changing it would introduce more problems than it would solve. Further, the API currently provides both context aware and context free construction -- all the tools needed are already there. Let's leave this alone and simply document the best practices (using unary plus after a precision change and constructing using create_decimal whenever context is important to construction). Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
On May 19, 2005, at 8:43 AM, Phillip J. Eby wrote: > At 06:09 PM 5/19/2005 +1200, Greg Ewing wrote: >> Guido van Rossum wrote: >>> - When a generator is GC'ed, its close() method is called (which >>> is a >>> no-op if it is already closed). >> >> Does this mean that all generators will be ineligible >> for cyclic garbage collection (since they implicitly >> have something equivalent to a __del__ method)? > > No, since it's implemented in C. (The C equivalent to __del__ does > not > interfere with cyclic GC.) But you're missing the point -- there's a *reason* that __del__ interferes with cyclic GC. It doesn't just do it for the heck of it! You can't simply have the C delete call into python code...the objects the generator has references to may be invalid objects already because they've been cleared to break a cycle. If you want to execute python code on collection of a generator, it must be done via __del__, or else it'll be horribly, horribly broken. James ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
At 12:36 PM 5/19/2005 -0400, James Y Knight wrote: >On May 19, 2005, at 8:43 AM, Phillip J. Eby wrote: >>At 06:09 PM 5/19/2005 +1200, Greg Ewing wrote: >>>Guido van Rossum wrote: - When a generator is GC'ed, its close() method is called (which is a no-op if it is already closed). >>> >>>Does this mean that all generators will be ineligible >>>for cyclic garbage collection (since they implicitly >>>have something equivalent to a __del__ method)? >> >>No, since it's implemented in C. (The C equivalent to __del__ does >>not >>interfere with cyclic GC.) > >But you're missing the point -- there's a *reason* that __del__ >interferes with cyclic GC. It doesn't just do it for the heck of it! >You can't simply have the C delete call into python code...the >objects the generator has references to may be invalid objects >already because they've been cleared to break a cycle. If you want to >execute python code on collection of a generator, it must be done via >__del__, or else it'll be horribly, horribly broken. Eeyowch. Good point. OTOH, the only way a generator-iterator can become part of a cycle is via an action taken outside the generator. (E.g. passing it into itself via 'continue', creating a link from one of its arguments to it, etc.) So, it's probably not a terrible limitation in practice. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[James Y Knigh] > But you're missing the point -- there's a *reason* that __del__ > interferes with cyclic GC. It doesn't just do it for the heck of it! > You can't simply have the C delete call into python code...the > objects the generator has references to may be invalid objects > already because they've been cleared to break a cycle. If you want to > execute python code on collection of a generator, it must be done via > __del__, or else it'll be horribly, horribly broken. Thank you for reminding me -- that's indeed the reason, and it applies here. I think in the past I've unsuccessfully tried to argue that if a cycle contains exactly one object with a Python-invoking finalizer, that finalizer could be invoked before breaking the cycle. I still think that's a sensible proposal, and generators may be the use case to finally implement it. All this suggests that generators should indeed have a __del__() method which is synonymous with close() (I want close() to be the user-facing API). BTW I think that close() and __del__() should raise an exception when the throw(GeneratorExit) call doesn't end up either re-raising GeneratorExit or raising StopIteration. The framework for calling __del__() takes care of handling this exception (by printing and then ignoring it). Raymond take notice if you're still working on the PEP. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[Phillip J. Eby]the only way a generator-iterator can become > part of a cycle is via an action taken outside the generator. (E.g. > passing it into itself via 'continue', creating a link from one of its > arguments to it, etc.) So, it's probably not a terrible limitation in > practice. It's enough to store a reference to the generator in a global (or in anything that's reachable from a global). The generator's frame's f_globals pointer then ensures the cycle. Throwing an exception also provides ample opportunity for creating cycles, since the frame hold a reference to the most recent traceback. Ironically, throwing an exception with a traceback into a generator is likely to cause a cycle because the traceback likely references the throwing frame, which certainly has a reference to the generator... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325:generator exceptions and cleanup
> BTW I think that close() and __del__() should raise an exception when > the throw(GeneratorExit) call doesn't end up either re-raising > GeneratorExit or raising StopIteration. The framework for calling > __del__() takes care of handling this exception (by printing and then > ignoring it). Raymond take notice if you're still working on the PEP. Got it. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
At 10:09 AM 5/19/2005 -0700, Guido van Rossum wrote: >[Phillip J. Eby]the only way a generator-iterator can become > > part of a cycle is via an action taken outside the generator. (E.g. > > passing it into itself via 'continue', creating a link from one of its > > arguments to it, etc.) So, it's probably not a terrible limitation in > > practice. > >It's enough to store a reference to the generator in a global (or in >anything that's reachable from a global). The generator's frame's >f_globals pointer then ensures the cycle. Well, if the generator was defined in the same module, I suppose. This would probably only happen with short scripts, where the lack of GC is unlikely to be an issue. However, at least it would also be possible to explicitly close the generator, which wasn't possible before. >Throwing an exception also provides ample opportunity for creating >cycles, since the frame hold a reference to the most recent traceback. >Ironically, throwing an exception with a traceback into a generator is >likely to cause a cycle because the traceback likely references the >throwing frame, which certainly has a reference to the generator... *head exploding* Double ouch. Wait a minute... those cycles don't include the generator, do they? Let me think. Frame A has a reference to the generator iterator, and invokes throw() (directly or indirectly) on it. Frame B, the generator frame, gets its f_back set to point to Frame A, but presumably that link is cleared on exit? (If it isn't, it probably should be). Anyway, frame B throws an exception, and the traceback is created. The traceback has a reference to frame B. We return to frame A, and add it to the traceback as well, and a reference to the traceback goes into the frame too. Hm. Still no cycle passing through the *generator iterator*, unless the generator's frame's f_back is still pointing to the frame that it was last called from. This holds even if the generator's frame holds the traceback that was current at the time of the error, because that traceback only includes the generator's frame, not the caller's frame. So, as long as the generator guarantees its frame's f_back is empty while the generator is not actually executing, the cycle should not include the generator, so it will still be GC'able. (Note, by the way, that this property is probably another good reason for using the yield expression as the source location for a throw()!) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fwd: Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
Here's the word on GC vs. __del__, by the way. -- Forwarded message -- From: Tim Peters <[EMAIL PROTECTED]> Date: May 19, 2005 10:24 AM Subject: Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup To: [EMAIL PROTECTED] > Remind me. Why again is GC afraid to touch objects with a __del__? > (There's a good reason, it's so subtle I keep forgetting it and I > can't seem to reconstruct the argument from first principles this > morning.) tp_clear is called on the objects in a cycle in an arbitrary order, and it's possible for a __del__ method to (e.g.) resurrect any object in the cycle. But obj.tp_clear() doesn't necessarily leave obj in a sane state, so we could end up resurrecting insane objects. > Would the same reasoning apply to a generator that's part of a cycle if > deleting the generator would cause more Python code to run? The general rule now is that gc must guarantee that no object it decided is trash can be reachable from any Python code by the time a gc pass first calls tp_clear. Calling tp_clear can certainly trigger __del__ methods (and weakref callbacks), so it's not (a common misunderstanding) the rule that __del__ methods can't run at all during gc. The real rule is subtler than that: gc is happy to run a __del__ method (or wr callback), provided that no trash is reachable from such a method. I haven't had the bandwidth to follow this discussion, but I sure _suppose_ that other trash objects could be reachable from a trash generator in a cycle. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[Guido] > ... > I think in the past I've unsuccessfully tried to argue that if a > cycle contains exactly one object with a Python-invoking finalizer, > that finalizer could be invoked before breaking the cycle. I still > think that's a sensible proposal, and generators may be the use case > to finally implement it. You have argued it, and I've agreed with it. The primary hangup is that there's currently no code capable of doing it. gc currently determines the set of objects that must be part of cyclic trash, or reachable only from cyclic trash, but has no relevant knowledge beyond that. For example, it doesn't know the difference between an object that's in a trash cycle, and an object that's not in a trash cycle but is reachable only from trash cycles. In fact, it doesn't know anything about the cycle structure. That would require some sort of new SCC (strongly connected component) analysis. The graph derived from an arbitrary object graph by considering each SCC to be "a node" is necessarily a DAG (contains no cycles), and the general way to approach what you want here is to clear trash in a topological sort of the SCC DAG: so long as an SCC contains only one object that may execute Python code, it's safe to run that object's cleanup code first (for a meaning of "safe" that may not always coincide with "explainable" or "predictable" <0.9 wink>). gc would probably need to give up after the first such thingie is run, if any SCCs are reachable from the SCC X containing that thingie (gc can no longer be sure that successor SCCs _are_ still trash: they too are reachable from X, so may have been resurrected by the Python code X ran). There's currently no forcing at all of the order in which tp_clear gets called, and currently no analysis done sufficient to support forcing a relevant ordering. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[Phillip J. Eby] > >Throwing an exception also provides ample opportunity for creating > >cycles, since the frame hold a reference to the most recent traceback. > >Ironically, throwing an exception with a traceback into a generator is > >likely to cause a cycle because the traceback likely references the > >throwing frame, which certainly has a reference to the generator... > > *head exploding* Double ouch. > > Wait a minute... those cycles don't include the generator, do they? Let > me think. Frame A has a reference to the generator iterator, and invokes > throw() (directly or indirectly) on it. Frame B, the generator frame, gets > its f_back set to point to Frame A, but presumably that link is cleared on > exit? (If it isn't, it probably should be). > > Anyway, frame B throws an exception, and the traceback is created. The > traceback has a reference to frame B. We return to frame A, and add it to > the traceback as well, and a reference to the traceback goes into the frame > too. Hm. Still no cycle passing through the *generator iterator*, unless > the generator's frame's f_back is still pointing to the frame that it was > last called from. This holds even if the generator's frame holds the > traceback that was current at the time of the error, because that traceback > only includes the generator's frame, not the caller's frame. > > So, as long as the generator guarantees its frame's f_back is empty while > the generator is not actually executing, the cycle should not include the > generator, so it will still be GC'able. (Note, by the way, that this > property is probably another good reason for using the yield expression as > the source location for a throw()!) Hm. The way I see it, as soon as a generator raises an exception, its frame is part of a cycle: the frame's f_exc_traceback points to the traceback object, and the traceback object's tb_frame points back to the frame. So that's a cycle right there. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
At 10:48 AM 5/19/2005 -0700, Guido van Rossum wrote: >Hm. The way I see it, as soon as a generator raises an exception, its >frame is part of a cycle: the frame's f_exc_traceback points to the >traceback object, and the traceback object's tb_frame points back to >the frame. So that's a cycle right there. But that cycle doesn't include the generator-iterator object, and it's not a collectable cycle while the iterator still lives. Once the iterator itself goes away, that frame cycle is collectable. However, Tim's new post brings up a different issue: if the collector can't tell the difference between a cycle participant and an object that's only reachable from a cycle, then the mere existence of a generator __del__ will prevent the cycle collection of the entire traceback/frame system that includes a generator-iterator reference anywhere! And that's a pretty serious problem. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > However, Tim's new post brings up a different issue: if the collector can't > tell the difference between a cycle participant and an object that's only > reachable from a cycle, Uh, that's not what he meant: />> class C: |.. def __del__(self): |.. print 'bye' \__ ->> a = [C()] ->> a.append(a) ->> del a ->> gc.collect() bye 1 Cheers, mwh -- Now this is what I don't get. Nobody said absolutely anything bad about anything. Yet it is always possible to just pull random flames out of ones ass. -- http://www.advogato.org/person/vicious/diary.html?start=60 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
At 07:18 PM 5/19/2005 +0100, Michael Hudson wrote: >"Phillip J. Eby" <[EMAIL PROTECTED]> writes: > > > However, Tim's new post brings up a different issue: if the collector > can't > > tell the difference between a cycle participant and an object that's only > > reachable from a cycle, > >Uh, that's not what he meant: > >/>> class C: >|.. def __del__(self): >|.. print 'bye' >\__ >->> a = [C()] >->> a.append(a) >->> del a >->> gc.collect() >bye >1 Now you've shaken my faith in Uncle Timmy. :) Seriously, he did *say*: """For example, it doesn't know the difference between an object that's in a trash cycle, and an object that's not in a trash cycle but is reachable only from trash cycles.""" So now I wonder what he *did* mean. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[Phillip J. Eby]
> ...
> However, Tim's new post brings up a different issue: if the collector can't
> tell the difference between a cycle participant and an object that's only
> reachable from a cycle, then the mere existence of a generator __del__ will
> prevent the cycle collection of the entire traceback/frame system that
> includes a generator-iterator reference anywhere! And that's a pretty
> serious problem.
It's not that simple . If an object with a __del__ is not part
of a cycle, but is reachable only from trash cycles, that __del__ does
not inhibit garbage collection. Like:
A<->B -> C -> D -> E
where C, D and E have __del__, but A and B don't, and all are trash.
Relatively early on, gc "moves" C, D, and E into a special
"finalizers" list, and doesn't look at this list again until near the
end. Then A.tp_clear() and B.tp_clear() are called in some order. As
a *side effect* of calling B.tp_clear(), C's refcount falls to 0, and
Python's normal refcount-based reclamation (probably) recovers all of
C, D and E, and runs their __del__ methods. Note that refcount-based
reclamation necessarily follows a DAG order: E is still intact when
D.__del__ is called, and likewise D is still intact when C.__del__ is
called. It's possible that C.__del__ will resurrect D and/or E, and
that D.__del__ will resurrect E. In such cases, D and/or E's
refcounts don't fall to 0, and their __del__ methods won't be called
then.
Cyclic gc doesn't force any of that, though -- it's all a side effect
of the clear() in gcmodule.c's:
if ((clear = op->ob_type->tp_clear) != NULL) {
Py_INCREF(op);
clear(op);
Py_DECREF(op);
In turn, one of A and B get reclaimed as a side effect of the
Py_DECREF there -- it's one of the delights of gcmodule.c that if you
don't know the trick, you can stare at it for hours and never discover
where exactly it is anything gets released <0.9 wink>. In fact, it
doesn't release anything directly -- "all it really does" now is break
reference cycles, so that Py_DECREF can do its end-of-life thing.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Combining the best of PEP 288 and PEP 325: generator exceptions and cleanup
[Phillip J. Eby] > Now you've shaken my faith in Uncle Timmy. :) Now, now, a mere technical matter is no cause for soul-damning heresy! > Seriously, he did *say*: > > """For example, it doesn't know the difference between an object > that's in a trash cycle, and an object that's not in a trash cycle but > is reachable only from trash cycles.""" > > So now I wonder what he *did* mean. What I said, of course ;-) I hope my later email clarified it. gc knows which trash objects have __del__ methods, and which don't. That's all it needs to know so that a __del__ method on an object that's not in a trash cycle but is reachable only from trash cycles will get reclaimed (provided that no __del__ method on a predecessor object that's not in a trash cycle but is reachable only from trash cycles resurrects it). gc doesn't know whether the set of objects it _intends_ to call tp_clear on are or aren't in cycles, but all objects directly in __del__ free cycles are included in that set. That's enough to ensure that trash "hanging off of them" sees its refcounts fall to 0 as gc applies tp_clear to the objects in that set. Note that this set can mutate as gc goes along: calling tp_clear on one object can (also as a side effect of refcounts falling to 0) remove any number of other objects from that set (that's why I said "intends" above: there's no guarantee that gc will end up calling tp_clear on any object other than "the first" one in the set, where "the first" is utterly arbitrary now). If an object in a trash cycle has a __del__ method, this is why the cycle won't be reclaimed: all trash objects with __del__ methods, and everything transitively reachable from them, are moved to the "finalizers" list early on. If that happens to include a trash cycle C, then all of C ends up in the "finalizers" list, and no amount of tp_clear'ing on the objects that remain can cause the refcount on any object in C to fall to 0. gc has no direct knowledge of cycles in this case either. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
[Raymond Hettinger] > For brevity, the above example used the context free > constructor, but the point was to show the consequence > of a precision change. Yes, I understood your point. I was making a different point: "changing precision" isn't needed _at all_ to get surprises from a constructor that ignores context. Your example happened to change precision, but that wasn't essential to getting surprised by feeding strings to a context-ignoring Decimal constructor. In effect, this creates the opportunity for everyone to get suprised by something only experts should need to deal with. There seems to be an unspoken "wow that's cool!" kind of belief that because Python's Decimal representation is _potentially_ unbounded, the constructor should build an object big enough to hold any argument exactly (up to the limit of available memory). And that would be appropriate for, say, an unbounded rational type -- and is appropriate for Python's unbounded integers. But Decimal is a floating type with fixed (albeit user-adjustable) precision, and ignoring that mixes arithmetic models in a fundamentally confusing way. I would have no objection to a named method that builds a "big as needed to hold the input exactly" Decimal object, but it shouldn't be the behavior of the everyone-uses-it-constructor. It's not an oversight that the IBM standard defines no operations that ignore context (and note that string->float is a standard operation): it's trying to provide a consistent arithmetic, all the way from input to output. Part of consistency is applying "the rules" everywhere, in the absence of killer-strong reasons to ignore them. Back to your point, maybe you'd be happier if a named (say) apply_context() method were added? I agree unary plus is a funny-looking way to spell it (although that's just another instance of applying the same rules to all operations). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
[Tim suggesting that I'm clueless and dazzled by sparkling lights] > There seems to be an unspoken "wow that's cool!" kind of belief > that because Python's Decimal representation is _potentially_ > unbounded, the constructor should build an object big enough to > hold any argument exactly (up to the limit of available memory). > And that would be appropriate for, say, an unbounded rational > type -- and is appropriate for Python's unbounded integers. I have no such thoughts but do strongly prefer the current design. I recognize that it allows a user to specify an input at a greater precision than the current context (in fact, I provided the example). The overall design of the module and the spec is to apply context to the results of operations, not their inputs. In particular, the spec recognizes that contexts can change and rather than specifying automatic or implicit context application to all existing values, it provides the unary plus operation so that such an application is explicit. The use of extra digits in a calculation is not invisible as the calculation will signal Rounded and Inexact (if non-zero digits are thrown away). One of the original motivating examples was "schoolbook" arithmetic where the input string precision is incorporated into the calculation. IMO, input truncation/rounding is inconsistent with that motivation. Likewise, input rounding runs contrary to the basic goal of eliminating representation error. With respect to integration with the rest of Python (everything beyond that spec but needed to work with it), I suspect that altering the Decimal constructor is fraught with issues such as the string-to-decimal-to-string roundtrip becoming context dependent. I haven't thought it through yet but suspect that it does not bode well for repr(), pickling, shelving, etc. Likewise, I suspect that traps await multi-threaded or multi-context apps that need to share data. Also, adding another step to the constructor is not going to help the already disasterous performance. I appreciate efforts to make the module as idiot-proof as possible. However, that is a pipe dream. By adopting and exposing the full standard instead of the simpler X3.274 subset, using the module is a non-trivial exercise and, even for experts, is a complete PITA. Even a simple fixed-point application (money, for example) requires dealing with quantize(), normalize(), rounding modes, signals, etc. By default, outputs are not normalized so it is difficult even to recognize what a zero looks like. Just getting output without exponential notation is difficult. If someone wants to craft another module to wrap around and candy-coat the Decimal API, I would be all for it. Just recognize that the full spec doesn't have a beginner mode -- for better or worse, we've simulated a hardware FPU. Lastly, I think it is a mistake to make a change at this point. The design of the constructor survived all drafts of the PEP, comp.lang.python discussion, python-dev discussion, all early implementations, sandboxing, the Py2.4 alpha/beta, cookbook contributions, and several months in the field. I say we document a recommendation to use Context.create_decimal() and get on with life. Clueless in Boston P.S. With 28 digit default precision, the odds of this coming up in practice are slim (when was the last time you typed in a floating point value with more than 28 digits; further, if you had, would it have ruined your day if your 40 digits were not first rounded to 28 before being used). IOW, bug tracker lists hundreds of bigger fish to fry without having to change a published API (pardon the mixed metaphor). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
Sorry, I simply can't make more time for this. Shotgun mode: [Raymond] > I have no such thoughts but do strongly prefer the current > design. How can you strongly prefer it? You asked me whether I typed floats with more than 28 significant digits. Not usually . Do you? If you don't either, how can you strongly prefer a change that makes no difference to what you do? > ... > The overall design of the module and the spec is to apply > context to the results of operations, not their inputs. But string->float is an _operation_ in the spec, as it has been since 1985 in IEEE-754 too. The float you get is the result of that operation, and is consistent with normal numeric practice going back to the first time Fortran grew a distinction between double and single precision. There too the common practice was to write all literals as double-precision, and leave it to the compiler to round off excess bits if the assignment target was of single precision. That made it easy to change working precision via fiddling a single "implicit" (a kind of type declaration) line. The same kind of thing would be pleasantly applicable for decimal too -- if the constructor followed the rules. > In particular, the spec recognizes that contexts can change |> and rather than specifying automatic or implicit context > application to all existing values, it provides the unary plus > operation so that such an application is explicit. The use > of extra digits in a calculation is not invisible as the > calculation will signal Rounded and Inexact (if non-zero digits > are thrown away). Doesn't change that the standard rigorously specifies how strings are to be converted to decimal floats, or that our constructor implementation doesn't do that. > One of the original motivating examples was "schoolbook" > arithmetic where the input string precision is incorporated > into the calculation. Sorry, doesn't ring a bell to me. Whose example was this? > IMO, input truncation/rounding is inconsistent with that > motivation. Try keying more digits into your hand calculator than it can hold <0.5 wink>. > Likewise, input rounding runs contrary to the basic goal of > eliminating representation error. It's no surprise that an exact value containing more digits than current precision gets rounded. What _is_ surprising is that the decimal constructor doesn't follow that rule, instead making up its own rule. It's an ugly inconsistency at best. > With respect to integration with the rest of Python (everything > beyond that spec but needed to work with it), I suspect that > altering the Decimal constructor is fraught with issues such > as the string-to-decimal-to-string roundtrip becoming context > dependent. Nobody can have a reasonable expectation that string -> float -> string is an identity for any fixed-precision type across all strings. That's just unrealistic. You can expect string -> float -> string to be an identity if the string carries no more digits than current precision. That's how a bounded type works. Trying to pretend it's not bounded in this one case is a conceptual mess. > I haven't thought it through yet but suspect that it does not > bode well for repr(), pickling, shelving, etc. The spirit of the standard is always to deliver the best possible approximation consistent with current context. Unpickling and unshelving should play that game too. repr() has a special desire for round-trip fidelity. > Likewise, I suspect that traps await multi-threaded or multi- > context apps that need to share data. Like what? Thread-local context precision is a reality here, going far beyond just string->float. > Also, adding another step to the constructor is not going to > help the already disasterous performance. (1) I haven't found it to be a disaster. (2) Over the long term, the truly speedy implementations of this standard will be limited to a fixed set of relatively small precisions (relative to, say, 100, not to 28 ). In that world it would be unboundedly more expensive to require the constructor to save every bit of every input: rounding string->float is a necessity for speedy operation over the long term. > I appreciate efforts to make the module as idiot-proof as > possible. That's not my interest here. My interest is in a consistent, std-conforming arithmetic, and all fp standards since IEEE-754 recognized that string->float is "an operation" much like every other fp operation. Consistency helps by reducing complexity. Most users will never bump into this, and experts have a hard enough job without gratuitous deviations from a well-defined spec. What's the _use case_ for carrying an unbounded amount of information into a decimal instance? It's going to get lost upon the first operation anyway. > However, that is a pipe dream. By adopting and exposing the > full standard instead of the simpler X3.274 subset, using the > module is a non-trivial exercise and, even for experts, is a > complete PIT
Re: [Python-Dev] [Python-checkins] python/nondist/peps pep-0343.txt, 1.11, 1.12
Tim Peters wrote:
> [Raymond Hettinger]
>from decimal import getcontext, Decimal as D
>getcontext().prec = 3
>D('3.104') + D('2.104')
>>
>>Decimal("5.21")
>>
>D('3.104') + D('0.000') + D('2.104')
>>
>>Decimal("5.20")
>
> the results differ here because D(whatever)
> ignores context settings; having a common operation ignore context is
> ugly and error-prone).
I don't see it's because of that. Even if D(whatever)
didn't ignore the context settings, you'd get the same
oddity if the numbers came from somewhere else with a
different precision.
I'm very uncomfortable about the whole idea of a
context-dependent precision. It just seems to be
asking for trouble.
--
Greg Ewing, Computer Science Dept, +--+
University of Canterbury, | A citizen of NewZealandCorp, a |
Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. |
[EMAIL PROTECTED] +--+
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] python/nondist/peps pep-0343.txt, 1.11, 1.12
[Greg Ewing] > I don't see it's because of that. Even if D(whatever) > didn't ignore the context settings, you'd get the same > oddity if the numbers came from somewhere else with a > different precision. Most users don't change context precision, and in that case there is no operation defined in the standard that can _create_ a decimal "with different precision". Python's Decimal constructor, however, can (Python's Decimal constructor performs an operation that's not in the standard -- it's a Python-unique extension to the standard). > I'm very uncomfortable about the whole idea of a > context-dependent precision. It just seems to be > asking for trouble. If you're running on a Pentium box, you're using context-dependent precision a few million times per second. Most users will be as blissfully unaware of decimal's context precsion as you are of the Pentium FPU's context precision. Most features in fp standards are there for the benefit of experts. You're not required to change context; those who need such features need them desperately, and don't care whether you think they should . An alternative is a God-awful API that passes a context object explicitly to every operation. You can, e.g., kiss infix "+" goodbye then. Some implementations of the standard do exactly that. You might want to read the standard before getting carried off by gut reactions: http://www2.hursley.ibm.com/decimal/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
I know I should stay out of here, but isn't Decimal() with a string
literal as argument a rare case (except in examples)? It's like
float() with a string argument -- while you *can* write float("1.01"),
nobody does that. What people do all the time is parse a number out of
some larger context into a string, and then convert the string to a
float by passing it to float(). I assume that most uses of the
Decimal() constructor will be similar. In that case, it makes total
sense to me that the context's precision should be used, and if the
parsed string contains an insane number of digits, it will be rounded.
I guess the counter-argument is that because we don't have Decimal
literals, Decimal("12345") is used as a pseudo-literal, so it actually
occurs more frequently than float("12345"). Sure. But the same
argument applies: if I write a floating point literal in Python (or C,
or Java, or any other language) with an insane number of digits, it
will be rounded.
So, together with the 28-digit default precision, I'm fine with
changing the constructor to use the context by default. If you want
all the precision given in the string, even if it's a million digits,
set the precision to the length of the string before you start; that's
a decent upper bound. :-)
--
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
[Guido van Rossum]
> I know I should stay out of here,
Hey, it's still your language .
> but isn't Decimal() with a string literal as argument a rare
> case (except in examples)? It's like float() with a string
> argument -- while you *can* write float("1.01"), nobody does
> that. What people do all the time is parse a number out
> of some larger context into a string, and then convert the
> string to a float by passing it to float(). I assume that most
> uses of the Decimal() constructor will be similar.
I think that's right. For example, currency exchange rates, and stock
prices, are generally transmitted as decimal strings now, and those
will get fed to a Decimal constructor.
OTOH, in scientific computing it's common to specify literals to very
high precision (like 40 decimal digits). Things like pi, e, sqrt(2),
tables of canned numeric quadrature points, canned coefficients for
polynomial approximations of special functions, etc. The authors
don't expect "to get" all they specify, what they expect is that
various compilers on various platforms will give them as much
precision as they're capable of using efficiently. Rounding is
expected then, and indeed pragmatically necessary (carrying precision
beyond that natively supported comes with high runtime costs -- and
that can be equally true of Decimal literals carried with digits
beyond context precision: the standard requires that results be
computed "as if to infinite precision then rounded once" using _all_
digits in the inputs).
> In that case, it makes total sense to me that the context's
> precision should be used, and if the parsed string contains
> an insane number of digits, it will be rounded.
That's the IBM standard's intent (and mandatory in its string->float operation).
> I guess the counter-argument is that because we don't have
> Decimal literals, Decimal("12345") is used as a pseudo-literal,
> so it actually occurs more frequently than float("12345"). Sure.
> But the same argument applies: if I write a floating point literal
> in Python (or C, or Java, or any other language) with an insane
> number of digits, it will be rounded.
Or segfault <0.9 wink>.
> So, together with the 28-digit default precision, I'm fine with
> changing the constructor to use the context by default. If you
> want all the precision given in the string, even if it's a million
> digits, set the precision to the length of the string before you
> start; that's a decent upper bound. :-)
That is indeed the intended way to do it. Note that this also applies
to integers passed to a Decimal constructor.
Maybe it's time to talk about an unbounded rational type again .
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adventures with Decimal
I sense a religious fervor about this so go ahead and do whatever you want. Please register my -1 for the following reasons: a.) It re-introduces representation error into a module that worked so hard to overcome that very problem. The PEP explicitly promises that a transformation from a literal involves no loss of information. Likewise, it promises that "context just affects operations' results". b.) It is inconsistent with the idea of having the input specify its own precision: http://www2.hursley.ibm.com/decimal/decifaq1.html#tzeros c.) It is both untimely and unnecessary. The module is functioning according to its tests, the specification test suite, and the PEP. Anthony should put his foot down as this is NOT a bugfix, it is a change in concept. The Context.create_decimal() method already provides a standard conforming implementation of the to-number conversion. http://www.python.org/peps/pep-0327.html#creating-from-context . d.) I believe it will create more problems than it would solve. If needed, I can waste an afternoon coming up with examples. Likewise, I think it will make the module more difficult to use (esp. when experimenting with the effect of results of changing precision). e.) It does not eliminate the need to use the plus operation to force rounding/truncation when switching precision. f.) To be consistent, one would need to force all operation inputs to have the context applied before their use. The standard specifically does not do this and allows for operation inputs to be of a different precision than the current context (that is the reason for the plus operation). g.) It steers people in the wrong direction. Increasing precision is generally preferable to rounding or truncating explicit inputs. I included two Knuth examples in the docs to show the benefits of bumping up precision when needed. h.) It complicates the heck out of storage, retrieval, and input. Currently, decimal objects have a meaning independent of context. With the proposed change, the meaning becomes context dependent. i.) After having been explicitly promised by the PEP, discussed on the newsgroup and python-dev, and released to the public, a change of this magnitude warrants a newsgroup announcement and a comment period. A use case: --- The first use case that comes to mind is in the math.toRadians() function. When originally posted, there was an objection that the constant degToRad was imprecise to the last bit because it was expressed as the ratio of two literals that compiler would have rounded, resulting in a double rounding. Link to rationale for the spec: --- http://www2.hursley.ibm.com/decimal/IEEE-cowlishaw-arith16.pdf See the intro to section 4 which says: The digits in decimal are not significands; rather, the numbers are exact. The arithmetic on those numbers is also exact unless rounding to a given precision is specified. Link to the discussion relating decimal design rationale to schoolbook math --- I can't find this link. If someone remembers, please post it. Okay, I've said my piece. Do what you will. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
