Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Fri, Mar 19, 2010 at 1:17 PM, Case Vanhorsen wrote: > On Fri, Mar 19, 2010 at 3:07 AM, Mark Dickinson wrote: >> On Fri, Mar 19, 2010 at 9:37 AM, Mark Dickinson wrote: >>> Making hashes of int, >>> float, Decimal *and* Fraction all compatible with one another, >>> efficient for ints and floats, and not grossly inefficient for >>> Fractions and Decimals, is kinda hairy, though I have a couple of >>> ideas of how it could be done. >> >> To elaborate, here's a cute scheme for the above, which is actually >> remarkably close to what Python already does for ints and floats, and >> which doesn't require any of the numeric types to figure out whether >> it's exactly equal to an instance of some other numeric type. > Will this change the result of hashing a long? I know that both gmpy > and SAGE use their own hash implementations for performance reasons. I > understand that CPython's hash function is an implementation detail, > but there are external modules that rely on the existing hash > behavior. Yes, it would change the hash of a long. What external modules are there that rely on existing hash behaviour? And exactly what behaviour do they rely on? Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 12:10 AM, Greg Ewing wrote: > Nick Coghlan wrote: >> >> Mark Dickinson wrote: > >> It seems to me that given the existing conflation of numeric equivalence >> and containment testing, going the whole hog and fixing the set >> membership problem for all of our rational types would be the right >> thing to do. > > Isn't this only solving half the problem, though? Yes. > Even if > you find that the hashes are equal, you still have to decide > whether the values themselves are equal. True. The reason I was concentrating on the hashes is that it's not immediately obvious that it's even *possible* to find a decent hash function that's efficient to compute and gives equal results for numerically equal inputs (regardless of type); this is especially true if you don't want to significantly slow down the existing hashes for int and float. But once that problem is solved, it shouldn't be too hard to implement all the comparisons. It *is* kinda messy, because as far as I can see the oddities of the various types mean that you end up producing specialized code for comparing each pair of types (one block of code for float<->Fraction comparisons, another for float<->Decimal, yet another for Decimal<->Fraction, and so on), but it's doable. Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Fraction arithmetic (Was: Decimal ... float comparisons in py3k)
On 20 March 2010 04:20, Nick Coghlan wrote: > In the case of floats and Decimals, there's no ambiguity here that > creates any temptation to guess - to determine a true/false result for a > comparison, floats can be converted explicitly to Decimals without any > loss of accuracy. For Fractions, the precedent has already been set by > allowing implicit (potentially lossy) conversion to binary floats - a > lossy conversion to Decimal wouldn't be any worse. Hmm, given that a float can be converted losslessly to a fraction, why was the decision taken to convert the fraction to a float rather than the other way round? I don't see a PEP for the fractions module, and my google-fu has failed to find anything. Was there a discussion on this? Paul. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fraction arithmetic (Was: Decimal ... float comparisons in py3k)
On Sat, Mar 20, 2010 at 11:41 AM, Paul Moore wrote: > On 20 March 2010 04:20, Nick Coghlan wrote: >> In the case of floats and Decimals, there's no ambiguity here that >> creates any temptation to guess - to determine a true/false result for a >> comparison, floats can be converted explicitly to Decimals without any >> loss of accuracy. For Fractions, the precedent has already been set by >> allowing implicit (potentially lossy) conversion to binary floats - a >> lossy conversion to Decimal wouldn't be any worse. > > Hmm, given that a float can be converted losslessly to a fraction, why > was the decision taken to convert the fraction to a float rather than > the other way round? I'm not sure of the actual reason for this decision, but one argument I've seen used for other languages is that it's desirable for the inexactness of the float type to be contagious: rationals are perceived as exact, while floats are perceived as approximations. Note that this only applies to arithmetic operations: for comparisons, an exact conversion *is* performed. This is much like what currently happens with ints and floats in the core: a mixed-type arithmetic operation between an int and a float first converts the int to a float (possibly changing the value in the process). A mixed-type comparison makes an exact comparison without doing such a conversion. For example (in any non-ancient version of Python): >>> n = 2**53 + 1 >>> x = 2.**53 >>> n > x # compares exact values; no conversion performed True >>> n - x# converts n to a float before subtracting 0.0 > I don't see a PEP for the fractions module, and my google-fu has > failed to find anything. Was there a discussion on this? There's PEP 3141 (http://www.python.org/dev/peps/pep-3141/), which was the motivation for adding the fractions module in the first place, and there's the issue tracker item for the fractions module (http://bugs.python.org/issue1682). Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Attribute lookup ambiguity
Michael Foord a écrit :
On 19/03/2010 18:58, Pascal Chambon wrote:
Hello
I've already crossed a bunch of articles detailing python's attribute
lookup semantic (__dict__, descriptors, order of base class
traversing...), but I have never seen, so far, an explanation of
WHICH method did waht, exactly.
I assumed that getattr(a, b) was the same as a.__getattribute__(b),
and that this __getattribute__ method (or the hidden routine
replacing it when we don't override it in our class) was in charge of
doing the whole job of traversing the object tree, checking
descriptors, binding methods, calling __getattr__ on failure etc.
However, the test case below shows that __getattribute__ does NOT
call __getattr__ on failure. So it seems it's an upper levl
machinery, in getattr(), which is in chrge of that last action.
Python 3 has the behavior you are asking for. It would be a backwards
incompatible change to do it in Python 2 as __getattribute__ *not*
calling __getattr__ is the documented behaviour.
Python 3.2a0 (py3k:78770, Mar 7 2010, 20:32:50)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
>>> class x:
... def __getattribute__(s, name):
... print ('__getattribute__', name)
... raise AttributeError
... def __getattr__(s, name):
... print ('__getattr__', name)
...
>>> a = x()
>>> a.b
__getattribute__ b
__getattr__ b
I'm confused there, because the script you gave behaves the same in
python 2.6. And according to the doc, it's normal, getattr() reacts to
an AttributeError from __getattribute__, by calling __getattr__ :
"""
Python 2.6.5 documentation
object.__getattribute__(/self/, /name/)
Called unconditionally to implement attribute accesses for instances
of the class. If the class also defines __getattr__()
,
the latter will not be called unless __getattribute__()
either calls it explicitly or raises an AttributeError
.
This method should return the (computed) attribute value or raise an
AttributeError
exception. In order to avoid infinite recursion in this method, its
implementation should always call the base class method with the
same name to access any attributes it needs, for example,
object.__getattribute__(self, name).
"""
But the point which for me is still unclear, is : does the default
implementation of __getattribute__ (the one of "object") call
__getattr__ by himself, or does it rely on its caller for that, by
raising an AttributeError ? For Python2, it's blatantly the latter case
which is favoured, but since it looks like an implementation detail at
the moment, I propose we settle it (and document it) once for all.
This list is not really an appropriate place to ask questions like
this though, comp.lang.python would be better.
All the best,
Michael Fooord
Sorry if I misposted, I just (wrongly ?) assumed that it was more an
undecided, implementation-specific point (since the doc gave possible
behaviours for __getattribute__, without precising which one was the
default one), and thus targetted the hands-in-core-code audience only.
Regards,
Pascal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Mark Dickinson gmail.com> writes: > > On Fri, Mar 19, 2010 at 9:50 PM, Guido van Rossum python.org> wrote: > > There is one choice which I'm not sure about. Should a mixed > > float/Decimal operation return a float or a Decimal? > > I'll just say that it's much easier to return a Decimal if you want to > be able to make guarantees about rounding behaviour, basically because > floats can be converted losslessly to Decimals. I also like the fact > that the decimal module offers more control (rounding mode, precision, > flags, wider exponent range) than float. A problem, though, is that decimals are much slower than floats. If you have a decimal creeping in some part of a calculation it could degrade performance quite a bit. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 4:06 AM, Mark Dickinson wrote: > On Fri, Mar 19, 2010 at 1:17 PM, Case Vanhorsen wrote: >> On Fri, Mar 19, 2010 at 3:07 AM, Mark Dickinson wrote: >>> On Fri, Mar 19, 2010 at 9:37 AM, Mark Dickinson wrote: Making hashes of int, float, Decimal *and* Fraction all compatible with one another, efficient for ints and floats, and not grossly inefficient for Fractions and Decimals, is kinda hairy, though I have a couple of ideas of how it could be done. >>> >>> To elaborate, here's a cute scheme for the above, which is actually >>> remarkably close to what Python already does for ints and floats, and >>> which doesn't require any of the numeric types to figure out whether >>> it's exactly equal to an instance of some other numeric type. > >> Will this change the result of hashing a long? I know that both gmpy >> and SAGE use their own hash implementations for performance reasons. I >> understand that CPython's hash function is an implementation detail, >> but there are external modules that rely on the existing hash >> behavior. > > Yes, it would change the hash of a long. > > What external modules are there that rely on existing hash behaviour? I'm only aware of gmpy and SAGE. > And exactly what behaviour do they rely on? Instead of calculating hash(long(mpz)), they calculate hash(mpz) directly. It avoids creation of a temporary object that could be quite large and is faster than the two-step process. I would need to modify the code so that it continues to produce the same result. casevh > Mark > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker reviews workflow and flags
On Fri, 19 Mar 2010 23:20:52 -0400, Alexander Belopolsky wrote: > I will not go into details here beyond referring to > http://bugs.python.org/issue8154, but if you follow the link, you'll > see that there was not a consensus on how the issue should be > addressed and even whether or not it was a bug. Nevertheless the > patch was committed both to the trunk and to 2.6 without any answer to > my concerns and without even an rNNN link to the committed revision. > > I think it would be nice if committers would not cut the discussion > short without at least a note explaining their decision. I think the reason this happened in this bug is that it *appeared* as though the same problem had been encountered and already fixed in python3, and all that was needed was a backport. We were groping our way toward deciding that that wasn't actually the case when Matthias committed the patch. Possibly my fault, since I said that I agreed that it should be fixed. As I've now noted on the tracker (and in the Ubuntu tracker just for good measure), I don't think that commit was appropriate, at least for 2.6, and probably not for 2.7 either. I also hope that Matthias will put in commit numbers in the future, since it is very helpful. Even better, I hope that we can automate this after the switch to Mercurial (but someone will need to write the code to do it, of course...) --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On 3/20/2010 7:06 AM, Mark Dickinson wrote: Will this change the result of hashing a long? I know that both gmpy and SAGE use their own hash implementations for performance reasons. I understand that CPython's hash function is an implementation detail, but there are external modules that rely on the existing hash behavior. Yes, it would change the hash of a long. What external modules are there that rely on existing hash behaviour? And exactly what behaviour do they rely on? Depending on specifics of CPython's hash is a 'do at your own risk' enterprise, like CPython bytecode hacks. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug? syslog.openlog using ident " python" by default.
Sean Reifschneider tummy.com> writes: > > I would propose changing the Python syslog() call to do the C equivalent of: > >if openlog_hasnt_been_called_before_now: > if sys.argv[0]: > syslog.openlog(os.path.basename(sys.argv[0])) > > In other words, if there's a script name and openlog hasn't already been > called, call openlog with the basename of the script name. This looks fine to me. The compatibility breakage doesn't look too serious, given that "python" is an inappropriate name and therefore people would have changed it anyway. By the way, why does logging have its own syslog implementation? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Tracker opened/closed issue list: missing?
I did not receive the usual Friday tracker post on issues opened and closed during the past week. Did anyone else? I am reading via gmane. Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Sat, Mar 20, 2010 at 09:11, Antoine Pitrou wrote: > Mark Dickinson gmail.com> writes: >> >> On Fri, Mar 19, 2010 at 9:50 PM, Guido van Rossum python.org> > wrote: >> > There is one choice which I'm not sure about. Should a mixed >> > float/Decimal operation return a float or a Decimal? >> >> I'll just say that it's much easier to return a Decimal if you want to >> be able to make guarantees about rounding behaviour, basically because >> floats can be converted losslessly to Decimals. I also like the fact >> that the decimal module offers more control (rounding mode, precision, >> flags, wider exponent range) than float. > > A problem, though, is that decimals are much slower than floats. If you have a > decimal creeping in some part of a calculation it could degrade performance > quite a bit. For a little context, we have this numeric tower: int -> Fraction -> float -> complex And coincidentally (or not), we have an unspoken rule that you can go right, but never left (int/int -> float, int/Fraction -> Fraction, Fraction/float -> float, Fraction/complex -> complex, etc). This gives us a preference for fast, inexact results. Decimal is more precise, and pays a performance cost for it. It also seems odd to stick it between float and complex (nobody's planning a ComplexDecimal, right?) That suggests it should go between Fraction and float. Decimal/float -> float. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 3:17 PM, Case Vanhorsen wrote: > On Sat, Mar 20, 2010 at 4:06 AM, Mark Dickinson wrote: >> What external modules are there that rely on existing hash behaviour? > > I'm only aware of gmpy and SAGE. > >> And exactly what behaviour do they rely on? > > Instead of calculating hash(long(mpz)), they calculate hash(mpz) > directly. It avoids creation of a temporary object that could be quite > large and is faster than the two-step process. I would need to modify > the code so that it continues to produce the same result. Does gmpy only do this for Python 2.6? Or does it use different algorithms for 2.4/2.5 and 2.6? As far as I can tell, there was no reasonable way to compute long_hash directly at all before the algorithm was changed for 2.6, unless you imitate exactly what Python was doing (break up into 15-bit pieces, and do all the rotation and addition exactly the same way), in which case you might as well be calling long_hash directly. Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker opened/closed issue list: missing?
On Sat, Mar 20, 2010 at 11:02, Terry Reedy wrote: > I did not receive the usual Friday tracker post on issues opened and closed > during the past week. > Did anyone else? > I am reading via gmane. > > Terry Jan Reedy > I actually haven't seen that mail in a few weeks (subscribed to the list via my gmail account). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 10:05 AM, Mark Dickinson wrote: > On Sat, Mar 20, 2010 at 3:17 PM, Case Vanhorsen wrote: >> On Sat, Mar 20, 2010 at 4:06 AM, Mark Dickinson wrote: >>> What external modules are there that rely on existing hash behaviour? >> >> I'm only aware of gmpy and SAGE. >> >>> And exactly what behaviour do they rely on? >> >> Instead of calculating hash(long(mpz)), they calculate hash(mpz) >> directly. It avoids creation of a temporary object that could be quite >> large and is faster than the two-step process. I would need to modify >> the code so that it continues to produce the same result. > > Does gmpy only do this for Python 2.6? Or does it use different > algorithms for 2.4/2.5 and 2.6? As far as I can tell, there was no > reasonable way to compute long_hash directly at all before the > algorithm was changed for 2.6, unless you imitate exactly what Python > was doing (break up into 15-bit pieces, and do all the rotation and > addition exactly the same way), in which case you might as well be > calling long_hash directly. > > Mark > It does the later: it converts from GMP's internal format to CPython's long format and calculates the hash along the way. I may (should :) ) revert back to converting to long and then calling long_hash. The majority of the speed increase came from the conversion improvements, not the hash calculation. I am in favor of any change that makes 2.7 and 3.2 behave the same. casevh ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker opened/closed issue list: missing?
On Sat, 20 Mar 2010 11:11:17 -0600, Brian Curtin wrote: > On Sat, Mar 20, 2010 at 11:02, Terry Reedy wrote: > > I did not receive the usual Friday tracker post on issues opened and closed > > during the past week. > > Did anyone else? > > I am reading via gmane. > > > > Terry Jan Reedy > > > > I actually haven't seen that mail in a few weeks (subscribed to the list via > my gmail account). Darn. I bet I broke it when I added the 'languishing' status. I even thought about that at the time but didn't follow up on it. I'll see if I can figure out where the script lives. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fraction arithmetic (Was: Decimal ... float comparisons in py3k)
On Sat, Mar 20, 2010 at 4:58 AM, Mark Dickinson wrote: > On Sat, Mar 20, 2010 at 11:41 AM, Paul Moore wrote: >> On 20 March 2010 04:20, Nick Coghlan wrote: >>> In the case of floats and Decimals, there's no ambiguity here that >>> creates any temptation to guess - to determine a true/false result for a >>> comparison, floats can be converted explicitly to Decimals without any >>> loss of accuracy. For Fractions, the precedent has already been set by >>> allowing implicit (potentially lossy) conversion to binary floats - a >>> lossy conversion to Decimal wouldn't be any worse. >> >> Hmm, given that a float can be converted losslessly to a fraction, why >> was the decision taken to convert the fraction to a float rather than >> the other way round? > > I'm not sure of the actual reason for this decision, but one argument > I've seen used for other languages is that it's desirable for the > inexactness of the float type to be contagious: rationals are > perceived as exact, while floats are perceived as approximations. You hit the nail on the head. > Note that this only applies to arithmetic operations: for > comparisons, an exact conversion *is* performed. This is much like > what currently happens with ints and floats in the core: a mixed-type > arithmetic operation between an int and a float first converts the int > to a float (possibly changing the value in the process). A mixed-type > comparison makes an exact comparison without doing such a conversion. > For example (in any non-ancient version of Python): > n = 2**53 + 1 x = 2.**53 n > x # compares exact values; no conversion performed > True n - x # converts n to a float before subtracting > 0.0 > >> I don't see a PEP for the fractions module, and my google-fu has >> failed to find anything. Was there a discussion on this? > > There's PEP 3141 (http://www.python.org/dev/peps/pep-3141/), which was > the motivation for adding the fractions module in the first place, and > there's the issue tracker item for the fractions module > (http://bugs.python.org/issue1682). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker opened/closed issue list: missing?
> Darn. I bet I broke it when I added the 'languishing' status. I even > thought about that at the time but didn't follow up on it. I'll see > if I can figure out where the script lives. My guess is that the Debian upgrade killed it. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 4:16 AM, Mark Dickinson wrote: > True. The reason I was concentrating on the hashes is that it's not > immediately obvious that it's even *possible* to find a decent hash > function that's efficient to compute and gives equal results for > numerically equal inputs (regardless of type); this is especially true > if you don't want to significantly slow down the existing hashes for > int and float. But once that problem is solved, it shouldn't be too > hard to implement all the comparisons. > > It *is* kinda messy, because as far as I can see the oddities of the > various types mean that you end up producing specialized code for > comparing each pair of types (one block of code for float<->Fraction > comparisons, another for float<->Decimal, yet another for > Decimal<->Fraction, and so on), but it's doable. I propose to reduce all hashes to the hash of a normalized fraction, which we can define as a combination of the hashes for the numerator and the denominator. Then all we have to do is figure fairly efficient ways to convert floats and decimals to normalized fractions (not necessarily Fractions). I may be naive but this seems doable: for a float, the denominator is always a power of 2 and removing factors of 2 from the denominator is easy (just right-shift until the last bit is zero). For Decimal, the unnormalized denominator is always a power of 10, and the normalization is a bit messier, but doesn't seem excessively so. The resulting numerator and denominator may be large numbers, but for typical use of Decimal and float they will rarely be excessively large, and I'm not too worried about slowing things down when they are (everything slows down when you're using really large integers anyway). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Sat, Mar 20, 2010 at 7:56 PM, Guido van Rossum wrote:
> I propose to reduce all hashes to the hash of a normalized fraction,
> which we can define as a combination of the hashes for the numerator
> and the denominator. Then all we have to do is figure fairly efficient
> ways to convert floats and decimals to normalized fractions (not
> necessarily Fractions). I may be naive but this seems doable: for a
> float, the denominator is always a power of 2 and removing factors of
> 2 from the denominator is easy (just right-shift until the last bit is
> zero). For Decimal, the unnormalized denominator is always a power of
> 10, and the normalization is a bit messier, but doesn't seem
> excessively so. The resulting numerator and denominator may be large
> numbers, but for typical use of Decimal and float they will rarely be
> excessively large, and I'm not too worried about slowing things down
> when they are (everything slows down when you're using really large
> integers anyway).
I *am* worried about slowing things down for large Decimals: if you
can't put Decimal('1e1234567') into a dict or set without waiting for
an hour for the hash computation to complete (because it's busy
computing 10**1234567), I consider that a problem.
But it's solvable! I've just put a patch on the bug tracker:
http://bugs.python.org/issue8188
It demonstrates how hashes can be implemented efficiently and
compatibly for all numeric types, even large Decimals like the above.
It needs a little tidying up, but it works.
Mark
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Silencing DeprecationWarning by default in Python 2.7 is silencing division warnings
On Sat, Mar 6, 2010 at 12:43, Brett Cannon wrote: > I see two possible fixes for this. One is to not silence DeprecationWarning > if Py_DivisionWarningFlag is set to >= 1. The other is to introduce a new > subclass of DeprecationWarning called IntegerDivisionWarning and have that > added to the warnings filter so that if it is triggered it is handled > separately from what DeprecationWarning triggers. > The former means that you might get more than you bargained for in terms of > warnings as you are suddenly switching on all DeprecationWarnings on top of > division warnings. The latter means that you now have to explicit care about > IntegerDivisionWarning on top of DeprecationWarning (to minimize this I > could have IntegerDivisionWarning added to the warnings filter only in the > case of when Py_DivisionWarningFlag is set instead of blindly adding it). > Thoughts? > -Brett Just to follow up on this with my current thinking, I am leaning towards having Py_DivisionWarningFlag disable the suppression of DeprecationWarning. Since the silencing is new in 2.7, people using -Q will not see any different semantics than they were with Python 2.6. Plus if they are bothering to be notified about division warnings they probably want to know about other warnings as well. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & amp; lt; -& amp; gt; float comparisons in py3k.
Glenn Linderman wrote: One can argue that either way, that it is completely different, or completely the same. An important difference is that there is no intermediate type that can be compared with both ints and strings. Another relevant difference is that numbers are just one of many possible things that a string could be interpreted as representing. Decimals, on the other hand, are just numbers and nothing else. (Even strings which happen to look like numbers might not be -- e.g. telephone "numbers", which are better thought of as strings with a limited character set... despite some misguided souls occasionally storing them in floating point. :-) -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Adam Olsen wrote: For a little context, we have this numeric tower: int -> Fraction -> float -> complex Decimal is more precise, and pays a performance cost for it. It also seems odd to stick it between float and complex (nobody's planning a ComplexDecimal, right?) That suggests it should go between Fraction and float. Decimal/float -> float. There are two ways in which that linear tower is overly simplistic: * It conflates the notions of exactness and width. They're really orthogonal concepts, and to reflect this you would need two parallel towers, with exact and inexact versions of each type. * Decimal and float really belong side-by-side in the tower, rather than one above the other. Neither of them is inherently any more precise or exact than the other. There doesn't seem to be any good solution here. For every use case in which Decimal+float->float appears better, there seems to be another one for which Decimal+float->Decimal appears better. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Sat, Mar 20, 2010 at 4:20 PM, Greg Ewing wrote: > Adam Olsen wrote: > >> For a little context, we have this numeric tower: >> >> int -> Fraction -> float -> complex >> >> Decimal is more precise, and pays a performance cost for it. It also >> seems odd to stick it between float and complex (nobody's planning a >> ComplexDecimal, right?) That suggests it should go between Fraction >> and float. Decimal/float -> float. > > There are two ways in which that linear tower is overly > simplistic: > > * It conflates the notions of exactness and width. They're > really orthogonal concepts, and to reflect this you would > need two parallel towers, with exact and inexact versions > of each type. It's representing the mathematical concepts Integral -> Rational -> Real -> Complex When designing it, I tried to include a notion of exact/inexact types, but we couldn't find anything practical to do with them, so we took them out. It's reasonably easy to design inexact Integral and Rational types, but pretty hard to design a useful, exact Real type (things like '==' get uncomputable quickly), so we probably couldn't actually implement two whole parallel towers. > * Decimal and float really belong side-by-side in the > tower, rather than one above the other. Yep. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Sat, Mar 20, 2010 at 11:20 PM, Greg Ewing wrote: > * Decimal and float really belong side-by-side in the > tower, rather than one above the other. Neither of them is > inherently any more precise or exact than the other. Except that float is fixed-width (typically 53 bits of precision), while Decimal allows a user-specified, arbitrarily large, precision; so in that sense the two floating-point types aren't on an equal footing. Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Sat, Mar 20, 2010 at 4:46 PM, Mark Dickinson wrote: > On Sat, Mar 20, 2010 at 11:20 PM, Greg Ewing > wrote: >> * Decimal and float really belong side-by-side in the >> tower, rather than one above the other. Neither of them is >> inherently any more precise or exact than the other. > > Except that float is fixed-width (typically 53 bits of precision), > while Decimal allows a user-specified, arbitrarily large, precision; > so in that sense the two floating-point types aren't on an equal > footing. But this doesn't really help deciding which should be positioned at the end does it? Both Fraction and Decimal can represent every float value (barring NaN/Inf). I wonder if we need to look at use cases or see what other languages do for guidance. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Sat, Mar 20, 2010 at 17:20, Greg Ewing wrote: > There are two ways in which that linear tower is overly > simplistic: > > * It conflates the notions of exactness and width. They're > really orthogonal concepts, and to reflect this you would > need two parallel towers, with exact and inexact versions > of each type. > > * Decimal and float really belong side-by-side in the > tower, rather than one above the other. Neither of them is > inherently any more precise or exact than the other. > > There doesn't seem to be any good solution here. For every > use case in which Decimal+float->float appears better, there > seems to be another one for which Decimal+float->Decimal > appears better. Sure, from a purist point of view my post is completely wrong. It doesn't correspond to the mathematical reality. What it does correspond to is the code. Only going rightward through the types is what we have today. A linear progression is a lot simpler to understand than any sort of cycle; parallel progressions isn't even on the table. float has been the king of inexact types right for a long time. All other things being equal, that's good enough for me. -- Adam Olsen, aka Rhamphoryncus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & amp; lt; -& amp; gt; float comparisons in py3k.
Greg Ewing wrote:
> Antoine Pitrou wrote:
>
>> We forbid comparisons when there is a real danger or ambiguity, such
>> as unicode
>> vs. bytes. There is no such danger or ambiguity when comparing a
>> decimal with a
>> float.
>
> So do you think that float("0.1") and Decimal("0.1") should be
> equal or not, and why?
Note that Antoine's point was that float("0.1") and
Decimal.from_float(0.1) should compare equal. The latter exactly matches
the underlying binary floating point value:
>>> dec.from_float(0.1)
Decimal('0.155511151231257827021181583404541015625')
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
---
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & amp; lt; -& amp; gt; float comparisons in py3k.
Glenn Linderman wrote: > Thanks, though for the nice chart from an internals guy, you confirmed > all that I thought I knew about how this presently works, from PEP and > manual reading, a bit of experimentation, and the discussions here. I'll confess to a bit of interpreter prompt experimentation to confirm the behaviour, especially in the relation to Fractions - I didn't know all of that off the top of my head. The implicit coercion from Fraction -> float definitely surprised me a bit. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Attribute lookup ambiguity
On 20/03/2010 12:00, Pascal Chambon wrote:
Michael Foord a écrit :
On 19/03/2010 18:58, Pascal Chambon wrote:
Hello
I've already crossed a bunch of articles detailing python's
attribute lookup semantic (__dict__, descriptors, order of base
class traversing...), but I have never seen, so far, an explanation
of WHICH method did waht, exactly.
I assumed that getattr(a, b) was the same as a.__getattribute__(b),
and that this __getattribute__ method (or the hidden routine
replacing it when we don't override it in our class) was in charge
of doing the whole job of traversing the object tree, checking
descriptors, binding methods, calling __getattr__ on failure etc.
However, the test case below shows that __getattribute__ does NOT
call __getattr__ on failure. So it seems it's an upper levl
machinery, in getattr(), which is in chrge of that last action.
Python 3 has the behavior you are asking for. It would be a backwards
incompatible change to do it in Python 2 as __getattribute__ *not*
calling __getattr__ is the documented behaviour.
Python 3.2a0 (py3k:78770, Mar 7 2010, 20:32:50)
[GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
>>> class x:
... def __getattribute__(s, name):
... print ('__getattribute__', name)
... raise AttributeError
... def __getattr__(s, name):
... print ('__getattr__', name)
...
>>> a = x()
>>> a.b
__getattribute__ b
__getattr__ b
I'm confused there, because the script you gave behaves the same in
python 2.6. And according to the doc, it's normal, getattr() reacts to
an AttributeError from __getattribute__, by calling __getattr__ :
"""
Python 2.6.5 documentation
object.__getattribute__(/self/, /name/)
Called unconditionally to implement attribute accesses for
instances of the class. If the class also defines __getattr__()
,
the latter will not be called unless __getattribute__()
either calls it explicitly or raises an AttributeError
.
This method should return the (computed) attribute value or raise
an AttributeError
exception. In order to avoid infinite recursion in this method,
its implementation should always call the base class method with
the same name to access any attributes it needs, for example,
object.__getattribute__(self, name).
"""
But the point which for me is still unclear, is : does the default
implementation of __getattribute__ (the one of "object") call
__getattr__ by himself, or does it rely on its caller for that, by
raising an AttributeError ? For Python2, it's blatantly the latter
case which is favoured, but since it looks like an implementation
detail at the moment, I propose we settle it (and document it) once
for all.
Ah right, my apologies. So it is still documented behaviour -
__getattr__ is obviously called by the Python runtime and not by
__getattribute__. (It isn't just by getattr as the same behaviour is
shown when doing a normal attribute lookup and not via the getattr
function.)
This list is not really an appropriate place to ask questions like
this though, comp.lang.python would be better.
All the best,
Michael Fooord
Sorry if I misposted, I just (wrongly ?) assumed that it was more an
undecided, implementation-specific point (since the doc gave possible
behaviours for __getattribute__, without precising which one was the
default one), and thus targetted the hands-in-core-code audience only.
Well, the documentation you pointed to specifies that __getattr__ will
be called if __getattribute__ raises an AttributeError, it just doesn't
specify that it is done by object.__getattribute__ (which it isn't).
Michael Foord
Regards,
Pascal
--
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog
READ CAREFULLY. By accepting and reading this email you agree, on behalf of
your employer, to release me from all obligations and waivers arising from any
and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap,
clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and
acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in perpetuity, without
prejudice to my ongoing rights and privileges. You further represent that you
have the authority to release me from any BOGUS AGREEMENTS on behalf of your
employer.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Guido van Rossum wrote: > I think we should seriously reconsider allowing mixed arithmetic > involving Decimal, not just mixed comparisons. I'm glad I'm not the only one that started wondering that. I wasn't quite game enough to actually suggest it though :) > There is one choice which I'm not sure about. Should a mixed > float/Decimal operation return a float or a Decimal? I note that > Fraction (which *is* properly embedded in the numeric tower) supports > this and returns a float result in this case. While I earlier > proposed to return the most "complicated" type of the two, i.e. > Decimal, I now think it may also make sense to return a float, being > the most "fuzzy" type in the numeric tower. This would also make > checking for accidental floats easier, since floats now propagate > throughout the computation (like NaN) and a simple assertion that the > result is a Decimal instance suffices to check that no floats were > implicitly mixed into the computation. To blend a couple of different ideas from the thread reboot together: I suggest a 'linearised' numeric tower that looks like: int -> Decimal -> Fraction -> float -> complex As Adam stated, this is a pragmatic tower stating which implicit coercions are defined by the code, not a formal mathematical relationship. Note that this would involve adding mixed Fraction/Decimal arithmetic as well as Decimal/float arithmetic. I placed Decimal to the left of Fraction to keep Decimal's dependencies clear and because Decimal -> Fraction conversions appear straightforward (using power of 10 denominators) without introducing the precision issues that would arise in Decimal -> Fraction conversions. I also like the idea of adding the decimal context signal that Facundo suggests (e.g. under the name ImplicitCoercionToBinaryFloat). So "quick and dirty don't need a perfect answer" operations end up with ordinary binary floats, while more precise code can enable the signal trap to ensure the calculation fails noisily if binary floats are introduced. Allowing implicit operations would also finally allow Decimal to be registered with numbers.Real. Whatever we decide to do will need to be codified in a PEP though. "Cleaning Up Python's Numeric Tower" or something along those lines. > The implementation of __hash__ will be complicated, and it may make > sense to tweak the hash function of float, Fraction and Decimal to > make it easier to ensure that for values that can be represented in > either type the hash matches the equality. But this sounds a > worthwhile price to pay for proper embedding in the numeric tower. And Mark appears to already have a good answer to that problem. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Attribute lookup ambiguity
Michael Foord wrote: > Well, the documentation you pointed to specifies that __getattr__ will > be called if __getattribute__ raises an AttributeError, it just doesn't > specify that it is done by object.__getattribute__ (which it isn't). And as for why not: because __getattribute__ implementations need to be able to call object.__getattribute__ without triggering the fallback behaviour. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Mar 20, 2010, at 6:54 PM, Nick Coghlan wrote: > > I suggest a 'linearised' numeric tower that looks like: > > int -> Decimal -> Fraction -> float -> complex Is that a typo? Shouldn't Decimal and float go between Fraction and complex? The abstract numeric tower is: Number Complex Real Rational Integral where both Decimal and float have operations associated with reals. Raymond___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
On Mar 20, 2010, at 4:27 PM, Jeffrey Yasskin wrote: > > When designing it, I tried to include a notion of exact/inexact types, > but we couldn't find anything practical to do with them, so we took > them out. The were also other reasons that they were taken out. The notion of inexactness is a taint, not a property of a type. The design documents for the decimal spec aimed for the ability to do either inexact or exact calculations (and to encompass integer and fixed-point arithmetic). That is in-part why decimal is suitable for accounting work. A person can forbid any inexactness by setting a context flag that would raise an exception if any inexact calculation occurs. Another design principle for decimal is the notion that numbers are always exact, it is the results of operations that are subject to rounding. That is why the decimal constructor is not context sensitive. So, as Jeffrey says, the notion of exactness and inexactness is not built in to the numeric tower. Instead, it is about which operations or methods can be expected to apply to a given type (i.e. both float and Decimal support the necessary methods to register as a Real). In terms of interoperability of concrete types, we're in the fortunate position that any float can be exactly converted to a Decimal and any Decimal can be exactly converted to a Fraction. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Mark Dickinson wrote: Except that float is fixed-width (typically 53 bits of precision), while Decimal allows a user-specified, arbitrarily large, precision; Yes, but it still has *some* fixed limit at any given moment, so the result of an operation on Decimals always has the potential to produce an inexact result. It's not like an int or Fraction where the result can expand to whatever size is needed. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & amp; lt; -& amp; gt; float comparisons in py3k.
Nick Coghlan wrote:
Note that Antoine's point was that float("0.1") and
Decimal.from_float(0.1) should compare equal.
That would mean that Decimal("0.1") != float("0.1"), which might be
surprising to someone who didn't realise they were mixing floats
and decimals.
--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Attribute lookup ambiguity
Michael Foord wrote: Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't). If __getattribute__ raises an exception, it won't get a chance to do anything else, so something outside of __getattribute__ must catch the AttributeError and calling __getattr__. So I think the docs *are* specifying the behaviour here, if only by implication. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Mixing float and Decimal -- thread reboot
Nick Coghlan wrote: int -> Decimal -> Fraction -> float -> complex I don't think it's a good idea to put Decimal below Fraction, because Decimal has to be considered an implicitly inexact type like float, and we don't want to coerce from an inexact type to an exact one. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal & amp; lt; -& amp; gt; float comparisons in py3k.
On Sat, Mar 20, 2010 at 22:48, Greg Ewing wrote:
> Nick Coghlan wrote:
>
>> Note that Antoine's point was that float("0.1") and
>> Decimal.from_float(0.1) should compare equal.
>
> That would mean that Decimal("0.1") != float("0.1"), which might be
> surprising to someone who didn't realise they were mixing floats
> and decimals.
Much like Fraction("0.1") != float("0.1") is surprising.
The only way to get rid of surprises around float is to get rid of
float, and that ain't happening.
--
Adam Olsen, aka Rhamphoryncus
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
