Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, 17 Mar 2010 12:27:01 pm Raymond Hettinger wrote:
> On Mar 16, 2010, at 3:16 PM, Greg Ewing wrote:
> > Seems to me that this education would mostly consist of saying
> > "don't compare floats and decimals", which is why I think that
> > disallowing them in the first place would be better.
>
> That makes sense.
>
> I do worry that 2.x currently does make the comparison
> and gives the wrong answer. We have the ability to
> make it a correct answer. But, it seems like the mood
> here is that wrong-is-better-than-right for an action
> that someone shouldn't be doing in the first place.
I don't get this.
Why is it "wrong" to compare Decimals to floats, and why shouldn't I do
so? What harm is there?
If the argument is that naive users who don't understand floats may be
confused by the results, then the problem lies with floats, and if you
really want to avoid confusing the float-naive, then we should prohibit
all comparisons on floats:
>>> 1e20 + 1e4 < 1e20 + 2e4
False
I don't mean that as a serious suggestion -- it would be absurd to
cripple floats for the sake of avoiding confusion of those who don't
understand floats. Why are Decimals different?
I can't see why comparing Decimal(1) to float(1) is wrong in any sense.
I can see that comparing Decimal("1.1") to float("1.1") may confuse the
float-naive, but the float naive will be confused by this too:
>>> x = 1.0/3
>>> x + 1.0 - 1.0 == x
False
There's an awful lot about floats that is confusing to naive users, I
don't see that the behaviour of Decimals will make it worse.
--
Steven D'Aprano
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Fixing" the new GIL
I could also point you at EVE online, a large scale IO machine based on stackless python. Our game proxies (front end servers) handle 30.000 client connections each using tasklets. The problem is coarse grained scheduling of IO and other tasks, same as with threads in regular Python. We have found that adhering to FIFO scheduling provides good all round performance, particularly the variance of IO latency is very low. As I‘v mentioned in another message, I am toying with the idea of adding another priority layer to the Stackless scheduler for IO tasks. It´s not trivial though (requires me to do some stackless surgery) and the turnaround time to test new approaches in production for us is a bit high ☺ K From: [email protected] [mailto:[email protected]] On Behalf Of Robert Hancock Sent: 16. mars 2010 20:10 To: Peter Portante Cc: David Beazley; [email protected] Subject: Re: [Python-Dev] "Fixing" the new GIL The Linux kernel scheduler deals with two types of ratings and has a heuristic algorithm that rewards and penalizes on a rapid basis. To determine the next thread it inspects the priority array to find the highest priority task that is runnable and then selects the first task in that priority. This is a gross simplification of an extremely complex system, but it is not simply a simple queue. To duplicate this exact type of scheduling in the interpreter would be a daunting task, but Dave's example illustrates that some system of reward and penalty can be beneficial. Peter has brought up the point, both here and at the Open Space at Pycon, that before attempting a downsized version of the a priority thread scheduler that we should look at other programs that have had to deal with similar problems. Peter, since you have dealt with the nitty-gritty of concurrency before, can you suggest some applications that could serve as models for research? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Fixing" the new GIL
> -Original Message- > From: Nick Coghlan [mailto:[email protected]] > Sent: 16. mars 2010 22:08 > To: Kristján Valur Jónsson > Cc: Antoine Pitrou; [email protected] > Subject: Re: [Python-Dev] "Fixing" the new GIL > > Kristján Valur Jónsson wrote: > > We have never had high priority for IO threads in > > python (and its not-by-design round robin scheduler on single core > > cpus) and it is unclear why that should be required now as a fix. > > David explained that in the issue tracker - 2.x typically doesn't do > that much work per bytecode instruction, so the interpreter releases > and > reacquires the GIL far more frequently in a CPU-bound thread than it > does under the new time-based check interval. > > Okay, but my original post in this thread was "why don't we fix the original problem with the GIL as implemented in 2.x" rather than to try to beat the new, more complicated, GIL into submission? In fact, I probably should submit a patch to do that since 2.x isn't disappearing quite yet. Cheers, K ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, 17 Mar 2010 10:01:12 am Guido van Rossum wrote:
> On Tue, Mar 16, 2010 at 2:32 PM, Steven D'Aprano
wrote:
> > But mixed arithmetic runs into the problem, what do you want the
> > result type to be? Given (say) decimal+float, returning either a
> > Decimal or a float will be the wrong thing to do some of the time,
> > so better to prohibit mixed arithmetic and let folks handle their
> > own conversions. So +1 on continuing to prohibit mixed arithmetic.
>
> I'm not disagreeing, but I really wonder, what is the value of
> supporting mixed comparisons then? Just because you *can* assign a
> meaning to it doesn't mean you should.
Any mixed comparison with floats is already risky:
>>> 1e20 + 1e4 == 10**20 + 10**4
False
but we allow them anyway because they're too useful and too obvious not
to. The problems with (say) floating point equality are far too well
known to need me give examples, but I will anyway *wink*:
>>> 1.3 == 3.7 - 2.4
False
but we allow it anyway, partly from tradition ("floats have always been
like that") and partly because, frankly, laziness is a virtue. It would
be painful not to be able to say (e.g.):
>>> 1.25 == 3.75 - 2.5
True
>>> 10.0 > 1
True
My defence of mixed comparisons is the same. If I'm doing serious maths
work, then I'm going to be comparing numbers carefully, and not just
tossing a comparison operator between them. But for simple
calculations, or using the interactive interpreter as a calculator, or
if I need to sort a list of mixed numbers without caring whether they
are ints or floats or Decimals, laziness is a virtue. If your needs for
accuracy aren't high, you can go far by comparing floats with ==. Why
shouldn't the same apply to mixed floats and Decimals?
Having said all that, I've just re-read the PEP, and spotted a fly in
the ointment... hash.
If we allow Decimals to compare equal to floats, that presumably implies
that they need to hash equal. That may be simple enough for integer
valued floats, but what non-integer values?
--
Steven D'Aprano
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] "Fixing" the new GIL
I'm not sure I see any inherent problem in putting all I/O bound threads in some kind of FIFO queue. However, there still should be some mechanism that gives those threads priority over a CPU-bound thread that never wants to give up the GIL. A common OS solution is to simply have multiple FIFO queues, each with varying priorities. Threads with low priority don't get to run unless the higher priority queues are empty.One could probably try it with Python by simply having two queues (I/O and CPU) and separating threads into two categories depending on whether they are forcefully timed out or not.An OS would tend to have many more levels, but maybe just two queues would be enough to solve the thread scheduling problems being discussed here. Cheers, Dave On Mar 17, 2010, at 6:01 AM, Kristján Valur Jónsson wrote: > I could also point you at EVE online, a large scale IO machine based on > stackless python. Our game proxies (front end servers) handle 30.000 client > connections each using tasklets. The problem is coarse grained scheduling of > IO and other tasks, same as with threads in regular Python. We have found > that adhering to FIFO scheduling provides good all round performance, > particularly the variance of IO latency is very low. > As I‘v mentioned in another message, I am toying with the idea of adding > another priority layer to the Stackless scheduler for IO tasks. It´s not > trivial though (requires me to do some stackless surgery) and the turnaround > time to test new approaches in production for us is a bit high J > > K > > From: [email protected] > [mailto:[email protected]] On Behalf Of > Robert Hancock > Sent: 16. mars 2010 20:10 > To: Peter Portante > Cc: David Beazley; [email protected] > Subject: Re: [Python-Dev] "Fixing" the new GIL > > The Linux kernel scheduler deals with two types of ratings and has a > heuristic algorithm that rewards and penalizes on a rapid basis. To > determine the next thread it inspects the priority array to find the highest > priority task that is runnable and then selects the first task in that > priority. This is a gross simplification of an extremely complex system, but > it is not simply a simple queue. > > To duplicate this exact type of scheduling in the interpreter would be a > daunting task, but Dave's example illustrates that some system of reward and > penalty can be beneficial. > > Peter has brought up the point, both here and at the Open Space at Pycon, > that before attempting a downsized version of the a priority thread scheduler > that we should look at other programs that have had to deal with similar > problems. > > Peter, since you have dealt with the nitty-gritty of concurrency before, can > you suggest some applications that could serve as models for research? > > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Tue, Mar 16, 2010 at 10:16 PM, Greg Ewing
wrote:
> Mark Dickinson wrote:
>>
>> On Tue, Mar 16, 2010 at 3:58 PM, P.J. Eby wrote:
>>
>>> If not, it might be confusing if a number that prints as '.1' compares
>>> unequal to Decimal('.1').
>>
>> Agreed, but this is just your everyday floating-point confusion, to be
>> dealt with by social means (e.g., educating the programmer).
>
> Seems to me that this education would mostly consist of saying
> "don't compare floats and decimals", which is why I think that
> disallowing them in the first place would be better.
I was thinking of something more along the lines of: "Sure, go ahead
and compare floats and decimals, but be aware that float('1.1') is not
exactly 1.1, so don't complain when 1.1 == Decimal('1.1') returns
False." For me, this actually a plus of allowing these comparisons:
it makes the education easier. "Look, the binary float stored for 1.1
is actually larger than 1.1, and here's the proof: >>> 1.1 >
Decimal('1.1') -> True."
--
Mark
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Tue, Mar 16, 2010 at 10:32 PM, Steven D'Aprano wrote:
> On Wed, 17 Mar 2010 03:23:30 am Mark Dickinson wrote:
>> On Tue, Mar 16, 2010 at 4:11 PM, Mark Dickinson
>> wrote: [...]
>>
>> Decimal.from_float(1.1) == 1.1
>> >
>> > False
>>
>> Whoops. To clarify, this is the pre-patch behaviour; post-patch,
>> this gives True.
>
> Whew! You had me worried there for a second. Just to clarify, you are
> proposing:
>
> Decimal.from_float(1.1) == 1.1
> Decimal.('1.1') != float('1.1')
Exactly, yes.
--
Mark
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, Mar 17, 2010 at 12:36 AM, Raymond Hettinger wrote: > > On Mar 16, 2010, at 9:41 AM, Guido van Rossum wrote: >> >> Also supporting comparisons but not other mixed operations is going to >> be confusing. If you are sticking to that behavior I think mixed >> comparisons should also be ruled out. > > The difference is that mixed comparisons currently do give a result, > but one that is non-sensical. The proposal is a make in give a > meaningful result, not as an extra feature, but in an effort to not > be wrong. Yes, exactly. A number of people (including me, early in the issue tracker discussion) have suggested that it would be confusing/inconsistent/strange to allow mixed-type comparisons but not mixed-type arithmetic. But it's not clear to me exactly why this would be a bad thing. It seems likely that for serious applications there's little benefit in allowing float<->Decimal comparisons if mixed-type arithmetic isn't allowed: if you're checking that b > a, it's probably because you're about to subtract b from a, or divide b by a, or do some other piece of arithmetic that depends on that condition being true. But for simply playing around with numbers on the command line, and for helping free people from their floating-point misconceptions, I think having these float<->Decimal comparisons available is potentially useful. I understand the argument about not adding this feature to 2.x if it's not going to go into 3.x; but I'm convinced enough that the 2.x behaviour should change that I'd prefer to add it to both 2.x and 3.x than to neither. The other option would be to leave 3.x alone and just add a py3k warning to 2.x, so that at least there's some indication of where strange results are coming from. -- Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, Mar 17, 2010 at 11:43 AM, Steven D'Aprano wrote: > Having said all that, I've just re-read the PEP, and spotted a fly in > the ointment... hash. > > If we allow Decimals to compare equal to floats, that presumably implies > that they need to hash equal. That may be simple enough for integer > valued floats, but what non-integer values? That is indeed an issue. It can be taken care by checking whether the Decimal instance is exactly representable as a float, and returning the hash of the corresponding float if so. From the patch (http://bugs.python.org/file16544/issue2531.patch): +self_as_float = float(self) +if Decimal.from_float(self_as_float) == self: +return hash(self_as_float) Strictly speaking this only works if the Decimal -> float conversion (which relies on the platform strtod) is accurate to within <1ulp, but that's going to be true in 2.7/3.x on all platforms using David Gay's str<->float conversion code, and is *probably* true for the system strtod on any sane platform, so I don't think it's a real issue. This also slows down the (already slow) hash computation for Decimal a touch. Until people start complaining, or show evidence of applications that require making large dictionaries of Decimal instances, I'm not going to worry about that either. It would be easy to cache the hash value of a Decimal instance if it became necessary. A less messy (but more intrusive) alternative would be to come up with a single sane hash function defined for all rational numbers (plus infinities and nans), and use this function for int, long, float, Decimal and Fraction types. I have a candidate function in mind... Mark ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Tue, Mar 16, 2010 at 5:15 PM, Guido van Rossum wrote:
> Definitely some. Stricter comparison rules are a frequent cause of
> problems when code is first ported to 3.x. While you'd think that code
> comparing a float and a Decimal is *already* broken, there's a
> surprising number of situations where that's not necessary the case,
> e.g. when an arbitrary but stable ordering is needed.
Hmm. Okay. It seems like backporting the exception isn't a real option, then.
A nitpick: the current 2.x behaviour fails to give an arbitrary but
stable ordering; it merely gives an arbitrary ordering:
>>> sorted([Decimal(1), 2, 3.0])
[Decimal('1'), 2, 3.0]
>>> sorted([2, Decimal(1), 3.0])
[3.0, Decimal('1'), 2]
So long as your list contains only floats and Decimals you're fine,
but for a list containing floats, integers and Decimals the ordering
is no longer stable: the problem, of course, being that int <-> float
and int <-> Decimal comparisons use a rule (compare by numeric value)
that's not compatible with the way that floats and Decimals are
compared. This seems like yet another possible cause of subtle bugs,
and again would be fixed by the proposed change in behaviour. On the
other hand, I've not seen any reports of anyone encountering this in
real life.
Mark
Mark
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, Mar 17, 2010 at 8:04 AM, Mark Dickinson wrote:
> On Tue, Mar 16, 2010 at 5:15 PM, Guido van Rossum wrote:
>> Definitely some. Stricter comparison rules are a frequent cause of
>> problems when code is first ported to 3.x. While you'd think that code
>> comparing a float and a Decimal is *already* broken, there's a
>> surprising number of situations where that's not necessary the case,
>> e.g. when an arbitrary but stable ordering is needed.
>
> Hmm. Okay. It seems like backporting the exception isn't a real option,
> then.
>
> A nitpick: the current 2.x behaviour fails to give an arbitrary but
> stable ordering; it merely gives an arbitrary ordering:
>
sorted([Decimal(1), 2, 3.0])
> [Decimal('1'), 2, 3.0]
sorted([2, Decimal(1), 3.0])
> [3.0, Decimal('1'), 2]
>
> So long as your list contains only floats and Decimals you're fine,
> but for a list containing floats, integers and Decimals the ordering
> is no longer stable: the problem, of course, being that int <-> float
> and int <-> Decimal comparisons use a rule (compare by numeric value)
> that's not compatible with the way that floats and Decimals are
> compared. This seems like yet another possible cause of subtle bugs,
> and again would be fixed by the proposed change in behaviour. On the
> other hand, I've not seen any reports of anyone encountering this in
> real life.
Ok, I'll try to stay out of the discussion of which solution is best
of our users, and if the outcome is that mixed operations in general
are bad but mixed comparisons are good, I'll trust you. However I want
to reiterate that you really shouldn't improve the situation for 2.7
unless you also forward-port the solution to 3.x.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] test_bigmem and test_bigaddrspace broken
Hello, The title says it all. test_bigmem and test_bigaddrspace are partly broken under py3k, and no buildbot ever tests that they run successfully. As an example, here is a test_bigmem run under a 16GB machine. It gets "killed" at some point, probably by the Linux kernel's OOM killer, although I instruct the test suite to only use a fraction of the available RAM. There are also a couple of failures: $ ./python -m test.regrtest -v -M 8Gb test_bigmem == CPython 3.2a0 (py3k, Mar 17 2010, 18:38:10) [GCC 4.3.2] == Linux-a.b.c.d-grsec--grs-ipv4-64-x86_64-with-debian-lenny-sid == /home/antoine/py3k/py3k/build/test_python_10071 test_bigmem test_capitalize (test.test_bigmem.StrTest) ... Skipping test_capitalize because of memory constraint ok test_center (test.test_bigmem.StrTest) ... ok test_compare (test.test_bigmem.StrTest) ... Skipping test_compare because of memory constraint ok test_concat (test.test_bigmem.StrTest) ... ok test_contains (test.test_bigmem.StrTest) ... Skipping test_contains because of memory constraint ok test_count (test.test_bigmem.StrTest) ... Skipping test_count because of memory constraint ok test_encode (test.test_bigmem.StrTest) ... ERROR test_encode_ascii (test.test_bigmem.StrTest) ... ok test_encode_raw_unicode_escape (test.test_bigmem.StrTest) ... ok test_encode_utf32 (test.test_bigmem.StrTest) ... ok test_encode_utf7 (test.test_bigmem.StrTest) ... ok test_endswith (test.test_bigmem.StrTest) ... Skipping test_endswith because of memory constraint ok test_expandtabs (test.test_bigmem.StrTest) ... Skipping test_expandtabs because of memory constraint ok test_find (test.test_bigmem.StrTest) ... Skipping test_find because of memory constraint ok test_format (test.test_bigmem.StrTest) ... Skipping test_format because of memory constraint ok test_hash (test.test_bigmem.StrTest) ... ok test_index (test.test_bigmem.StrTest) ... Skipping test_index because of memory constraint ok test_isalnum (test.test_bigmem.StrTest) ... Skipping test_isalnum because of memory constraint ok test_isalpha (test.test_bigmem.StrTest) ... Skipping test_isalpha because of memory constraint ok test_isdigit (test.test_bigmem.StrTest) ... Skipping test_isdigit because of memory constraint ok test_islower (test.test_bigmem.StrTest) ... Skipping test_islower because of memory constraint ok test_isspace (test.test_bigmem.StrTest) ... Skipping test_isspace because of memory constraint ok test_istitle (test.test_bigmem.StrTest) ... Skipping test_istitle because of memory constraint ok test_isupper (test.test_bigmem.StrTest) ... Skipping test_isupper because of memory constraint ok test_join (test.test_bigmem.StrTest) ... Skipping test_join because of memory constraint ok test_ljust (test.test_bigmem.StrTest) ... ok test_lower (test.test_bigmem.StrTest) ... Skipping test_lower because of memory constraint ok test_lstrip (test.test_bigmem.StrTest) ... ok test_repeat (test.test_bigmem.StrTest) ... ok test_replace (test.test_bigmem.StrTest) ... Skipping test_replace because of memory constraint ok test_repr_large (test.test_bigmem.StrTest) ... Skipping test_repr_large because of memory constraint ok test_repr_small (test.test_bigmem.StrTest) ... Skipping test_repr_small because of memory constraint ok test_rfind (test.test_bigmem.StrTest) ... Skipping test_rfind because of memory constraint ok test_rindex (test.test_bigmem.StrTest) ... Skipping test_rindex because of memory constraint ok test_rjust (test.test_bigmem.StrTest) ... ok test_rstrip (test.test_bigmem.StrTest) ... ok test_slice_and_getitem (test.test_bigmem.StrTest) ... Skipping test_slice_and_getitem because of memory constraint ok test_split_large (test.test_bigmem.StrTest) ... Skipping test_split_large because of memory constraint ok test_split_small (test.test_bigmem.StrTest) ... Skipping test_split_small because of memory constraint ok test_splitlines (test.test_bigmem.StrTest) ... Skipping test_splitlines because of memory constraint ok test_startswith (test.test_bigmem.StrTest) ... Skipping test_startswith because of memory constraint ok test_strip (test.test_bigmem.StrTest) ... ok test_swapcase (test.test_bigmem.StrTest) ... Skipping test_swapcase because of memory constraint ok test_title (test.test_bigmem.StrTest) ... Skipping test_title because of memory constraint ok test_translate (test.test_bigmem.StrTest) ... Skipping test_translate because of memory constraint ok test_unicode_repr (test.test_bigmem.StrTest) ... Skipping test_unicode_repr because of memory constraint ok test_unicode_repr_overflow (test.test_bigmem.StrTest) ... Skipping test_unicode_repr_overflow because of memory constraint ok test_unicode_repr_wide (test.test_bigmem.StrTest) ... Skipping test_unicode_repr_wide because of memory constraint ok test_upper (test.test_bigmem.StrTest) ... Skipping test_upper because of memory constraint ok test_zfill (test.test_bigmem.StrTest) ... ok test_capitalize (test.test_bigmem.BytesTest) ... ok te
Re: [Python-Dev] test_bigmem and test_bigaddrspace broken
Hello, Le mercredi 17 mars 2010 à 11:58 -0700, C. Titus Brown a écrit : > On Wed, Mar 17, 2010 at 06:54:33PM +, Antoine Pitrou wrote: > > The title says it all. test_bigmem and test_bigaddrspace are partly > > broken under py3k, and no buildbot ever tests that they run successfully. > > As an example, here is a test_bigmem run under a 16GB machine. It gets > > "killed" at some point, probably by the Linux kernel's OOM killer, > > although I instruct the test suite to only use a fraction of the > > available RAM. There are also a couple of failures: > > I can provide login access to a 24gb machine (Linux), if you or others are > interested. Sometimes it's running other stuff, but it's got lots of swap :) Thanks for the proposal. That wasn't really the point of my message, though :) As mentioned I have access to a 16GB machine which is able to run the tests. The problem is that nobody seems interested in maintaining these tests. As far as I can say, the initial committer is Thomas Wouters, and not much happened since then. It seems to me that big companies such as Google and Apple would be the primarily concerned ones by such scenarios, and willing to ensure they run fine. (by the way, even without swapping and in non-debug mode, the tests are quite long to run) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On 3/17/2010 1:09 PM, Guido van Rossum wrote: Ok, I'll try to stay out of the discussion of which solution is best of our users, and if the outcome is that mixed operations in general are bad but mixed comparisons are good, I'll trust you. However I want to reiterate that you really shouldn't improve the situation for 2.7 unless you also forward-port the solution to 3.x. I agree. Improving 2.7 and not 3.2+ would give people a reason to not move to 3.x. tjr ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Removing Windows 95-related code in posixmodule.c
Now that Windows 9x is no longer supported (per PEP 11), would it be
appropriate to replace the ANSI ("A") versions of Windows OS calls in
posixmodule.c with their Unicode ("W") equivalents and remove the
unicode_file_names() function that determines which version to call?
There are a number of places in that module where we try to decide whether
to call ANSI or UNICODE versions (because only the ANSI versions were fully
supported on Windows 9x). This is still fine on Win32, where the ANSI
versions are still supported for backwards compatibility, but it causes
issues with Windows CE, which doesn't have the ANSI versions at all.
I've been maintaining some fairly ugly patches to cope with this in my
company's Python 2.5.x for CE build, and I'd love to drop them as we move to
a post-2.6 version.
Thanks!
--
Tim Lesher
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Mar 17, 2010, at 12:34 PM, Terry Reedy wrote: > > I agree. Improving 2.7 and not 3.2+ would give people a reason to not move to > 3.x. > FWIW, I think this is mischaracterizing the proposal. The spectrum of options from worst to best is 1) compare but give the wrong answer 2) compare but give the right answer 3) refuse to compare. Py3.x is already in the best position, it refuses to compare. IOW, is already is as improved as it can get. P2.6 is in the worst position. The proposal is to make it better, but not as good as 3.x. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Thu, 18 Mar 2010 07:44:21 am Raymond Hettinger wrote: > The spectrum of options from worst to best is > 1) compare but give the wrong answer > 2) compare but give the right answer > 3) refuse to compare. Why is 3 the best? If there is a right answer to give, surely giving the right answer it is better than not? -- Steven D'Aprano ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_bigmem and test_bigaddrspace broken
> As mentioned I have access to a 16GB machine which is able to run the > tests. The problem is that nobody seems interested in maintaining these > tests. So what do you propose to do? I personally don't see that as a problem. Many rarely-used features get infrequent testing - that's the nature of rarely-used features. In turn, they tend to break over time, until somebody comes along and fixes them. That's open source: scratch your own itches. > As far as I can say, the initial committer is Thomas Wouters, and not > much happened since then. It seems to me that big companies such as > Google and Apple would be the primarily concerned ones by such > scenarios, and willing to ensure they run fine. I'm not sure Apple has really Python applications running that use a lot of memory - being a big company doesn't necessarily mean you operate large machines. I'm sure Google has applications that use that much memory, and I wouldn't be surprised if Google has also bug fixes for various bugs in large objects that they haven't been contributing (yet). In any case, I think it's a good thing that these tests are there for people to run if they have the hardware and the time. If somebody wanted to contribute such hardware as a build slave - that could certainly be arranged. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Removing Windows 95-related code in posixmodule.c
Tim Lesher wrote:
> Now that Windows 9x is no longer supported (per PEP 11), would it be
> appropriate to replace the ANSI ("A") versions of Windows OS calls in
> posixmodule.c with their Unicode ("W") equivalents and remove the
> unicode_file_names() function that determines which version to call?
Not directly. The A versions are still used if you pass byte strings as
file names.
It would be possible to convert byte strings to Unicode strings (using
CP_ACP), but I would only want to consider that if the code complexity
does not increase under that approach (which I doubt).
FWIW, the unicode_file_names function *is* removed in Python 2.7.
Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] test_bigmem and test_bigaddrspace brok en
Martin v. Löwis v.loewis.de> writes: > > > As mentioned I have access to a 16GB machine which is able to run the > > tests. The problem is that nobody seems interested in maintaining these > > tests. > > So what do you propose to do? Nothing. I was pointing out the issue here because it is not obvious that interested people would follow tracker updates. Of course, there is also the problem of having tests which never get run and whose failures don't get noticed (test_zipfile64 is another example, although the last time I tried it ran ok). And the additional problem of buggy tests (test_bigmem shouldn't OOM if I ask it to use 8GB on a machine with around 14GB of free mem). Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
Steven D'Aprano wrote: > On Thu, 18 Mar 2010 07:44:21 am Raymond Hettinger wrote: >> The spectrum of options from worst to best is >> 1) compare but give the wrong answer >> 2) compare but give the right answer >> 3) refuse to compare. > > Why is 3 the best? If there is a right answer to give, surely giving the > right answer it is better than not? I agree with Steven here - for mixed *arithmetic* refusing to get involved is a reasonable choice, because the whole point of Decimals is to get the answers according to base 10 expectations. Allowing implicit conversion of base 2 floats puts that foundation at risk. In what way do comparisons carry the same risk? Decimal.from_float shows that a perfectly adequate mapping from float into the Decimal space can be made, and the comparisons have a clear well-defined answer. It may be slightly confusing to those not familiar with the subtleties of limited precision base 2 vs larger precision base 10, but the boolean result places them in a clearly different category to my mind. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia --- ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Mar 17, 2010, at 1:59 PM, Steven D'Aprano wrote:
> On Thu, 18 Mar 2010 07:44:21 am Raymond Hettinger wrote:
>> The spectrum of options from worst to best is
>> 1) compare but give the wrong answer
>> 2) compare but give the right answer
>> 3) refuse to compare.
>
> Why is 3 the best? If there is a right answer to give, surely giving the
> right answer it is better than not?
From the early days of the decimal module,
we've thought that mixed float-decimal operations
are 1) a bit perilous and 2) have few, if any good
use cases.
Accordingly, any mixed operations should be explicit
rather than implicit:
Decimal('1.1') + Decimal.from_float(2.2)
is better than:
Decimal('1.1') + 2.2
To help the user avoid confusion, we flag the latter with a TypeError:
unsupported operand type(s) for +: 'Decimal' and 'float'.
Unfortunately, in Py2.x, implicit mixed comparisons do not
raise an exception, and instead will silently fail by giving
an incorrect answer:
>>> Decimal('1.1') < 2.2
False
IMO, raising the exception is the right thing to do.
Short of that though, if we're going to give a result,
it should at least be a correct one.
Raymond
New zen:
* right answers are better wrong
* but ill-formed questions are best not answered at all
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Wed, Mar 17, 2010 at 1:44 PM, Raymond Hettinger wrote: > > On Mar 17, 2010, at 12:34 PM, Terry Reedy wrote: > > I agree. Improving 2.7 and not 3.2+ would give people a reason to not move > to 3.x. > > > FWIW, I think this is mischaracterizing the proposal. > The spectrum of options from worst to best is > 1) compare but give the wrong answer > 2) compare but give the right answer > 3) refuse to compare. > Py3.x is already in the best position, it refuses to compare. > IOW, is already is as improved as it can get. > P2.6 is in the worst position. The proposal is to make > it better, but not as good as 3.x. Some people have argued that (2) is better than (3). At the very least, there is a marked discontinuity in this spectrum between (2) and (3). With (1), users are less likely to rely on this feature in 2.x and then have no recourse when converting to 3.x. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
Steven D'Aprano wrote: More explicit than someDecimal == someFloat? Seems pretty explicit to me. Yes. I mean at least specifying either float(someDecimal) == someFloat or someDecimal == Decimal(someFloat). Preferably also whether the conversion is to be as exact as possible or on a minimum-digits basis. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
Guido van Rossum wrote: in some "intuitive complexity" sense an int is a simpler type than a float and a float is a simpler type than a Decimal I don't think this analogy holds. In a mathematical sense, ints are a subset of reals, but binary and decimal floats are just alternative approximate representations of reals, neither one being inherently preferable over the other. One could argue that since all binary floats are exactly representable in decimal but not vice versa, decimal should be regarded as the wider type. But even this doesn't hold when you have a limited number of decimal digits available, which you always do at any given moment with the Decimal type. And even if there are enough digits, an exact conversion mightn't be what you really want. This problem doesn't arise with int->float conversion -- there is only one obvious way of chopping it to fit. -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
Raymond Hettinger wrote: Python 3 doesn't need it because it is possible to not give a result at all. Python 2 does need it because we have to give *some* result. That's not true -- it's possible for comparisons to raise an exception in 2.x, and they sometimes do already: Python 2.5.4 (r254:67916, May 15 2009, 15:21:20) [GCC 4.1.2 20070925 (Red Hat 4.1.2-33)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 1+2j < 3+4j Traceback (most recent call last): File "", line 1, in TypeError: no ordering relation is defined for complex numbers -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Decimal <-> float comparisons in py3k.
On Mar 17, 2010, at 5:02 PM, Greg Ewing wrote: > Raymond Hettinger wrote: > >> Python 3 doesn't need it because it is possible to not give a result >> at all. Python 2 does need it because we have to give *some* >> result. > > That's not true -- it's possible for comparisons to raise > an exception in 2.x, and they sometimes do already: Complex objects do not support __float__. Decimal objects do. If an object supports __float__, then a float comparison coerces its other argument via __float__ and the other argument never gets a chance to raise an exception. >>> class D: def __float__(self): return 3.14 >>> float(D()) 3.1401 >>> float(complex(3.14)) Traceback (most recent call last): File "", line 1, in float(complex(3.14)) TypeError: can't convert complex to float >>> D() < 10.0 True >>> complex(3.14) < 10.0 Traceback (most recent call last): File "", line 1, in complex(3.14) < 10.0 TypeError: no ordering relation is defined for complex numbers Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
