Re: [Python-Dev] new buffer in python2.7
Kristján Valur Jónsson, 27.10.2010 16:32: Sorry, here the tables properly formatted: Certainly looked better on your first try. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new buffer in python2.7
Kristján Valur Jónsson, 27.10.2010 16:28: Notice how a Slice object is generated. Then a PyObject_GetItem() is done. The salient code path is from apply_slice(). A slice object must be constructed and destroyed. If slice object creation bothers you here, it might be worth using a free list in PySlice_New() instead of creating new slice objects on request. Creating a slice of something is not necessarily such a costly operation that it dominates creating the slice object, so optimising the slice request itself sounds like a good idea. You can take a look at how it's done in tupleoject.c if you want to provide a patch. Then, please open a bug tracker ticket and attach the patch there (and post a link to the ticket in this thread). Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new buffer in python2.7
I've already created a patch. See http://bugs.python.org/issue10227. I was working with 2.7 where slicing sequences is done differently than in 3.2, so the difference is not that very great. I'm going to have another go at profiling the 3.2 version later and see why slicing a bytearray is so much more expensive than slicing a bytes object. K > -Original Message- > From: [email protected] > [mailto:[email protected]] On Behalf > Of Stefan Behnel > Sent: 1. nóvember 2010 16:45 > To: [email protected] > Subject: Re: [Python-Dev] new buffer in python2.7 > > Kristján Valur Jónsson, 27.10.2010 16:28: > > Notice how a Slice object is generated. Then a PyObject_GetItem() > is > > done. The salient code path is from apply_slice(). A slice object > must > > be constructed and destroyed. > > If slice object creation bothers you here, it might be worth using a > free > list in PySlice_New() instead of creating new slice objects on request. > > Creating a slice of something is not necessarily such a costly > operation > that it dominates creating the slice object, so optimising the slice > request itself sounds like a good idea. > > You can take a look at how it's done in tupleoject.c if you want to > provide > a patch. Then, please open a bug tracker ticket and attach the patch > there > (and post a link to the ticket in this thread). > > Stefan > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python- > dev/kristjan%40ccpgames.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new buffer in python2.7
Stefan Behnel, 01.11.2010 09:45: If slice object creation bothers you here, it might be worth using a free list in PySlice_New() instead of creating new slice objects on request. [...] You can take a look at how it's done in tupleoject.c if you want to provide a patch. Hmm, that's actually a particularly bad place to look. The implementation in listobject.c is much simpler. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new buffer in python2.7
Ah, yes. There are, in my case. (why do I always seem to be doing stuff that
is different from what you all are doing:)
The particular piece of code is from the chunked reader. It may be reading
rather large chunks at a time (several lots of Kb.):
def recvchunk(socket):
len = socket.unpack('i', recv_exactly(socket, 4))
return recv_exactly(len)
#old style
def recv_exactly(socket, length):
data = []
while length:
got = socket.receive(length)
if not got: raise EOFError
data.append(got)
length -= len(got)
return "".join(data)
#new style
def recv_exactly(socket, length):
data = bytearray(length)
view = memoryview(data)
while length:
got = socket.receive_into(view[-length:])
if not got: raise EOFError
length -= len(got)
return data
Here I spot another optimzation oppertunity: let memoryview[:] return self,
since the object is immutable, I believe.
K
> -Original Message-
> From: "Martin v. Löwis" [mailto:[email protected]]
> Sent: 1. nóvember 2010 14:22
> To: Kristján Valur Jónsson
> Cc: [email protected]
> Subject: Re: [Python-Dev] new buffer in python2.7
>
>
> Assuming there are multiple recv calls. For a typical struct, all data
> will come out of the stream with a single recv. so no join will be
> necessary.
>
> Regards,
> Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] str.format_from_mapping
On Mon, Nov 1, 2010 at 10:53 AM, Barry Warsaw wrote: > On Oct 31, 2010, at 04:39 PM, Eric Smith wrote: > >>What are your thoughts on adding a str.format_from_mapping (or similar >>name, maybe the suggested "format_map") to 3.2? > > +1 for the shorter name. +1 for a format_map() method that takes a single mapping argument (Eric's patch on the issue). -1 for the most recent patch attached to that issue that allows further positional arguments after the mapping object. (Raymond mentioned it on the issue, but I'll mention it here as well: this addition would fall under the "case-by-case exemption" clause in the moratorium PEP, since it adds a new method to a builtin type) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] new buffer in python2.7
2010/11/1 Kristján Valur Jónsson : > Ah, yes. There are, in my case. (why do I always seem to be doing stuff > that is different from what you all are doing:) I would guess that most of us aren't writing MMOs for a living. Gamers seem to be a particularly demanding breed of user :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Stable sort and partial order
On Mon, 01 Nov 2010 02:55:35 +
Michael Foord wrote:
> Having a more efficient 'slow-path' and moving to that by default would
> fix it. The bug is only a duplicate of the bug in sorted - caused by the
> fact that sets / frozensets can't be sorted in the standard Python way
> (their less than comparison adheres to the set definition). This is
> something that will probably surprise many Python developers:
>
> >>> a = [{2,4}, {1,2}]
> >>> b = a[::-1]
> >>> sorted(a)
> [set([2, 4]), set([1, 2])]
> >>> sorted(b)
> [set([1, 2]), set([2, 4])]
>
> (Fixing the bug in sorted would fix assertItemsEqual ;-)
How is this a bug? The sort algorithm is stable, which means the above
behaviour is a feature.
I see no easy way of eliminating the O(n*n) issue. Custom key functions
can't work in all cases.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On Mon, 01 Nov 2010 12:33:31 +0100, Antoine Pitrou wrote:
> On Mon, 01 Nov 2010 02:55:35 +
> Michael Foord wrote:
> > Having a more efficient 'slow-path' and moving to that by default would
> > fix it. The bug is only a duplicate of the bug in sorted - caused by the
> > fact that sets / frozensets can't be sorted in the standard Python way
> > (their less than comparison adheres to the set definition). This is
> > something that will probably surprise many Python developers:
> >
> > >>> a = [{2,4}, {1,2}]
> > >>> b = a[::-1]
> > >>> sorted(a)
> > [set([2, 4]), set([1, 2])]
> > >>> sorted(b)
> > [set([1, 2]), set([2, 4])]
> >
> > (Fixing the bug in sorted would fix assertItemsEqual ;-)
>
> How is this a bug? The sort algorithm is stable, which means the above
> behaviour is a feature. I see no easy way of eliminating the O(n*n)
> issue. Custom key functions can't work in all cases.
Even granting some theoretical way to sort sets by their contents, it
still wouldn't be a bug in sorted. Sorted is just using the results
returned by '__lt__', which is what it should do. Special casing sets
in sorted would be wrong.
--
R. David Murray www.bitdance.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
I've been sitting on the sideline seeing this unfold. We've seen some different viewpoints on the matter and I'm happy to see that I'm not the only one lamenting the proclaimed death of the 2.x linage. However, As correctly stated by Martin, I merely voiced a suggestion and I have gotten helpful counter-suggestions. A private branch is fine (More correctly a fork, even, as people have pointed out) and Hg is going to support user-branches. In the meantime, however, unless someone strongly objects, I'm probably going to set up a temporary branch off /release27-maint under /stackless/sandboxes/ until the Hg switchover. Name undecided yet. Cheers, Kristján > -Original Message- > From: [email protected] > [mailto:[email protected]] On Behalf > Of "Martin v. Löwis" > Sent: 29. október 2010 22:13 > This thread was started by a specific proposal from Kristjan, and > Kristjan got a specific suggestion on how to proceed (namely, wait > for the Mercurial switchover, then publish his changes in a branch). > So despite the more general subject (which I think is still mostly > hypothetical), the real issue Kristjan raised has been resolved, > AFAICT (although Kristjan has not yet voiced an opinion of whether > he finds that resolution acceptable). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] time.wallclock()
Working on Condition variables and semaphores (see http://bugs.python.org/issue10260) I noticed that time.time() was being used to correctly time blocking system calls. On windows, I would have used time.clock() but reading the documentation made me realize that on Unix that would return CPU seconds which are useless when blocking. However, on Windows, time.clock() has a much higher resolution, apart from being a "wallclock" time, and is thus better suited to timing that time.time(). In addition, time.time() has the potential of giving unexpected results if someone messes with the system clock. I was wondering if it were helpful to have a function such as time.wallclock() which is specified to give relative wallclock time between invocations or an approximation thereof, to the system's best ability? We could then choose this to be an alias of time.clock() on windows and time.time() on any other machine, or even have custom implementations on machines that support such a notion. Kristján ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Change to logging Formatters: support for alternative format styles
On Sun, Oct 31, 2010 at 9:55 AM, Vinay Sajip wrote:
> Olemis Lang gmail.com> writes:
>>
>> On Fri, Oct 29, 2010 at 10:07 AM, Barry Warsaw python.org> wrote:
>> > I haven't played with it yet, but do you think it makes sense to add a
>> > 'style' keyword argument to basicConfig()? That would make it pretty easy
>> > to get the formatting style you want without having to explicitly
>> > instantiate a Formatter, at least for simple logging clients.
>> >
>>
>> Since this may be considered as a little sophisticated, I'd rather
>> prefer these new classes to be added to configuration sections using
>> fileConfig (and default behavior if missing), and still leave
>> `basicConfig` unchanged (i.e. *basic*) .
>>
>
> Actually it's no biggie to have an optional style argument for basicConfig.
> People who don't use it don't have to specify it; the style argument would
> only
> apply if format was specified.
>
ok
> For some people, use of {} over % is more about personal taste than about the
> actual usage of str.format's flexibility;
Thought you were talking about me, you only needed to say «he has
black hair and blue eyes» ... ;o)
> we may as well accommodate that
> preference, as it encourages in a small way the use of {}-formatting.
>
ok , nevermind , it's ok for me anyway (provided that sections for
`fileConfig` will be available) .
--
Regards,
Olemis.
Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/
Featured article:
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] time.wallclock()
On 01/11/2010 14:00, Kristján Valur Jónsson wrote: Working on Condition variables and semaphores (see http://bugs.python.org/issue10260) I noticed that time.time() was being used to correctly time blocking system calls. On windows, I would have used time.clock() but reading the documentation made me realize that on Unix that would return CPU seconds which are useless when blocking. However, on Windows, time.clock() has a much higher resolution, apart from being a "wallclock" time, and is thus better suited to timing that time.time(). In addition, time.time() has the potential of giving unexpected results if someone messes with the system clock. I was wondering if it were helpful to have a function such as time.wallclock() which is specified to give relative wallclock time between invocations or an approximation thereof, to the system's best ability? We could then choose this to be an alias of time.clock() on windows and time.time() on any other machine, or even have custom implementations on machines that support such a notion. I think this would be helpful. Having to do platform specific checks to choose which time function to use is annoying. Michael Kristján ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On 01/11/2010 11:33, Antoine Pitrou wrote:
On Mon, 01 Nov 2010 02:55:35 +
Michael Foord wrote:
Having a more efficient 'slow-path' and moving to that by default would
fix it. The bug is only a duplicate of the bug in sorted - caused by the
fact that sets / frozensets can't be sorted in the standard Python way
(their less than comparison adheres to the set definition). This is
something that will probably surprise many Python developers:
>>> a = [{2,4}, {1,2}]
>>> b = a[::-1]
>>> sorted(a)
[set([2, 4]), set([1, 2])]
>>> sorted(b)
[set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-)
How is this a bug? The sort algorithm is stable, which means the above
behaviour is a feature.
Well, bug is the wrong word as it is obviously an intended feature (or
consequence of a feature). I still think, given the general behaviour of
Python sorting, that it is unexpected. It breaks what is usually an
invariant for sorting without an explicit key that sortable types
sorted(l) == sorted(l[::-1]).
There is however a note in the set documentation:
Since sets only define partial ordering (subset relationships), the
output of the list.sort() method is undefined for lists of sets.
I see no easy way of eliminating the O(n*n) issue. Custom key functions
can't work in all cases.
Right. Special casing sets and frozensets would be one (particularly
inelegant) way however.
All the best,
Michael
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
--
http://www.voidspace.org.uk/
READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] okay to remove argparse.__all__?
I think the easiest and most sensible way to address http://bugs.python.org/issue9353 is to simply remove the __all__ definition from argparse - everything that doesn't start with an underscore in the module is already meant to be exposed. But then I wonder - is __all__ considered part of the public API of a module? Or is it okay to just remove it and assume that no one should have been accessing it directly anyway? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On 01/11/2010 14:48, Steven Bethard wrote: I think the easiest and most sensible way to address http://bugs.python.org/issue9353 is to simply remove the __all__ definition from argparse - everything that doesn't start with an underscore in the module is already meant to be exposed. But then I wonder - is __all__ considered part of the public API of a module? Or is it okay to just remove it and assume that no one should have been accessing it directly anyway? Isn't it better to add the missing elements - what is the problem with that approach? Not defining __all__ will mean that "from argparse import *" will also export all the modules you import (copy, os, re, sys, textwrap). All the best, Michael Steve -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On Mon, Nov 1, 2010 at 2:53 PM, Michael Foord wrote: > On 01/11/2010 14:48, Steven Bethard wrote: >> >> I think the easiest and most sensible way to address >> http://bugs.python.org/issue9353 is to simply remove the __all__ >> definition from argparse - everything that doesn't start with an >> underscore in the module is already meant to be exposed. >> >> But then I wonder - is __all__ considered part of the public API of a >> module? Or is it okay to just remove it and assume that no one should >> have been accessing it directly anyway? > > Isn't it better to add the missing elements - what is the problem with that > approach? It just requires extra synchronization, and history shows that I always forget to add them. ;-) > Not defining __all__ will mean that "from argparse import *" will also > export all the modules you import (copy, os, re, sys, textwrap). That won't happen in the case of argparse - all modules are imported like "import os as _os". Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On Mon, Nov 1, 2010 at 7:53 AM, Michael Foord wrote: > On 01/11/2010 14:48, Steven Bethard wrote: >> >> I think the easiest and most sensible way to address >> http://bugs.python.org/issue9353 is to simply remove the __all__ >> definition from argparse - everything that doesn't start with an >> underscore in the module is already meant to be exposed. >> >> But then I wonder - is __all__ considered part of the public API of a >> module? Or is it okay to just remove it and assume that no one should >> have been accessing it directly anyway? > > Isn't it better to add the missing elements - what is the problem with that > approach? Agreed, that's what I would do. > Not defining __all__ will mean that "from argparse import *" will also > export all the modules you import (copy, os, re, sys, textwrap). Well, the copy of argparse.py that I have carefully renames those to _copy, _os etc. to avoid this. You never know. It is also possible to write automated tests that flag likely missing symbols in __all__ (as well as symbols in __all__ missing from the module). -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On Mon, Nov 1, 2010 at 2:57 PM, Guido van Rossum wrote: > On Mon, Nov 1, 2010 at 7:53 AM, Michael Foord > wrote: >> On 01/11/2010 14:48, Steven Bethard wrote: >>> But then I wonder - is __all__ considered part of the public API of a >>> module? Or is it okay to just remove it and assume that no one should >>> have been accessing it directly anyway? >> >> Isn't it better to add the missing elements - what is the problem with that >> approach? > > Agreed, that's what I would do. Ok, sounds good. > It is also possible to write automated tests that flag likely missing > symbols in __all__ (as well as symbols in __all__ missing from the > module). Yep, I plan on doing that. I already had a test something like this to remind me how I broke __all__ before. ;-) Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On 01/11/2010 14:57, Guido van Rossum wrote: [snip...] Not defining __all__ will mean that "from argparse import *" will also export all the modules you import (copy, os, re, sys, textwrap). Well, the copy of argparse.py that I have carefully renames those to _copy, _os etc. to avoid this. Bah Sorry about that. Michael -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On Tue, Nov 2, 2010 at 12:57 AM, Guido van Rossum wrote:
> It is also possible to write automated tests that flag likely missing
> symbols in __all__ (as well as symbols in __all__ missing from the
> module).
These days, test___all__ checks that everything in __all__ exists in
standard library modules. It is also possible for individual module
tests to include a check that goes the other way along the lines of:
def test_all_is_complete():
known_private = {"known", "unexported", "names"}
expected_public = (k for k in mod.__dict__ if k not in known_private
and not k.startswith("_"))
self.assertEqual(set(mod.__all__), expected_public)
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] time.wallclock()
Ok, please see http://bugs.python.org/issue10278 K From: Michael Foord [mailto:[email protected]] Sent: 1. nóvember 2010 22:05 To: Kristján Valur Jónsson Cc: [email protected] Subject: Re: [Python-Dev] time.wallclock() I think this would be helpful. Having to do platform specific checks to choose which time function to use is annoying. Michael ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On Mon, 01 Nov 2010 14:26:19 -, Michael Foord
wrote:
> On 01/11/2010 11:33, Antoine Pitrou wrote:
> > On Mon, 01 Nov 2010 02:55:35 +
> > Michael Foord wrote:
> >> Having a more efficient 'slow-path' and moving to that by default would
> >> fix it. The bug is only a duplicate of the bug in sorted - caused by the
> >> fact that sets / frozensets can't be sorted in the standard Python way
> >> (their less than comparison adheres to the set definition). This is
> >> something that will probably surprise many Python developers:
> >>
> >> >>> a =3D [{2,4}, {1,2}]
> >> >>> b =3D a[::-1]
> >> >>> sorted(a)
> >> [set([2, 4]), set([1, 2])]
> >> >>> sorted(b)
> >> [set([1, 2]), set([2, 4])]
> >>
> >> (Fixing the bug in sorted would fix assertItemsEqual ;-)
> > How is this a bug? The sort algorithm is stable, which means the above
> > behaviour is a feature.
>
> Well, bug is the wrong word as it is obviously an intended feature (or
> consequence of a feature). I still think, given the general behaviour of
> Python sorting, that it is unexpected. It breaks what is usually an
> invariant for sorting without an explicit key that sortable types
> sorted(l) = sorted(l[::-1]).
Well, as Antoine pointed out, Python's sorting algorithm is stable,
so that is in fact *not* an invariant:
>>> x = ['abcd', 'foo'*50, 'foo'*50, 'dkke']
>>> y = x[::-1]
>>> [id(n) for n in sorted(x)]
[3073747680, 3073747904, 3073747624, 3073747512]
>>> [id(n) for n in sorted(y)]
[3073747680, 3073747904, 3073747512, 3073747624]
Yes, == usually hides the fact that the *objects* are in a different
order, but obviously that doesn't apply to sets :)
--
R. David Murray www.bitdance.com
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On 01/11/2010 15:10, R. David Murray wrote:
On Mon, 01 Nov 2010 14:26:19 -, Michael Foord
wrote:
On 01/11/2010 11:33, Antoine Pitrou wrote:
On Mon, 01 Nov 2010 02:55:35 +
Michael Foord wrote:
Having a more efficient 'slow-path' and moving to that by default would
fix it. The bug is only a duplicate of the bug in sorted - caused by the
fact that sets / frozensets can't be sorted in the standard Python way
(their less than comparison adheres to the set definition). This is
something that will probably surprise many Python developers:
>>> a =3D [{2,4}, {1,2}]
>>> b =3D a[::-1]
>>> sorted(a)
[set([2, 4]), set([1, 2])]
>>> sorted(b)
[set([1, 2]), set([2, 4])]
(Fixing the bug in sorted would fix assertItemsEqual ;-)
How is this a bug? The sort algorithm is stable, which means the above
behaviour is a feature.
Well, bug is the wrong word as it is obviously an intended feature (or
consequence of a feature). I still think, given the general behaviour of
Python sorting, that it is unexpected. It breaks what is usually an
invariant for sorting without an explicit key that sortable types
sorted(l) = sorted(l[::-1]).
Well, as Antoine pointed out, Python's sorting algorithm is stable,
so that is in fact *not* an invariant:
x = ['abcd', 'foo'*50, 'foo'*50, 'dkke']
y = x[::-1]
[id(n) for n in sorted(x)]
[3073747680, 3073747904, 3073747624, 3073747512]
[id(n) for n in sorted(y)]
[3073747680, 3073747904, 3073747512, 3073747624]
Yes, == usually hides the fact that the *objects* are in a different
order, but obviously that doesn't apply to sets :)
Sorry, that should have been sorted(l) == sorted(l[::-1]) - which *is*
the case for your example above.
Michael
--
R. David Murray www.bitdance.com
--
http://www.voidspace.org.uk/
READ CAREFULLY. By accepting and reading this email you agree,
on behalf of your employer, to release me from all obligations
and waivers arising from any and all NON-NEGOTIATED agreements,
licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap,
confidentiality, non-disclosure, non-compete and acceptable use
policies (”BOGUS AGREEMENTS”) that I have entered into with your
employer, its partners, licensors, agents and assigns, in
perpetuity, without prejudice to my ongoing rights and privileges.
You further represent that you have the authority to release me
from any BOGUS AGREEMENTS on behalf of your employer.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PyMem_MALLOC vs PyMem_Malloc
On 2010/10/31 2:32, M.-A. Lemburg wrote: M.-A. Lemburg wrote: Hirokazu Yamamoto wrote: Hello. I found several codes using PyMem_Free to free allocated memory with PyMem_MALLOC (ie: PyUnicode_AsWideCharString) Is it safe? Within the interpreter: yes. In extensions: depends on the platform, but probably not. The macros provide faster access to the C lib malloc calls. The functions need to be used in extensions in case the interpreter will free the resource or the extension wants to free an interpreter allocated resource. They provide access to the malloc calls used by the interpreter, which may operate on a different heap than the extensions. Within an extension the macros use the extension heap. A subtle, but important difference. BTW: If you were referring to extensions using PyMem_Free() to deallocate memory allocated in the interpreter using PyMem_MALLOC(), then that's exactly how things should be done. I was referring to use of the two mentioned APIs within an extension. Thank you for reply, probably I could understand. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] okay to remove argparse.__all__?
On Mon, Nov 01, 2010 at 02:55:25PM +, Steven Bethard wrote:
> On Mon, Nov 1, 2010 at 2:53 PM, Michael Foord
> wrote:
> > Isn't it better to add the missing elements - what is the problem with that
> > approach?
>
> It just requires extra synchronization, and history shows that I
> always forget to add them. ;-)
Automate:
for key, value in globals().items():
if not key.startswith('_'):
__all__.append(key)
Further filter (by key or value) to your needs.
Oleg.
--
Oleg Broytmanhttp://phd.pp.ru/[email protected]
Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On Mon, 01 Nov 2010 15:14:36 -, Michael Foord wrote: > On 01/11/2010 15:10, R. David Murray wrote: > > On Mon, 01 Nov 2010 14:26:19 -, Michael > > Foord wrote: > >> Well, bug is the wrong word as it is obviously an intended feature (or > >> consequence of a feature). I still think, given the general behaviour of > >> Python sorting, that it is unexpected. It breaks what is usually an > >> invariant for sorting without an explicit key that sortable types > >> sorted(l) = sorted(l[::-1]). > > Well, as Antoine pointed out, Python's sorting algorithm is stable, > > so that is in fact *not* an invariant: > > > x = ['abcd', 'foo'*50, 'foo'*50, 'dkke'] > y = x[::-1] > [id(n) for n in sorted(x)] > > [3073747680, 3073747904, 3073747624, 3073747512] > [id(n) for n in sorted(y)] > > [3073747680, 3073747904, 3073747512, 3073747624] > > > > Yes, == usually hides the fact that the *objects* are in a different > > order, but obviously that doesn't apply to sets :) > > > > Sorry, that should have been sorted(l) == sorted(l[::-1]) - which *is* > the case for your example above. Yes, I know that's what you meant, that's why I said "== usually hides this". If you are restricting yourself to built in types, then your invariant is mostly true but (IMO) misleading, as set demonstrates. And it certainly doesn't have to be true for custom types, even if they don't redefine __lt__. You can argue that in a good design it should be, but as the set example indicates, there are problem domains where it is useful for it not to be. Or, to put it another way, *if* there is a bug here it would be in set, not sorted. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r85987 - python/branches/py3k/Lib/test/test_os.py
On 2010/10/31 6:24, brian.curtin wrote:
Author: brian.curtin
Date: Sat Oct 30 23:24:21 2010
New Revision: 85987
Log:
Fix #10257. Clear resource warnings by using os.popen's context manager.
Modified:
python/branches/py3k/Lib/test/test_os.py
Modified: python/branches/py3k/Lib/test/test_os.py
==
--- python/branches/py3k/Lib/test/test_os.py(original)
+++ python/branches/py3k/Lib/test/test_os.pySat Oct 30 23:24:21 2010
@@ -406,17 +406,19 @@
os.environ.clear()
if os.path.exists("/bin/sh"):
os.environ.update(HELLO="World")
-value = os.popen("/bin/sh -c 'echo $HELLO'").read().strip()
-self.assertEquals(value, "World")
+with os.popen("/bin/sh -c 'echo $HELLO'") as popen:
+value = popen.read().strip()
+self.assertEquals(value, "World")
Does this really cause resource warning? I think os.popen instance
won't be into traceback because it's not declared as variable. So I
suppose it will be deleted by reference count == 0 even when exception
occurs.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On Tue, Nov 2, 2010 at 1:33 AM, R. David Murray wrote: > Or, to put it another way, *if* there is a bug here it would be in set, > not sorted. Put me in the "it's not a bug, it's a feature" camp. Providing a "elements equal" check that doesn't rely on LT providing a total ordering is a non-trivial exercise. Looking at assertItemsEqual, I'd be inclined to insert a check that falls back to the "unorderable_list_difference" approach in the case where "expected != sorted(reversed(expected))" (only need to check the one, since if the expected values are totally ordered, while the actual values are not, this should show up when comparing the elements). It slows down the fast path a bit, but the updated function should at least handle partial orderings more correctly than it does now. Cheers, Nick. P.S. Late night post, so I may be missing something obvious... -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On 01/11/2010 16:23, Nick Coghlan wrote: On Tue, Nov 2, 2010 at 1:33 AM, R. David Murray wrote: Or, to put it another way, *if* there is a bug here it would be in set, not sorted. Put me in the "it's not a bug, it's a feature" camp. Providing a "elements equal" check that doesn't rely on LT providing a total ordering is a non-trivial exercise. Looking at assertItemsEqual, I'd be inclined to insert a check that falls back to the "unorderable_list_difference" approach in the case where "expected != sorted(reversed(expected))" If that is sufficient then it would be a nice way of keeping the fast path. (I'm not arguing that Antoine and R. David aren't correct in what they're saying about set ordering - I'm just saying that I was surprised and bet I'm not the only one. Bit of a dead end discussion. :-) Michael (only need to check the one, since if the expected values are totally ordered, while the actual values are not, this should show up when comparing the elements). It slows down the fast path a bit, but the updated function should at least handle partial orderings more correctly than it does now. Cheers, Nick. P.S. Late night post, so I may be missing something obvious... -- http://www.voidspace.org.uk/ READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r85987 - python/branches/py3k/Lib/test/test_os.py
On Tue, Nov 2, 2010 at 2:10 AM, Hirokazu Yamamoto wrote: > Does this really cause resource warning? I think os.popen instance > won't be into traceback because it's not declared as variable. So I > suppose it will be deleted by reference count == 0 even when exception > occurs. Any time __del__ has to close the resource triggers ResourceWarning, regardless of whether that is due to the cyclic garbage collector or the refcount naturally falling to zero. In the past dealing with this was clumsy, so it made sense to rely on CPython's refcounting to do the work. However, we have better tools for deterministic resource management now (in the form of context managers), so these updates help make the standard library and its test suite more suitable for use with non-refcounting Python implementations (such as PyPy, Jython and IronPython). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Stable sort and partial order
On Tue, Nov 2, 2010 at 2:26 AM, Michael Foord wrote:
> On 01/11/2010 16:23, Nick Coghlan wrote:
>> Looking at assertItemsEqual, I'd be inclined to insert a check that
>> falls back to the "unorderable_list_difference" approach in the case
>> where "expected != sorted(reversed(expected))"
>
> If that is sufficient then it would be a nice way of keeping the fast path.
As far as I can tell, that check is a valid partial ordering detector
for any sequence that contains one or more pairs of items for which
LT, EQ and GE are all False:
>>> seq = [{1}, {2}]
>>> seq[0] < seq[1]
False
>>> seq[0] == seq[1]
False
>>> seq[0] > seq[1]
False
>>> sorted(seq)
[{1}, {2}]
>>> sorted(reversed(sorted(seq)))
[{2}, {1}]
Obviously, if the sequence doesn't contain any such items (e.g. all
subsets of each other), then it will look like a total ordering and
use the fast path. I see that as an upside :)
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Continuing 2.x
2010/11/1 Kristján Valur Jónsson : > I've been sitting on the sideline seeing this unfold. > We've seen some different viewpoints on the matter and I'm happy to see that > I'm not the only one lamenting the proclaimed death of the 2.x linage. > However, As correctly stated by Martin, I merely voiced a suggestion and I > have gotten helpful counter-suggestions. > A private branch is fine (More correctly a fork, even, as people have pointed > out) and Hg is going to support user-branches. > In the meantime, however, unless someone strongly objects, I'm probably going > to set up a temporary branch off /release27-maint under /stackless/sandboxes/ > until the Hg switchover. Name undecided yet. No objection from me; branches in svn are for experimental stuff and this is what you are proposing. -Brett > > Cheers, > Kristján > > >> -Original Message- >> From: [email protected] >> [mailto:[email protected]] On Behalf >> Of "Martin v. Löwis" >> Sent: 29. október 2010 22:13 >> This thread was started by a specific proposal from Kristjan, and >> Kristjan got a specific suggestion on how to proceed (namely, wait >> for the Mercurial switchover, then publish his changes in a branch). >> So despite the more general subject (which I think is still mostly >> hypothetical), the real issue Kristjan raised has been resolved, >> AFAICT (although Kristjan has not yet voiced an opinion of whether >> he finds that resolution acceptable). > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Change to logging Formatters: support for alternative format styles
Olemis Lang gmail.com> writes:
>
> > For some people, use of {} over % is more about personal taste than about
> > the
> > actual usage of str.format's flexibility;
>
> Thought you were talking about me, you only needed to say «he has
> black hair and blue eyes» ... ;o)
>
No, it was a general comment; I don't know your preferences. The basicConfig()
change has now been checked into the py3k branch.
Regards,
Vinay Sajip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cleaning-up the new unittest API
On 10/31/2010 10:55 PM, Michael Foord wrote:
fact that sets / frozensets can't be sorted in the standard Python way
(their less than comparison adheres to the set definition). This is
something that will probably surprise many Python developers:
Any programmer who sorts (or uses functions that depend on proper
sorting) should know and respect the difference between partial orders,
such as set inclusion, and total orders, such as lex order of sequences.
So I am surprised by the above claim ;-).
>>> a = [{2,4}, {1,2}]
>>> b = a[::-1]
>>> sorted(a)
[set([2, 4]), set([1, 2])]
>>> sorted(b)
[set([1, 2]), set([2, 4])]
The bug is not in the sort method, but the attempt to sort partially
ordered items, which are not properly sortable.
a = [{2,4}, {1,2}]
b = a[::-1]
print(sorted(a,key=sorted))
#[{1, 2}, {2, 4}]
print(sorted(b,key=sorted))
#[{1, 2}, {2, 4}]
A test method (or internal branch) that depends on sorting to work
properly could just refuse to work with sets (and frozensets).
--
Terry Jan Reedy
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cleaning-up the new unittest API
On Mon, Nov 01, 2010 at 02:37:33PM -0400, Terry Reedy wrote: > On 10/31/2010 10:55 PM, Michael Foord wrote: > >> fact that sets / frozensets can't be sorted in the standard Python way >> (their less than comparison adheres to the set definition). This is >> something that will probably surprise many Python developers: > > Any programmer who sorts (or uses functions that depend on proper > sorting) should know and respect the difference between partial orders, > such as set inclusion, and total orders, such as lex order of sequences. > So I am surprised by the above claim ;-). Huh. Count me out. I guess I don't live up to your standards. --titus p.s. Seriously? I can accept that there's a rational minimalist argument for this "feature", but arguing that it's somehow the responsibility of a programmer to *expect* this seems kind of whack. -- C. Titus Brown, [email protected] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] On breaking modules into packages Was: [issue10199] Move Demo/turtle under Lib/
On Wed, Oct 27, 2010 at 03:42, Antoine Pitrou wrote: > On Tue, 26 Oct 2010 22:06:37 -0400 > Alexander Belopolsky wrote: >> >> While I appreciate your and Michael's eloquence, I don't really see >> why five 400-line modules are necessarily easier to maintain than one >> 2000-line module. Splitting code into modules is certainly a good >> thing when the resulting modules can be used independently. This >> helps users write leaner programs, reduces mental footprint of >> individual modules, etc, etc. The split unittest module does not >> bring any such benefits. It still presents a single "big-ball-of-mud" >> namespace, only rather than implemented in a single file, it is now >> swept in from eight different files. > > Are you saying that it has become a pile of medium-sized balls of mud? > I would like to say thanks for the mud, Michael! It's high quality mud > for sure. I realize I am a little late in this reply but issue 10273 linked to this and so now I am actually bothering to read this thread since it felt like bikeshedding when the thread began. I think the issue here is that the file structure of the code no longer matches the public API documented by unittest. Personally I, like most people it seems, prefer source files to be structured in a way to match the public API. In the case of unittest Michael didn't. He did ask python-dev if it was okay to do what he did, we all kept quiet, and now we have realized that most of us prefer to have files that mirror the API; lesson learned. But Python 2.7 shipped with this file layout so we have to stick with it lest we break any imports out there that use the package-like file structure Michael went with (which we could actually document and use if we wanted now that Michael has already broken things up). Reversing the trend by sticking all the code into unittest/__init__.py and then sticking import shims into the existing modules would be a stupid waste of time, especially considering the head maintainer of the package likes it the way it is. So basically it seems like we have learned a lesson: we prefer to have our code structured in files that match the public API. I think that is a legitimate design rule for the stdlib to follow from now on, but in the case of unittest it's too late to change it back (and it's a minor price to pay to learn this lesson and to have Michael maintaining unittest like he has been, plus we could consider using the new structure so that the public API matches the file structure when the need arises). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
