Re: [Python-Dev] 2.5a2 try/except slow-down: Convert to type?
This is probably orthogonal to the problem, however, you may want to look into trying to speed up Python/errors.c. This link will probably not work due to sessions, but click on the latest run for python and Python/errors.c http://coverage.livinglogic.de/coverage/web/selectEntry.do?template=2850&entryToSelect=547460 If you look at the coverage numbers, many functions are called millions of times. In many cases traceback is passed as NULL in other functions within errors.c. If you could get rid of that check and possibly others and do some fast pathing in there, it might speed up exceptions some (or normal cases where exceptions are raised internally, then discarded). n -- On 5/24/06, Sean Reifschneider <[EMAIL PROTECTED]> wrote: > We've done some more research on it, and Richard Jones is working on it > right now. We'll see how it works, probably tomorrow. > > Thanks, > Sean > -- > If you don't have time to do it right, when will you ever find time to do > it over? > Sean Reifschneider, Member of Technical Staff <[EMAIL PROTECTED]> > tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] extensions and multiple source files with the same basename
Hi, I'm trying to port ctypes to darwin/x86 (aka the new intel macs), which went pretty smooth. I am running into some odd behaviour of distutils now that I'm trying to port those changes to the trunk. ctypes uses libffi, which contains source files in various platform- specific directories, such as: libffi/src/powerpc/ffi_darwin.c libffi/src/x86/ffi_darwin.c To make it possible to build a univeral binary of ctypes when -- enable-universalsdk is specified I'm using some preprocessor trickery and add both versions of the file to the source list of the _ctypes extension. So far so good. This works just fine when building the out-of-tree copy of ctypes. However, when I build the in-tree copy of ctypes distutils suddenly tries compile both these source files into the same object file (that is both get compiled into temp.../libffi/ ffi_darwin.o). That obviously isn't what I want to happen. The cause of this behaviour is CCompiler._setup_compile: this calls self.object_filenames with strip_dir=True when doing a python build. Removing this argument (and therefore having object_filenames fall back to the default value False for strip_dir) doesn't seem to have bad effects on the build on OSX. Does anyone know why distutils behaves like this and if it would be safe to change this behaviour? Ronald ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Cost-Free Slice into FromString constructors--Long
We've been talking this week about ideas for speeding up the parsing of Longs coming out of files or network. The use case is having a large string with embeded Long's and parsing them to real longs. One approach would be to use a simple slice: long(mystring[x:y]) an expensive operation in a tight loop. The proposed solution is to add further keyword arguments to Long (such as):long(mystring, base=10, start=x, end=y)The start/end would allow for negative indexes, as slices do, but otherwise simply limit the scope of the parsing. There are other solutions, using buffer-like objects and such, but this seems like a simple win for anyone parsing a lot of text. I implemented it in a branch runar-longslice-branch, but it would need to be updated with Tim's latest improvements to long. Then you may ask, why not do it for everything else parsing from string--to which I say it should. Thoughts? -- Runar PeturssonBetur[EMAIL PROTECTED] -- http://betur.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On Thu, May 25, 2006 at 03:01:36PM +, Runar Petursson wrote: >simply limit the scope of the parsing. There are other solutions, using >buffer-like objects and such, but this seems like a simple win for anyone >parsing a lot of text. I implemented it in a branch I'll expand on that a little bit, since I talked to Runar yesterday about it. My original blurb on the #nfs IRC channel was that this would seem to have the "camel's nose under the tent" problem, that once this is in you probably want it in lots of other places where slices may be used. Another interface would be to have something like a lazy string slice that functions that understood it could directly in C operate on the sub-buffer, and oblivious functions would have to create a new slice. Later, John showed Martin and I the Java byte buffer class, and Martin is working on that now. This would probably provide a solution. The Java interface allows you to push a new start and end, and the buffer then acts like it's the string in that range. Useful for parsing without having to create new objects in a tight loop. Conversion functions taking a start and end are the low-hanging fruit, but I think in the long term something that does that described I would prefer. I believe that Martin is expecting expecting to have something this week to try. Thanks, Sean -- A computer lets you make more mistakes faster than any invention in human history -- with the possible exceptions of handguns and tequila. -- Mitch Ratcliffe Sean Reifschneider, Member of Technical Staff <[EMAIL PROTECTED]> ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On Thu, 25 May 2006 15:01:36 +, Runar Petursson <[EMAIL PROTECTED]> wrote:
>We've been talking this week about ideas for speeding up the parsing of
>Longs coming out of files or network. The use case is having a large string
>with embeded Long's and parsing them to real longs. One approach would be
>to use a simple slice:
>long(mystring[x:y])
>
>an expensive operation in a tight loop. The proposed solution is to add
>further keyword arguments to Long (such as):
>
>long(mystring, base=10, start=x, end=y)
>
>The start/end would allow for negative indexes, as slices do, but otherwise
>simply limit the scope of the parsing. There are other solutions, using
>buffer-like objects and such, but this seems like a simple win for anyone
>parsing a lot of text. I implemented it in a branch runar-longslice-
>branch,
>but it would need to be updated with Tim's latest improvements to long.
>Then you may ask, why not do it for everything else parsing from string--to
>which I say it should. Thoughts?
This really seems like a poor option. Why fix the problem with a hundred
special cases instead of a single general solution?
Hmm, one reason could be that the general solution doesn't work:
[EMAIL PROTECTED]:~$ python
Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> long(buffer('1234', 0, 3))
Traceback (most recent call last):
File "", line 1, in ?
ValueError: null byte in argument for long()
>>> long(buffer('123a', 0, 3))
Traceback (most recent call last):
File "", line 1, in ?
ValueError: invalid literal for long(): 123a
>>>
Still, fixing that seems like a better idea. ;)
Jean-Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On 5/25/06, Sean Reifschneider <[EMAIL PROTECTED]> wrote: > Conversion functions taking a start and end are the low-hanging fruit, but > I think in the long term something that does that described I would prefer. > I believe that Martin is expecting expecting to have something this week > to try. I'm -1 on this. It adds considerable API complications for something that 99.99% of users can do without regrets by using a slice. Yes, I know regex and string searches already have this API complication. They have a much more important use case. I don't want every API that takes a string argument to also take slice arguments; then everybody ends up doing duplicate work. If you really need this for speed, I recommend you make it a custom extension. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
Runar Petursson wrote: > We've been talking this week about ideas for speeding up the parsing > of Longs coming out of files or network. The use case is having a > large string with embeded Long's and parsing them to real longs. One > approach would be to use a simple slice: > long(mystring[x:y]) > > an expensive operation in a tight loop. The proposed solution is to > add further keyword arguments to Long (such as): > > long(mystring, base=10, start=x, end=y) > > The start/end would allow for negative indexes, as slices do, but > otherwise simply limit the scope of the parsing. There are other > solutions, using buffer-like objects and such, but this seems like a > simple win for anyone parsing a lot of text. I implemented it in a > branch runar-longslice-branch, but it would need to be updated with > Tim's latest improvements to long. Then you may ask, why not do it > for everything else parsing from string--to which I say it should. > Thoughts? -1 This is a somewhat specialized parsing application and should not be allowed to muck-up an otherwise simple, general-purpose API. Micro-optimizations do not warrant API changes. Certainly, it should not be propogated to everything that can parse from a string. Also, you are likely to find that the cost of varargs and kwargs will offset the slicing gains. I think the whole notion is off base. The int(mystring[x:y]) situation is only important when you're doing it many times. Often, you'll be doing other conversions as well. So, you would be better-off creating a text version of the struct module that would enable you to extract and convert many elements from a long record stored as text. Alternately, you could expose a function styled after fscanf(). IOW, better to provide to general purpose text parsing tool than to muck-up the whole language for one specialized application. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On May 25, 2006, at 3:28 PM, Jean-Paul Calderone wrote:
> On Thu, 25 May 2006 15:01:36 +, Runar Petursson
> <[EMAIL PROTECTED]> wrote:
>> We've been talking this week about ideas for speeding up the
>> parsing of
>> Longs coming out of files or network. The use case is having a
>> large string
>> with embeded Long's and parsing them to real longs. One approach
>> would be
>> to use a simple slice:
>> long(mystring[x:y])
>>
>> an expensive operation in a tight loop. The proposed solution is
>> to add
>> further keyword arguments to Long (such as):
>>
>> long(mystring, base=10, start=x, end=y)
>>
>> The start/end would allow for negative indexes, as slices do, but
>> otherwise
>> simply limit the scope of the parsing. There are other solutions,
>> using
>> buffer-like objects and such, but this seems like a simple win for
>> anyone
>> parsing a lot of text. I implemented it in a branch runar-
>> longslice-
>> branch,
>> but it would need to be updated with Tim's latest improvements to
>> long.
>> Then you may ask, why not do it for everything else parsing from
>> string--to
>> which I say it should. Thoughts?
>
> This really seems like a poor option. Why fix the problem with a
> hundred special cases instead of a single general solution?
>
> Hmm, one reason could be that the general solution doesn't work:
>
> [EMAIL PROTECTED]:~$ python
> Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
> Type "help", "copyright", "credits" or "license" for more
> information.
long(buffer('1234', 0, 3))
> Traceback (most recent call last):
> File "", line 1, in ?
> ValueError: null byte in argument for long()
long(buffer('123a', 0, 3))
> Traceback (most recent call last):
> File "", line 1, in ?
> ValueError: invalid literal for long(): 123a
One problem with buffer() is that it does a memcpy of the buffer. A
zero-copy version of buffer (a view on some object that implements
the buffer API) would be nice.
-bob
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
Bob Ippolito <[EMAIL PROTECTED]> wrote: > One problem with buffer() is that it does a memcpy of the buffer. A > zero-copy version of buffer (a view on some object that implements > the buffer API) would be nice. Did buffers change? I seem to remember that there were segfaulting conditions when using array and buffer due to arrays being resizable and leaving buffers pointing at ether...or is the memcpy part of the buffer 'fix'? If so, the changes in performance and memory use may surprise long-time users of buffer. - Josiah P.S. There is a possible solution to the "buffer pointing at ether due to realloc" problem, but it would rely on a non-existing (from what I understand about memory allocation) "would a realloc of pointer foo point to the same address?" Of course the other alternative is to just always alloc when resizing arrays, etc., leaving a version of the old object around until all buffers pointing to that object have been freed. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On 5/25/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: If you really need this for speed, I recommend you make it a custom extension. The custom extension is a good idea, and what we do now. But now that the long algorithm is so fast, it would be nice to expose it at least at the C level to use a substring. Any solution--be it buffer based or even an module could have no reuse of the code without an End Pointer. If we added the end pointer to a FromSubstring (or somesuch) function (leaving PyLong_FromString alone), it would be easily reused from any parsing module. Assuming it didn't have any negative performance implication. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
On 5/25/06, Runar Petursson <[EMAIL PROTECTED]> wrote: > > On 5/25/06, Guido van Rossum <[EMAIL PROTECTED]> wrote: > > > > If you really need this for speed, I recommend you make it a custom > extension. > > > > > The custom extension is a good idea, and what we do now. But now that the > long algorithm is so fast, it would be nice to expose it at least at the C > level to use a substring. Any solution--be it buffer based or even an > module could have no reuse of the code without an End Pointer. If we added > the end pointer to a FromSubstring (or somesuch) function (leaving > PyLong_FromString alone), it would be easily reused from any parsing module. > Assuming it didn't have any negative performance implication. This discussion about conversion of longs is very much related to the new hot buffer class I'm currently adding in the bytebuf branch, which will directly tackle the issue of I/O on a fixed buffer without intermediate string creation or memory copy. You will be able to pack and unpack structs directly into and out of a fixed memory buffer. In case you're interested, it is a clone of the Java NIO ByteBuffer class for Python. I'm making minor modifications to the socket and struct modules to support I/O using the buffer protocol. The hot buffer class will live in a module and neither of the socket nor struct modules will depend on it. Following Fredrik's suggestion I will write up a mini-PEP for discussion of this idea and will move the code in the sandbox (and out of a branch). Additionally, I must say, all that hot water in the blue lagoon this morning was a really hot justficiation for the "hot buffer" name. http://furius.ca/tmp/nfs3/html/2006-05-25.11185801.html -- Martin Blais ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
[Jean-Paul Calderone]
>> ...
>> Hmm, one reason could be that the general solution doesn't work:
>>
>> [EMAIL PROTECTED]:~$ python
>> Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
>> [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>> >>> long(buffer('1234', 0, 3))
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> ValueError: null byte in argument for long()
>> >>> long(buffer('123a', 0, 3))
>> Traceback (most recent call last):
>> File "", line 1, in ?
>> ValueError: invalid literal for long(): 123a
[Bob Ippolito]
> One problem with buffer() is that it does a memcpy of the buffer.
It does not, at least not in the cases above. In fact, that's
essentially _why_ they fail now. Here's what actually happens (in
outline) for the first example: a buffer object is created that
merely points to the "1234" string object, recording the offset of 0
and the length of 3. PyLong_FromString() only sees the starting
address, and-- as it always does --parses until it hits a character
that doesn't make sense for the input base. That happens to be the
NUL at the end of the "1234" string object guts (Python string objects
are always NUL-terminated, BTW).
PyLong_FromString() is perfectly happy, but converted "1234" rather
than "123". It's PyLong_FromString()'s _caller_ that's unhappy,
because PyLong_FromString also told the caller that it stopped parsing
at offset 4, and then
if (end != s + len) {
PyErr_SetString(PyExc_ValueError,
"null byte in argument for long()");
The assumption here is bad: the caller assumes that if end != s+len,
it must be the case that PyLong_FromString() stopped parsing _before_
hitting the end, and did so _because_ it hit an embedded NUL byte.
Neither is true in this case, so the error message is senseless.
In any case, none of this would have happened if the buffer object had
done a memcpy of the 3 requested bytes, and NUL-terminated it.
> A zero-copy version of buffer (a view on some object that implements
> the buffer API) would be nice.
Above we already had that. The internal parsing APIs don't currently
support the buffer's "offset & length" view of the world, so have no
chance of working as hoped with any kind of buffer object now.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Import hooks falling back on built-in import machinery?
While working on a patch involving sys.path_importer_cache, I discovered that if a path_hooks import hook has been found for a given sys.path entry, but isn't able to import a specific module, find_module() tries to import the module from this sys.path entry as a regular file. This results in e.g. doing an open call to /usr/lib/python25.zip/x.py when I do "import x", while I think that once a sys.path entry has been identified as belonging to a sys.path_hooks importer instance, it shouldn't be handled by the built-in machinery anymore since the path string could be anything. PEP 302 says "...it was chosen to ... simply fall back to the built-in logic if no hook on sys.path_hooks could handle the path item", but that's refering to sys.path entries, not individual module names to be imported. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import hooks falling back on built-in import machinery?
Good catch. I think this would save a certain number of unnecessary stat() calls. But it does change semantics, so we can't fix this in 2.4. In 2.5, we should warn hook authors that this might affect them. The PEP ought to be updated to clarify this. --Guido On 5/25/06, Georg Brandl <[EMAIL PROTECTED]> wrote: > While working on a patch involving sys.path_importer_cache, I discovered > that if a path_hooks import hook has been found for a given sys.path entry, > but isn't able to import a specific module, find_module() tries to import > the module from this sys.path entry as a regular file. > > This results in e.g. doing an open call to /usr/lib/python25.zip/x.py when > I do "import x", while I think that once a sys.path entry has been identified > as belonging to a sys.path_hooks importer instance, it shouldn't be handled > by the built-in machinery anymore since the path string could be anything. > > PEP 302 says "...it was chosen to ... simply fall back to the built-in logic > if no hook on sys.path_hooks could handle the path item", but that's > refering to sys.path entries, not individual module names to be imported. > > Georg > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Import hooks falling back on built-in import machinery?
Guido van Rossum wrote: > Good catch. I think this would save a certain number of unnecessary > stat() calls. > > But it does change semantics, so we can't fix this in 2.4. In 2.5, we > should warn hook authors that this might affect them. > > The PEP ought to be updated to clarify this. Okay. The patch fixing this is at #921466. I'll see what I can do about the PEP. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] A Horrible Inconsistency
>>> -1 * (1, 2, 3) () >>> -(1, 2, 3) Traceback (most recent call last): File "", line 1, in TypeError: bad operand type for unary - We Really Need To Fix This! [\F] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fredrik Lundh wrote: > >>> -1 * (1, 2, 3) > () > >>> -(1, 2, 3) > Traceback (most recent call last): >File "", line 1, in > TypeError: bad operand type for unary - > > We Really Need To Fix This! I can't find this inconsistency horrible. py> +"Hello" Traceback (most recent call last): File "", line 1, in ? TypeError: bad operand type for unary + py> +1*"Hello" 'Hello' Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Martin v. Löwis wrote: > Fredrik Lundh wrote: >> >>> -1 * (1, 2, 3) >> () >> >>> -(1, 2, 3) >> Traceback (most recent call last): >>File "", line 1, in >> TypeError: bad operand type for unary - >> >> We Really Need To Fix This! > > I can't find this inconsistency horrible. > > py> +"Hello" > Traceback (most recent call last): > File "", line 1, in ? > TypeError: bad operand type for unary + > py> +1*"Hello" > 'Hello' Don't tell me that! I was actually working on a patch right now... Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
You're joking right? On 5/25/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > >>> -1 * (1, 2, 3) > () > >>> -(1, 2, 3) > Traceback (most recent call last): >File "", line 1, in > TypeError: bad operand type for unary - > > We Really Need To Fix This! > > [\F] Doesn't the real effbot have /F as sig? -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On 25-mei-2006, at 23:04, Martin v. Löwis wrote: > Fredrik Lundh wrote: > -1 * (1, 2, 3) >> () > -(1, 2, 3) >> Traceback (most recent call last): >>File "", line 1, in >> TypeError: bad operand type for unary - >> >> We Really Need To Fix This! > > I can't find this inconsistency horrible. > > py> +"Hello" > Traceback (most recent call last): > File "", line 1, in ? > TypeError: bad operand type for unary + > py> +1*"Hello" > 'Hello' I don't know which one Fredrik thinks is wrong, but I think the result of -1*(1,2,3) is very surprising. I'd expect an exception here. Ronald > > Regards, > Martin > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > ronaldoussoren%40mac.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Ronald Oussoren wrote: > I don't know which one Fredrik thinks is wrong, but I think the result > of -1*(1,2,3) is very surprising. I'd expect an exception here. I agree, but this has nothing to do with whether or not the unary - is supported. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On Thu, May 25, 2006 at 09:06:49PM +, Georg Brandl wrote: >Don't tell me that! I was actually working on a patch right now... While undoubtedly a performance patch, it wasn't on the list of tasks to do today. You risk Steve's wrath! Thanks, Sean -- In the end, we will remember not the words of our enemies, but the silence of our friends. -- Martin Luther King Jr. Sean Reifschneider, Member of Technical Staff <[EMAIL PROTECTED]> tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fredrik Lundh wrote: > >>> -1 * (1, 2, 3) >() > >>> -(1, 2, 3) >Traceback (most recent call last): > File "", line 1, in >TypeError: bad operand type for unary - > >We Really Need To Fix This! > > The second one doesn't bug me. Unary minus on a sequence is meaningless. The first is a bit odd. When using the * operator for sequence repetition, I don't expect it to have the same commutative property as multiplication. IOW, "seq * n" makes sense but "n * seq" is a bit weird. Also, I'm not clear on the rationale for transforming negative repetition counts to zero instead of raising an exception. OTOH, neither of these has EVER been an issue for me or anyone I know. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
[Fredrik] > >>> -1 * (1, 2, 3) > () > >>> -(1, 2, 3) > Traceback (most recent call last): >File "", line 1, in > TypeError: bad operand type for unary - > > We Really Need To Fix This! What's broken? It's generally true that n*s == s*n == empty_container_of_type_type(s) whenever s is a sequence and n is an integer <= 0. The above is just an instance of that. See footnote 2 in Library Ref section 2.3.6 Sequence Types. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
On 5/25/06, Raymond Hettinger <[EMAIL PROTECTED]> wrote: > Fredrik Lundh wrote: > > > >>> -1 * (1, 2, 3) > >() > > >>> -(1, 2, 3) > >Traceback (most recent call last): > > File "", line 1, in > >TypeError: bad operand type for unary - > > > >We Really Need To Fix This! > > > > > > The second one doesn't bug me. Unary minus on a sequence is meaningless. > > The first is a bit odd. When using the * operator for sequence > repetition, I don't expect it to have the same commutative property as > multiplication. IOW, "seq * n" makes sense but "n * seq" is a bit > weird. Also, I'm not clear on the rationale for transforming negative > repetition counts to zero instead of raising an exception. OTOH, > neither of these has EVER been an issue for me or anyone I know. It would be very strange if n*s didn't return the same thing as s*n. In fact, I can't even decide which one feels more natural! As to truncation of 0, that's debatable but can't be fixed until py3k -- surely lots of code (perhaps accidentally) depends on it. Still unclear which one \F wanted fixed... -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
[Raymond Hettinger] ... > Also, I'm not clear on the rationale for transforming negative > repetition counts to zero instead of raising an exception. There are natural use cases. Here's one: you have a string and want to right-justify it to 80 columns with blanks if it's shorter than 80. s = " " * (80 - len(s)) + s does the right thing in all cases, because the blank repetition gracefully collapses to an empty string whenever len(s) >= 80. I've used that (and variants thereof) dozens of times. Then again, I don't believe I've ever used the "integer * sequence" form (i.e., with the integer on the left). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Guido van Rossum wrote: >> We Really Need To Fix This! >> >> [\F] > > Doesn't the real effbot have /F as sig? yeah, we've had some trouble with fake bots lately. I mean, there's a timbot posting to this thread, but I know for sure that the real Tim got tired of hacking on Python earlier tonight, and retired to his hotel room. too much silica mud, I suppose. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Fredrik Lundh wrote: > Guido van Rossum wrote: > > >>>We Really Need To Fix This! >>> >>>[\F] >> >>Doesn't the real effbot have /F as sig? > > > yeah, we've had some trouble with fake bots lately. I mean, there's a > timbot posting to this thread, but I know for sure that the real Tim got > tired of hacking on Python earlier tonight, and retired to his hotel > room. too much silica mud, I suppose. > In actual fact the effbot has lately found itself so permeated with Windows that it has become constituionally incapable of using a forward slash. Don't know what's with the square brackets though ... regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Tim Peters wrote: > [Raymond Hettinger] > ... >> Also, I'm not clear on the rationale for transforming negative >> repetition counts to zero instead of raising an exception. > > There are natural use cases. Here's one: you have a string and want > to right-justify it to 80 columns with blanks if it's shorter than 80. > > s = " " * (80 - len(s)) + s > > does the right thing in all cases, because the blank repetition > gracefully collapses to an empty string whenever len(s) >= 80. I've > used that (and variants thereof) dozens of times. Then again, I don't > believe I've ever used the "integer * sequence" form (i.e., with the > integer on the left). I originally brought that up because decimal.py uses it. (try to translate this to C code...) Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Georg Brandl wrote: > Martin v. Löwis wrote: >> Fredrik Lundh wrote: >>> >>> -1 * (1, 2, 3) >>> () >>> >>> -(1, 2, 3) >>> Traceback (most recent call last): >>>File "", line 1, in >>> TypeError: bad operand type for unary - >>> >>> We Really Need To Fix This! >> >> I can't find this inconsistency horrible. >> >> py> +"Hello" >> Traceback (most recent call last): >> File "", line 1, in ? >> TypeError: bad operand type for unary + >> py> +1*"Hello" >> 'Hello' > > Don't tell me that! I was actually working on a patch right now... Since I've already been bombarded with questions by some fellow Germans (which once again prove that they've got no sense of humour at all ;): this was a joke. georG ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] whatever happened to string.partition ?
does anyone remember? given what we're currently working on, implementing it would take roughly no time at all. do people still think it's a good idea ? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] This
Raymond Hettinger wrote: > Fredrik Lundh wrote: > > >>does anyone remember? given what we're currently working on, >>implementing it would take roughly no time at all. do people still >>think it's a good idea ? >> >> >> >> >> > > > Yes. It went to my todo list and is awaiting some free raymond-cycles > to finish it up. I've been task saturated of late but would like to get > this a number of other patches complete for Py2.5. > > Also, Crutcher had long ago posted a patch for exec/eval to accept > generic mapping arguments. If someone like Tim or /F has the time, > feel free to pick it up. > This reminds me I am tasked with trying to find out what the interface to timeit.py is supposed to look like. Raymond, your name has been mentioned as someone who took part int he discussions. Google hasn't given me a lot to go on. Anyone? [Follow-ups to python-dev would be best, I suspect]. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Love me, love my blog http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5a2 try/except slow-down: Convert to type?
>Brett provided the following direction: > >Right, I meant change how it (BaseException) is written. Right now > >it uses PyMethodDef for magic methods and just uses PyType_New() > >as a constructor. I was wondering if, for some reason, it would be > >faster if you used a PyType_Type definition for BaseException and > >used the proper C struct to associate the methods with the class. Just a status update. Richard has implemented enough of this that I've been able to run some performance tests, and this changes it this pybench test from being 57% slower than 2.4.3 to being 32% faster. Go Richard. It's looking pretty good so far. Now we just have to keep it from exploding. Thanks, Sean -- Denial is more than just a river in Egypt, you know? Sean Reifschneider, Member of Technical Staff <[EMAIL PROTECTED]> tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability Back off man. I'm a scientist. http://HackingSociety.org/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whatever happened to string.partition ?
Fredrik Lundh wrote: > no need to wait for any raymond-cycles here; just point me to the latest > version of the proposal, and it'll be in the trunk within 30 minutes. are these still valid? http://mail.python.org/pipermail/python-dev/2005-August/055764.html http://mail.python.org/pipermail/python-dev/2005-August/055770.html ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] whatever happened to string.partition ?
Fredrik Lundh wrote: Yes. It went to my todo list and is awaiting some free raymond-cycles to finish it up. I've been task saturated of late but would like to get this a number of other patches complete for Py2.5. no need to wait for any raymond-cycles here; just point me to the latest version of the proposal, and it'll be in the trunk within 30 minutes. IIRC, Skip had developed a smart version that returned lazy string objects that kept a reference and pointers to the original string (instead of making its own copy of the string components). The string subclass would expand itself and free the reference if necessary for a subsequent string operation. The main purpose was to handle the cases where one fragment of the other was never used or just had a length check. Also it was helpful when partition was used lisp-style to repeatedly break-off head/tail fragments. > are these still valid? > > http://mail.python.org/pipermail/python-dev/2005-August/055764.html > http://mail.python.org/pipermail/python-dev/2005-August/055770.html Yes. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whatever happened to string.partition ?
Fredrik Lundh wrote: > does anyone remember? given what we're currently working on, > implementing it would take roughly no time at all. do people still > think it's a good idea ? > > I had enquired this before but got no answer. I assume nobody's working on it at the moment. I also proposed a slightly different semantic which would prevent much boilerplate in the stdlib: http://mail.python.org/pipermail/python-dev/2006-March/062582.html -- Giovanni Bajo ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] This
Steve Holden wrote: > > This reminds me I am tasked with trying to find out what the interface > to timeit.py is supposed to look like. Raymond, your name has been > mentioned as someone who took part int he discussions. Google hasn't > given me a lot to go on. Anyone? > IIRC, Guido's words were that too much to the smarts in timeit.py were in the command-line interface and not in the Timer object were in should be. So, this should be a simple refactoring exercise to expose more of the existing API. No inventiveness is required. While you're at it, please document the import trick or testing existing code without copying it in to a text string. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whatever happened to string.partition ?
Giovanni Bajo wrote: Fredrik Lundh wrote: does anyone remember? given what we're currently working on, implementing it would take roughly no time at all. do people still think it's a good idea ? I had enquired this before but got no answer. I assume nobody's working on it at the moment. I also proposed a slightly different semantic which would prevent much boilerplate in the stdlib: http://mail.python.org/pipermail/python-dev/2006-March/062582.html Please no. This was discussed and settled long ago. It takes less time to implement this than to go through the whole API debate again. Raymond ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Returning int instead of long from struct when possible for performance
Python ints are a heck of a lot faster to work with than Python longs and have the additional benefit that psyco can optimize the hell out of int but can't do anything at all for long. This is important because psyco is currently in pretty wide-spread use amongst people who need to squeeze every bit of performance out of their Python programs, and often yields 4x or better performance improvement in real-world tests. A more far reaching change would be to add a new PyInteger_From* API for code that doesn't care if long or int is returned, but I don't see much of a reason to go that far at this point since the rest of the standard library wouldn't benefit much. Unfortunately, this change to the struct module slightly alters the documented API for the following format codes: I, L, q, Q. Currently it is documented that those format codes will always return longs, regardless of their value. I've prototyped this change on the trunk (behind a currently undefined PY_USE_INT_WHEN_POSSIBLE macro). The standard library test suite passes with this enabled, but it would break any doctests in third party code that check the result of an unpack operation with one of those format codes. Given the interchangeability of int and long, I don't foresee any other complications with this change. Thoughts? -bob ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Returning int instead of long from struct when possible for performance
[Bob Ippolito] > ... > Unfortunately, this change to the struct module slightly alters the > documented API for the following format codes: I, L, q, Q. Currently > it is documented that those format codes will always return longs, > regardless of their value. I view that more as having documented the CPython implementation du jour than as documenting the language. IOW, it was a doc bug <0.5 wink>. > I've prototyped this change on the trunk (behind a currently > undefined PY_USE_INT_WHEN_POSSIBLE macro). The standard library test > suite passes with this enabled, but it would break any doctests in > third party code that check the result of an unpack operation with > one of those format codes. Given the interchangeability of int and > long, I don't foresee any other complications with this change. > > Thoughts? +1, and for 2.5. Even int() doesn't always return an int anymore, and it's just stupid to bear the burden of an unbounded long when it's not really needed. As to doctest breakage, I'm wholly unsympathetic to any doctest user except me, and I don't have any doctests that would break. So that's a total non-issue :-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] whatever happened to string.partition ?
I will certainly look forward to using it. On May 25, 2006, at 16:21, Fredrik Lundh wrote: > does anyone remember? given what we're currently working on, > implementing it would take roughly no time at all. do people still > think it's a good idea ? > > > > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ > shane.holloway%40ieee.org ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Cost-Free Slice into FromString constructors--Long
Tim Peters wrote: > PyLong_FromString() only sees the starting > address, and-- as it always does --parses until it hits a character > that doesn't make sense for the input base. This is the bug, then. long() shouldn't be using PyLong_FromString() to convert its argument, but something that takes an address and a size. (Is there a PyLong_FromStringAndSize()? If not, there should be.) -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A Horrible Inconsistency
Steve Holden wrote: > In actual fact the effbot has lately found itself so permeated with > Windows that it has become constituionally incapable of using a forward > slash. Don't know what's with the square brackets though ... I was thinking maybe that message had resulted from a Windows and a VMS port of the effbot that got mixed together somehow... -- Greg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] whatever happened to string.partition ?
On 5/25/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > Raymond Hettinger wrote: > > > IIRC, Skip had developed a smart version that returned lazy string > > objects that kept a reference and pointers to the original string > > (instead of making its own copy of the string components). The string > > subclass would expand itself and free the reference if necessary for a > > subsequent string operation. The main purpose was to handle the cases > > where one fragment of the other was never used or just had a length > > check. Also it was helpful when partition was used lisp-style to > > repeatedly break-off head/tail fragments. > > that sounds nice in theory, but I completely fail to see how that can be > implemented without ripping out the existing string object and replacing > it with something entirely different, or (via horrible macro tricks) > slow down virtually all other use of PyStringObject. > > skip? was this a real design or a Py3K bluesky idea? I suggest to forget about that particular idea and just do what you were thinking of originally. str.partition() and unicode.partition() should be as simple as possible, to cover the 80% use case FAST. -- --Guido van Rossum (home page: http://www.python.org/~guido/) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] whatever happened to string.partition ?
>> skip? was this a real design or a Py3K bluesky idea? Just about nothing I do is a real design. I don't remember the weather conditions that day either. I was probably just noodling around based upon discussions on python-dev that were mutated in my brain by gamma rays from Alpha Centauri. Guido> I suggest to forget about that particular idea and just do what Guido> you were thinking of originally. str.partition() and Guido> unicode.partition() should be as simple as possible, to cover the Guido> 80% use case FAST. Agreed. Get it right then optimize it during the next NeedForSpeed sprint. Skip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
