date:20120618

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Jim Jewett

On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan  wrote:
> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  wrote:

>>    *Every* Parameter attribute is optional, even name.
>>    (Think of  builtins, even if they aren't automatically
>>supported yet.)   So go ahead and define some others
>>that are sometimes useful.

> Add only stuff we know is interesting and useful.

Agreed, but it doesn't have to be useful in all cases, or even
available on all Signatures; if users are already prepared for missing
data, it is enough that the attribute be well-defined, and be useful
when it does appear.  That said, it looks like is_implemented isn't
sufficiently well-defined.

> - kind
> - name (should be given meaningful content, even for POSITIONAL_ONLY 
> parameters)

I agree that it *should* be given meaningful content, but I don't
think the Parameter (or Signature) should be blocked without it.  I
also don't think that a documentation-only name that cannot be used
for keyword calls should participate in equality.  The existence of
the parameter should participate, and its annotation is more important
than usual, but its name is not.

> - default (may be missing, since "None" is allowed as a default value)
> - annotation (may be missing, since "None" is allowed as an annotation)

Position is also important, but I'm not certain whether it should be
represented in the Parameter, or only in the Signature.

copy(source, target)
copy(target, source)

have different signatures, but I'm not sure whether it would be
appropriate to reuse the same parameter objects.

>>    Instead of defining a BoundArguments class, just return
>>a copy  of the Signature, with "value" attributes added to
>>the Parameters.

> No, the "BoundArguments" class is designed to be easy to
> feed to a function call as f(*args, **kwds)

Why does that take a full class, as opposed to a method returning a
tuple and a dict?

>>    Use subclasses to distinguish the parameter kind.

> Please, no, using subclasses when there is no behavioural
> change is annoying.

A **kwargs argument is very different from an ordinary parameter.  Its
name doesn't matter (and therefore should not be considered in
__eq__), it can only appear once per signature, and the possible
location of its appearance is different.  It is formatted differently
(which I would prefer to do in the Parameter, rather than in
Signature).  It also holds very different data, and must be treated
specially by several Signature methods, particularly when either
validating or binding.  (It is bound to a Mapping, rather than to a
single value, so you have to keep it around longer and use a different
"bind method".)

>>> A Signature object has the following public attributes and methods:

The more I try to work with it, the more I want direct references to
the two special arguments (*args, **kwargs) if they exist.  FWIW, the
current bind logic to find them -- particularly kwargs -- seems
contorted, compared to self.kwargsparameter.

>> (3rd edition)
>>> * is_keyword_only : bool ...
>>> * is_args : bool ...
>>> * is_kwargs : bool ...

>> (4th edition)
>>> ... Parameter.POSITIONAL_ONLY ...
>>> ... Parameter.POSITIONAL_OR_KEYWORD ...
>>> ... Parameter.KEYWORD_ONLY ...
>>> ... Parameter.VAR_POSITIONAL ...
>>> ... Parameter.VAR_KEYWORD ...

>> This set has already grown, and I can think of others I would like to
>> use.  (Pseudo-parameters, such as a method's self instance, or an
>> auxiliary variable.)

> No. This is the full set of binding behaviours. "self" is just an
> ordinary "POSITIONAL_OR_KEYWORD" argument (or POSITIONAL_ONLY, in some
> builtin cases).

Or no longer a "parameter" at all, once the method is bound.  Except
it sort of still is.  Same for the space parameter in PyPy.  I don't
expect the stdlib implementation to support them initially, but I
don't want it to get in the way, either.  A supposedly closed set gets
in the way.

>> I'm not sure
>> if positional parameters should also check position, or if that
>> can be left to the Signature.

> Positional parameters don't know their relative position, so it *has*
> to be left to the signature.

But perhaps they *should* know their relative position.  Also,
positional_only, *args, and **kwargs should be able to remove name
from the list of compared attributes.

-jJ
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] 3.3 beta in one week

2012-06-18 Thread Georg Brandl

Hi all,

this is just a quick reminder that the feature freeze for 3.3
will start next weekend with the release of beta1.  Since  I
won't be able to shift that date for short periods (the next
possible date for me would be around July 16), I hope that
everybody has planned ahead accordingly.

Let me also say that it's great to see how 3.3 is shaping up;
the team is doing very well and I want to thank everybody for
pushing, but not rushing, awesome features :)

cheers,
Georg

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.3 beta in one week

2012-06-18 Thread Martin v. Löwis

> this is just a quick reminder that the feature freeze for 3.3
> will start next weekend with the release of beta1.  Since  I
> won't be able to shift that date for short periods (the next
> possible date for me would be around July 16), I hope that
> everybody has planned ahead accordingly.

Expect some rushing of PEP 397 then.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Martin v. Löwis

> The default should be what we've had though.
> The new settings cause a lot more collisions
> and resizes.

Raymond, can you kindly point to an application that demonstrates this
claim (in particular the "a lot more" part, which I'd translate to
"more than 20% more").

I'm fine with reverting changes, but I agree that any benchmarking
performed should be repeatable, and public. I agree it's sad to see
a month worth of benchmarking reverted - but had that benchmarking
been documented publicly, rather than just reporting the outcome,
such reversal might have been avoided.

> Dicts get their efficiency from sparseness.
> Reducing the mindict size from 8 to 4 causes
> substantially more collisions in small dicts
> and gets closer to a linear search of a small tuple.

Why do you think the dictsize has been reduced from 8 to 4? It has not.

Regards,
Martin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Nick Coghlan

On Mon, Jun 18, 2012 at 5:08 PM, Jim Jewett  wrote:
> But perhaps they *should* know their relative position.

No, relative position is a property of the Signature - a parameter has
no position until it is made part of a signature.

> Also,
> positional_only, *args, and **kwargs should be able to remove name
> from the list of compared attributes.

As you yourself said, removing the name from consideration for
positional arguments is potentially dangerous - a function that
accepts (source, dest) is very different from one that accepts (dest,
source). If you don't care about names, then call bind() to see if it
works or write your own more permissive comparison operation. If you
do care about names, then the default equality definition is
appropriate.

Ultimately, people are going to be free to do their own thing. The
heart of the new API is to get all of this already available
information out into a more conveniently manipulable format. Once we
see what people do with it, *then* it is time to discuss additional
convenience APIs (such as Signature level properties for parameter
subsets).

Don't overengineer it at the foundational level - let people build
their own additional layers of customisation on top.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the best way to debug python3 source code? (for this bug: http://bugs.python.org/issue15068)

2012-06-18 Thread gmspro

@martin,

I'm working on this bug, http://bugs.python.org/issue15068
I tried this with gdb
(gdb)run
>>>from sys import stdin
>>>str=sys.stdin.read()
blabla
blabla
blabla
CTRL+D
CTRL+D
>>>CTRL+C
(gdb)backtrace

0xb7f08348 in ___newselect_nocancel () at ../sysdeps/unix/syscall-template.S:82
82    ../sysdeps/unix/syscall-template.S: No such file or directory.
    in ../sysdeps/unix/syscall-template.S
Current language:  auto
The current source language is "auto; currently asm".
(gdb) backtrace
#0  0xb7f08348 in ___newselect_nocancel () at 
../sysdeps/unix/syscall-template.S:82
#1  0xb7b43f9f in readline_until_enter_or_signal (prompt=0xb7b578d0 ">>> ", 
signal=0xbfffed54)
    at /home/user1/python/Python-3.2.3/Modules/readline.c:987
#2  0xb7b440ed in call_readline (sys_stdin=0xb7f84420, sys_stdout=0xb7f844c0, 
prompt=0xb7b578d0 ">>> ")
    at /home/user1/python/Python-3.2.3/Modules/readline.c:1082
#3  0x0811a15b in PyOS_Readline (sys_stdin=0xb7f84420, sys_stdout=0xb7f844c0, 
prompt=0xb7b578d0 ">>> ")
    at ../Parser/myreadline.c:200
#4  0x0811b92d in tok_nextc (tok=0x82e1d08) at ../Parser/tokenizer.c:897
#5  0x0811c39b in tok_get (tok=0x82e1d08, p_start=0xbfffef20, p_end=0xbfffef1c) 
at ../Parser/tokenizer.c:1306
#6  0x0811cda2 in PyTokenizer_Get (tok=0x82e1d08, p_start=0xbfffef20, 
p_end=0xbfffef1c) at ../Parser/tokenizer.c:1687
#7  0x08119866 in parsetok (tok=0x82e1d08, g=0x81df0c0, start=256, 
err_ret=0xbfffefd8, flags=0xbfffefd4)
    at ../Parser/parsetok.c:150
#8  0x081197a6 in PyParser_ParseFileFlagsEx (fp=0xb7f84420, filename=0x818ac7a 
"", enc=0xb7c57750 "UTF-8", g=
    0x81df0c0, start=256, ps1=0xb7b578d0 ">>> ", ps2=0xb7b578e8 "... ", 
err_ret=0xbfffefd8, flags=0xbfffefd4)
    at ../Parser/parsetok.c:100
#9  0x080cb96d in PyParser_ASTFromFile (fp=0xb7f84420, filename=0x818ac7a 
"", enc=0xb7c57750 "UTF-8", start=256, ps1=
    0xb7b578d0 ">>> ", ps2=0xb7b578e8 "... ", flags=0xb194, 
errcode=0xb044, arena=0x8285a50)
    at ../Python/pythonrun.c:1941
#10 0x080c9c9c in PyRun_InteractiveOneFlags (fp=0xb7f84420, filename=0x818ac7a 
"", flags=0xb194)
    at ../Python/pythonrun.c:1175
#11 0x080c99ed in PyRun_InteractiveLoopFlags (fp=0xb7f84420, filename=0x818ac7a 
"", flags=0xb194)
    at ../Python/pythonrun.c:1086
#12 0x080c98b5 in PyRun_AnyFileExFlags (fp=0xb7f84420, filename=0x818ac7a 
"", closeit=0, flags=0xb194)
    at ../Python/pythonrun.c:1055
#13 0x080deb5c in run_file (fp=0xb7f84420, filename=0x0, p_cf=0xb194) at 
../Modules/main.c:307
#14 0x080df5c0 in Py_Main (argc=1, argv=0x81f7008) at ../Modules/main.c:733
#15 0x0805c3d9 in main (argc=1, argv=0xb2f4) at ../Modules/python.c:63

But i can't get any clue which file to look at. 

Thanks
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] CFFI released

2012-06-18 Thread Armin Rigo

Hi all,

We (=fijal and myself) finally released the beta-0.1 version of CFFI.

http://cffi.readthedocs.org/

It is a(nother) simple Foreign Function Interface for Python calling C
code.  I talked about it with a few python core people during the
PyCon sprint; now it's done, with a pure Python part and a compact
(but still 3000 lines) piece of C code.  The goal is for it to be
simple yet as complete as possible; it can be used in places where
ctypes (say) is not applicable or only with platform-specific
difficulties, e.g. to rewrite a "_curses" module in pure Python, or
access the X libraries, etc.

Of course I'm not going to suggest that it should be part of the
standard library right now, but I do hope that over time, should it
prove useful and used, I could come back and make such a suggestion.
In any case it looks like we are going to write native and JITted PyPy
support for it, and change our pure Python "_ctypes" implementation to
be based on "cffi".  As it is much more compact to support than the
full _ctypes, it is also good for Jython and IronPython.


A bientôt,

Armin.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Mark Shannon


Martin v. Löwis wrote:

The default should be what we've had though.
The new settings cause a lot more collisions
and resizes.


Raymond, can you kindly point to an application that demonstrates this
claim (in particular the "a lot more" part, which I'd translate to
"more than 20% more").


It is quite easy to find a program that results in a lots more resizes.
The issue is not whether there are more resizes, but whether it is
faster or slower. The evidence suggests that the new default settings
are no slower and reduce memory use.



I'm fine with reverting changes, but I agree that any benchmarking
performed should be repeatable, and public. I agree it's sad to see
a month worth of benchmarking reverted - but had that benchmarking
been documented publicly, rather than just reporting the outcome,
such reversal might have been avoided.


Dicts get their efficiency from sparseness.


But do they? The results of benchmarking would seem to suggest (at least
on my test machine) that overly-sparse dicts are slower.
Possibly due to increased cache misses.


Reducing the mindict size from 8 to 4 causes
substantially more collisions in small dicts
and gets closer to a linear search of a small tuple.


Why do you think the dictsize has been reduced from 8 to 4? It has not.


For combined tables it remains 8 as before.

For split tables it *has* been reduced to 4.
This will increase collisions, and it is possible that a linear search would
be faster for these very small dictionaries.
However, a 4 entry table fits into a single cache line (for a 64 byte cache
line on a 32 bit machine) which may save a lot of cache misses.
But this all conjecture. Whatever the reason, the current parameters give
the best performance empirically.

Cheers,
Mark.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

Jim,

On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan  wrote:
>> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  wrote:
> 
>>>Instead of defining a BoundArguments class, just return
>>>   a copy  of the Signature, with "value" attributes added to
>>>   the Parameters.
> 
>> No, the "BoundArguments" class is designed to be easy to
>> feed to a function call as f(*args, **kwds)
> 
> Why does that take a full class, as opposed to a method returning a
> tuple and a dict?

Read this thread, please: 
http://mail.python.org/pipermail/python-dev/2012-June/12.html
And also take a look at the check types example in the pep.

In short - it's easy to work with 'BoundArguments.arguments' list,
but it is not enough for invocation, thus 'BoundArguments.args' & 
'.kwargs' properties.

>>>Use subclasses to distinguish the parameter kind.
> 
>> Please, no, using subclasses when there is no behavioural
>> change is annoying.
> 
> A **kwargs argument is very different from an ordinary parameter.  Its
> name doesn't matter (and therefore should not be considered in
> __eq__),

The importance of its name depends hugely on the use context.  In some
it may be very important.

> it can only appear once per signature, and the possible
> location of its appearance is different.  

I'll fix the __eq__ to ignore positions of **kwargs & keyword-only 
parameters.

> It is formatted differently
> (which I would prefer to do in the Parameter, rather than in
> Signature).  

I think we'll remove the 'Signature.format()' from the PEP, leaving just
'Signature.__str__'.  And just for '__str__' I don't think it's a bad
idea to let Signature format its parameters.

> It also holds very different data, and must be treated
> specially by several Signature methods, particularly when either
> validating or binding.  (It is bound to a Mapping, rather than to a
> single value, so you have to keep it around longer and use a different
> "bind method".)

And it is treated specially, along with the *args.

 A Signature object has the following public attributes and methods:
> 
> The more I try to work with it, the more I want direct references to
> the two special arguments (*args, **kwargs) if they exist.  FWIW, the
> current bind logic to find them -- particularly kwargs -- seems
> contorted, compared to self.kwargsparameter.

Well, 'self.kwargsparameter'  will break 'self.parameters' collection,
unless you want one parameter to be in two places.  I've already had
three or four implementations of this PEP, with the first couple having
**kwargs parameter stored separately, but keeping all parameters in one 
collection turned out to be the most elegant solution.  

In fact, the check types example (in the PEP) is currently shorter and 
easier to read with 'Signature.parameters' than with dedicated property 
for '**kwargs' parameter.

And if after all you need direct references to *args or **kwargs - write 
a little helper, which finds them in 'Signature.parameters'.

>>>  I'm not sure
>>> if positional parameters should also check position, or if that
>>> can be left to the Signature.
> 
>> Positional parameters don't know their relative position, so it *has*
>> to be left to the signature.
> 
> But perhaps they *should* know their relative position.

I disagree here.  That will just complicate things.

>  Also,
> positional_only, *args, and **kwargs should be able to remove name
> from the list of compared attributes.

I still believe in the most contexts the name of a parameter matters 
(even if it's **kwargs).  Besides, how can we make __eq__ to be
configurable?  Let's make it do the most explicit and simple logic,
and those who need a custom one will implement such for themselves.

Thank you,
-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou

On Mon, 18 Jun 2012 15:28:24 +0100
Mark Shannon  wrote:
> 
> But do they? The results of benchmarking would seem to suggest (at least
> on my test machine) that overly-sparse dicts are slower.
> Possibly due to increased cache misses.

Or, at least, they are not faster. See the synthetic experiments in
http://bugs.python.org/issue10408

That said, Raymond might have witnessed different results at the time.
Hardware evolves quickly and the parameters change (memory latency
today is at least 50+ CPU cycles, which is quite a lot of wasted work on
a pipelined superscalar CPU).

Regards

Antoine.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Raw string syntax inconsistency

2012-06-18 Thread Guido van Rossum

Ok, banning ru"..." and ur"..." altogether is fine too (assuming it's fine
with the originators of the PEP).

On Sun, Jun 17, 2012 at 11:31 PM, Nick Coghlan  wrote:

> On Mon, Jun 18, 2012 at 3:59 PM, "Martin v. Löwis" 
> wrote:
> > On 17.06.2012 22:41, Guido van Rossum wrote:
> >> Would it make sense to detect and reject these in 3.3 if the 2.7 syntax
> >> is used?
> >
> > Maybe we are talking about different things: The (new) proposal is that
> > the ur prefix in 3.3 is a syntax error (again, as it was before PEP
> > 414). So, yes: the raw unicode literals will be rejected (not by
> > explicitly detecting them, though).
>
> I think GvR was replying to my email where I was briefly reconsidering
> the idea of keeping them around (because the unicode_literals future
> import already suffers from this problem of literals that don't mean
> the same things in 2.x and in 3.x). However, that was flawed reasoning
> on my part - simply banning them altogether in 3.x is the simplest
> option to ensure this particular error doesn't pass silently,
> especially since there are alternate forward compatible ways to write
> them, such as:
>
> Python 2.7.3 (default, May 29 2012, 14:54:22)
> >>> from __future__ import unicode_literals
> >>> print(u"\u03b3" r"\n")
> γ\n
> >>> print(u"\u03b3\\n")
> γ\n
>
> Python 3.3.0a4 (default:f1dd70bfb4c5, May 31 2012, 09:47:51)
> >>> print(u"\u03b3" r"\n")
> γ\n
> >>> print(u"\u03b3\\n")
> γ\n
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   [email protected]   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Raw string syntax inconsistency

2012-06-18 Thread Guido van Rossum

On Sun, Jun 17, 2012 at 10:59 PM, Terry Reedy  wrote:

> On 6/17/2012 9:07 PM, Guido van Rossum wrote:
>
>> On Sun, Jun 17, 2012 at 4:55 PM, Nick Coghlan >
>
> So, perhaps the answer is to leave this as is, and try to make 2to3
>>smart enough to detect such escapes and replace them with their
>>properly encoded (according to the source code encoding) Unicode
>>equivalent?
>>
>>
>> But the whole point of the reintroduction of u"..." is to support code
>> that isn't run through 2to3.
>>
>
> People writing 2&3 code sometimes use 2to3 once (or a few times) on their
> 2.6/7 version during development to find things they must pay attention to.
> So Nick's idea could be helpful to people who do not want to use 2to3
> routinely either in development or deployment.
>
>
> > Frankly, I don't care how it's done, but
>
>> I'd say it's important not to silently have different behavior for the
>> same notation in the two versions.
>>
>
> The fundamental problem was giving the 'u' prefix two different meanings
> in 2.x: 'change the storage type from bytes to unicode', and 'change the
> contents by partially cooking the literal even when raw processing is
> requested'*. The only way to silently have the same behavior is to
> re-introduce the second meaning of partial cooking. (But I would rather
> make it unnecessary.) But that would freeze the 'u' prefix, or at least
> 'ur' ('un-raw') forever. It would be better to introduce a new, separate
> 'p' prefix, to mean partially raw, partially cooked. (But I am opposes to
>
> *I think this non-orthogonal interaction effect was a design mistake and
> that it would have been better to have re do all the cooking needed by also
> interpreting \u and \U sequences. I also think we should add this now for
> 3.3 if possible, to make partial cooking at the parsing stage unnecessary.
> Putting the processing in re makes it work for all strings, not just those
> given as literals.
>
>
> > If that means we have to add an extra
>
>> step to the compiler to reject r"\u03b3", so be it.
>>
>
> I do not get this. Surely you cannot mean to suddenly start rejecting, in
> 3.3, a large set of perfectly legal and sensible 6 and 10 character
> sequences when embedded in literals?
>

Sorry, I meant rejecting ru"" (and ur"") if it contains a \u or \U
escape that would be expanded by Python 2.

 Hm. I still encounter enough environments that don't know how to display
> such characters that I would prefer to have a rock solid \u escape
> mechanism. I can think of two ways to support "expanded" unicode
> characters in raw strings a la Python 2;
>

(a) let the re module interpret the escapes (like it does for \r and \n);

As said above, I favor this. The 2.x partial cooking (with 'ur' prefix) was
> primarily a substitute for this.
>
>
> (b) the user can write r"someblah" "\u03b3" r"moreblah".
>
> This is somewhat orthogonal to (a). Users can this whenever they want
> partial processing of backslashes without doubling those they want left as
> is. A generic example is r'someraw' 'somecooked' r'moreraw' 'morecooked'.
>
> --
> Terry Jan Reedy
>
>
>
>
> __**_
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.3 beta in one week

2012-06-18 Thread Brian Curtin

On Mon, Jun 18, 2012 at 2:17 AM, "Martin v. Löwis"  wrote:
>> this is just a quick reminder that the feature freeze for 3.3
>> will start next weekend with the release of beta1.  Since  I
>> won't be able to shift that date for short periods (the next
>> possible date for me would be around July 16), I hope that
>> everybody has planned ahead accordingly.
>
> Expect some rushing of PEP 397 then.

FYI: Martin requested that I be the PEP czar for 397, so the rush
kicks off...now :)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the best way to debug python3 source code? (for this bug: http://bugs.python.org/issue15068)

2012-06-18 Thread Terry Reedy

On 6/18/2012 7:10 AM, gmspro wrote:

I'm working on this bug, http://bugs.python.org/issue15068

Oh. From your first message, I thought you were asking about a personal 
bug. I will mention that real names are customary on this list. (Unless 
you have a good professional reason otherwise.) They are also necessary 
for contributor forms, which we need from anyone submitting an 
acceptable patch large enough to be covered by copyright.

I tried this with gdb
(gdb)run
 >>>from sys import stdin
 >>>str=sys.stdin.read()
blabla
blabla
blabla
CTRL+D
CTRL+D
 >>>CTRL+C
(gdb)backtrace

The backtrace is from when you hit ^C after the prompt and should have 
nothing to do with the double ^D behavior, except that it shows what we 
would like to the state after the first ^D.

0xb7f08348 in ___newselect_nocancel () at
../sysdeps/unix/syscall-template.S:82
82../sysdeps/unix/syscall-template.S: No such file or directory.
 in ../sysdeps/unix/syscall-template.S
Current language:  auto
The current source language is "auto; currently asm".

It appears you interrupted in a system call written in assembler.

(gdb) backtrace
#0  0xb7f08348 in ___newselect_nocancel () at
../sysdeps/unix/syscall-template.S:82
#1  0xb7b43f9f in readline_until_enter_or_signal (prompt=0xb7b578d0 ">>>
", signal=0xbfffed54)
 at /home/user1/python/Python-3.2.3/Modules/readline.c:987

As far as Python goes, you were at line 987 of readline.c in 
readline_until_enter_or_signal(prompt, signal) where prompt was ">>>".

I would try again twice, hitting ^C once before and once after the first 
^D. This should tell you where Python and the system is when it receives 
^D and should return to the prompt (as in the backtrace you got) and 
where Python went instead after receiving ^D and where it is when it 
gets the second ^D and does return.

--
Terry Jan Reedy

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Terry Reedy


On 6/18/2012 9:14 AM, Armin Rigo wrote:

Hi all,

We (=fijal and myself) finally released the beta-0.1 version of CFFI.

http://cffi.readthedocs.org/

It is a(nother) simple Foreign Function Interface for Python calling C
code.  I talked about it with a few python core people during the
PyCon sprint; now it's done, with a pure Python part and a compact
(but still 3000 lines) piece of C code.  The goal is for it to be
simple yet as complete as possible; it can be used in places where
ctypes (say) is not applicable or only with platform-specific
difficulties, e.g. to rewrite a "_curses" module in pure Python, or
access the X libraries, etc.

Of course I'm not going to suggest that it should be part of the
standard library right now, but I do hope that over time, should it
prove useful and used, I could come back and make such a suggestion.


Make cffi less buggy (check the tracker for new test cases ;-), faster 
(closer to swig type wrappers), and easier to use than ctypes, and I am 
sure there will be interest.


--
Terry Jan Reedy



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Raw string syntax inconsistency

2012-06-18 Thread Terry Reedy


On 6/18/2012 11:12 AM, Guido van Rossum wrote:

Ok, banning ru"..." and ur"..." altogether is fine too (assuming it's
fine with the originators of the PEP).


The original PEP never proposed ur or ru , only u/U.

It turns out that ur is problematical even in 2.x, as its meaning is 
changed by the future import. 2&3 code should skip the convenience of 
the r prefix and just use u and doubled \s. The PEP should probably say 
that.


--
Terry Jan Reedy



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread PJ Eby

On Fri, Jun 15, 2012 at 5:03 PM, R. David Murray wrote:

> On Fri, 15 Jun 2012 22:48:42 +0200, Victor Stinner <
> [email protected]> wrote:
> > > 1. Should we keep 'Parameter.implemented' or not.  *Please vote*
>
> -1 to implemented.
>
> > I still disagree with the deepcopy. I read somewhere that Python
> > developers are consenting adult. If someone really want to modify a
> > Signature, it would be nice to provide a simple method to copy it. But
> > I don't see why it should be copied *by default*. I expect that
> > modifying a signature is more rare than just reading a signature.
>
> The issue isn't "consenting adults", the issue is consistency.
> Without the deepcopy, sometimes what you get back from the
> inspect function is freely modifiable and sometimes it is not.
> That inconsistency is a bad thing.
>

Then just copy the signature itself; as currently written, this is going to
copy the annotation objects, which could produce weird side-effects from
introspection.  Using deepcopy seems like overkill when all that's needed
is a new Signature instance with a fresh OrderedDict.

Or, better yet: make signature and parameter objects immutable (along with
the OrderedDict) and the whole problem of modification and copying goes
away altogether.  Or is there some reason not mentioned in the PEP why
mutability is necessary?  (The PEP provides no rationale at present for
making any part of a signature mutable)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 3.3 beta in one week

2012-06-18 Thread Terry Reedy


On 6/18/2012 11:23 AM, Brian Curtin wrote:

On Mon, Jun 18, 2012 at 2:17 AM, "Martin v. Löwis"  wrote:

this is just a quick reminder that the feature freeze for 3.3
will start next weekend with the release of beta1.  Since  I
won't be able to shift that date for short periods (the next
possible date for me would be around July 16), I hope that
everybody has planned ahead accordingly.


Expect some rushing of PEP 397 then.


FYI: Martin requested that I be the PEP czar for 397, so the rush
kicks off...now :)


Some people who have downloaded the standalone version have praised it 
and recommended it on python-list. So I hope you include something for 
testing, even if details are changed later. It seems to solve real 
problems on Windows for many people.


A couple of comments on the PEP.

"Independent installations will always only overwrite newer versions of 
the launcher with older versions." 'always only' is a bit awkward and 
the sentence looks backwards to me. I would expect only overwriting 
older versions with newer versions.


---
These seem contradictory:

"The 32-bit distribution of Python will not install a 32-bit version of 
the launcher on a 64-bit system."


I presume this mean that it will install the 64-bit version and that 
there will always be only one version of the launcher on the system.


"On 64bit Windows with both 32bit and 64bit implementations of the 
same (major.minor) Python version installed, the 64bit version will 
always be preferred.  This will be true for both 32bit and 64bit 
implementations of the launcher - a 32bit launcher will prefer to 
execute a 64bit Python installation of the specified version if 
available."


This implies to me that the 32bit installation *will* install a 32bit 
launcher and that there could be both versions of the launcher installed.



--
Terry Jan Reedy




___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

On 2012-06-18, at 1:35 PM, PJ Eby wrote:

> On Fri, Jun 15, 2012 at 5:03 PM, R. David Murray  
> wrote:
> On Fri, 15 Jun 2012 22:48:42 +0200, Victor Stinner  
> wrote:
> > > 1. Should we keep 'Parameter.implemented' or not.  *Please vote*
> 
> -1 to implemented.
> 
> > I still disagree with the deepcopy. I read somewhere that Python
> > developers are consenting adult. If someone really want to modify a
> > Signature, it would be nice to provide a simple method to copy it. But
> > I don't see why it should be copied *by default*. I expect that
> > modifying a signature is more rare than just reading a signature.
> 
> The issue isn't "consenting adults", the issue is consistency.
> Without the deepcopy, sometimes what you get back from the
> inspect function is freely modifiable and sometimes it is not.
> That inconsistency is a bad thing.
> 
> Then just copy the signature itself; as currently written, this is going to 
> copy the annotation objects, which could produce weird side-effects from 
> introspection.  Using deepcopy seems like overkill when all that's needed is 
> a new Signature instance with a fresh OrderedDict.

That's an excerpt from Signature.__deepcopy__:

 cls = type(self)
 sig = cls.__new__(cls)
 sig.parameters = OrderedDict((name, param.__copy__()) \
   for name, param in self.parameters.items())

And Parameter.__copy__:

cls = type(self)
copy = cls.__new__(cls)
copy.__dict__.update(self.__dict__)
return copy

So we don't recursively deepcopy parameters in Signature.__deepcopy__
(I hope that we don't violate the deepcopy meaning here)

> Or, better yet: make signature and parameter objects immutable (along with 
> the OrderedDict) and the whole problem of modification and copying goes away 
> altogether.  Or is there some reason not mentioned in the PEP why mutability 
> is necessary?  (The PEP provides no rationale at present for making any part 
> of a signature mutable)

The rationale is that sometimes you need to modify signatures.
For instance, in decorators.

-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Maciej Fijalkowski

On Mon, Jun 18, 2012 at 7:02 PM, Terry Reedy  wrote:

> On 6/18/2012 9:14 AM, Armin Rigo wrote:
>
>> Hi all,
>>
>> We (=fijal and myself) finally released the beta-0.1 version of CFFI.
>>
>> http://cffi.readthedocs.org/
>>
>> It is a(nother) simple Foreign Function Interface for Python calling C
>> code.  I talked about it with a few python core people during the
>> PyCon sprint; now it's done, with a pure Python part and a compact
>> (but still 3000 lines) piece of C code.  The goal is for it to be
>> simple yet as complete as possible; it can be used in places where
>> ctypes (say) is not applicable or only with platform-specific
>> difficulties, e.g. to rewrite a "_curses" module in pure Python, or
>> access the X libraries, etc.
>>
>> Of course I'm not going to suggest that it should be part of the
>> standard library right now, but I do hope that over time, should it
>> prove useful and used, I could come back and make such a suggestion.
>>
>
> Make cffi less buggy (check the tracker for new test cases ;-), faster
> (closer to swig type wrappers), and easier to use than ctypes, and I am
> sure there will be interest.
>
>
I would say it's already fulfilling those three, but I suppose you should
try for yourself.

Cheers,
fijal
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Maciej Fijalkowski

On Mon, Jun 18, 2012 at 5:04 PM, Antoine Pitrou  wrote:

> On Mon, 18 Jun 2012 15:28:24 +0100
> Mark Shannon  wrote:
> >
> > But do they? The results of benchmarking would seem to suggest (at least
> > on my test machine) that overly-sparse dicts are slower.
> > Possibly due to increased cache misses.
>
> Or, at least, they are not faster. See the synthetic experiments in
> http://bugs.python.org/issue10408
>
> That said, Raymond might have witnessed different results at the time.
> Hardware evolves quickly and the parameters change (memory latency
> today is at least 50+ CPU cycles, which is quite a lot of wasted work on
> a pipelined superscalar CPU).
>
> Regards
>
> Antoine.
>
>
More like 200-500 CPU cycles on modern CPUs.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou

On Mon, 18 Jun 2012 21:31:27 +0200
Maciej Fijalkowski  wrote:
> On Mon, Jun 18, 2012 at 5:04 PM, Antoine Pitrou  wrote:
> 
> > On Mon, 18 Jun 2012 15:28:24 +0100
> > Mark Shannon  wrote:
> > >
> > > But do they? The results of benchmarking would seem to suggest (at least
> > > on my test machine) that overly-sparse dicts are slower.
> > > Possibly due to increased cache misses.
> >
> > Or, at least, they are not faster. See the synthetic experiments in
> > http://bugs.python.org/issue10408
> >
> > That said, Raymond might have witnessed different results at the time.
> > Hardware evolves quickly and the parameters change (memory latency
> > today is at least 50+ CPU cycles, which is quite a lot of wasted work on
> > a pipelined superscalar CPU).
> >
> > Regards
> >
> > Antoine.
> >
> >
> More like 200-500 CPU cycles on modern CPUs.

You are right. I was thinking 50 nanoseconds (which for a - relatively
high-end - 3GHz CPU puts us at 150 cycles).

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Raw string syntax inconsistency

2012-06-18 Thread Guido van Rossum

Cool.

On Mon, Jun 18, 2012 at 10:19 AM, Terry Reedy  wrote:

> On 6/18/2012 11:12 AM, Guido van Rossum wrote:
>
>> Ok, banning ru"..." and ur"..." altogether is fine too (assuming it's
>> fine with the originators of the PEP).
>>
>
> The original PEP never proposed ur or ru , only u/U.
>
> It turns out that ur is problematical even in 2.x, as its meaning is
> changed by the future import. 2&3 code should skip the convenience of the r
> prefix and just use u and doubled \s. The PEP should probably say that.
>
> --
> Terry Jan Reedy
>
>
>
>
> __**_
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/**mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Guido van Rossum

On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
 wrote:
> The rationale is that sometimes you need to modify signatures.
> For instance, in decorators.

A decorator should make a modified copy, not modify it in place (since
the signature of the decorated function does not change, and you have
no guarantee that that function is no longer accessible.)

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Raymond Hettinger

On Jun 18, 2012, at 12:35 PM, Antoine Pitrou wrote:

> You are right. I was thinking 50 nanoseconds (which for a - relatively
> high-end - 3GHz CPU puts us at 150 cycles).

The last guidance I read from Intel said that
a cache miss was roughly as expensive as a floating-point divide.

When a dictionary is less sparse, it has more collisions
which means there are more cache misses.

Resizing into a dictionary growing at 2x means that we're going
from 2/3 full to 1/3 full and resizing at 4x means going from 2/3
full to 1/6 full.  That latter will have fewer collisions (that said,
one-third full is still very good).  So, the performance is worse
but not much worse.

For dictionaries large enough to require multiple resizes,
the 4x factor cuts the number of resizes in half and makes
each resize faster (because of few collisions).   Only the 
growth phase is affected though.

It is more problematic for use cases such as caching where
a dict is constantly deleting old entries to make space for
new ones.  Such a dict never reaches a steady-state
because the dummy entries accumulate and trigger a resize.
Under the 2x scheme this happens much more often.

Under the 4x scheme, some dicts are left larger (more sparse)
than they otherwise would be (i.e. formerly it grew 8, 32, 128, ...)
and now it grows to (8, 16, 32, 64, ...).  Some dicts will end-up
the same dicts.  Others might fit in 16 rather than 32.  That
decreases their sparsity, increases the number of collisions,
and slows the lookup speed.  The effect is not large though
(the number of collisions between 1/4 full and 1/2 full is better
but 1/2 is still pretty good).

In the timings, I had done a few years ago, the results were that
just about anything that increased the number of collisions or
resizings would impact performance.   I expect that effect will
be accentuated on modern processors but I'll have to do updated
tests to be sure.

From a high-level view, I question efforts to desparsify dictionaries.
When you have a bucket of water, the weight is in the water, not
in the bucket itself.   The actual keys and values dominate dict size
unless you're reusing the same values over and over again.

That said, the 4x growth factor was capped at 50,000.  For larger
dicts it fell back to 2x.  Some the only dicts affected by the 2x vs 4x
decision lie by in the 6 to 50k ranges.  The only apps that see any
noticeable difference in memory size are ones that have many
dicts of that size range alive at the same time.

Sorry I can make a more detailed post right now.  I'll make time in
the next couple of weeks to post some code and timings that
document the collision counts, total memory size, and its affect
on various dict use cases.

Raymond

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] What's the best way to debug python3 source code? (for this bug: http://bugs.python.org/issue15068)

2012-06-18 Thread Martin v. Löwis

> But i can't get any clue which file to look at.

I personally wouldn't use gdb but strace to establish what system calls
are being made, and decide whether these system calls are correct or
not. If you then think that some call is being made that shouldn't,
set a breakpoint on the syscall function, and see when it gets hit.

I'd also attach to a running Python interpreter instead of running
python from gdb, since the interaction between gdb's stdin
and python's stdin may be confusing.

Regards,
Martin

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Jim Jewett

On Mon, Jun 18, 2012 at 10:37 AM, Yury Selivanov
 wrote:
> Jim,
>
> On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
>> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan  wrote:
>>> On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  wrote:

    Instead of defining a BoundArguments class, just return
   a copy  of the Signature, with "value" attributes added to
   the Parameters.

>>> No, the "BoundArguments" class is designed to be easy to
>>> feed to a function call as f(*args, **kwds)

>> Why does that take a full class, as opposed to a method returning a
>> tuple and a dict?

> Read this thread, please: 
> http://mail.python.org/pipermail/python-dev/2012-June/12.html

I reread that.  I still don't see why it needs to be an instance of a
specific independent class, as opposed to a Signature method that
returns a (tuple of) a tuple and a dict.

   ((arg1, arg2, arg3...), {key1: val2, key2: val2})

    Use subclasses to distinguish the parameter kind.

>>> Please, no, using subclasses when there is no behavioural
>>> change is annoying.

[Examples of how the "kinds" of parameters are qualitatively different.]

>> A **kwargs argument is very different from an ordinary parameter.  Its
>> name doesn't matter (and therefore should not be considered in
>> __eq__),

> The importance of its name depends hugely on the use context.  In some
> it may be very important.

The name of kwargs can only be for documentation purposes.  Like an
annotation or a docstring, it won't affect the success of an attempted
call.  Annotations are kept because (often) their entire purpose is to
document the signature.  But docstrings are being dropped, because
they often serve other purposes.  I've had far more use for docstrings
than for the names of positional-only parameters.  (In fact, knowing
the name of a positional-only parameter has sometimes been an
attractive nuisance.)

> And it is treated specially, along with the *args.

Right -- but this was in response to Nick's claim that the
distinctions should not be represented as a subclass, because the
behavior wasn't different.

I consider different __eq__ implementations or formatting concers to
be sufficient on their own; I also consider different possible use
locations and counts, different used-by-the-system attributes (name),
or different value types (object vs collection) to be sufficiently
behavioral.

> A Signature object has the following public attributes and methods:

>> The more I try to work with it, the more I want direct references to
>> the two special arguments (*args, **kwargs) if they exist.  FWIW, the
>> current bind logic to find them -- particularly kwargs -- seems
>> contorted, compared to self.kwargsparameter.

> Well, 'self.kwargsparameter'  will break 'self.parameters' collection,
> unless you want one parameter to be in two places.

Correct; it should be redundant.  Signature.kwargsparameter should be
the same object that occurs as the nth element of
Signature.parameters.values().  It is just more convenient to retrieve
the parameter directly than it is to iterate through a collection
inspecting each element for the value of a specific attribute.

> In fact, the check types example (in the PEP) is currently shorter and
> easier to read with 'Signature.parameters' than with dedicated property
> for '**kwargs' parameter.

Agreed; the short-cuts *args and **kwargs are only useful because they
are special; they aren't needed when you're doing the same thing to
all parameters regardless of type.

> And if after all you need direct references to *args or **kwargs - write
> a little helper, which finds them in 'Signature.parameters'.

Looking at http://bugs.python.org/review/15008/diff/5143/Lib/inspect.py
you already need one in _bind; it is just that saving the info when
you pass it isn't too bad if you're already iterating through the
whole collection anyhow.

>>  Also,
>> positional_only, *args, and **kwargs should be able to remove name
>> from the list of compared attributes.

> I still believe in the most contexts the name of a parameter matters
> (even if it's **kwargs).  Besides, how can we make __eq__ to be
> configurable?

__eq__ can can an _eq_fields attribute to see which other attributes
matter -- but it makes more sense for that to be (sub-) class
property.

-jJ
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Steven D'Aprano


Raymond Hettinger wrote:


Sorry I can make a more detailed post right now.  I'll make time in
the next couple of weeks to post some code and timings that
document the collision counts, total memory size, and its affect
on various dict use cases.



Is there some way to instrument dictionary sparseness, number of hits and 
misses, etc. from Python?


A secret command-line switch, perhaps, or a compile-time option?

And if there isn't, perhaps there should be.


--
Steven



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Antoine Pitrou

On Mon, 18 Jun 2012 13:46:25 -0700
Raymond Hettinger  wrote:
> 
> On Jun 18, 2012, at 12:35 PM, Antoine Pitrou wrote:
> 
> > You are right. I was thinking 50 nanoseconds (which for a - relatively
> > high-end - 3GHz CPU puts us at 150 cycles).
> 
> The last guidance I read from Intel said that
> a cache miss was roughly as expensive as a floating-point divide.

Floating-point divides are not performance-critical in most Python code,
so the comparison is not very useful. Besides, this statement does not
state which kind of cache would be involved in the cache miss.

On recent Intel CPUs a floating-point divide takes between 10
and 24 cycles (*), which could be in line with a L3 cache access (i.e. a
L2 cache miss), but certainly not a main memory access (150+ cycles).

(*) according to http://www.agner.org/optimize/instruction_tables.pdf

(also, modern CPUs have automatic prefetchers which try to hide the
latency of accessing main memory, but AFAIK this only really helps with
large streaming accesses)

> When a dictionary is less sparse, it has more collisions
> which means there are more cache misses.

Only in the eventuality that a single dict access is done (or a very
small number of them compared to the number of stored elements). But if
you are doing many accesses with good temporal locality, then the
sparse dict's bigger size implies a bigger cache footprint and
therefore a lesser efficiency - not only for the dict accesses
themselves, but for the rest of the code since more data will get
evicted to make place for the dict.

Furthermore, total memory consumption can be important regardless of
execution time. RAM is cheap but VMs are common where memory is much
smaller than on entire systems.

> For dictionaries large enough to require multiple resizes,
> the 4x factor cuts the number of resizes in half and makes
> each resize faster (because of few collisions).   Only the 
> growth phase is affected though.

Indeed.

> From a high-level view, I question efforts to desparsify dictionaries.
> When you have a bucket of water, the weight is in the water, not
> in the bucket itself.

But caches store whole cache lines, not individual bytes, so you do care
about the bucket's size. With 16-byte or 24-byte dict entries, and
64-byte cache lines, it is easy to see that a sparse dict could result
in a significant waste of the cache lines' storage if e.g. only 1 of 3
entries were used (and used entries were distributed in a regular
manner).

> The actual keys and values dominate dict size
> unless you're reusing the same values over and over again.

It certainly depends a lot on the use case.

Regards

Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Armin Rigo

Hi,

On Mon, Jun 18, 2012 at 9:10 PM, Maciej Fijalkowski  wrote:
>> Make cffi less buggy (check the tracker for new test cases ;-), faster
>> (closer to swig type wrappers), and easier to use than ctypes, and I am sure
>> there will be interest.
>
> I would say it's already fulfilling those three, but I suppose you should
> try for yourself.

I don't think the first is fulfilled so far, as we found various
issues on various Linux and non-Linux platforms (and fixed them, so I
suppose that release 0.1.1 is coming soon).  But I agree with Fijal
about speed and ease of use.  Like SWIG it generates wrappers in the
form of a CPython C extension with built-in functions, so I suppose
the performance is similar to SWIG and not ctypes.  Well, SWIG
wrappers typically have custom logic written in C, whereas in cffi
this logic is typically written as Python code, so I suppose it ends
up being slower (on CPython; on PyPy small wrapping functions are
inlined and have little cost).  But the same argument can be pushed
further to "why did you use a slow language like Python to write your
app in the first place", which I hope we agree is bogus :-)

A bientôt,

Armin.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Tunable parameters in dictobject.c (was dictnotes.txt out of date?)

2012-06-18 Thread Martin v. Löwis

On 18.06.2012 23:08, Steven D'Aprano wrote:
> Raymond Hettinger wrote:
> 
>> Sorry I can make a more detailed post right now.  I'll make time in
>> the next couple of weeks to post some code and timings that
>> document the collision counts, total memory size, and its affect
>> on various dict use cases.
> 
> 
> Is there some way to instrument dictionary sparseness, number of hits
> and misses, etc. from Python?
> 
> A secret command-line switch, perhaps, or a compile-time option?

Not that I know of, no.

> And if there isn't, perhaps there should be.

If so, only compile time options could be acceptable. However,
in my experience with profiling, the specific statistics that you
want to obtain are rarely known in advance, so you have to write
specific instrumentation every time you want to do an experiment -
and then the instrumentation is only good for that single experiment.

Thus, nobody publishes the instrumentation, since it would accumulate
as clutter.

Regards,
Martin

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

Jim,

On 2012-06-18, at 5:06 PM, Jim Jewett wrote:
> On Mon, Jun 18, 2012 at 10:37 AM, Yury Selivanov
>  wrote:
>> Jim,
>> 
>> On 2012-06-18, at 3:08 AM, Jim Jewett wrote:
>>> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan  wrote:
 On Sat, Jun 16, 2012 at 1:56 PM, Jim J. Jewett  
 wrote:
> 
>Instead of defining a BoundArguments class, just return
>   a copy  of the Signature, with "value" attributes added to
>   the Parameters.
> 
 No, the "BoundArguments" class is designed to be easy to
 feed to a function call as f(*args, **kwds)
> 
>>> Why does that take a full class, as opposed to a method returning a
>>> tuple and a dict?
> 
>> Read this thread, please: 
>> http://mail.python.org/pipermail/python-dev/2012-June/12.html
> 
> I reread that.  I still don't see why it needs to be an instance of a
> specific independent class, as opposed to a Signature method that
> returns a (tuple of) a tuple and a dict.
> 
>   ((arg1, arg2, arg3...), {key1: val2, key2: val2})

I'll try to explain with the code:

def foo(a, *args, b, **kwargs): pass
sig = signature(foo)

* Case one.  We have BoundArguments:

ba = sig.bind(1, 2, 3, b=123, c='foo', d='bar')

Now, in 'ba.arguments':
   
{'a': 1, 'args': (2, 3), 'b': 123, 'kwargs': {'c': 'foo', 'd': 'bar'}}

It's easy to work with 'ba.arguments', just traverse it as follows:

for arg_name, arg_value in ba.arguments:
param = sig.parameters[arg_name]

So you have argument name and value, and the corresponding parameter.

* Case two.  We return tuple and dict:

ba = sig.bind(1, 2, 3, b=123, c='foo', d='bar')
((1, 2, 3), {'b': 123, 'c': 'foo', 'd': 'bar'})

Now, how are you going to work with that?  How will you map those
values to the corresponding parameters?  The whole point of having
'Signature.bind()' is to provide you that mapping.

In the link I gave you, I also explained why we need 'BoundArguments.args'
and 'BoundArguments.kwargs'.

>Use subclasses to distinguish the parameter kind.
> 
 Please, no, using subclasses when there is no behavioural
 change is annoying.
> 
> [Examples of how the "kinds" of parameters are qualitatively different.]
> 
>>> A **kwargs argument is very different from an ordinary parameter.  Its
>>> name doesn't matter (and therefore should not be considered in
>>> __eq__),
> 
>> The importance of its name depends hugely on the use context.  In some
>> it may be very important.
> 
> The name of kwargs can only be for documentation purposes.

Not really.  I've seen once the code where types of acceptable values
were encoded parameter names (after '__', like 'kwargs__int').
This is a rather extreme example, but it illustrates that names
may be important.

>  Like an
> annotation or a docstring, it won't affect the success of an attempted
> call.  

It very well may affect it.  For instance, I use annotations to
specify arguments types in RPC dispatch.  The call will not be dispatched
in case of a wrong type.  So 'foo(a:int)' in my context does not
equal to 'foo(a:str)'.

>> And it is treated specially, along with the *args.
> 
> Right -- but this was in response to Nick's claim that the
> distinctions should not be represented as a subclass, because the
> behavior wasn't different.

Yes, it's behaviour of the outer code (Signature) that is different, not 
the Parameter instances.  It's Signature who makes a decision of how to map
parameters, not parameters themselves.

> I consider different __eq__ implementations or formatting concers to
> be sufficient on their own; I also consider different possible use
> locations and counts, different used-by-the-system attributes (name),
> or different value types (object vs collection) to be sufficiently
> behavioral.
> 
>> A Signature object has the following public attributes and methods:
> 
>>> The more I try to work with it, the more I want direct references to
>>> the two special arguments (*args, **kwargs) if they exist.  FWIW, the
>>> current bind logic to find them -- particularly kwargs -- seems
>>> contorted, compared to self.kwargsparameter.
> 
>> Well, 'self.kwargsparameter'  will break 'self.parameters' collection,
>> unless you want one parameter to be in two places.
> 
> Correct; it should be redundant.  Signature.kwargsparameter should be
> the same object that occurs as the nth element of
> Signature.parameters.values().  

It will be always the last parameter (if specified at all)

> It is just more convenient to retrieve
> the parameter directly than it is to iterate through a collection
> inspecting each element for the value of a specific attribute.

Again, depends on the use case.  Can you show a realistic use case,
that needs that?  (The use case should be also common and widespread, 
i.e. it should worth it to uglify Signature class structure)

>> In fact, the check types example (in the PEP) is currently shorter and
>> easier to read with 'Signature.parameters' than with dedicated property
>> for

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

On 2012-06-18, at 4:25 PM, Guido van Rossum wrote:

> On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
>  wrote:
>> The rationale is that sometimes you need to modify signatures.
>> For instance, in decorators.
> 
> A decorator should make a modified copy, not modify it in place (since
> the signature of the decorated function does not change, and you have
> no guarantee that that function is no longer accessible.)

It seems that we have the following options for 'signature(obj)':

1. If 'obj' has a '__signature__' attribute - return a copy of it,
if not - create a new one.

2. If 'obj' has a '__signature__' attribute - return it,
if not - create a new one.

3. Same as '2', but Signature is also immutable.

The first option is the one currently implemented.  Its advantage
is consistency - we always have a Signature we can safely modify.

The second option has a design flaw - sometimes the result Signature
is safe to modify, sometimes not, you never know.

The third option is hard to work with.
Instead of:

sig = signature(wrapper)
sig.parameters.popitem(last=False)
decorator.__signature__ = sig

We will have (because Signature is immutable):

sig = signature(wrapper)
params = OrderedDict(sig.parameters.items())
params.popitem(last=False)

attrs = {'parameters': params}
try:
ra = sig.return_annotation
except AttributeError:
pass
else:
attrs['return_annotation'] = ra

decorator.__signature__ = Signature.from_attrs(**attrs)

It looks like a total overkill (unless we can come up with a nicer
API).

So it was decided to go with the first option, as it has the least
complications.  Plus, the copying itself should be fast, as
Signatures contain little information.

What do you think?

-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Nick Coghlan

On Mon, Jun 18, 2012 at 5:08 PM, Jim Jewett  wrote:
> On Sat, Jun 16, 2012 at 11:27 AM, Nick Coghlan  wrote:
>> No. This is the full set of binding behaviours. "self" is just an
>> ordinary "POSITIONAL_OR_KEYWORD" argument (or POSITIONAL_ONLY, in some
>> builtin cases).
>
> Or no longer a "parameter" at all, once the method is bound.  Except
> it sort of still is.  Same for the space parameter in PyPy.  I don't
> expect the stdlib implementation to support them initially, but I
> don't want it to get in the way, either.  A supposedly closed set gets
> in the way.

It's not supposedly closed, it *is* closed: Python doesn't support any
other ways of binding arguments to parameters. Now, you can have
additional state on a callable that gets used by that callable (such
as __closure__ and __globals__ on a function, or __self__ on a method,
or arbitrary state on an object that implements __call__) but that
extra state is not part of the call *signature*, and thus should not
be exposed on the result of inspect.getsignature(). Remember, this
object is not meant to be a representation of the full state of a
callable, it's solely about providing a consistent introspection
mechanism that allows arbitrary callables to define how they will bind
arguments to parameters.

That's why I keep pointing out that there will always need to be a
higher level object that brings in other related information, such as
the docstring, the name of the callable, etc. This is not an API that
describes *everything* that is worth knowing about an arbitrary
callable, nor is it intended to be.

I believe you have raised a legitimate question regarding whether or
not there is sufficient variation in behaviour amongst the parameter
kinds for it to be worth using a subclassing approach to override
__str__ and __eq__, compared to just implementing a few "kind" checks
in the base implementation.

However, given that the possible binding behaviours (positional only,
keyword-or-positional, excess positional, keyword-only, excess
keywords) *is* a closed set, I think a subclass based solution is
overkill and adds excessive complexity to the public API.

Cheers,
Nick.

P.S. A more complete response to the question of what constitutes
"suitable complexity" for this API.

Consider the following implementation sketch for a kind based
implementation that pushes as much behaviour as is reasonable into the
Parameter object (including the definition of equality and decisions
on how the parameter should be displayed):

_sentinel = object()
class Parameter:
 def __init__(self, name, kind, default=_sentinel,
annotation=_sentinel):
 if not name:
 raise ValueError("All parameters must be named for
introspection purposes (even positional-only parameters)")
 self.name = name
 if kind not in Parameter.KINDS:
 raise ValueError("Unrecognised parameter binding type
{}".format(kind))
 self.kind = kind
 if default is not _sentinel:
 if kind.startswith("VAR"):
 raise ValueError("Cannot specify default value
for {} parameter".format(kind))
 self.default = default
 if annotation is not _sentinel:
 self.annotation = annotation

def _get_key(self):
default = getattr(self, "default", _sentinel)
annotation = getattr(self, "annotation", _sentinel)
if self.kind in (Parameter.KEYWORD_OR_POSITIONAL,
Parameter.KEYWORD_ONLY):
# The name is considered significant for parameters
that can be specified
# as keyword arguments
return (self.name, self.kind, default, annotation)
# Otherwise, we don't really care about the name
return (self.kind, default, annotation)

def __eq__(self, other):
if not isinstance(other, Parameter):
return NotImplemented
return self._get_key() == other._get_key()

def __str__(self):
kind = self.kind
components = []
if kind == Parameter.VAR_POSITIONAL:
components += ["*"]
elif kind == Parameter.VAR_KEYWORD:
components += ["**"]
if kind == Parameter.POSITIONAL_ONLY:
components += ["<", self.name, ">"]
else:
components += [self.name]
try:
default = self.default
except AttributeError:
pass
else:
components += ["=", repr(default)]
try:
annotation = self.annotation
except AttributeError:
pass
else:
components += [":", repr(annotation)]
return "".join(components)

The points of variation:
- VAR_POSITIONAL and VAR_KEYWORD do not permit a "default" attribute
- VAR_POSITIONAL adds a "*" before the name when pr

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Ethan Furman


Yury Selivanov wrote:

On 2012-06-18, at 4:25 PM, Guido van Rossum wrote:


On Mon, Jun 18, 2012 at 11:09 AM, Yury Selivanov
 wrote:

The rationale is that sometimes you need to modify signatures.
For instance, in decorators.

A decorator should make a modified copy, not modify it in place (since
the signature of the decorated function does not change, and you have
no guarantee that that function is no longer accessible.)




It seems that we have the following options for 'signature(obj)':

1. If 'obj' has a '__signature__' attribute - return a copy of it,
if not - create a new one.

2. If 'obj' has a '__signature__' attribute - return it,
if not - create a new one.

3. Same as '2', but Signature is also immutable.


The first option is the one currently implemented.  Its advantage
is consistency - we always have a Signature we can safely modify.

The second option has a design flaw - sometimes the result Signature
is safe to modify, sometimes not, you never know.

The third option is hard to work with.
Instead of:

sig = signature(wrapper)
sig.parameters.popitem(last=False)
decorator.__signature__ = sig

We will have (because Signature is immutable):

sig = signature(wrapper)
params = OrderedDict(sig.parameters.items())
params.popitem(last=False)

attrs = {'parameters': params}
try:
ra = sig.return_annotation
except AttributeError:
pass
else:
attrs['return_annotation'] = ra

decorator.__signature__ = Signature.from_attrs(**attrs)

It looks like a total overkill (unless we can come up with a nicer
API).

So it was decided to go with the first option, as it has the least
complications.  Plus, the copying itself should be fast, as
Signatures contain little information.

What do you think?


Option 1 makes sense to me -- we already know we'll have cases where we 
want to modify a given signature, so why make it hard on ourselves?


~Ethan~
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Nick Coghlan

On Tue, Jun 19, 2012 at 7:06 AM, Jim Jewett  wrote:
> Correct; it should be redundant.  Signature.kwargsparameter should be
> the same object that occurs as the nth element of
> Signature.parameters.values().  It is just more convenient to retrieve
> the parameter directly than it is to iterate through a collection
> inspecting each element for the value of a specific attribute.

I suspect in 3.4 we will add the following additional convenience properties:

 Signature.positional -> list[Parameter]
List of POSITIONAL_ONLY and KEYWORD_OR_POSITIONAL parameters
 Signature.var_positional -> None or Parameter
Reference to the VAR_POSITIONAL parameter, if any
 Signature.keyword -> dict{name:Parameter}
Mapping of all KEYWORD_ONLY and KEYWORD_OR_POSITIONAL parameters
 Signature.var_keyword -> None or Parameter
Reference to the VAR_KEYWORD parameter, if any

However, I don't think we should add such convenience properties
*right now*. One step at a time.

> __eq__ can can an _eq_fields attribute to see which other attributes
> matter -- but it makes more sense for that to be (sub-) class
> property.

Only if you accept the premise that there are other possible parameter
binding behaviours beyond the five already defined.
Hypergeneralisation is a great way to make an API far more complex
than it needs to be.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

On 2012-06-18, at 9:29 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 7:06 AM, Jim Jewett  wrote:
>> Correct; it should be redundant.  Signature.kwargsparameter should be
>> the same object that occurs as the nth element of
>> Signature.parameters.values().  It is just more convenient to retrieve
>> the parameter directly than it is to iterate through a collection
>> inspecting each element for the value of a specific attribute.
> 
> I suspect in 3.4 we will add the following additional convenience properties:
> 
> Signature.positional -> list[Parameter]
>List of POSITIONAL_ONLY and KEYWORD_OR_POSITIONAL parameters
> Signature.var_positional -> None or Parameter
>Reference to the VAR_POSITIONAL parameter, if any
> Signature.keyword -> dict{name:Parameter}
>Mapping of all KEYWORD_ONLY and KEYWORD_OR_POSITIONAL parameters
> Signature.var_keyword -> None or Parameter
>Reference to the VAR_KEYWORD parameter, if any

Maybe.  But I'd suggest to avoid the intersection of 'Signature.positional'
and 'Signature.keyword'.  Better to have 'Signature.keywordonly'.

> However, I don't think we should add such convenience properties
> *right now*. One step at a time.

+1.  'Signature.parameters' seems to be enough right now.

-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Nick Coghlan

On Tue, Jun 19, 2012 at 4:09 AM, Yury Selivanov  wrote:
> On 2012-06-18, at 1:35 PM, PJ Eby wrote:
>> Then just copy the signature itself; as currently written, this is going to 
>> copy the annotation objects, which could produce weird side-effects from 
>> introspection.  Using deepcopy seems like overkill when all that's needed is 
>> a new Signature instance with a fresh OrderedDict.
>
> That's an excerpt from Signature.__deepcopy__:
>
>     cls = type(self)
>     sig = cls.__new__(cls)
>     sig.parameters = OrderedDict((name, param.__copy__()) \
>                           for name, param in self.parameters.items())
>
> And Parameter.__copy__:
>
>        cls = type(self)
>        copy = cls.__new__(cls)
>        copy.__dict__.update(self.__dict__)
>        return copy
>
> So we don't recursively deepcopy parameters in Signature.__deepcopy__
> (I hope that we don't violate the deepcopy meaning here)

In my opinion, It's better to redefine what you mean by a shallow copy
(making it a bit deeper than just the direct attributes) rather than
making a so-called deep copy shallower.

So keep the current copying semantics for Signature objects (i.e.
creating new copies of the Parameter objects as well), but call it a
shallow copy rather than a deep copy. Make it clear in the
documentation that any defaults and annotations are still shared with
the underlying callable.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Steven D'Aprano

On Mon, Jun 18, 2012 at 07:10:33PM -0400, Yury Selivanov wrote:

> It seems that we have the following options for 'signature(obj)':
> 
> 1. If 'obj' has a '__signature__' attribute - return a copy of it,
> if not - create a new one.
> 
> 2. If 'obj' has a '__signature__' attribute - return it,
> if not - create a new one.
> 
> 3. Same as '2', but Signature is also immutable.

There's a slight ambiguity there. Do you mean, create a __signature__ 
attribute, or just create a new Signature instance?

I presume you mean the later, a Signature instance, and not cache it in 
__signature__.



-- 
Steven

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

On 2012-06-18, at 9:50 PM, Steven D'Aprano wrote:

> On Mon, Jun 18, 2012 at 07:10:33PM -0400, Yury Selivanov wrote:
> 
>> It seems that we have the following options for 'signature(obj)':
>> 
>> 1. If 'obj' has a '__signature__' attribute - return a copy of it,
>> if not - create a new one.
>> 
>> 2. If 'obj' has a '__signature__' attribute - return it,
>> if not - create a new one.
>> 
>> 3. Same as '2', but Signature is also immutable.
> 
> There's a slight ambiguity there. Do you mean, create a __signature__ 
> attribute, or just create a new Signature instance?
> 
> I presume you mean the later, a Signature instance, and not cache it in 
> __signature__.

Right. No implicit caching to __signature__ by the signature() function.

So, more verbose:

1. If 'obj' has a '__signature__' attribute - return a copy of its value, 
if not - create a new Signature and return it.

2. If 'obj' has a '__signature__' attribute - return a copy of its value, 
if not - create a new Signature and return it.

3. Same as '2', but Signature is also immutable.

Thanks!
-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov

On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 4:09 AM, Yury Selivanov  
> wrote:
>> On 2012-06-18, at 1:35 PM, PJ Eby wrote:
>>> Then just copy the signature itself; as currently written, this is going to 
>>> copy the annotation objects, which could produce weird side-effects from 
>>> introspection.  Using deepcopy seems like overkill when all that's needed 
>>> is a new Signature instance with a fresh OrderedDict.
>> 
>> That's an excerpt from Signature.__deepcopy__:
>> 
>> cls = type(self)
>> sig = cls.__new__(cls)
>> sig.parameters = OrderedDict((name, param.__copy__()) \
>>   for name, param in self.parameters.items())
>> 
>> And Parameter.__copy__:
>> 
>>cls = type(self)
>>copy = cls.__new__(cls)
>>copy.__dict__.update(self.__dict__)
>>return copy
>> 
>> So we don't recursively deepcopy parameters in Signature.__deepcopy__
>> (I hope that we don't violate the deepcopy meaning here)
> 
> In my opinion, It's better to redefine what you mean by a shallow copy
> (making it a bit deeper than just the direct attributes) rather than
> making a so-called deep copy shallower.

Agree. That's the only thing about the implementation that I really didn't 
like - deepcopy that's not exactly deep.

> So keep the current copying semantics for Signature objects (i.e.
> creating new copies of the Parameter objects as well), but call it a
> shallow copy rather than a deep copy. Make it clear in the
> documentation that any defaults and annotations are still shared with
> the underlying callable.

So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'?  Or make
it private - 'Signature._shallow_copy()'?

-
Yury

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Steven D'Aprano

On Mon, Jun 18, 2012 at 02:09:17PM -0400, Yury Selivanov wrote:

> That's an excerpt from Signature.__deepcopy__:
> 
>  cls = type(self)
>  sig = cls.__new__(cls)
>  sig.parameters = OrderedDict((name, param.__copy__()) \
>for name, param in self.parameters.items())
> 
> And Parameter.__copy__:
> 
> cls = type(self)
> copy = cls.__new__(cls)
> copy.__dict__.update(self.__dict__)
> return copy
> 
> So we don't recursively deepcopy parameters in Signature.__deepcopy__
> (I hope that we don't violate the deepcopy meaning here)

I think you are. I would describe the above as a shallow copy, not a 
deep copy. I expect a deep copy to go *all the way down*, as deep as 
possible.

Even if it makes no practical difference, I think it will be less 
confusing to just describe it as a "copy" rather than a deep copy, 
unless you recursively copy everything all the way down.

-- 
Steven

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Nick Coghlan

On Tue, Jun 19, 2012 at 12:00 PM, Yury Selivanov
 wrote:
> On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:
>> So keep the current copying semantics for Signature objects (i.e.
>> creating new copies of the Parameter objects as well), but call it a
>> shallow copy rather than a deep copy. Make it clear in the
>> documentation that any defaults and annotations are still shared with
>> the underlying callable.
>
> So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'?  Or make
> it private - 'Signature._shallow_copy()'?

I'd just call it Signature.__copy__ :)

You're not doing anything unusual here, just declaring that the list
of parameters is a part of the Signature object's state, and thus even
a shallow copy shouldn't share the parameter objects.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 362: 4th edition

2012-06-18 Thread Yury Selivanov


On 2012-06-18, at 10:06 PM, Nick Coghlan wrote:

> On Tue, Jun 19, 2012 at 12:00 PM, Yury Selivanov
>  wrote:
>> On 2012-06-18, at 9:36 PM, Nick Coghlan wrote:
>>> So keep the current copying semantics for Signature objects (i.e.
>>> creating new copies of the Parameter objects as well), but call it a
>>> shallow copy rather than a deep copy. Make it clear in the
>>> documentation that any defaults and annotations are still shared with
>>> the underlying callable.
>> 
>> So, 'Signature.__deepcopy__()' -> 'Signature.shallow_copy()'?  Or make
>> it private - 'Signature._shallow_copy()'?
> 
> I'd just call it Signature.__copy__ :)
> 
> You're not doing anything unusual here, just declaring that the list
> of parameters is a part of the Signature object's state, and thus even
> a shallow copy shouldn't share the parameter objects.

OK, done ;)

BTW, http://bugs.python.org/issue15008 has the latest implementation
attached (if anybody wants to play with it)

-
Yury
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Terry Reedy


On 6/18/2012 5:29 PM, Armin Rigo wrote:

On Mon, Jun 18, 2012 at 9:10 PM, Maciej Fijalkowski  wrote:

>> Me

Make cffi less buggy (check the tracker for new test cases ;-), faster
(closer to swig type wrappers), and easier to use than ctypes, and I am sure
there will be interest.


I would say it's already fulfilling those three, but I suppose you should
try for yourself.





I don't think the first is fulfilled so far, as we found various
issues on various Linux and non-Linux platforms (and fixed them, so I
suppose that release 0.1.1 is coming soon).  But I agree with Fijal
about speed and ease of use.  Like SWIG it generates wrappers in the
form of a CPython C extension with built-in functions, so I suppose
the performance is similar to SWIG and not ctypes.  Well, SWIG
wrappers typically have custom logic written in C, whereas in cffi
this logic is typically written as Python code, so I suppose it ends
up being slower (on CPython; on PyPy small wrapping functions are
inlined and have little cost).  But the same argument can be pushed
further to "why did you use a slow language like Python to write your
app in the first place", which I hope we agree is bogus :-)


Yes, because languages have no speed, only implementations do; and yes,
because when CPython really is too slow for a particular task, it can be 
pushed onto C. But some people (pygame, others on python-list) have 
reported that for their project, ctypes negates too much of the C 
speedup, relative to swig or similar. So it has not been quite the C 
wrapper generator killer that some people hoped for. (This is not to say 
that is not great for uses it does succeed at.)


--
Terry Jan Reedy



___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Nick Coghlan

On Tue, Jun 19, 2012 at 12:35 PM, Terry Reedy  wrote:
> Yes, because languages have no speed, only implementations do; and yes,
> because when CPython really is too slow for a particular task, it can be
> pushed onto C. But some people (pygame, others on python-list) have reported
> that for their project, ctypes negates too much of the C speedup, relative
> to swig or similar. So it has not been quite the C wrapper generator killer
> that some people hoped for. (This is not to say that is not great for uses
> it does succeed at.)

There's also another reason ctypes hasn't taken over from Cython and
SWIG: because it's entirely ABI based and doesn't look at the C header
files, it loses even what little type safety C possesses.

SWIG and Cython, on the other hand, suffer from the fact that you
can't just decide to wrap an arbitrary DLL on the fly *without*
predefining an extension module, and you also can't just use C syntax
to define the ABI you want to access (although SWIG actually gets
pretty close in many cases).

The approach Armin and Maciej have chosen here (using C declarations
to define the ABI, and supporting verification against the C headers
as a separate step) looks very promising.

Cheers,
Nick.

-- 
Nick Coghlan   |   [email protected]   |   Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 397 - Last Comments

2012-06-18 Thread Brian Curtin

Martin approached me earlier and requested that I act as PEP czar for
397. I haven't been involved in the writing of the PEP and have been
mostly observing from the outside, so I accepted and hope to get this
wrapped up quickly and implemented in time for the beta. The PEP is
pretty complete, but there are a few outstanding issues.

On Mon, Jun 18, 2012 at 1:05 PM, Terry Reedy  wrote:
> "Independent installations will always only overwrite newer versions of the
> launcher with older versions." 'always only' is a bit awkward and the
> sentence looks backwards to me. I would expect only overwriting older
> versions with newer versions.

Agreed, I would expect the same. I would think taking out the word
"only" and then flipping newer and older in the sentence would correct
it.

On Mon, Jun 18, 2012 at 1:05 PM, Terry Reedy  wrote:
> These seem contradictory:
>
> "The 32-bit distribution of Python will not install a 32-bit version of the
> launcher on a 64-bit system."
>
> I presume this mean that it will install the 64-bit version and that there
> will always be only one version of the launcher on the system.
>
> "On 64bit Windows with both 32bit and 64bit implementations of the same
> (major.minor) Python version installed, the 64bit version will always be
> preferred.  This will be true for both 32bit and 64bit implementations of
> the launcher - a 32bit launcher will prefer to execute a 64bit Python
> installation of the specified version if available."
>
> This implies to me that the 32bit installation *will* install a 32bit
> launcher and that there could be both versions of the launcher installed.

I took that as covering an independently-installed launcher.

You could always install your own 32-bit launcher, and it'd prefer to
launch a binary matching the machine type. So yes, there could be
multiple launchers installed for different machine types, and I'm not
sure why we'd want to (or how we could) prevent people from installing
them. You could have a 64-bit launcher available system-wide in your
Windows folder, then you could have a 32-bit launcher running out of
C:\Users\Terry for some purposes.

Martin - is that correct?

===

Outside of Terry's concerns, I find the updated PEP almost ready to go
as-is. Many of the updates were in line with what Martin and I briefly
talked about at PyCon, and I believe some of them came out of previous
PEP discussions on here, so I see nothing unexpected at this point.

My only additional comment would be to have the "Configuration file"
implementation details supplemented with a readable example of where
the py.ini file should be placed. On my machine that is
"C:\Users\brian\AppData\Local", rather than making people have to run
that parameter through the listed function via pywin32.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Stefan Behnel

Armin Rigo, 18.06.2012 23:29:
> On Mon, Jun 18, 2012 at 9:10 PM, Maciej Fijalkowski wrote:
>>> Make cffi less buggy (check the tracker for new test cases ;-), faster
>>> (closer to swig type wrappers), and easier to use than ctypes, and I am sure
>>> there will be interest.
>>
>> I would say it's already fulfilling those three, but I suppose you should
>> try for yourself.
> 
> I don't think the first is fulfilled so far, as we found various
> issues on various Linux and non-Linux platforms (and fixed them, so I
> suppose that release 0.1.1 is coming soon).  But I agree with Fijal
> about speed and ease of use.  Like SWIG it generates wrappers in the
> form of a CPython C extension with built-in functions, so I suppose
> the performance is similar to SWIG and not ctypes.  Well, SWIG
> wrappers typically have custom logic written in C, whereas in cffi
> this logic is typically written as Python code, so I suppose it ends
> up being slower

Any reason you didn't write the C parts in Cython? Skipping through the
code, it seems like it could benefit, both in terms of code overhead and
performance. For something that heavily relies on Python calls, reducing
the call overhead some more should be quite interesting.

It would also be cool to have Cython integration in the sense that cffi
could generate a .pxd file from the parsed C declarations (and eventually
pass the declarations directly into the compiler), which Cython could use
for its one-the-fly code compilation (similar to what cffi does in the
background, as I understand it) or even in its pure Python mode to allow
for C code interaction (which currently requires Cython syntax).

Both could then support the same interface for C declarations. That would
allow users to start with cffi and then compile the performance critical
parts of their code (e.g. a couple of functions in a Python module) through
Cython without major changes - in some cases, applying the "cython.compile"
decorator could be enough already. Sounds great to me.

Stefan

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 397 - Last Comments

2012-06-18 Thread Martin v. Löwis

> Agreed, I would expect the same. I would think taking out the word
> "only" and then flipping newer and older in the sentence would correct
> it.

Will change.

>> "On 64bit Windows with both 32bit and 64bit implementations of the same
>> (major.minor) Python version installed, the 64bit version will always be
>> preferred.  This will be true for both 32bit and 64bit implementations of
>> the launcher - a 32bit launcher will prefer to execute a 64bit Python
>> installation of the specified version if available."
>>
>> This implies to me that the 32bit installation *will* install a 32bit
>> launcher and that there could be both versions of the launcher installed.

No - this paragraph talks about the Python being launched, not the
bitness of the launcher. As (currently) the launcher creates a
subprocess always, this is quite feasible.

The bitness of the launcher really doesn't matter, except that a 32-bit
launcher cannot access all directories, and a 64-bit launcher does not
work on a 32-bit system.

Now that I think about it, it might be that it's best to always have the
launcher as a 32-bit binary. It could disable the filesystem and
registry redirection if it really wanted to, and would work on both
32-bit and 64-bit systems.

> I took that as covering an independently-installed launcher.
> 
> You could always install your own 32-bit launcher, and it'd prefer to
> launch a binary matching the machine type.

No, that's not the plan. The binary being launched is entirely
controlled by command line arguments, ini files, and shebang lines.

I personally find it sad that it always creates a subprocess, and it
could avoid doing so if the launched Python has the same bitness, but
alas, the problems with doing so are mostly convincing.

> So yes, there could be
> multiple launchers installed for different machine types, and I'm not
> sure why we'd want to (or how we could) prevent people from installing
> them. You could have a 64-bit launcher available system-wide in your
> Windows folder, then you could have a 32-bit launcher running out of
> C:\Users\Terry for some purposes.

The PEP doesn't really consider launcher binaries not installed into
the standard location. It would work, but it's out of scope of the PEP.

The PEP actually only talks about launcher binaries in c:\windows, and
essentially says that they must match the bitness of the system.

> My only additional comment would be to have the "Configuration file"
> implementation details supplemented with a readable example of where
> the py.ini file should be placed. On my machine that is
> "C:\Users\brian\AppData\Local", rather than making people have to run
> that parameter through the listed function via pywin32.

Will do.

Martin

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] CFFI released

2012-06-18 Thread Greg Ewing


Is there any provision for keeping the compiled
C code and distributing it along with an application?
Requiring a C compiler to be present at all times
could be a difficulty for Windows.

--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

50 matches

Mail list logo