Re: [Python-Dev] PEP 447: add type.__locallookup__
Perhaps "__getdescriptor__" would work as the method name? Yes, it can technically return a non-descriptor, but the *primary* purpose is to customise the retrieval of objects that will be checked to see if they're descriptors. It *won't* be invoked when looking for ordinary attributes in an instance dict, but *will* be invoked when looking on the class object. Cheers, Nick. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On Fri, Sep 13, 2013 at 08:42:46PM +1000, Nick Coghlan wrote: > Perhaps "__getdescriptor__" would work as the method name? Yes, it can > technically return a non-descriptor, So technically that name is, um, what's the term... oh yes, "a lie". :-) > but the *primary* purpose is to > customise the retrieval of objects that will be checked to see if they're > descriptors. If that's the case, the PEP should make that clear. [Aside: the PEP states that the method shouldn't invoke descriptors. What's the reason for that? If I take the statement literally, doesn't it mean that the method mustn't use any other methods at all? Surely that can't be what is intended, but I'm not sure what is intended.] > It *won't* be invoked when looking for ordinary attributes in > an instance dict, but *will* be invoked when looking on the class object. Just to be clear, if I have: instance = MyClass() x = instance.name and "name" is found in instance.__dict__, then this special method will not be invoked. But if "name" is not found in the instance dict, then "name" will be looked up on the class object MyClass, which may invoke this special method. Am I correct? -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.3 -> default): Fix http.server's request handling case on trailing '/'.
On 9/13/2013 3:22 AM, senthil.kumaran wrote:
> http://hg.python.org/cpython/rev/b85c9d2a5227
> changeset: 85672:b85c9d2a5227
> parent: 85668:66ec8431032d
> parent: 85671:1fcccbbe15e2
> user:Senthil Kumaran
> date:Fri Sep 13 00:22:45 2013 -0700
> summary:
> Fix http.server's request handling case on trailing '/'.
>
> Patch contributed by Vajrasky Kok. Addresses Issue #17324
> +trailing_slash = True if path.rstrip().endswith('/') else False
Wouldn't this be better just as:
trailing_slash = path.rstrip().endswith('/')
--
Eric.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
From: Steven D'Aprano
> On Fri, Sep 13, 2013 at 04:26:06AM +, Steve Dower wrote:
>
>> Last I checked, looking up in the instance dict us exactly what it
>> does. Even the example you posted is doing that.
>
> The example from the PEP shows:
>
> return cls.__dict__[name]
>
> not "self.__dict__[name]". It is true that "the instance" in this case refers
> to
> it being an instance of the metaclass, but that instance is, in fact, a
> class/type. That's why we normally call it "cls" in a metaclass method rather
> than "self".
Right, but where's the difference between these two?
class A:
def __tdb__(self, name):
if name == 'some_attribute_on_my_instance_of_A':
return its_value
try:
return self.__dict__[name]
except KeyError:
raise AttributeError(name)
class MetaB:
def __tdb__(cls, name):
if name == 'some_attribute_on_my_class_B':
return its_value
try:
return cls.__dict__[name]
except KeyError:
raise AttributeError(name)
(Yes, either of these could be written with __getattribute__, but that function
cannot be called by super().)
As I see it, there are two (correct) ways to interpret what this method is for,
which influences what it should be called.
1. It directly replaces obj.__dict__[name] everywhere that is done, including
internally in the interpreter.
2. It is the same as __getattribute__ without the final call to
object.__getattribute__
I guess it's also correct to view it as a special helper for super(), but it is
more generally applicable than that.
[...]
> By the way, I think the PEP should have a more complex example. The
> SillyObject
> example is nice and easy to understand, but it doesn't really help with the
> motivating use-case "dynamic classes that can grow new methods on demand".
> Ronald, if you're reading this, can you add such an example please? Also,
> there's a typo in the SillyObject M method ("fourtytwo" should not have a U in
> it).
Agreed. No harm in more examples.
Here's a quick example of code that does not behave correctly at present.
class A:
def __getattribute__(self, name):
if name == 'foo':
return 'A.foo'
return object.__getattribute__(self, name)
class B(A):
def get_foo(self):
return super().foo
>>> B().get_foo() # skips A.__getattribute__
Traceback (most recent call last):
File "", line 1, in
File "", line 3, in get_foo
AttributeError: 'super' object has no attribute 'foo'
>>> B().foo
A.foo
After changing to use the __tbd__ method:
class A:
def __getattribute__(self, name):
'''Not strictly necessary for this example, but I would expect
that most types overriding __tbd__ also want to override
__getattribute__ to use it. Or maybe object.__getattribute__
should be changed to use __tbd__ too...?'''
try:
return self.__tbd__(name)
except AttributeError:
return object.__getattribute__(self, name)
def __tbd__(self, name):# CHANGED
if name == 'foo':
return 'A.foo'
try:
return self.__dict__[name]
except KeyError:
raise AttributeError(name) # CHANGED
class B(A):
def get_foo(self):
return super().foo
>>> B().get_foo() # does not skip A.__tbd__
A.foo # hopefully this is the result :)
>>> B().foo
A.foo
A full example of where this may realistically be needed is longer and
certainly involves metaclasses, but fundamentally it's just the same as
__getattribute__ with slightly different semantics.
Cheers,
Steve
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2013-09-06 - 2013-09-13)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open4212 (+17)
closed 26571 (+48)
total 30783 (+65)
Open issues with patches: 1935
Issues opened (46)
==
#18939: Venv docs regarding original python install
http://bugs.python.org/issue18939 reopened by gwideman
#18948: deliberately crashing tests should prevent core dumps
http://bugs.python.org/issue18948 opened by pitrou
#18950: Miscellaneous fixes for the sunau module
http://bugs.python.org/issue18950 opened by serhiy.storchaka
#18951: In unittest.TestCase.assertRegex change "re" and "regex" to "r
http://bugs.python.org/issue18951 opened by py.user
#18955: Confusing documentation in Lib/importlib/util.py
http://bugs.python.org/issue18955 opened by vajrasky
#18956: Document useful functions in âpydocâ module
http://bugs.python.org/issue18956 opened by bignose
#18958: Exception('No JSON object could be decoded') when parsing a va
http://bugs.python.org/issue18958 opened by Gallaecio
#18959: Create a "Superseded modules" section in standard library ToC
http://bugs.python.org/issue18959 opened by ncoghlan
#18960: First line can be executed twice
http://bugs.python.org/issue18960 opened by serhiy.storchaka
#18961: Non-UTF8 encoding line
http://bugs.python.org/issue18961 opened by serhiy.storchaka
#18965: 2to3 can produce illegal bytes literals
http://bugs.python.org/issue18965 opened by serhiy.storchaka
#18966: Threads within multiprocessing Process terminate early
http://bugs.python.org/issue18966 opened by pietvo
#18967: Find a less conflict prone approach to Misc/NEWS
http://bugs.python.org/issue18967 opened by ncoghlan
#18968: Find a way to detect incorrectly skipped tests
http://bugs.python.org/issue18968 opened by ncoghlan
#18969: test suite: enable faulthandler timeout in assert_python
http://bugs.python.org/issue18969 opened by neologix
#18970: run_setup() behavior differs from cli invocation of setup.py
http://bugs.python.org/issue18970 opened by l
#18971: Use argparse in the profile/cProfile modules
http://bugs.python.org/issue18971 opened by serhiy.storchaka
#18972: Use argparse in email example scripts
http://bugs.python.org/issue18972 opened by serhiy.storchaka
#18973: Use argparse in the calendar module
http://bugs.python.org/issue18973 opened by serhiy.storchaka
#18974: Use argparse in the diff script
http://bugs.python.org/issue18974 opened by serhiy.storchaka
#18975: timeit: Use thousands separators and print number of loops per
http://bugs.python.org/issue18975 opened by jstasiak
#18976: distutils/command/build_ext passes wrong linker flags
http://bugs.python.org/issue18976 opened by Benedikt.Morbach
#18977: The -t option has no effect in for uu command-line
http://bugs.python.org/issue18977 opened by serhiy.storchaka
#18978: Allow urllib.request.Request subclasses to override method
http://bugs.python.org/issue18978 opened by jason.coombs
#18979: Use argparse in the uu module
http://bugs.python.org/issue18979 opened by serhiy.storchaka
#18981: Typo in the ctypes tests
http://bugs.python.org/issue18981 opened by Anoop.Thomas.Mathew
#18982: Add tests for CLI of the calendar module
http://bugs.python.org/issue18982 opened by serhiy.storchaka
#18983: Specify time unit for timeit CLI
http://bugs.python.org/issue18983 opened by serhiy.storchaka
#18985: Improve the documentation in fcntl module
http://bugs.python.org/issue18985 opened by vajrasky
#18986: Add a case-insensitive case-preserving dict
http://bugs.python.org/issue18986 opened by pitrou
#18987: distutils.utils.get_platform() for 32-bit Python on a 64-bit m
http://bugs.python.org/issue18987 opened by sferencik
#18989: reuse of enum names in class creation inconsistent
http://bugs.python.org/issue18989 opened by ethan.furman
#18990: Remove unnecessary API inconsistency from ElementTree.XMLPullP
http://bugs.python.org/issue18990 opened by scoder
#18993: There is an overshadowed and invalid test in testmock.py
http://bugs.python.org/issue18993 opened by vajrasky
#18994: Inside fcntl module, we does not check the return code of all_
http://bugs.python.org/issue18994 opened by vajrasky
#18995: Enum does not work with reversed
http://bugs.python.org/issue18995 opened by vajrasky
#18996: unittest: more helpful truncating long strings
http://bugs.python.org/issue18996 opened by serhiy.storchaka
#18998: iter() not working in ElementTree
http://bugs.python.org/issue18998 opened by Kronuz
#18999: Robustness issues in multiprocessing.{get,set}_start_method
http://bugs.python.org/issue18999 opened by larsmans
#19001: test_gdb fails on Fedora buildbot
http://bugs.python.org/issue19001 opened by pitrou
#19003: email.generator.BytesGenerator corrupts data by changing line
http://bugs.python.org/issue19003 opene
[Python-Dev] PEP 455: TransformDict
Hello,
Following the python-dev discussion, I've written a PEP to recap the
proposal and the various arguments. It's inlined below, and it will
probably appear soon at http://www.python.org/dev/peps/pep-0455/, too.
Regards
Antoine.
PEP: 455
Title: Adding a key-transforming dictionary to collections
Version: $Revision$
Last-Modified: $Date$
Author: Antoine Pitrou
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 13-Sep-2013
Python-Version: 3.4
Post-History:
Abstract
This PEP proposes a new data structure for the ``collections`` module,
called "TransformDict" in this PEP. This structure is a mutable mapping
which transforms the key using a given function when doing a lookup, but
retains the original key when reading.
Rationale
=
Numerous specialized versions of this pattern exist. The most common
is a case-insensitive case-preserving dict, i.e. a dict-like container
which matches keys in a case-insensitive fashion but retains the
original casing. It is a very common need in network programming, as
many protocols feature some arrays of "key / value" properties in their
messages, where the keys are textual strings whose casing isn't
relevant.
Another common request is an identity dict, where keys are matched
according to their respective id()s instead of normal matching.
Both are instances of a more general pattern, where a given
transformation function is applied to keys when looking them up: that
function being ``str.lower`` in the former example and the built-in
``id`` function in the latter.
(it can be said that the pattern *projects* keys from the user-visible
set onto the internal lookup set, hence this PEP's title)
Semantics
=
TransformDict is a ``MutableMapping`` implementation: it faithfully
implements the well-known API of mutable mappings, as ``dict`` itself
and other dict-like classes in the standard library. Therefore, this
PEP won't rehash the semantics of most TransformDict methods.
The transformation function needn't be bijective, it can be strictly
surjective as in the case-insensitive example::
>>> d = TransformDict(str.lower)
>>> d['SomeKey'] = 5
>>> d['somekey']
5
>>> d['SOMEKEY']
5
TransformDict retains the first key used when creating an entry::
>>> d = TransformDict(str.lower)
>>> d['SomeKey'] = 1
>>> d['somekey'] = 2
>>> list(d.items())
[('SomeKey', 2)]
The original keys needn't be hashable, as long as the transformation
function returns a hashable one::
>>> d = TransformDict(id)
>>> l = [None]
>>> d[l] = 5
>>> l in d
True
Constructor
---
As shown in the example aboves, creating a TransformDict requires
passing the key transformation function as the first argument (much
like creating a ``defaultdict`` requires passing the factory function
as first argument).
The constructor also takes other optional arguments which can be used
to initialize the TransformDict with certain key-value pairs. Those
optional arguments are the same as in the ``dict`` and ``defaultdict``
constructors::
>>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
>>> sorted(d.items())
[('Bar', 2), ('Foo', 1)]
Alternative proposals and questions
===
Retaining the last original key
---
Most python-dev respondents found retaining the first user-supplied key
more intuitive than retaining the last. Also, it matches the dict
object's own behaviour when using different but equal keys::
>>> d = {}
>>> d[1] = 'hello'
>>> d[1.0] = 'world'
>>> d
{1: 'world'}
Furthermore, explicitly retaining the last key in a first-key-retaining
scheme is still possible using the following approach::
d.pop(key, None)
d[key] = value
while the converse (retaining the first key in a last-key-retaining
scheme) doesn't look possible without rewriting part of the container's
code.
Using an encoder / decoder pair
---
Using a function pair isn't necessary, since the original key is
retained by the container. Moreover, an encoder / decoder pair would
require the transformation to be bijective, which prevents important
use cases like case-insensitive matching.
Providing a transformation function for values
--
Dictionary values are not used for lookup, their semantics are totally
irrelevant to the container's operation. Therefore, there is no point
in having both an "original" and a "transformed" value: the transformed
value wouldn't be used for anything.
Providing a specialized container, not generic
--
It was asked why we would provide the generic TransformDict construct
rather than a specialized case-insensitive dict variant. The answer
is that it's nearly as cheap (code-wise and performance-wise) to provide
the generic construct, and it can fill more use cases.
Implementation
==
A patch
Re: [Python-Dev] PEP 455: TransformDict
13.09.13 21:40, Antoine Pitrou написав(ла):
Both are instances of a more general pattern, where a given
transformation function is applied to keys when looking them up: that
function being ``str.lower`` in the former example and the built-in
``id`` function in the latter.
Please use str.casefold in examples.
>>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
{'Foo': 1} or [('Foo', 1)].
Providing a specialized container, not generic
--
It was asked why we would provide the generic TransformDict construct
rather than a specialized case-insensitive dict variant. The answer
is that it's nearly as cheap (code-wise and performance-wise) to provide
the generic construct, and it can fill more use cases.
Except lightweight IdentityDict which can be implemented more efficient
than TransformDict(id). It doesn't need in calling the transform
function, computing the hash of transformed key, comparing keys. But
perhaps in many cases TransformDict(id) is enough.
Python's own pickle module uses identity lookups for object
memoization:
http://hg.python.org/cpython/file/0e70bf1f32a3/Lib/pickle.py#l234
Also copy, json, cProfile, doctest and _threading_local.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
+1 :) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 22:31:02 +0300
Serhiy Storchaka wrote:
> 13.09.13 21:40, Antoine Pitrou написав(ла):
> > Both are instances of a more general pattern, where a given
> > transformation function is applied to keys when looking them up: that
> > function being ``str.lower`` in the former example and the built-in
> > ``id`` function in the latter.
>
> Please use str.casefold in examples.
>
> > >>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
>
> {'Foo': 1} or [('Foo', 1)].
Ok, thanks.
> > Providing a specialized container, not generic
> > --
> >
> > It was asked why we would provide the generic TransformDict construct
> > rather than a specialized case-insensitive dict variant. The answer
> > is that it's nearly as cheap (code-wise and performance-wise) to provide
> > the generic construct, and it can fill more use cases.
>
> Except lightweight IdentityDict which can be implemented more efficient
> than TransformDict(id). It doesn't need in calling the transform
> function, computing the hash of transformed key, comparing keys.
> But
> perhaps in many cases TransformDict(id) is enough.
That's true. But it's only important if TransformDict is the
bottleneck. I doubt the memoizing dictionary is a bottleneck in
e.g. the pure Python implementation of pickle or json.
> > Python's own pickle module uses identity lookups for object
> > memoization:
> > http://hg.python.org/cpython/file/0e70bf1f32a3/Lib/pickle.py#l234
>
> Also copy, json, cProfile, doctest and _threading_local.
Thanks, will add them.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
I see the PEP does not contemplate a way to retrieve the original key
- like we've talked about somewhere along the thread.
On 13 September 2013 16:37, Antoine Pitrou wrote:
> On Fri, 13 Sep 2013 22:31:02 +0300
> Serhiy Storchaka wrote:
>> 13.09.13 21:40, Antoine Pitrou написав(ла):
>> > Both are instances of a more general pattern, where a given
>> > transformation function is applied to keys when looking them up: that
>> > function being ``str.lower`` in the former example and the built-in
>> > ``id`` function in the latter.
>>
>> Please use str.casefold in examples.
>>
>> > >>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
>>
>> {'Foo': 1} or [('Foo', 1)].
>
> Ok, thanks.
>
>> > Providing a specialized container, not generic
>> > --
>> >
>> > It was asked why we would provide the generic TransformDict construct
>> > rather than a specialized case-insensitive dict variant. The answer
>> > is that it's nearly as cheap (code-wise and performance-wise) to provide
>> > the generic construct, and it can fill more use cases.
>>
>> Except lightweight IdentityDict which can be implemented more efficient
>> than TransformDict(id). It doesn't need in calling the transform
>> function, computing the hash of transformed key, comparing keys.
>> But
>> perhaps in many cases TransformDict(id) is enough.
>
> That's true. But it's only important if TransformDict is the
> bottleneck. I doubt the memoizing dictionary is a bottleneck in
> e.g. the pure Python implementation of pickle or json.
>
>> > Python's own pickle module uses identity lookups for object
>> > memoization:
>> > http://hg.python.org/cpython/file/0e70bf1f32a3/Lib/pickle.py#l234
>>
>> Also copy, json, cProfile, doctest and _threading_local.
>
> Thanks, will add them.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 16:54:01 -0300 "Joao S. O. Bueno" wrote: > I see the PEP does not contemplate a way to retrieve the original key > - like we've talked about somewhere along the thread. Indeed. If that's important I can add it. I was hoping to keep very close to the MutableMapping API, to make the PEP as few controversial as possible. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
13.09.13 21:40, Antoine Pitrou написав(ла): Alternative proposals and questions === Yet one alternative proposal is to add the dict.__transform__() method. Actually this not contradict TransformDict, but generalize it. TransformDict will be just handly interface to __transform__() as defaultdict to __missing__(). It provides only constructor, repr and pickling. My second patch for http://bugs.python.org/issue18986 shows that as implementation of dict.__transform__(), so implementation of TransformDict using dict.__transform__() are simple enough. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 23:16:10 +0300 Serhiy Storchaka wrote: > 13.09.13 21:40, Antoine Pitrou написав(ла): > > Alternative proposals and questions > > === > > Yet one alternative proposal is to add the dict.__transform__() method. > Actually this not contradict TransformDict, but generalize it. > TransformDict will be just handly interface to __transform__() as > defaultdict to __missing__(). It provides only constructor, repr and > pickling. Is it an alternative proposal or is it compatible with the PEP? The PEP specifies the API, not the implementation. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 16:09:10 -0400 "R. David Murray" wrote: > On Fri, 13 Sep 2013 20:40:58 +0200, Antoine Pitrou > wrote: > > Rationale > > = > > > > Numerous specialized versions of this pattern exist. The most common > > is a case-insensitive case-preserving dict, i.e. a dict-like container > > which matches keys in a case-insensitive fashion but retains the > > original casing. It is a very common need in network programming, as > > many protocols feature some arrays of "key / value" properties in their > > messages, where the keys are textual strings whose casing isn't > > relevant. > > This motivation would be stronger if the last phrase was something like > "where the keys are textual strings whose case is specified to be ignored > on receipt but by either specification or custom is to be preserved > or non-trivially canonicalized when retransmitted." Thanks, will add. > > (it can be said that the pattern *projects* keys from the user-visible > > set onto the internal lookup set, hence this PEP's title) > > Not clear what "projects" has to do with the PEP title. The PEP was originally titled "a key-projecting dictionary" and I've changed it as it was too obscure. I've now also removed the obsolete part of the sentence above :-) > > Alternative proposals and questions > > === > > You don't mention the alternate constructor discussion, or the > rationale for following the defaultdict pattern (ie: following the > established pattern :) Ok, will do that too. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
13.09.13 23:02, Antoine Pitrou написав(ла): On Fri, 13 Sep 2013 16:54:01 -0300 "Joao S. O. Bueno" wrote: I see the PEP does not contemplate a way to retrieve the original key - like we've talked about somewhere along the thread. Indeed. If that's important I can add it. I was hoping to keep very close to the MutableMapping API, to make the PEP as few controversial as possible. I think that's important. As OrderectDict has additional methods besides the MutableMapping API, so TransformDict should provide useful specialized methods. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
13.09.13 22:37, Antoine Pitrou написав(ла): That's true. But it's only important if TransformDict is the bottleneck. I doubt the memoizing dictionary is a bottleneck in e.g. the pure Python implementation of pickle or json. It can be used in the C implementation. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 20:40:58 +0200, Antoine Pitrou wrote:
> Rationale
> =
>
> Numerous specialized versions of this pattern exist. The most common
> is a case-insensitive case-preserving dict, i.e. a dict-like container
> which matches keys in a case-insensitive fashion but retains the
> original casing. It is a very common need in network programming, as
> many protocols feature some arrays of "key / value" properties in their
> messages, where the keys are textual strings whose casing isn't
> relevant.
This motivation would be stronger if the last phrase was something like
"where the keys are textual strings whose case is specified to be ignored
on receipt but by either specification or custom is to be preserved
or non-trivially canonicalized when retransmitted."
> (it can be said that the pattern *projects* keys from the user-visible
> set onto the internal lookup set, hence this PEP's title)
Not clear what "projects" has to do with the PEP title.
> Constructor
> ---
>
> As shown in the example aboves, creating a TransformDict requires
> passing the key transformation function as the first argument (much
> like creating a ``defaultdict`` requires passing the factory function
> as first argument).
>
> The constructor also takes other optional arguments which can be used
> to initialize the TransformDict with certain key-value pairs. Those
> optional arguments are the same as in the ``dict`` and ``defaultdict``
> constructors::
>
>>>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
>>>> sorted(d.items())
>[('Bar', 2), ('Foo', 1)]
>
>
> Alternative proposals and questions
> ===
You don't mention the alternate constructor discussion, or the
rationale for following the defaultdict pattern (ie: following the
established pattern :)
--David
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
13.09.13 23:21, Antoine Pitrou написав(ла): On Fri, 13 Sep 2013 23:16:10 +0300 Serhiy Storchaka wrote: 13.09.13 21:40, Antoine Pitrou написав(ла): Alternative proposals and questions === Yet one alternative proposal is to add the dict.__transform__() method. Actually this not contradict TransformDict, but generalize it. TransformDict will be just handly interface to __transform__() as defaultdict to __missing__(). It provides only constructor, repr and pickling. Is it an alternative proposal or is it compatible with the PEP? The PEP specifies the API, not the implementation. Both. On one side, with this proposition TransformDict itself doesn't deserve PEP. It will be trivial and obvious thing. Not very important. On other side, TransformDict can be implemented even without dict.__transform__(), on pure Python. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Sat, 14 Sep 2013 00:05:27 +0300 Serhiy Storchaka wrote: > 13.09.13 23:21, Antoine Pitrou написав(ла): > > On Fri, 13 Sep 2013 23:16:10 +0300 > > Serhiy Storchaka wrote: > >> 13.09.13 21:40, Antoine Pitrou написав(ла): > >>> Alternative proposals and questions > >>> === > >> > >> Yet one alternative proposal is to add the dict.__transform__() method. > >> Actually this not contradict TransformDict, but generalize it. > >> TransformDict will be just handly interface to __transform__() as > >> defaultdict to __missing__(). It provides only constructor, repr and > >> pickling. > > > > Is it an alternative proposal or is it compatible with the PEP? > > The PEP specifies the API, not the implementation. > > Both. On one side, with this proposition TransformDict itself doesn't > deserve PEP. It will be trivial and obvious thing. Well, TransformDict would still be the user-visible API, not __transform__; like defaultdict is the user-visible API, not __missing__. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
You know, I can think up several use cases for the case-insensitive idea.
Not sure if I missed something, but there should be a function to retrieve
the original key from the modified. i.e.:
>>> mydict['SomeStr'] = 5
>>> mydict.get_key('somestr')
'SomeStr'
>>>
On Fri, Sep 13, 2013 at 1:40 PM, Antoine Pitrou wrote:
>
> Hello,
>
> Following the python-dev discussion, I've written a PEP to recap the
> proposal and the various arguments. It's inlined below, and it will
> probably appear soon at http://www.python.org/dev/peps/pep-0455/, too.
>
> Regards
>
> Antoine.
>
>
> PEP: 455
> Title: Adding a key-transforming dictionary to collections
> Version: $Revision$
> Last-Modified: $Date$
> Author: Antoine Pitrou
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 13-Sep-2013
> Python-Version: 3.4
> Post-History:
>
>
> Abstract
>
>
> This PEP proposes a new data structure for the ``collections`` module,
> called "TransformDict" in this PEP. This structure is a mutable mapping
> which transforms the key using a given function when doing a lookup, but
> retains the original key when reading.
>
>
> Rationale
> =
>
> Numerous specialized versions of this pattern exist. The most common
> is a case-insensitive case-preserving dict, i.e. a dict-like container
> which matches keys in a case-insensitive fashion but retains the
> original casing. It is a very common need in network programming, as
> many protocols feature some arrays of "key / value" properties in their
> messages, where the keys are textual strings whose casing isn't
> relevant.
>
> Another common request is an identity dict, where keys are matched
> according to their respective id()s instead of normal matching.
>
> Both are instances of a more general pattern, where a given
> transformation function is applied to keys when looking them up: that
> function being ``str.lower`` in the former example and the built-in
> ``id`` function in the latter.
>
> (it can be said that the pattern *projects* keys from the user-visible
> set onto the internal lookup set, hence this PEP's title)
>
>
> Semantics
> =
>
> TransformDict is a ``MutableMapping`` implementation: it faithfully
> implements the well-known API of mutable mappings, as ``dict`` itself
> and other dict-like classes in the standard library. Therefore, this
> PEP won't rehash the semantics of most TransformDict methods.
>
> The transformation function needn't be bijective, it can be strictly
> surjective as in the case-insensitive example::
>
>>>> d = TransformDict(str.lower)
>>>> d['SomeKey'] = 5
>>>> d['somekey']
>5
>>>> d['SOMEKEY']
>5
>
> TransformDict retains the first key used when creating an entry::
>
>>>> d = TransformDict(str.lower)
>>>> d['SomeKey'] = 1
>>>> d['somekey'] = 2
>>>> list(d.items())
>[('SomeKey', 2)]
>
> The original keys needn't be hashable, as long as the transformation
> function returns a hashable one::
>
>>>> d = TransformDict(id)
>>>> l = [None]
>>>> d[l] = 5
>>>> l in d
>True
>
> Constructor
> ---
>
> As shown in the example aboves, creating a TransformDict requires
> passing the key transformation function as the first argument (much
> like creating a ``defaultdict`` requires passing the factory function
> as first argument).
>
> The constructor also takes other optional arguments which can be used
> to initialize the TransformDict with certain key-value pairs. Those
> optional arguments are the same as in the ``dict`` and ``defaultdict``
> constructors::
>
>>>> d = TransformDict(str.lower, [('Foo': 1)], Bar=2)
>>>> sorted(d.items())
>[('Bar', 2), ('Foo', 1)]
>
>
> Alternative proposals and questions
> ===
>
> Retaining the last original key
> ---
>
> Most python-dev respondents found retaining the first user-supplied key
> more intuitive than retaining the last. Also, it matches the dict
> object's own behaviour when using different but equal keys::
>
>>>> d = {}
>>>> d[1] = 'hello'
>>>> d[1.0] = 'world'
>>>> d
>{1: 'world'}
>
> Furthermore, explicitly retaining the last key in a first-key-retaining
> scheme is still possible using the following approach::
>
>d.pop(key, None)
>d[key] = value
>
> while the converse (retaining the first key in a last-key-retaining
> scheme) doesn't look possible without rewriting part of the container's
> code.
>
> Using an encoder / decoder pair
> ---
>
> Using a function pair isn't necessary, since the original key is
> retained by the container. Moreover, an encoder / decoder pair would
> require the transformation to be bijective, which prevents important
> use cases like case-insensitive matching.
>
> Providing a transformation function for values
> --
>
> Dictionary values are not used for lookup, their semantics are totally
> irr
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 23:18:39 +0300
Serhiy Storchaka wrote:
> 13.09.13 23:02, Antoine Pitrou написав(ла):
> > On Fri, 13 Sep 2013 16:54:01 -0300
> > "Joao S. O. Bueno" wrote:
> >> I see the PEP does not contemplate a way to retrieve the original key
> >> - like we've talked about somewhere along the thread.
> >
> > Indeed. If that's important I can add it. I was hoping to keep very
> > close to the MutableMapping API, to make the PEP as few
> > controversial as possible.
>
> I think that's important. As OrderectDict has additional methods besides
> the MutableMapping API, so TransformDict should provide useful
> specialized methods.
Ok, I have a better (IMO) proposal:
>>> d = TransformDict(str.casefold, {'Foo': 1})
>>> d.getitem('foo')
('Foo', 1)
>>> d.getitem('bar')
Traceback (most recent call last):
File "", line 1, in
KeyError: 'bar'
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 02:45 PM, Antoine Pitrou wrote:
Serhiy Storchaka wrote:
I think that's important. As OrderectDict has additional methods besides
the MutableMapping API, so TransformDict should provide useful
specialized methods.
Ok, I have a better (IMO) proposal:
>>> d = TransformDict(str.casefold, {'Foo': 1})
>>> d.getitem('foo')
('Foo', 1)
>>> d.getitem('bar')
Traceback (most recent call last):
File "", line 1, in
KeyError: 'bar'
-1
When I want the canonical key, I'm not going to care about the value; and if I
do, it's easy enough to get.
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, Sep 13, 2013 at 11:45:16PM +0200, Antoine Pitrou wrote:
> Ok, I have a better (IMO) proposal:
>
> >>> d = TransformDict(str.casefold, {'Foo': 1})
> >>> d.getitem('foo')
> ('Foo', 1)
> >>> d.getitem('bar')
> Traceback (most recent call last):
> File "", line 1, in
> KeyError: 'bar'
Is it more common to want both the canonical key and value at the same
time, or to just want the canonical key? My gut feeling is that I'm
likely to have code like this:
d = TransformDict(...)
for key in data:
key = d.get_canonical(key)
value = d[key]
print("{}: {}".format(key, value))
in which case having a single call to return both will be great:
for key in data:
key, value = d.getitem(key)
print("{}: {}".format(key, value))
but I'm really not sure. Maybe Ethan is right.
I think these sorts of associated questions are why some people
(Raymond, Nick) want to see the proposal live outside of the standard
library for a while first. The general idea is great, but we're going to
bike shed the API without having much idea of how it will actually be
used.
So, my suggestion is this:
- Let's add __transform__ to dicts for 3.4, similar to __missing__, and
see how people end up using it in the real world.
- Then, in 3.5, we can make a much better informed decision about the
best API for a TransformedDict front-end to __transform__.
+1 on __transform__ method on dicts.
+0 on TransformedDict in 3.4
+1 on waiting for 3.5 based on experience on using __transform__.
--
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 450 adding statistics module
On Sun, Sep 08, 2013 at 10:51:57AM -0700, Guido van Rossum wrote: > Never mind, I found the patch and the issue. I really think that the > *PEP* is ready for inclusion after the open issues are changed into > something like Discussion or Future Work, and after adding a more > prominent link to the issue with the patch. Then the *patch* can be > reviewed some more until it is ready -- it looks very close already. I've updated the PEP as requested. Is there anything further that needs to be done to have it approved? http://www.python.org/dev/peps/pep-0450/ -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Sat, 14 Sep 2013 10:49:52 +1000
Steven D'Aprano wrote:
>
> Is it more common to want both the canonical key and value at the same
> time, or to just want the canonical key? My gut feeling is that I'm
> likely to have code like this:
>
> d = TransformDict(...)
> for key in data:
> key = d.get_canonical(key)
> value = d[key]
> print("{}: {}".format(key, value))
>
> in which case having a single call to return both will be great:
>
> for key in data:
> key, value = d.getitem(key)
> print("{}: {}".format(key, value))
>
> but I'm really not sure. Maybe Ethan is right.
I don't think it really matters. From getitem() it's trivial to extract
the key alone.
> I think these sorts of associated questions are why some people
> (Raymond, Nick) want to see the proposal live outside of the standard
> library for a while first. The general idea is great, but we're going to
> bike shed the API without having much idea of how it will actually be
> used.
Even after it's used, it will still be bikeshedded: such is how
proposals are generally discussed. There really isn't very much to
decide here, and whether we only return a key or a (key, value) pair is
almost cosmetic: both APIs are reasonably convenient.
> So, my suggestion is this:
>
> - Let's add __transform__ to dicts for 3.4, similar to __missing__, and
> see how people end up using it in the real world.
Well, __missing__ isn't used very much, I think. People use defaultdict.
(note that I'm not even proposing __transform__ right now :-))
> +1 on __transform__ method on dicts.
>
> +0 on TransformedDict in 3.4
>
> +1 on waiting for 3.5 based on experience on using __transform__.
Ok, thanks.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, 13 Sep 2013 15:32:55 -0700
Ethan Furman wrote:
> On 09/13/2013 02:45 PM, Antoine Pitrou wrote:
> > Serhiy Storchaka wrote:
> >>
> >> I think that's important. As OrderectDict has additional methods besides
> >> the MutableMapping API, so TransformDict should provide useful
> >> specialized methods.
> >
> > Ok, I have a better (IMO) proposal:
> >
> > >>> d = TransformDict(str.casefold, {'Foo': 1})
> > >>> d.getitem('foo')
> > ('Foo', 1)
> > >>> d.getitem('bar')
> > Traceback (most recent call last):
> >File "", line 1, in
> > KeyError: 'bar'
>
> -1
>
> When I want the canonical key, I'm not going to care about the value; and if
> I do, it's easy enough to get.
But if you don't care about the value, the key is enough to get ;-)
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 14/09/2013 01:49, Steven D'Aprano wrote:
On Fri, Sep 13, 2013 at 11:45:16PM +0200, Antoine Pitrou wrote:
Ok, I have a better (IMO) proposal:
>>> d = TransformDict(str.casefold, {'Foo': 1})
>>> d.getitem('foo')
('Foo', 1)
>>> d.getitem('bar')
Traceback (most recent call last):
File "", line 1, in
KeyError: 'bar'
Is it more common to want both the canonical key and value at the same
time, or to just want the canonical key? My gut feeling is that I'm
likely to have code like this:
d = TransformDict(...)
for key in data:
key = d.get_canonical(key)
value = d[key]
print("{}: {}".format(key, value))
I think must be missing something. I thought that iterating over the
dict would yield the original keys, so if you wanted the original key
and value you would write:
for key, value in data.items():
print("{}: {}".format(key, value))
and if you wanted the transformed key you would apply the transform
function to the key.
in which case having a single call to return both will be great:
for key in data:
key, value = d.getitem(key)
print("{}: {}".format(key, value))
but I'm really not sure. Maybe Ethan is right.
I think these sorts of associated questions are why some people
(Raymond, Nick) want to see the proposal live outside of the standard
library for a while first. The general idea is great, but we're going to
bike shed the API without having much idea of how it will actually be
used.
So, my suggestion is this:
- Let's add __transform__ to dicts for 3.4, similar to __missing__, and
see how people end up using it in the real world.
- Then, in 3.5, we can make a much better informed decision about the
best API for a TransformedDict front-end to __transform__.
+1 on __transform__ method on dicts.
+0 on TransformedDict in 3.4
+1 on waiting for 3.5 based on experience on using __transform__.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 05:49 PM, Steven D'Aprano wrote: +1 on __transform__ method on dicts. What would __transform__ do? Just canonicalize the key, or also save the original key? How much front-end work will each user have to do to actually use this new magic method to good effect? Personally, if there's a bunch of push-back against just adding TransformDict directly, why don't we make it provisional? I thought that was what provisional was for (meaning: we're going to add it, PyPI is not really appropriate, there may be some API changes). -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 06:25 PM, MRAB wrote:
On 14/09/2013 01:49, Steven D'Aprano wrote:
Is it more common to want both the canonical key and value at the same
time, or to just want the canonical key? My gut feeling is that I'm
likely to have code like this:
d = TransformDict(...)
for key in data:
key = d.get_canonical(key)
value = d[key]
print("{}: {}".format(key, value))
I think I must be missing something. I thought that iterating over the
dict would yield the original keys, so if you wanted the original key
and value you would write:
for key, value in data.items():
print("{}: {}".format(key, value))
Well, that's certainly how I would do it. ;)
and if you wanted the transformed key you would apply the transform
function to the key.
Indeed. The question is: how? It is entirely possible that your function has a TransformDict alone, and no memory of
the transform function used to create the dict...
If the key transform function were saved directly on the TransformDict instance as, say, .transform_key, then problem
solved.
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 14/09/2013 02:40, Ethan Furman wrote:
On 09/13/2013 06:25 PM, MRAB wrote:
On 14/09/2013 01:49, Steven D'Aprano wrote:
Is it more common to want both the canonical key and value at the same
time, or to just want the canonical key? My gut feeling is that I'm
likely to have code like this:
d = TransformDict(...)
for key in data:
key = d.get_canonical(key)
value = d[key]
print("{}: {}".format(key, value))
I think I must be missing something. I thought that iterating over the
dict would yield the original keys, so if you wanted the original key
and value you would write:
for key, value in data.items():
print("{}: {}".format(key, value))
Well, that's certainly how I would do it. ;)
and if you wanted the transformed key you would apply the transform
function to the key.
Indeed. The question is: how? It is entirely possible that your function has
a TransformDict alone, and no memory of
the transform function used to create the dict...
If the key transform function were saved directly on the TransformDict instance
as, say, .transform_key, then problem
solved.
defaultdict has .default_factory, so having something like
.transform_key would have the added advantage of consistency.
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, Sep 13, 2013 at 06:00:18PM -0700, Ethan Furman wrote: > Personally, if there's a bunch of push-back against just adding > TransformDict directly, why don't we make it provisional? I thought that > was what provisional was for (meaning: we're going to add it, PyPI is not > really appropriate, there may be some API changes). Not according to PEP 411. It implies that only modules/packages can be provisional, not individual functions, and states that "most packages" are expected to be provisional. So either PEP 411 doesn't apply to TransformDict at all, or it applies by default. The PEP doesn't say. http://www.python.org/dev/peps/pep-0411/ Everything below the line is about PEP 411, not TransformDict. If you don't care about PEP 411, you can stop reading now. == Personally, I think it's a poor PEP. It doesn't document opposition to the idea, and if I recall the discussion at the time correctly, there was plenty of opposition. - Since people cannot rely on provisional features still being available in the future, if they care the slightest about forward compatibility, they dare not use them. - If people do use them, and we care about backwards compatibility, we dare not remove provisional packages without going through the same deprecation process as for ordinary packages. - It relies on people reading the documentation and noticing that a package is marked "provisional". Like that's going to happen. None of these arguments are documented in the PEP, let alone refuted. Even if the decision to approve the PEP ends up being vindicated, I think it was poor form for the PEP to ignore arguments against. I don't think that "provisional" helps end users at all. If anything, it hurts them -- it means more uncertainty and doubt. Packages may languish in provisional status indefinitely. Speaking as an end user, I used to know that once a feature hit the std lib, it was stable and wouldn't be removed without going through a lengthy period of deprecation (except under truly remarkable circumstances). But for provisional packages, that promise doesn't apply, and although PEP 411 says that packages won't be gratuitously removed, I have to assume that any provisional package in version 3.x could be gone without notice or warning in 3.x+1. I don't see how that helps me. It just means that for some indefinite period, there's a package in the std lib doing exactly what I want that I dare not use in production software because there are no stability guarantees. The PEP has a lengthy section that claims to be about how it will help end users. It actually isn't -- it is about how inclusion in the standard library helps end users. Not surprisingly, there's nothing about why reserving the right to rip out a package without warning is good for end uers, since it isn't. End of rant. -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Sat, Sep 14, 2013 at 02:25:42AM +0100, MRAB wrote:
> On 14/09/2013 01:49, Steven D'Aprano wrote:
> >d = TransformDict(...)
> >for key in data:
> > key = d.get_canonical(key)
> > value = d[key]
> > print("{}: {}".format(key, value))
> >
> I think must be missing something. I thought that iterating over the
> dict would yield the original keys, so if you wanted the original key
> and value you would write:
>
> for key, value in data.items():
> print("{}: {}".format(key, value))
You're missing that I'm not iterating over the entire dict, just some
subset ("data") that I got from elsewhere. In real code, of course I
would have to handle missing keys. My gut feeling is that normally I
would want both the canonical key and the value, not just any valid key:
# not this
for key in data:
value = d[key]
print("{}: {}".format(key, value))
# but this
for key in data:
key, value = d.getitem(key)
print("{}: {}".format(key, value))
but that's just a gut feeling, and I daresay that it would depend on the
application.
> and if you wanted the transformed key you would apply the transform
> function to the key.
The caller may not know or care what the transformation function is. The
caller may only care about one or two things:
- does this key match a key in the dict?
- what is the canonical version of this key?
For example, in a case-preserving file system, the canonical version of
a file name is whatever the user first typed when they created the file:
create file "SoMe FiLe"
so the canonical version is, "SoMe FiLe". But any of these may be
expected to match:
some file
SOME FILE
Some File
some%20file
etc. It is *not* simply the case that the canonical version is "some
file". Certainly that's not how case-preserving file systems work.
--
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, Sep 13, 2013 at 06:00:18PM -0700, Ethan Furman wrote: > On 09/13/2013 05:49 PM, Steven D'Aprano wrote: > > > >+1 on __transform__ method on dicts. > > What would __transform__ do? Just canonicalize the key, or also save the > original key? How much front-end work will each user have to do to > actually use this new magic method to good effect? Sorry, I missed replying to this part. See Serhiy's post: https://mail.python.org/pipermail/python-dev/2013-September/128633.html and suggested patch here: http://bugs.python.org/issue18986 -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On Fri, Sep 13, 2013 at 06:40:06PM -0700, Ethan Furman wrote: > >and if you wanted the transformed key you would apply the transform > >function to the key. > > Indeed. The question is: how? It is entirely possible that your function > has a TransformDict alone, and no memory of the transform function used to > create the dict... > > If the key transform function were saved directly on the TransformDict > instance as, say, .transform_key, then problem solved. While I think it is good and proper to have the transformation function available, if you're manually applying the transformation function then IMO chances are good that you've missed the whole point of the transformdict. Think of defaultdict. The purpose of defaultdict is to avoid needing to check whether the key is missing or not. If I suggested that the way to use a defaultdict was to write code like this: value = d[key] if key in d else d.default_factory() I'd be missing the point. Likewise, the point of transformdict is to avoid needing to care about the transformation function 99% of the time. -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 08:33 PM, Steven D'Aprano wrote: Likewise, the point of transformdict is to avoid needing to care about the transformation function 99% of the time. No one's arguing against that point. It's the 1% of the time that being able to get the canonical name back is a Good Thing, and the best way to do that is to have the transform_key function easily available. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 13 September 2013 22:40, Ethan Furman wrote:
> On 09/13/2013 06:25 PM, MRAB wrote:
>>
>> On 14/09/2013 01:49, Steven D'Aprano wrote:
>>>
>>>
>>> Is it more common to want both the canonical key and value at the same
>>> time, or to just want the canonical key? My gut feeling is that I'm
>>> likely to have code like this:
>>>
>>>
>>> d = TransformDict(...)
>>> for key in data:
>>> key = d.get_canonical(key)
>>> value = d[key]
>>> print("{}: {}".format(key, value))
>>>
>> I think I must be missing something. I thought that iterating over the
>>
>> dict would yield the original keys, so if you wanted the original key
>> and value you would write:
>>
>> for key, value in data.items():
>> print("{}: {}".format(key, value))
>
>
> Well, that's certainly how I would do it. ;)
I hope you are aware that this pattern does not help when one wants
_one_ canonical key having a non-canonical one, besides having to linearly
walk through all keys and check the "__transform__"
to each one.
I mean - given no function to retrieve the canonical key,
one would have to resort to:
my_key = data.__transform__(given_key)
for key, value in data.items():
if data.__transform__(key) == my_key:
WHich would defeat not only the purpose of a Transform dict, but the purpose
of a dict alltogether.
(of course, the one obvious way to do it would be to have the original key
stored along the value - which is just a bit less silly than the example above)
OTOH, having a `get_canonical_key` method or similar seens trivial enough-
if the only provided method retrieves the value as well, it would not
be that bad.
>
>
>
>> and if you wanted the transformed key you would apply the transform
>> function to the key.
>
>
> Indeed. The question is: how? It is entirely possible that your function
> has a TransformDict alone, and no memory of the transform function used to
> create the dict...
>
> If the key transform function were saved directly on the TransformDict
> instance as, say, .transform_key, then problem solved.
>
> --
> ~Ethan~
>
> ___
> Python-Dev mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 11:45 PM, Antoine Pitrou wrote:
> On Fri, 13 Sep 2013 23:18:39 +0300
> Serhiy Storchaka wrote:
>> 13.09.13 23:02, Antoine Pitrou написав(ла):
>> > On Fri, 13 Sep 2013 16:54:01 -0300
>> > "Joao S. O. Bueno" wrote:
>> >> I see the PEP does not contemplate a way to retrieve the original key
>> >> - like we've talked about somewhere along the thread.
>> >
>> > Indeed. If that's important I can add it. I was hoping to keep very
>> > close to the MutableMapping API, to make the PEP as few
>> > controversial as possible.
>>
>> I think that's important. As OrderectDict has additional methods besides
>> the MutableMapping API, so TransformDict should provide useful
>> specialized methods.
>
> Ok, I have a better (IMO) proposal:
>
> >>> d = TransformDict(str.casefold, {'Foo': 1})
> >>> d.getitem('foo')
> ('Foo', 1)
> >>> d.getitem('bar')
> Traceback (most recent call last):
> File "", line 1, in
> KeyError: 'bar'
Except that getitem is very close to __getitem__, which will be confusing.
(Although it would be the correct name, and __getitem__ is the wrong one).
cheers,
Georg
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 10:09 PM, R. David Murray wrote: > On Fri, 13 Sep 2013 20:40:58 +0200, Antoine Pitrou > wrote: >> Rationale >> = >> >> Numerous specialized versions of this pattern exist. The most common >> is a case-insensitive case-preserving dict, i.e. a dict-like container >> which matches keys in a case-insensitive fashion but retains the >> original casing. It is a very common need in network programming, as >> many protocols feature some arrays of "key / value" properties in their >> messages, where the keys are textual strings whose casing isn't >> relevant. > > This motivation would be stronger if the last phrase was something like > "where the keys are textual strings whose case is specified to be ignored > on receipt but by either specification or custom is to be preserved > or non-trivially canonicalized when retransmitted." > >> (it can be said that the pattern *projects* keys from the user-visible >> set onto the internal lookup set, hence this PEP's title) > > Not clear what "projects" has to do with the PEP title. Clearly it should be called ProjectionDict! cheers, Georg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 08:40 PM, Antoine Pitrou wrote: > > Hello, > > Following the python-dev discussion, I've written a PEP to recap the > proposal and the various arguments. It's inlined below, and it will > probably appear soon at http://www.python.org/dev/peps/pep-0455/, too. > Looks good to me. Georg ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/14/2013 03:40 AM, Antoine Pitrou wrote: Hello, Following the python-dev discussion, I've written a PEP to recap the proposal and the various arguments. It's inlined below, and it will probably appear soon at http://www.python.org/dev/peps/pep-0455/, too. Whenever I read a discussion about the dict, I always wonder whether the same thing applies to a set. Have you considered the utility of a TransformSet? Or is it YAGNI? Cheers, //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 08:18 PM, Steven D'Aprano wrote:
You're missing that I'm not iterating over the entire dict, just some
subset ("data") that I got from elsewhere.
Ah, okay. Between you and Antoine I am convinced that .getitem() is a good thing. So have that and .transform_key and
we're golden! :)
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 447: add type.__locallookup__
On 13 September 2013 22:23, Steven D'Aprano wrote:
> On Fri, Sep 13, 2013 at 08:42:46PM +1000, Nick Coghlan wrote:
>> Perhaps "__getdescriptor__" would work as the method name? Yes, it can
>> technically return a non-descriptor,
>
> So technically that name is, um, what's the term... oh yes, "a lie".
>
> :-)
In this case, "__getdescriptor__" means "I am looking for a
descriptor, please don't invoke the descriptor methods or traverse the
MRO, just return the raw object", not "you *must* give me a
descriptor".
The name suggestion comes from the fact that name bindings on the
instance and names bindings on the class are *different*, in that only
the latter participate in the descriptor protocol:
>>> class A:
... def m(self): pass
...
>>> A.m
>>> a = A()
>>> a.f = A.m
>>> a.m
>
>>> a.f
It's the same function object underneath, so what's going on?
The trick is that only *types* get to play the descriptor game, where
the interpreter looks for __get__, __set__ and __delete__ on the
returned object and invokes them with found. Ordinary instances don't
have that behaviour - instead, they go through type(self) to look for
descriptors, and anything they find in the instance dictionary is
returned unaltered.
This difference in how attribute lookups are handled is actually the
most fundamental difference between normal class instances and classes
themselves (which are instances of metaclasses).
>> but the *primary* purpose is to
>> customise the retrieval of objects that will be checked to see if they're
>> descriptors.
>
> If that's the case, the PEP should make that clear.
Technically, that's what "Currently object.__getattribute__ and
super.__getattribute__ peek in the __dict__ of classes on the MRO for
a class when looking for an attribute." means.
However, I agree the current wording only conveys that to the handful
of people that already know exactly when in the attribute lookup
sequence that step occurs, which is a rather niche audience :)
It's also why I like __getdescriptor__ as a name - it's based on *why*
we're doing the lookup, rather than *how* we expect it to be done.
> [Aside: the PEP states that the method shouldn't invoke descriptors.
> What's the reason for that? If I take the statement literally, doesn't
> it mean that the method mustn't use any other methods at all? Surely
> that can't be what is intended, but I'm not sure what is intended.]
It means it shouldn't invoke __get__, __set__ or __delete__ on the
returned object, since that's the responsibility of the caller.
For example, there are times when a descriptor will be retrieved to
check for the presence of a __set__ or __delete__ method, but never
actually have its methods invoked because it is shadowed in the
instance dictionary:
>>> class Shadowable:
... def __get__(self, *args):
... print("Shadowable.__get__ called!")
...
>>> class Enforced:
... def __get__(self, *args):
... print("Enforced.__get__ called!")
... def __set__(self, *args):
... print("Enforced.__set__ called!")
...
>>> class Example:
... s = Shadowable()
... e = Enforced()
... def __getattribute__(self, attr):
... print("Retrieving {} from class".format(attr))
... return super().__getattribute__(attr)
...
>>> x = Example()
>>> x.s
Retrieving s from class
Shadowable.__get__ called!
>>> x.s = 1
>>> x.s
Retrieving s from class
1
This is the key line: we retrieved 's' from the class, but *didn't*
invoke the __get__ method because it was shadowed in the instance
dictionary.
It works this way because *if* the descriptor defines __set__ or
__delete__, then Python will *ignore* the instance variable:
>>> x.e
Retrieving e from class
Enforced.__get__ called!
>>> x.__dict__["e"] = 1
Retrieving __dict__ from class
>>> x.e
Retrieving e from class
Enforced.__get__ called!
So my proposed name is based on the idea that what Ronald is after
with the PEP is a hook that *only* gets invoked when the interpreter
is doing this hunt for descriptors, but *not* for ordinary attribute
lookups.
>> It *won't* be invoked when looking for ordinary attributes in
>> an instance dict, but *will* be invoked when looking on the class object.
>
> Just to be clear, if I have:
>
> instance = MyClass()
> x = instance.name
>
> and "name" is found in instance.__dict__, then this special method will
> not be invoked. But if "name" is not found in the instance dict, then
> "name" will be looked up on the class object MyClass, which may invoke
> this special method. Am I correct?
Well, that's *my* proposal. While re-reading the current PEP, I
realised my suggested change actually goes quite a bit further than
just proposing a different name: unlike the current PEP, my advice is
that the new hook should NOT be invoked for instance attribute lookups
and should *not* replace looking directly into the class dict.
Instead, it would be the descriptor lookup counterpart to __getattr__:
whereas __getattr__ only fires for lookups on instan
Re: [Python-Dev] PEP 455: TransformDict
On 09/13/2013 09:53 PM, Joao S. O. Bueno wrote:
On 13 September 2013 22:40, Ethan Furman wrote:
On 09/13/2013 06:25 PM, MRAB wrote:
On 14/09/2013 01:49, Steven D'Aprano wrote:
Is it more common to want both the canonical key and value at the same
time, or to just want the canonical key? My gut feeling is that I'm
likely to have code like this:
d = TransformDict(...)
for key in data:
key = d.get_canonical(key)
value = d[key]
print("{}: {}".format(key, value))
I think I must be missing something. I thought that iterating over the
dict would yield the original keys, so if you wanted the original key
and value you would write:
for key, value in data.items():
print("{}: {}".format(key, value))
Well, that's certainly how I would do it. ;)
I hope you are aware that this pattern does not help when one wants
_one_ canonical key having a non-canonical one [...]
True, but I was thinking Steve was talking about printing the entire dict, in
which case that is, indeed, how I would do it.
I mean - given no function to retrieve the canonical key,
one would have to resort to:
my_key = data.__transform__(given_key)
for key, value in data.items():
if data.__transform__(key) == my_key:
Which is exactly why I, and others, would like to have the transform function easily available. Besides being able to
use it to get a canonical key, one could use it to get the function itself. Yay, introspection!
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
