Re: [Python-Dev] Add a "transformdict" to collections
Le Tue, 10 Sep 2013 21:40:36 -0500, Raymond Hettinger a écrit : > > On Sep 10, 2013, at 4:28 AM, Antoine Pitrou > wrote: > > > In http://bugs.python.org/issue18986 I proposed adding a new mapping > > type to the collections module. > > I would *really* like for this to start outside the standard library. From a quick search: - case-insensitive dicts (use cases and implementation attempts): http://twistedmatrix.com/documents/current/api/twisted.python.util.InsensitiveDict.html https://mail.python.org/pipermail/python-list/2013-May/647243.html https://mail.python.org/pipermail/python-list/2005-April/296208.html https://mail.python.org/pipermail/python-list/2004-June/241748.html http://bugs.python.org/msg197376 http://stackoverflow.com/a/2082169 http://stackoverflow.com/a/3296782 http://code.activestate.com/recipes/66315-case-insensitive-dictionary/ https://gist.github.com/babakness/3901174 http://www.wikier.org/blog/key-insensitive-dictionary-in-python http://en.sharejs.com/python/14534 http://www.voidspace.org.uk/python/archive.shtml#caseless - identity dicts: https://mail.python.org/pipermail/python-ideas/2010-May/007235.html http://www.gossamer-threads.com/lists/python/python/209527 Python's own pickle module: http://hg.python.org/cpython/file/0e70bf1f32a3/Lib/pickle.py#l234 > It needs to mature with user feedback before being dumped > in the collections module (which was never intended to be a > giant pile of every collection a person could think of). Well, thanks for the reminder, I was *indeed* going to dump all the collections I could think of in the collections module :-) (that would have been embarassing!) Seriously, I'm curious: what needs to mature, according to you? The proposed collection is a plain MutableMapping with the single addition of transforming the key on lookup. The use cases are well-defined and well-known. If you have any concrete questions or concerns then please offer them here. > Adding yet more dictionary variants is an example of > way-too-many-ways-to-do-it. So what is your proposal for what is definitely (see examples above, and this thread's and the tracker's responses) a very common need? Keep letting people write suboptimal, incomplete, buggy versions of the same thing? Thanks Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
There is a question about specifying the transform function. There are three ways to do this: 1. Positional argument of the constructor. d = TransformDict(str.casefold, Foo=5) 2. Subclassing. class CaseInsensitiveDict(TransformDict): def transform(self, key): return key.casefold() d = CaseInsensitiveDict(Foo=5) 3. Type generator. d = TransformDict(str.casefold)(Foo=5) Second method looks a little simpler to me from implementation side (What if you call __init__ for already initialized object? What if you used the object before calling __init__?). Third method allows you to customize other aspects of dict behavior (combine OrderedDict, defaultdict,..). First method looks less cumbersome from user side at first look. But how many different transform functions you use? Aren't they deserve named classes? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Le Wed, 11 Sep 2013 12:38:13 +0300, Serhiy Storchaka a écrit : > 2. Subclassing. > > class CaseInsensitiveDict(TransformDict): > def transform(self, key): > return key.casefold() > d = CaseInsensitiveDict(Foo=5) I thought about this first, and then I remembered that python-dev isn't generally very keen on subclassing-based APIs :-) > 3. Type generator. > > d = TransformDict(str.casefold)(Foo=5) > [...] > > Third method allows you to customize other aspects of dict behavior > (combine OrderedDict, defaultdict,..). Well, no, it's not that easy. Especially since OrderedDict and defaultdict weren't written with combination in mind. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Need testing audio files
I work on enhancement of audio modules testing [1], and I need free (in both senses) small sample audio files in different formats. We already have audiotest.au (mono, and sunau has a bug in processing multichannel files [2]) and Sine-1000Hz-300ms.aif, but this is not enough. I have generated a pack of files like Sine-1000Hz-300ms.aif by Python, it is enough for regression testing but only if current implementation is correct. I found some collections of sample files at [3], but I'm not sure about copyright, and perhaps they are a little too big. In ideal it should be one high-quality (float64?) multichannel (5+1?) but short master file and it's lower-quality copies made by third-party tools. In ideal the content should be related to Python. [1] http://bugs.python.org/issue18919 [2] http://bugs.python.org/issue18950 [3] http://www-mmsp.ece.mcgill.ca/documents/AudioFormats/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
11.09.13 12:52, Antoine Pitrou написав(ла): Le Wed, 11 Sep 2013 12:38:13 +0300, Serhiy Storchaka a écrit : 2. Subclassing. class CaseInsensitiveDict(TransformDict): def transform(self, key): return key.casefold() d = CaseInsensitiveDict(Foo=5) I thought about this first, and then I remembered that python-dev isn't generally very keen on subclassing-based APIs :-) Why? This way looks most natural to me. 3. Type generator. d = TransformDict(str.casefold)(Foo=5) [...] Third method allows you to customize other aspects of dict behavior (combine OrderedDict, defaultdict,..). Well, no, it's not that easy. Especially since OrderedDict and defaultdict weren't written with combination in mind. They can be rewritten. Actually the defaultdict is just simple wrapper around existing functionality of the __missing__ method. We can add the __transform__ method directly in the dict class. I think it will significant (2-3x) decrease a size of needed C code (no need in defining new type with constructors/destructors/etc, only lookup_transform). ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Le Wed, 11 Sep 2013 13:37:53 +0300, Serhiy Storchaka a écrit : > > Actually the defaultdict is just simple wrapper around existing > functionality of the __missing__ method. We can add the __transform__ > method directly in the dict class. I think it will significant (2-3x) > decrease a size of needed C code (no need in defining new type with > constructors/destructors/etc, only lookup_transform). Right now I am not proposing any C implementation of TransformDict. That could be added, but it's not what I'm pushing for here :-) But I don't think the "type generator" API would be easier to implement in C, anyway. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
> Seriously, I'm curious: what needs to mature, according to you? In my mind, its availability on PyPI along with demonstrated use in the wild (plus corresponding votes to demonstrate that people use/like it) would help. That you can find several implementations at this doesn't mean it's necessarily worth adding to the std lib. Once in, it is very difficult to evict something that is later deemed not to have belonged in the std lib, so I think some extra scrutiny is worthwhile. Is there some obvious advantage to having a single API for this available to all Python applications? Are the semantics well-defined (do all the implementations you cited offer basically the same semantics)? The discussion so far here suggest that the best semantics might not be completely settled. (I still don't care for the name. "Transform" != "case folding" in my mind. A quick scan of your links suggests most people think something like "cidict" or "CaseInsensitiveDict" would be more descriptive.) Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
11.09.13 14:07, Antoine Pitrou написав(ла): But I don't think the "type generator" API would be easier to implement in C, anyway. No, I mean subclassing approach. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On Wed, 11 Sep 2013 06:08:25 -0500, Skip Montanaro wrote: > > Seriously, I'm curious: what needs to mature, according to you? > > In my mind, its availability on PyPI along with demonstrated use in > the wild (plus corresponding votes to demonstrate that people use/like > it) would help. That you can find several implementations at this > doesn't mean it's necessarily worth adding to the std lib. Once in, it > is very difficult to evict something that is later deemed not to have > belonged in the std lib, so I think some extra scrutiny is worthwhile. The problem is that it is hard to get traction on pypi for a "small" feature like this. Most people will just implement a special purpose version that solves their specific need (possibly in a somewhat broken fashion) rather than add yet another dependency. Our approach in cases like this has (I think) been to learn from the existing implementations and build our own based on that (isn't that what happened with OrderedDict?) > Is there some obvious advantage to having a single API for this > available to all Python applications? Are the semantics well-defined > (do all the implementations you cited offer basically the same > semantics)? The discussion so far here suggest that the best semantics > might not be completely settled. I think the only question was what happens to the key on assignment, and that has been settled. Well, I guess there's also a question of how you create one, but that's not a question a pypi version would be any help in resolving: we'd still bikeshed it upon addition to the stdlib. In fact, I would not be surprised if *all* of the bikeshedding we are doing now (except this bit :) would take place if an existing pypi module for this were proposed for addition. I think the bigger question is composition of different dict types, and that is again something that isn't going to be solved by a pypi module. > (I still don't care for the name. "Transform" != "case folding" in my > mind. A quick scan of your links suggests most people think something > like "cidict" or "CaseInsensitiveDict" would be more descriptive.) Except it is wider than that: the transform function can be anything, not just case folding. I suggested surjectiondict or ontodict, but Antoine didn't like those :) (I had to look up the terms...it's been a long time since I studied math.) --David ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
Le Wed, 11 Sep 2013 13:09:36 +0300, Serhiy Storchaka a écrit : > I work on enhancement of audio modules testing [1], and I need free > (in both senses) small sample audio files in different formats. We > already have audiotest.au (mono, and sunau has a bug in processing > multichannel files [2]) and Sine-1000Hz-300ms.aif, but this is not > enough. I have generated a pack of files like Sine-1000Hz-300ms.aif > by Python, it is enough for regression testing but only if current > implementation is correct. If you want to edit, shorten, convert sounds between different formats, you can try Audacity, a free sound editor: http://audacity.sourceforge.net/ Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
Use your microphone, say "python" and save the file in your favorite file format. Try for example Audacity. I suppose that you don't need specific audio content and you don't need a huge file. Victor 2013/9/11 Serhiy Storchaka : > I work on enhancement of audio modules testing [1], and I need free (in both > senses) small sample audio files in different formats. We already have > audiotest.au (mono, and sunau has a bug in processing multichannel files > [2]) and Sine-1000Hz-300ms.aif, but this is not enough. I have generated a > pack of files like Sine-1000Hz-300ms.aif by Python, it is enough for > regression testing but only if current implementation is correct. > > I found some collections of sample files at [3], but I'm not sure about > copyright, and perhaps they are a little too big. > > In ideal it should be one high-quality (float64?) multichannel (5+1?) but > short master file and it's lower-quality copies made by third-party tools. > In ideal the content should be related to Python. > > [1] http://bugs.python.org/issue18919 > [2] http://bugs.python.org/issue18950 > [3] http://www-mmsp.ece.mcgill.ca/documents/AudioFormats/ > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 11 September 2013 21:57, R. David Murray wrote: > Except it is wider than that: the transform function can be anything, > not just case folding. > > I suggested surjectiondict or ontodict, but Antoine didn't like those :) > (I had to look up the terms...it's been a long time since I studied > math.) I'll join the chorus requesting that this live on PyPI for a while first. I think this is a case similar to what happened with contextlib.ExitStack: I'm not sure if anyone actually *used* contextlib2 for anything significant (if they did, they didn't tell me about it), but just going through the process of properly documenting, publishing and testing it as a relatively independent project forced *me* to think through the design. Add in a couple of interesting conversation with Brandon Rhodes and others about the API design, and the end result was a *vast* improvement over what originally went up on PyPI. The barriers to making use of PyPI libraries are also falling with time, so a solid implementation may still see adoption, even if it's in the form of copy-and-paste programming rather than actual dependencies. I think there are also additional API decisions to be made beyond just the one about how to declare the mapping function, related to how to get the mapped key values *out*, as well as how to find out whether or not two potential key values map to the same actual key. As my preferred bikeshed colour, I'm going to suggest "MappedKeyDict" (using a similar naming style to OrderedDict), since the purpose of the container is to map the domain of the supplied keys to a different range before doing the value lookup. Suggested additional methods: md.map_key(key) # Applies the mapping function to the supplied key md.mapped_keys() # Like keys(), but with the key mapping function applied md.mapped_items() # Like items(), but with the key mapping function applied Another (more dubious) possible method: md.same_key(key1, key2) # "md.map_key(key1) == md.map_key(key2)" Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Le Wed, 11 Sep 2013 22:47:12 +1000, Nick Coghlan a écrit : > > I'll join the chorus requesting that this live on PyPI for a while > first. > > I think this is a case similar to what happened with > contextlib.ExitStack: I'm not sure if anyone actually *used* > contextlib2 for anything significant (if they did, they didn't tell me > about it), but just going through the process of properly documenting, > publishing and testing it as a relatively independent project forced > *me* to think through the design. ExitStack was quite a new thing, API-wise. The proposal here is to generalize something which already exists in various forms all over the Internet, and respecting a well-known API, MutableMapping. There are not many possible APIs to create case-insensitive dicts, or identity dicts. The only API contention until now has been about whether the first or last key would be retained (which settled on the former), and Serhiy's three instantiation schemes. The additional methods you suggest may be nice to have, but they are not necessary and (deliberately) not part of the original proposal. That is, they can be grafted later on. > As my preferred bikeshed colour, I'm going to suggest "MappedKeyDict" > (using a similar naming style to OrderedDict), Ha, another colour! Thanks :-) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On Wed, Sep 11, 2013 at 06:08:25AM -0500, Skip Montanaro wrote:
> (I still don't care for the name. "Transform" != "case folding" in my
> mind. A quick scan of your links suggests most people think something
> like "cidict" or "CaseInsensitiveDict" would be more descriptive.)
But the proposal is not for a case-insensitive dict. It is more general
than that, with case-insensitivity just one specific use-case for such
a transformative dict. Arguably the most natural, or at least obvious,
such transformation, but there are others.
I have code that does something like this:
MAPPING = {'spam': 23, 'ham': 42, 'eggs': 17}
result = MAPPING[key.strip()]
# later...
answer = MAPPING[key] # Oops, forgot to strip! This is broken.
Using Antoine's proposal:
MAPPING = TransformDict(str.strip)
MAPPING.update({'spam': 23, 'ham': 42, 'eggs': 17})
result = MAPPING[key]
# later...
answer = MAPPING[key] # Fine now.
so that the mapping handles stripping the keys, not the caller.
This isn't just about strings, and certainly not just
case-insensitivity.
--
Steven
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Antoine Pitrou writes: > ExitStack was quite a new thing, API-wise. The proposal here is to > generalize something which already exists in various forms all over the > Internet, and respecting a well-known API, MutableMapping. What Nick said was "I was too close to the design, and time and a very few good comments greatly improved the product." That's worth considering as generic advice rather than for the specifics of the case he faced compared to yours. Which modules in the stdlib would benefit from rewriting using "transformdict"? How about on PyPI? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
On Wed, Sep 11, 2013 at 8:46 AM, Victor Stinner wrote: > Use your microphone, say "python" and save the file in your favorite > file format. Try for example Audacity. I suppose that you don't need > specific audio content and you don't need a huge file. > And if you don't want your voice in the test suite forever you can see if you can convince Guido to say "Python" into a microphone. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On Wed, Sep 11, 2013 at 10:47:12PM +1000, Nick Coghlan wrote: > I'll join the chorus requesting that this live on PyPI for a while first. Another alternative would be ActiveState's Python recipes: http://code.activestate.com/recipes/langs/python which is a lot lighter weight than creating a package on PyPI. I believe that ChainMap, namedtuple, groupby, lru_cache and even math.fsum started life on ActiveState. -- Steven ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
2013/9/11 Steven D'Aprano :
> But the proposal is not for a case-insensitive dict. It is more general
> than that, with case-insensitivity just one specific use-case for such
> a transformative dict. Arguably the most natural, or at least obvious,
> such transformation, but there are others.
>
> I have code that does something like this:
>
> MAPPING = {'spam': 23, 'ham': 42, 'eggs': 17}
> result = MAPPING[key.strip()]
> # later...
> answer = MAPPING[key] # Oops, forgot to strip! This is broken.
For this use case, you should not keep the key unchanged, but it's
better to store the stripped key (so MAPPING.keys() gives you the
expected result). The transformdict type proposed by Antoine cannot be
used for this use case.
The os.environ mapping uses a subprocess of MutableMapping which
accepts 4 functions: encoder/decoder for the key and encoder/decoder
for the value. Such type is even more generic. transformdict cannot
replace os._Environ.
(Or do you prefer to store the ' eggs' key?)
Victor
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
On Wed, Sep 11, 2013 at 09:54:44AM -0400, Brett Cannon wrote: > On Wed, Sep 11, 2013 at 8:46 AM, Victor Stinner > wrote: > And if you don't want your voice in the test suite forever you can see if > you can convince Guido to say "Python" into a microphone. Wouldn't his name be enough? http://www.python.org/~guido/guido.au Oleg. -- Oleg Broytmanhttp://phdru.name/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
On Sep 11, 2013, at 01:09 PM, Serhiy Storchaka wrote: >In ideal it should be one high-quality (float64?) multichannel (5+1?) but >short master file and it's lower-quality copies made by third-party tools. In >ideal the content should be related to Python. I have some pro-audio recording capabilities and would be happy to generate some copyright-donated clips for Python. Please contact me off-list if needed. -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] importance of dir
http://docs.python.org/3/library/functions.html#dir: Note: Because dir() is supplied primarily as a convenience for use at an interactive prompt [...] I suspect this comment is out of date, as there are two functions in the inspect module that rely on dir(), which also means that help indirectly relies on dir(). And we also now have the __dir__ method that we can use to customize what gets returned. [...] more than it tries to supply a rigorously or consistently defined set of names, [...] If this is still true, should inspect be relying on dir()? -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Le Wed, 11 Sep 2013 22:50:08 +0900, "Stephen J. Turnbull" a écrit : > Antoine Pitrou writes: > > > ExitStack was quite a new thing, API-wise. The proposal here is to > > generalize something which already exists in various forms all > > over the Internet, and respecting a well-known API, MutableMapping. > > What Nick said was "I was too close to the design, and time and a very > few good comments greatly improved the product." That's worth > considering as generic advice rather than for the specifics of the > case he faced compared to yours. "Time and comments" is why I'm posting a discussion thread here and waiting for responses. There are currently 34517 packages on PyPI. A new 34518th one won't magically get a lot of attention :-) (for example, I got very few comments when posting pathlib on PyPI - despite making several actual releases there -, compared to what I got from the PEP 428 discussion on python-ideas) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 05:47 AM, Nick Coghlan wrote: On 11 September 2013 21:57, R. David Murray wrote: Except it is wider than that: the transform function can be anything, not just case folding. I suggested surjectiondict or ontodict, but Antoine didn't like those :) (I had to look up the terms...it's been a long time since I studied math.) I'll join the chorus requesting that this live on PyPI for a while first. Personally, I would not add a PyPI dependency for a single object like transformdict. If we are that nervous about it we can make it provisional, but I don't see that as necessary. I think this is a case similar to what happened with contextlib.ExitStack: I'm not sure if anyone actually *used* contextlib2 for anything significant (if they did, they didn't tell me about it), but just going through the process of properly documenting, publishing and testing it as a relatively independent project forced *me* to think through the design. I had the opposite experience. Going through PyDev to get Enum was *far* more educational, informative, and useful than creating an independent package. Plus, transformdict is not that new or different from existing dicts, and most decisions (which key to keep? how to initialize?) can be answered by simply being consistent with what we already have. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 02:38 AM, Serhiy Storchaka wrote:
There is a question about specifying the transform function.
There are three ways to do this:
1. Positional argument of the constructor.
d = TransformDict(str.casefold, Foo=5)
This method follows the precedent of defaultdict:
--> from collections import defaultdict
--> l = defaultdict(list, foo=['ham','eggs'])
--> l
defaultdict(, {'foo': ['ham', 'eggs']})
Without a compelling reason to change, we should keep it consistent.
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] importance of dir
>> Note: Because dir() is supplied primarily as a convenience for >> use at an interactive prompt [...] This was always my interpretation of its intent. In fact, I use a customized dir() for my own needs which would probably break inspect (elides _-prefixed functions by default, notes modules or packages within other packages which haven't been imported yet). I never realized that module used it. Skip ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 06:58 AM, Victor Stinner wrote:
2013/9/11 Steven D'Aprano :
But the proposal is not for a case-insensitive dict. It is more general
than that, with case-insensitivity just one specific use-case for such
a transformative dict. Arguably the most natural, or at least obvious,
such transformation, but there are others.
I have code that does something like this:
MAPPING = {'spam': 23, 'ham': 42, 'eggs': 17}
result = MAPPING[key.strip()]
# later...
answer = MAPPING[key] # Oops, forgot to strip! This is broken.
For this use case, you should not keep the key unchanged, but it's
better to store the stripped key (so MAPPING.keys() gives you the
expected result).
He isn't keeping the key unchanged (notice no white space in MAPPING), he's merely providing a function that will
automatically strip the whitespace from key lookups.
The transformdict type proposed by Antoine cannot be
used for this use case.
Yes, it can.
--> from collections import transformdict
--> MAPPING = transformdict(str.strip)
--> MAPPING.update({'spam': 23, 'ham': 42, 'eggs': 17})
--> MAPPING
transformdict(, {'ham': 42, 'spam': 23,
'eggs': 17})
--> MAPPING[' eggs ']
17
--
~Ethan~
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 06:58 AM, Victor Stinner wrote: The os.environ mapping uses a subclass of MutableMapping which accepts 4 functions: encoder/decoder for the key and encoder/decoder for the value. Such type is even more generic. transformdict cannot replace os._Environ. True, it's more generic -- but is it used more often? Or often enough to have those four functions be part of transformdict instead of just the one encodekey function? (I mean the idea of os._Environ, not that specific implementation.) Personally, I wouldn't mind having all four; for one thing, the name 'transformdict' would then be entirely appropriate. ;) -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 08:49 AM, Victor Stinner wrote: 2013/9/11 Ethan Furman : He isn't keeping the key unchanged (notice no white space in MAPPING), he's merely providing a function that will automatically strip the whitespace from key lookups. transformdict keeps the key unchanged, see the first message: >>> d = transformdict(str.lower) >>> d['Foo'] = 5 >>> d['foo'] 5 >>> d['FOO'] 5 >>> list(d) ['Foo'] 'Foo' is stored as 'Foo', not as 'foo'. So for stripped keys: d=transformdict(str.strip); d[' abc ']; print(list(d)) should print "[' abc ']", not "['abc']". And indeed it does: Python 3.4.0a1+ (default:833246d42825+, Aug 31 2013, 14:17:59) [GCC 4.7.3] on linux Type "help", "copyright", "credits" or "license" for more information. --> from collections import transformdict --> d=transformdict(str.strip); d[' abc '] = 42; print(list(d)) [' abc '] Is it the expected result? Yup! :) -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Le Wed, 11 Sep 2013 07:48:56 -0700, Ethan Furman a écrit : > On 09/11/2013 06:58 AM, Victor Stinner wrote: > > > > The os.environ mapping uses a subclass of MutableMapping which > > accepts 4 functions: encoder/decoder for the key and encoder/decoder > > for the value. Such type is even more generic. transformdict cannot > > replace os._Environ. > > True, it's more generic -- but is it used more often? Or often > enough to have those four functions be part of transformdict instead > of just the one encodekey function? (I mean the idea of os._Environ, > not that specific implementation.) > > Personally, I wouldn't mind having all four; for one thing, the name > 'transformdict' would then be entirely appropriate. ;) The key decoder function is quite useless since the original key is retained. As for the value encoder/decoder, I don't really see the point: just store whichever value you want to retrieve later. The main point of transformdict is that the *lookup* is done using a different key than the user-provided key. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 08:48 AM, Antoine Pitrou wrote: Le Wed, 11 Sep 2013 07:48:56 -0700, Ethan Furman a écrit : Personally, I wouldn't mind having all four; for one thing, the name 'transformdict' would then be entirely appropriate. ;) The key decoder function is quite useless since the original key is retained. As for the value encoder/decoder, I don't really see the point: just store whichever value you want to retrieve later. The main point of transformdict is that the *lookup* is done using a different key than the user-provided key. Very good points. I am convinced. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
2013/9/11 Ethan Furman : > He isn't keeping the key unchanged (notice no white space in MAPPING), he's > merely providing a function that will automatically strip the whitespace > from key lookups. transformdict keeps the key unchanged, see the first message: >>> d = transformdict(str.lower) >>> d['Foo'] = 5 >>> d['foo'] 5 >>> d['FOO'] 5 >>> list(d) ['Foo'] 'Foo' is stored as 'Foo', not as 'foo'. So for stripped keys: d=transformdict(str.strip); d[' abc ']; print(list(d)) should print "[' abc ']", not "['abc']". Is it the expected result? Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
11.09.13 16:50, Stephen J. Turnbull написав(ла): Which modules in the stdlib would benefit from rewriting using "transformdict"? How about on PyPI? At least _threading_local, cProfile, doctest, json, and perhaps future implementations of __sizeof__ for some classes would benefit from rewriting using IdentityDict. Unfortunately copy and pickle expose their replacements of IdentityDict to public and can't change them. I don't known anything about general TransformDict use cases. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
11.09.13 15:45, Antoine Pitrou написав(ла): If you want to edit, shorten, convert sounds between different formats, you can try Audacity, a free sound editor: http://audacity.sourceforge.net/ Yes, Audacity is great. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
11.09.13 15:46, Victor Stinner написав(ла): Use your microphone, say "python" and save the file in your favorite file format. Try for example Audacity. I suppose that you don't need specific audio content and you don't need a huge file. My voice is even more ugly than my English. I don't want perpetuate it in Python test suite. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
11.09.13 17:10, Oleg Broytman написав(ла): Wouldn't his name be enough? http://www.python.org/~guido/guido.au It is Lib/test/audiotest.au. 1-channel and 8-bit. No, it wouldn't enough. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
Am 11.09.13 15:04, schrieb Antoine Pitrou: > There are not many possible APIs to create case-insensitive dicts, or > identity dicts. That is certainly not true. Most obviously, you have the choice of a specialized case-mapping dict, or a generalized type that can be used for case mapping also. Does any of the referenced use cases actually use the API that you are proposing (i.e. with a key transformation function)? FWIW, I can think of yet another API for this: write a Proxy class class TransformedKey: # untested def __init__(self, original): self.original = original self.transformed = self.transform(original) def __hash__(self): return hash(self.transformed) def __eq__(self, other): return self.transformed == other.transformed def transform(self, key): raise NotImplementedError If there is then disciplined use in the form d[TransformedKey(key)] == value then you can use the existing dictionary type. Notice that this can do both case-insensitive and identity dicts (plus there are multiple choices of getting the transform function into it, as well as variations of the __eq__ implementation). There is evidence in this thread that people "grasp" case-insensitive more easily than the generalized API. For the record, I initially typed two responses into the tracker which I found to be incorrect before posting them, so I ended up posting neither. The "transformdict" type is not at all "natural", even though it may be useful. If you really want to "push" this API into 3.4, I think you will need to write a PEP, and find a PEP dictator who is willing to approve it. As you seem to dislike the idea of writing a PEP, I suggest to follow the idea of publishing it on PyPI now, and then proposing it for inclusion into 3.5. Regards, Martin ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On Wed, 11 Sep 2013 19:31:56 +0200 "Martin v. Löwis" wrote: > Am 11.09.13 15:04, schrieb Antoine Pitrou: > > There are not many possible APIs to create case-insensitive dicts, or > > identity dicts. > > That is certainly not true. Most obviously, you have the choice of a > specialized case-mapping dict, or a generalized type that can be used > for case mapping also. Does any of the referenced use cases actually > use the API that you are proposing (i.e. with a key transformation > function)? Well, when you have a specialized need, you don't implement a generic version (*), so I don't think anybody implemented it ;-). The point of a generic API is to help replace all specialized implementations at once, rather than a small subset of them. (*) even the Twisted people don't seem to go that far > FWIW, I can think of yet another API for this: write a Proxy class > > class TransformedKey: # untested > def __init__(self, original): > self.original = original > self.transformed = self.transform(original) > def __hash__(self): > return hash(self.transformed) > def __eq__(self, other): > return self.transformed == other.transformed > def transform(self, key): > raise NotImplementedError > > If there is then disciplined use in the form > > d[TransformedKey(key)] == value The problem is "disciplined use" here. This means any consumer of your API has to know about that quirk, so it's an inferior solution. (TransformedKey could be an internal detail of the implementation, as with weak dictionaries; but as a public API it doesn't seem very desirable) > The "transformdict" type > is not at all "natural", even though it may be useful. By that token, nothing is "natural" ;-) defaultdict isn't natural, yet it went in without a PEP. However, to all persons who already had that need and responded to the proposal, the generic version *does* seem natural enough, AFAICT. Only people who've never had such a need seem to fail to "grasp" it (and I only counted one such person, but my memory may be failing me). You know, I'm not against having "only" a case-insensitive dict. But that would be less useful all the while not being significantly easier to use, so I'm not really seeing the point. > If you really want to "push" this API into 3.4, I think you will need > to write a PEP, and find a PEP dictator who is willing to approve it. > As you seem to dislike the idea of writing a PEP, I suggest to follow > the idea of publishing it on PyPI now, and then proposing it for > inclusion into 3.5. What I dislike is the idea of doing additional work because some barriers are imposed ;-). PEP or PyPI are on a similar scale here. At least a PEP would help record the various discussion details, so I'd favour that over the PyPI path. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 10:49 AM, Antoine Pitrou wrote: What I dislike is the idea of doing additional work because some barriers are imposed ;-). PEP or PyPI are on a similar scale here. At least a PEP would help record the various discussion details, so I'd favour that over the PyPI path. I would think a good summary on the bug tracker would preserve the detail just fine. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
On 9/11/2013 10:10 AM, Barry Warsaw wrote: On Sep 11, 2013, at 01:09 PM, Serhiy Storchaka wrote: In ideal it should be one high-quality (float64?) multichannel (5+1?) but short master file and it's lower-quality copies made by third-party tools. In ideal the content should be related to Python. How about "Python is a great programming language." Long enough? I have some pro-audio recording capabilities and would be happy to generate some copyright-donated clips for Python. Please contact me off-list if needed. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
11.09.13 17:10, Barry Warsaw написав(ла): I have some pro-audio recording capabilities and would be happy to generate some copyright-donated clips for Python. Please contact me off-list if needed. Thank you. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 12 September 2013 02:03, Ethan Furman wrote: > On 09/11/2013 08:49 AM, Victor Stinner wrote: > >> 2013/9/11 Ethan Furman : >> >>> He isn't keeping the key unchanged (notice no white space in MAPPING), >>> he's >>> merely providing a function that will automatically strip the whitespace >>> from key lookups. >>> >> >> transformdict keeps the key unchanged, see the first message: >> >> >>> d = transformdict(str.lower) >> >>> d['Foo'] = 5 >> >>> d['foo'] >> 5 >> >>> d['FOO'] >> 5 >> >>> list(d) >> ['Foo'] >> > That seems backwards to me. I would think that retrieving the keys from the dict would return the transformed keys (I'd call them canonical keys). That way there's no question about which key is stored - it's *always* the transformed key. In fact, I think this might get more traction if it were referred to as a canonicalising dictionary (bikeshedding, I know). Tim Delaney ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Add a "transformdict" to collections
On 09/11/2013 02:39 PM, Tim Delaney wrote: On 12 September 2013 02:03, Ethan Furman mailto:[email protected]>> wrote: On 09/11/2013 08:49 AM, Victor Stinner wrote: 2013/9/11 Ethan Furman mailto:[email protected]>>: He isn't keeping the key unchanged (notice no white space in MAPPING), he's merely providing a function that will automatically strip the whitespace from key lookups. transformdict keeps the key unchanged, see the first message: >>> d = transformdict(str.lower) >>> d['Foo'] = 5 >>> d['foo'] 5 >>> d['FOO'] 5 >>> list(d) ['Foo'] That seems backwards to me. I would think that retrieving the keys from the dict would return the transformed keys (I'd call them canonical keys). That way there's no question about which key is stored - it's *always* the transformed key. At this point there is still no question: it's the first version of the key seen. For a stupid example: --> d = transformdict(str.lower) --> d['ThePyramid'] = 'Game Show' --> d['AtOnce'] = now() --> for k, v in d.items(): ... print(k, v) Imagine writing a function to get that capitalization right. In fact, I think this might get more traction if it were referred to as a canonicalising dictionary (bikeshedding, I know). Whoa, that's way harder to spell! ;) Drop the 'ising', though, and I'm in. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Need testing audio files
Serhiy Storchaka writes: > I work on enhancement of audio modules testing [1], and I need free > (in both senses) small sample audio files in different formats. The Internet Archive https://archive.org/> has a wide selection of free-software media, many of which have free license terms that would be suitable for inclusion in Python. I haven't used it, but http://www.pdsounds.org/> appears to be a collection of sounds in the public domain. -- \“The whole area of [treating source code as intellectual | `\property] is almost assuring a customer that you are not going | _o__) to do any innovation in the future.” —Gary Barnett | Ben Finney ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (merge 3.3 -> default): merge from 3.3
On Thu, 12 Sep 2013 07:57:25 +0200 (CEST) senthil.kumaran wrote: > > +<<< local > Optional argument random is a 0-argument function returning a > random float in [0.0, 1.0); if it is the default None, the > standard random.random will be used. > +=== > +Optional arg random is a 0-argument function returning a random > +float in [0.0, 1.0); by default, the standard random.random. > + > +Do not supply the 'int' argument. > +>>> other Can you fix this? Thanks Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
