Re: [Python-Dev] Bilingual scripts
On Fri, 2013-05-24 at 15:56 -0400, Barry Warsaw wrote: > Here's something that seems to come up from time to time in Debian. > > Take a Python application like tox, nose, or pyflakes. Their executables work > with both Python 2 and 3, but require a #! line to choose which interpreter to > invoke. You probably already know this, but I'll mention it anyway. This probably matters a lot for nose and pyflakes, but I'd say that for tox it should not, it basically just scripts execution of shell commands. I'd think maybe in cases like tox (and others that are compatible with both Python 2 and 3) the hashbang should just be set to "#!/usr/bin/python" unconditionally. Maybe we could also think about modifying pyflakes so that it can validate both 2 and 3 code (choosing one or the other based on a header line in the validated files and defaulting to the version of Python being run). This is kind of the right thing anyway. Nose is a bit of a special case. I personally never run nosetests directly, I always use setup.py nosetests, which makes it not matter. In general, I'd like to think that scripts that get installed to global bindirs will execute utilities that are useful independent of the version of Python being used to execute them. - C ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Fri, 24 May 2013 15:56:29 -0400 Barry Warsaw wrote: > Here's something that seems to come up from time to time in Debian. > > Take a Python application like tox, nose, or pyflakes. Their executables work > with both Python 2 and 3, but require a #! line to choose which interpreter to > invoke. > > When we add Python 3 support in Debian for such a script, all the library > installations are handled just fine, but we have conflicts about what to name > the thing that lands in /usr/bin. Do we have /usr/bin/pyflakes and > /usr/bin/pyflakes3? Do we call the latter py3flakes (as has been convention > with some other scripts, but which breaks locate(1))? Do we simply remove the > /usr/bin scripts and encourage people to use something like `$python -m nose`? How about always running the version specific targets, e.g. nosetests-2.7? Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Sat, May 25, 2013 at 5:56 AM, Barry Warsaw wrote: > Have any other *nix distros addressed this, and if so, how do you solve it? I believe Fedora follows the lead set by our own makefile and just appends a "3" to the script name when there is also a Python 2 equivalent (thus ``pydoc3`` and ``pyvenv``). (I don't have any other system provided Python 3 scripts on this machine, though) > It would be nice if we could have some cross-platform recommendations so > things work the same wherever you go. To that end, if we can reach some > consensus, I'd be willing to put together an informational PEP and some > scripts that might be of general use. It seems to me the existing recommendation to use ``#!/usr/bin/env python`` instead of referencing a particular binary already covers the general case. The challenge for the distros is that we want a solution that *ignores* user level virtual environments. I think the simplest thing to do is just append the "3" to the binary name (as we do ourselves for pydoc) and then abide by the recommendations in PEP 394 to reference the correct system executable. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Sat, 2013-05-25 at 17:57 +1000, Nick Coghlan wrote: > I think the simplest thing to do is just append the "3" to the binary > name (as we do ourselves for pydoc) and then abide by the > recommendations in PEP 394 to reference the correct system executable. I'm curious if folks have other concrete examples of global bindir executables other than nosetests and pydoc that need to be disambiguated by Python version. I'd hate to see it become standard practice to append "3" to scripts generated by packages which happen to use Python 3, as it will just sort of perpetuate its otherness. - C ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bilingual scripts
On Sat, May 25, 2013 at 8:17 PM, Chris McDonough wrote: > On Sat, 2013-05-25 at 17:57 +1000, Nick Coghlan wrote: >> I think the simplest thing to do is just append the "3" to the binary >> name (as we do ourselves for pydoc) and then abide by the >> recommendations in PEP 394 to reference the correct system executable. > > I'm curious if folks have other concrete examples of global bindir > executables other than nosetests and pydoc that need to be disambiguated > by Python version. Single source Python 2/3 packages don't have the problem. They can either run with the system Python explicitly, or use "/usr/bin/env python" in order to respect virtual environments. The issue only exists for projects where Python 2 and Python 3 are separate code bases with distinct scripts to be installed on the target system. In my opinion, is just one more reason why single source is a vastly superior alternative to treating the Python 2 and Python 3 versions as separate applications. > I'd hate to see it become standard practice to > append "3" to scripts generated by packages which happen to use Python > 3, as it will just sort of perpetuate its otherness. Fedora only does it for stuff that has to straddle the two with a parallel install as a system binary. If something is exclusive to Py3 (like "pyvenv") it doesn't get the suffix. It's certainly not an elegant solution, but Python is far from the only runtime platform afflicted by similar issues when it comes to balancing the interests of distro developers wanting to build a single integrated system against those of application developers wanting to use more recent versions of their dependencies. Longer term, we (Red Hat) want to better support application stacks that are more independent of the system versions, while still being fully under the control of the system package manager (to ease security updates). The mechanism underlying that is Software Collections, which is better described here: http://developerblog.redhat.com/2013/01/28/software-collections-on-red-hat-enterprise-linux/ (The software collection system also exists in Fedora, as that's where it was developed, but the main benefit there is to allow application developers to use something *older* than what Fedora provides in the system packages if there are backwards compatibility issues with an update) I have no idea if Debian are contemplating anything similar, but the current situation where application developers have to choose between letting distro vendors control the upgrade cycle of key dependencies and divorcing themselves entirely from the benefits offered by the system package manager is unsustainable in the long term. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
Hello,
Since the initial version, several minor changes have been made to the
PEP. The history is visible on hg.python.org. The most important
change in this version is that I introduced ABC support and completed
a reference implementation.
No open issues remain from my point of view.
PEP: 443
Title: Single-dispatch generic functions
Version: $Revision$
Last-Modified: $Date$
Author: Łukasz Langa
Discussions-To: Python-Dev
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 22-May-2013
Post-History: 22-May-2013, 25-May-2013
Replaces: 245, 246, 3124
Abstract
This PEP proposes a new mechanism in the ``functools`` standard library
module that provides a simple form of generic programming known as
single-dispatch generic functions.
A **generic function** is composed of multiple functions sharing the
same name. Which form should be used during a call is determined by the
dispatch algorithm. When the implementation is chosen based on the type
of a single argument, this is known as **single dispatch**.
Rationale and Goals
===
Python has always provided a variety of built-in and standard-library
generic functions, such as ``len()``, ``iter()``, ``pprint.pprint()``,
``copy.copy()``, and most of the functions in the ``operator`` module.
However, it currently:
1. does not have a simple or straightforward way for developers to
create new generic functions,
2. does not have a standard way for methods to be added to existing
generic functions (i.e., some are added using registration
functions, others require defining ``__special__`` methods, possibly
by monkeypatching).
In addition, it is currently a common anti-pattern for Python code to
inspect the types of received arguments, in order to decide what to do
with the objects. For example, code may wish to accept either an object
of some type, or a sequence of objects of that type.
Currently, the "obvious way" to do this is by type inspection, but this
is brittle and closed to extension. Abstract Base Classes make it easier
to discover present behaviour, but don't help adding new behaviour.
A developer using an already-written library may be unable to change how
their objects are treated by such code, especially if the objects they
are using were created by a third party.
Therefore, this PEP proposes a uniform API to address dynamic
overloading using decorators.
User API
To define a generic function, decorate it with the ``@singledispatch``
decorator. Note that the dispatch happens on the type of the first
argument, create your function accordingly::
>>> from functools import singledispatch
>>> @singledispatch
... def fun(arg, verbose=False):
... if verbose:
... print("Let me just say,", end=" ")
... print(arg)
To add overloaded implementations to the function, use the
``register()`` attribute of the generic function. It takes a type
parameter::
>>> @fun.register(int)
... def _(arg, verbose=False):
... if verbose:
... print("Strength in numbers, eh?", end=" ")
... print(arg)
...
>>> @fun.register(list)
... def _(arg, verbose=False):
... if verbose:
... print("Enumerate this:")
... for i, elem in enumerate(arg):
... print(i, elem)
To enable registering lambdas and pre-existing functions, the
``register()`` attribute can be used in a functional form::
>>> def nothing(arg, verbose=False):
... print("Nothing.")
...
>>> fun.register(type(None), nothing)
The ``register()`` attribute returns the undecorated function which
enables decorator stacking, pickling, as well as creating unit tests for
each variant independently::
>>> @fun.register(float)
... @fun.register(Decimal)
... def fun_num(arg, verbose=False):
... if verbose:
... print("Half of your number:", end=" ")
... print(arg / 2)
...
>>> fun_num is fun
False
When called, the generic function dispatches on the first argument::
>>> fun("Hello, world.")
Hello, world.
>>> fun("test.", verbose=True)
Let me just say, test.
>>> fun(42, verbose=True)
Strength in numbers, eh? 42
>>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
Enumerate this:
0 spam
1 spam
2 eggs
3 spam
>>> fun(None)
Nothing.
>>> fun(1.23)
0.615
To get the implementation for a specific type, use the ``dispatch()``
attribute::
>>> fun.dispatch(float)
>>> fun.dispatch(dict)
The proposed API is intentionally limited and opinionated, as to ensure
it is easy to explain and use, as well as to maintain consistency with
existing members in the ``functools`` module.
Implementation Notes
The functionality described in this PEP is already implemented in the
``pkgutil`` standard library module as ``simplegeneric``. Because this
implementation is mature, the goal is to move it largely as-is. The
reference implementation is available on hg.python.org [#ref-impl]_.
The dispatc
[Python-Dev] __subclasses__() return order
Hello, In http://bugs.python.org/issue17936, I proposed making tp_subclasses (the internal container implementing object.__subclasses__) a dict. This would make the return order of __subclasses__ completely undefined, while it is right now slightly predictable. I have never seen __subclasses__ actually used in production code, so I'm wondering whether someone might be affected by such a change. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On Sat, May 25, 2013 at 6:18 AM, Antoine Pitrou wrote: > > Hello, > > In http://bugs.python.org/issue17936, I proposed making tp_subclasses > (the internal container implementing object.__subclasses__) a dict. > This would make the return order of __subclasses__ completely > undefined, while it is right now slightly predictable. I have never seen > __subclasses__ actually used in production code, so I'm wondering > whether someone might be affected by such a change. > > Regards > Personally I never used it, but it's now explicitly documented as returning a list. Not sure what's the right thing to do here, but perhaps returning an OrderedDict can eliminate the order problem? Eli ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On Sat, 25 May 2013 06:23:56 -0700 Eli Bendersky wrote: > On Sat, May 25, 2013 at 6:18 AM, Antoine Pitrou wrote: > > > > > Hello, > > > > In http://bugs.python.org/issue17936, I proposed making tp_subclasses > > (the internal container implementing object.__subclasses__) a dict. > > This would make the return order of __subclasses__ completely > > undefined, while it is right now slightly predictable. I have never seen > > __subclasses__ actually used in production code, so I'm wondering > > whether someone might be affected by such a change. > > > > Regards > > > > Personally I never used it, but it's now explicitly documented as returning > a list. Not sure what's the right thing to do here, but perhaps returning > an OrderedDict can eliminate the order problem? It would still return a list. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On Sat, 25 May 2013 15:26:58 +0200 Antoine Pitrou wrote: > On Sat, 25 May 2013 06:23:56 -0700 > Eli Bendersky wrote: > > On Sat, May 25, 2013 at 6:18 AM, Antoine Pitrou wrote: > > > > > > > > Hello, > > > > > > In http://bugs.python.org/issue17936, I proposed making tp_subclasses > > > (the internal container implementing object.__subclasses__) a dict. > > > This would make the return order of __subclasses__ completely > > > undefined, while it is right now slightly predictable. I have never seen > > > __subclasses__ actually used in production code, so I'm wondering > > > whether someone might be affected by such a change. > > > > > > Regards > > > > > > > Personally I never used it, but it's now explicitly documented as returning > > a list. Not sure what's the right thing to do here, but perhaps returning > > an OrderedDict can eliminate the order problem? > > It would still return a list. I guess I should explain myself more clearly: __subclasses__() already computes its result on-the-fly (it must weed out dead weakrefs) (*). So the visible behaviour of __subclasses__ wouldn't change, except for ordering. (*) >>> object.__subclasses__() is object.__subclasses__() False Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On 25 maj 2013, at 15:45, Antoine Pitrou wrote: > On Sat, 25 May 2013 15:26:58 +0200 > Antoine Pitrou wrote: > >> On Sat, 25 May 2013 06:23:56 -0700 >> Eli Bendersky wrote: >>> On Sat, May 25, 2013 at 6:18 AM, Antoine Pitrou wrote: >>> Hello, In http://bugs.python.org/issue17936, I proposed making tp_subclasses (the internal container implementing object.__subclasses__) a dict. This would make the return order of __subclasses__ completely undefined, while it is right now slightly predictable. I have never seen __subclasses__ actually used in production code, so I'm wondering whether someone might be affected by such a change. Regards >>> >>> Personally I never used it, but it's now explicitly documented as returning >>> a list. Not sure what's the right thing to do here, but perhaps returning >>> an OrderedDict can eliminate the order problem? >> >> It would still return a list. > > I guess I should explain myself more clearly: __subclasses__() already > computes its result on-the-fly (it must weed out dead weakrefs) (*). So > the visible behaviour of __subclasses__ wouldn't change, except for > ordering. +1 Makes sense to me. As currently defined, you cannot rely on the item order anyway. -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sat, May 25, 2013 at 8:08 AM, Łukasz Langa wrote: > The most important > change in this version is that I introduced ABC support and completed > a reference implementation. Excellent! A couple of thoughts on the implementation... While the dispatch() method allows you to look up what implementation would be *selected* for a target type, it does not let you figure out whether a particular method has been *registered* for a type. That is, if I have a class MyInt that subclasses int, I can't use dispatch() to check whether a MyInt implementation has been registered, because I might get back an implementation registered for int or object. ISTM there should be some way to get at the raw registration info, perhaps by exposing a dictproxy for the registry. Second, it should be possible to memoize dispatch() using a weak key dictionary that is cleared if new ABC implementations have been registered or when a call to register() is made. The way to detect ABC registrations is via the ABCMeta._abc_invalidation_counter attribute: if its value is different than the previous value saved with the cache, the cache must be cleared, and the new value stored. (Unfortunately, this is a private attribute at the moment; it might be a good idea to make it public, however, because it's needed for any sort of type dispatching mechanism, not just this one particular generic function implementation.) Anyway, doing the memoizing in the wrapper function should bring the overall performance very close to a hand-written type dispatch. Code might look something like: # imported inside closure so that functools module # doesn't force import of these other modules: # from weakref import ref, WeakKeyDictionary from abc import ABCMeta cache = WeakKeyDictionary() valid_as_of = ABCMeta._abc_invalidation_counter def wrapper(*args, **kw): nonlocal valid_as_of if valid_as_of != ABCMeta._abc_invalidation_counter: cache.clear() valid_as_of = ABCMeta._abc_invalidation_counter cls = args[0].__class__ try: impl = cache.data[ref(cls)] except KeyError: impl = cache[cls] = dispatch(cls) return impl(*args, **kw) def register(typ, func=None): ... cache.clear() ... This would basically eliminate doing any extra (Python) function calls in the common case, and might actually be faster than my current simplegeneric implementation on PyPI (which doesn't even do ABCs at the moment). ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On 25 maj 2013, at 16:08, PJ Eby wrote: > ISTM there should be some way to get at the raw > registration info, perhaps by exposing a dictproxy for the registry. Is that really useful? Just today Antoine asked about changing behaviour of __subclasses__(), suspecting it isn't used in real world code anyway. What you're proposing is the functional equivalent of __subclasses__(). If you need direct access to the registry, do you think the ability to specify your own registry container isn't enough? > The way to detect ABC registrations is via the > ABCMeta._abc_invalidation_counter attribute: if its value is different > than the previous value saved with the cache, the cache must be > cleared, and the new value stored. Wow, I was looking at it just today morning and somehow missed how it can be used. Great idea, I'll play around with it later today. > (Unfortunately, this is a private attribute at the moment; it might be > a good idea to make it public, however, because it's needed for any > sort of type dispatching mechanism, not just this one particular > generic function implementation.) I think we can safely use it within the standard library, anyway. As for making it public, it's an idea for a separate discussion. > This would basically eliminate doing any extra (Python) function calls > in the common case, and might actually be faster than my current > simplegeneric implementation on PyPI (which doesn't even do ABCs at > the moment). Yes, that sounds neat. Thanks for feedback! -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sun, May 26, 2013 at 12:08 AM, PJ Eby wrote: > On Sat, May 25, 2013 at 8:08 AM, Łukasz Langa wrote: >> The most important >> change in this version is that I introduced ABC support and completed >> a reference implementation. > > Excellent! A couple of thoughts on the implementation... > > While the dispatch() method allows you to look up what implementation > would be *selected* for a target type, it does not let you figure out > whether a particular method has been *registered* for a type. > > That is, if I have a class MyInt that subclasses int, I can't use > dispatch() to check whether a MyInt implementation has been > registered, because I might get back an implementation registered for > int or object. ISTM there should be some way to get at the raw > registration info, perhaps by exposing a dictproxy for the registry. I like that idea - types.MappingProxyType makes it straightforward to expose a read-only view of the dispatch registry. > Second, it should be possible to memoize dispatch() using a weak key > dictionary that is cleared if new ABC implementations have been > registered or when a call to register() is made. The way to detect > ABC registrations is via the ABCMeta._abc_invalidation_counter > attribute: if its value is different than the previous value saved > with the cache, the cache must be cleared, and the new value stored. > > (Unfortunately, this is a private attribute at the moment; it might be > a good idea to make it public, however, because it's needed for any > sort of type dispatching mechanism, not just this one particular > generic function implementation.) I think I added an issue on the tracker for that somewhere... yup: http://bugs.python.org/issue16832 Given the global nature of the cache invalidation, it may be better as a module level abc.get_cache_token() function. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On 25 maj 2013, at 16:59, Nick Coghlan wrote: > I think I added an issue on the tracker for that somewhere... yup: > http://bugs.python.org/issue16832 > > Given the global nature of the cache invalidation, it may be better as > a module level abc.get_cache_token() function. I assigned myself to the issue. I hope it doesn't require a PEP? ;-) -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sun, May 26, 2013 at 12:53 AM, Łukasz Langa wrote: > On 25 maj 2013, at 16:08, PJ Eby wrote: > >> ISTM there should be some way to get at the raw >> registration info, perhaps by exposing a dictproxy for the registry. > > Is that really useful? Just today Antoine asked about changing > behaviour of __subclasses__(), suspecting it isn't used in real world > code anyway. What you're proposing is the functional equivalent of > __subclasses__(). > > If you need direct access to the registry, do you think the ability to > specify your own registry container isn't enough? I'm actually wary about allowing that - letting people pass in the registry namespace the can alter it directly and thus bypass any extra code in register (such as setting the flag to enable the ABC support in the reference impl, or clearing the cache in PJE's suggested update). We don't allow custom namespaces for classes either. Sure, we allow them for the class *body*, but the contents of that get copied to a new dict when creating the class instance, precisely so we can ensure user code never gets a direct reference to the underlying mapping that defines the class behaviour. I actually patched Python once to to remove that copy operation while tinkering with the PEP 422 implementation. I wish I had kept a recording of the subsequent terminal session, as the terrible interactions with the method lookup caching were thoroughly confusing, but quite entertaining if you're into tinkering with programming languages :) So I think I'd prefer flipping this around - you can't provide a custom registry mapping, but you *can* get access to a read only view of it through a "registry" attribute on the generic function. >> The way to detect ABC registrations is via the >> ABCMeta._abc_invalidation_counter attribute: if its value is different >> than the previous value saved with the cache, the cache must be >> cleared, and the new value stored. > > Wow, I was looking at it just today morning and somehow missed how it > can be used. Great idea, I'll play around with it later today. Yeah, this is exactly how the ABC code itself uses it - sorry I wasn't clearer about why I suggested you look at the caching code for ABC instance and subtype checks :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sun, May 26, 2013 at 1:09 AM, Łukasz Langa wrote: > On 25 maj 2013, at 16:59, Nick Coghlan wrote: > > I think I added an issue on the tracker for that somewhere... yup: > http://bugs.python.org/issue16832 > > Given the global nature of the cache invalidation, it may be better as > a module level abc.get_cache_token() function. > > > I assigned myself to the issue. I hope it doesn't require a PEP? ;-) Heh, I think you're safe on that one :) Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sat, May 25, 2013 at 10:59 AM, Nick Coghlan wrote: > Given the global nature of the cache invalidation, it may be better as > a module level abc.get_cache_token() function. Well, since the only reason to ever use it is to improve performance, it'd be better to expose it as an attribute than as a function. ;-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On Sat, May 25, 2013 at 9:18 AM, Antoine Pitrou wrote: > In http://bugs.python.org/issue17936, I proposed making tp_subclasses > (the internal container implementing object.__subclasses__) a dict. > This would make the return order of __subclasses__ completely > undefined, while it is right now slightly predictable. I have never seen > __subclasses__ actually used in production code, so I'm wondering > whether someone might be affected by such a change. FWIW, when I've used __subclasses__, I've never depended on it having a stable or predictable order. (I find it somewhat difficult to imagine *why* one would do that, but of course that doesn't mean nobody has done it.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sun, May 26, 2013 at 2:48 AM, PJ Eby wrote: > On Sat, May 25, 2013 at 10:59 AM, Nick Coghlan wrote: >> Given the global nature of the cache invalidation, it may be better as >> a module level abc.get_cache_token() function. > > Well, since the only reason to ever use it is to improve performance, > it'd be better to expose it as an attribute than as a function. ;-) A single function call is hardly in the same league as arbitrary traversal of the object graph. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On 25 maj 2013, at 17:13, Nick Coghlan wrote: > So I think I'd prefer flipping this around - you can't provide a > custom registry mapping, but you *can* get access to a read only view > of it through a "registry" attribute on the generic function. You guys convinced me. Both the PEP and the implementation are updated accordingly. I also introduced the caching as suggested by PJ. So, the latest document is live: http://www.python.org/dev/peps/pep-0443/ The code is here: http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363 The documentation here: http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l189 The tests here: http://hg.python.org/features/pep-443/file/tip/Lib/test/test_functools.py#l855 I say we're ready. -- Best regards, Łukasz Langa WWW: http://lukasz.langa.pl/ Twitter: @llanga IRC: ambv on #python-dev ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sat, 25 May 2013 22:16:04 +0200 Łukasz Langa wrote: > On 25 maj 2013, at 17:13, Nick Coghlan wrote: > > > So I think I'd prefer flipping this around - you can't provide a > > custom registry mapping, but you *can* get access to a read only view > > of it through a "registry" attribute on the generic function. > > You guys convinced me. Both the PEP and the implementation are updated > accordingly. I also introduced the caching as suggested by PJ. > > So, the latest document is live: > http://www.python.org/dev/peps/pep-0443/ > > The code is here: > http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363 > > The documentation here: > http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l189 > > The tests here: > http://hg.python.org/features/pep-443/file/tip/Lib/test/test_functools.py#l855 > > I say we're ready. Let it cook for a week or two. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sat, May 25, 2013 at 4:16 PM, Łukasz Langa wrote: > So, the latest document is live: > http://www.python.org/dev/peps/pep-0443/ > > The code is here: > http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363 > > The documentation here: > http://hg.python.org/features/pep-443/file/tip/Doc/library/functools.rst#l189 Code and tests look great! Nitpick on the docs and PEP, though: generic functions are not composed of functions sharing the same name; it would probably be more correct to say they're composed of functions that perform the same operations on different types. (I think the "names" language might be left over from discussion of *overloaded* functions in PEP 3124 et al; in any case we're actually recommending people *not* use the same names now, so it's confusing.) We should probably also standardize on the term used for the registered functions. The standard terminology is "method", but that would be confusing in Python, where methods usually have a self argument. The PEP uses the term "implementation", and I think that actually makes a lot of sense: a generic function is composed of functions that implement the same operation for different types. So I suggest changing this: """ Transforms a function into a single-dispatch generic function. A **generic function** is composed of multiple functions sharing the same name. Which form should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Adding an overload to a generic function is achieved by using the :func:`register` attribute of the generic function. The :func:`register` attribute is a decorator, taking a type paramater and decorating a function implementing the overload for that type.""" to: """ Transforms a function into a single-dispatch generic function. A **generic function** is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Adding an implementation to a generic function is achieved by using the :func:`register` attribute of the generic function. The :func:`register` attribute is a decorator, taking a type paramater and decorating a function implementing the operation for that type.""" And replacing "overload" with "implementation" in the remainder of the docs and code. Last, but not least, there should be a stacking example somewhere in the doc, as in the PEP, and perhaps the suggestion to name individual implementations differently from each other and the main function -- perhaps as an adjunct to documenting that register() always returns its argument unchanged. (Currently, it doesn't mention what register()'s return value is.) (It may also be useful to note somewhere that, due to caching, changing the base classes of an existing class may not change what implementation is selected the next time the generic function is invoked with an argument of that type or a subclass thereof.) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On Sun, May 26, 2013 at 9:07 AM, PJ Eby wrote: > On Sat, May 25, 2013 at 4:16 PM, Łukasz Langa wrote: >> So, the latest document is live: >> http://www.python.org/dev/peps/pep-0443/ >> >> The code is here: >> http://hg.python.org/features/pep-443/file/tip/Lib/functools.py#l363 Hmm, I find the use of the variable name "dispatch_cache" for a cache that dispatch() doesn't actually use to be confusing. It also doesn't make sense to me that dispatch() itself bypasses the cache - I would expect all the cache manipulation to be in dispatch(), and there to be a separate "_find_impl()" function that is invoked to handle cache misses. If there's a good reason for dispatch() to bypass the cache without refreshing it, then I suggest renaming the cache variable to "impl_cache". > We should probably also standardize on the term used for the > registered functions. The standard terminology is "method", but that > would be confusing in Python, where methods usually have a self > argument. The PEP uses the term "implementation", and I think that > actually makes a lot of sense: a generic function is composed of > functions that implement the same operation for different types. +1 on consistently using "implementation" as the name for individual functions that are passed to register() and that dispatch() may return. It's also consistent with the terminology we (or at least I) tend to use about finding the implementation of methods based on the type hierarchy. Also +1 to your other docs comments - that info is in the PEP, and is relevant to actually using the new generics in practice. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 443 - Single-dispatch generic functions (including ABC support)
On 26/05/13 09:07, PJ Eby wrote: """ Transforms a function into a single-dispatch generic function. A **generic function** is composed of multiple functions implementing the same operation for different types. Which implementation should be used during a call is determined by the dispatch algorithm. When the implementation is chosen based on the type of a single argument, this is known as **single dispatch**. Adding an implementation to a generic function is achieved by using the :func:`register` attribute of the generic function. The :func:`register` attribute is a decorator, taking a type paramater Typo: /s/paramater/parameter/ and decorating a function implementing the operation for that type.""" Otherwise, +1 on the doc changes suggested. Thanks PJ and Łukasz for seeing this one through. -- Steven ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On 2013-05-25 09:18, Antoine Pitrou wrote: Hello, In http://bugs.python.org/issue17936, I proposed making tp_subclasses (the internal container implementing object.__subclasses__) a dict. This would make the return order of __subclasses__ completely undefined, while it is right now slightly predictable. I have never seen __subclasses__ actually used in production code, so I'm wondering whether someone might be affected by such a change. I do use a package that does use __subclasses__ in production code, but the order is unimportant. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] __subclasses__() return order
On Sat, May 25, 2013 at 11:49 PM, Łukasz Langa wrote: > I guess I should explain myself more clearly: __subclasses__() already > computes its result on-the-fly (it must weed out dead weakrefs) (*). So > the visible behaviour of __subclasses__ wouldn't change, except for > ordering. > > > +1 > > Makes sense to me. As currently defined, you cannot rely on the item order > anyway. Another concurrence here - if any code in the world depends on __subclasses__ always returning entries in the exact order they happen to be returned in right now, I'm quite happy to declare that code implementation dependent and thus exempt from the normal backwards compatibility guarantees :) Cheers, Nick. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
