One thing I'd like to add real quick to this (I'm on my phone so apologies for crappy quoting):
Although there are existing cases where there is a loss of efficiency over Python 2 map() when dealing with the opaque, iterable Python 3 map(), the latter also presents many opportunities for enhancements that weren't possible before. For example, previously a user might pass map(func, some_list) where func is some pure function and the iterable is almost always a list of some kind. Previously that map() call would be evaluated (often slowly) first. But now we can treat a map as something a little more formal, as a container for a function and one or more iterables, which happens to have this special functionality when you iterate over it, but is otherwise just a special container. This is technically already the case, we just can't directly access it as a container. If we could, it would be possible to implement various optimizations that a user might not have otherwise been obvious to the user. This is especially the case of the iterable is a simple list, which is something we can check. The function in this case very likely might actually be a C function that was wrapped with Cython. I can easily convert this on the user's behalf to a simple C loop or possibly even some other more optimal vectorized code. These are application-specific special cases of course, but many such cases become easily accessible if map() and friends are usable as specialized containers. On Wed, Nov 28, 2018, 16:31 E. Madison Bray <[email protected] wrote: > On Wed, Nov 28, 2018 at 4:24 PM Chris Angelico <[email protected]> wrote: > > > > On Thu, Nov 29, 2018 at 2:19 AM E. Madison Bray <[email protected]> > wrote: > > > > > > On Wed, Nov 28, 2018 at 4:14 PM Steven D'Aprano <[email protected]> > wrote: > > > > > > > > On Wed, Nov 28, 2018 at 04:04:33PM +0100, E. Madison Bray wrote: > > > > > > > > > That effort is already mostly done and adding a helper function > would > > > > > not have worked as users *passing* map(...) as an argument to some > > > > > function just expect it to work. > > > > > > > > Ah, that's what I was missing. > > > > > > > > But... surely the function will still work if they pass an opaque > > > > iterator *other* than map() and/or filter? > > > > > > > > it = (func(x) for x in something if condition(x)) > > > > some_sage_function(it) > > > > > > That one is admittedly tricky. For that matter it might be nice to > > > have more introspection of generator expressions too, but there at > > > least we have .gi_code if nothing else. > > > > Considering that a genexp can do literally anything, I doubt you'll > > get anywhere with that introspection. > > > > > But those are a far less common example in my case, whereas map() is > > > *everywhere* in math code :) > > > > Perhaps then, the problem is that math code treats "map" as something > > that is more akin to "instrumented list" than it is to a generator. If > > you know for certain that you're mapping a low-cost pure function over > > an immutable collection, the best solution may be to proxy through to > > the original list than to generate values on the fly. And if that's > > the case, you don't want the Py3 map *or* the Py2 one, although the > > Py2 one can behave this way, at the cost of crazy amounts of > > efficiency. > > Yep, that's a great example where it might be possible to introspect a > given `map` object and take it apart to do something more efficient > with it. This is less of a problem with internal code where it's easy > to just not use map() at all, and that is often the case. But a lot > of the people who develop code for Sage are mathematicians, not > engineers, and they may not be aware of this, so they write code that > passes `map()` objects to more internal machinery. And users will do > the same even moreso. > > I can (and have) written horrible C-level hacks--not for this specific > issue, but others like it--and am sometimes tempted to do the same > here :( >
_______________________________________________ Python-ideas mailing list [email protected] https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
