David Lukeš <dafydd.lu...@gmail.com> added the comment:
Yes, sorry for not being quite clear the first time around :)
I eventually found out about Pool.imap (see item 3 on list in OP) and indeed it
fits my use case very nicely, but my point was that the documentation is
somewhat misleading with respect to the semantics of built-in `map()` in Python
3.
Specifically, I would argue that it is unexpected for a function which claims
to be "Equivalent to map(func, *iterables)" to require allocating a list the
length of the shortest iterable.
Maybe a code example will make this clearer for potential newcomers to the
discussion -- this is what I would expect to happen (= the behavior of built-in
`map()` itself), yielding values from the iterable is interleaved with calls to
the mapped function:
```
>>> def gen():
... for i in range(3):
... print("yielding", i)
... yield i
...
>>> def add1(i):
... print("adding 1 to", i)
... return i + 1
...
>>> list(map(add1, gen()))
yielding 0
adding 1 to 0
yielding 1
adding 1 to 1
yielding 2
adding 1 to 2
[1, 2, 3]
```
This is what happens instead with `concurrent.futures.Executor.map()`:
```
>>> def my_map(fn, iterable):
... lst = list(iterable)
... for i in lst:
... yield fn(i)
...
>>> list(my_map(add1, gen()))
yielding 0
yielding 1
yielding 2
adding 1 to 0
adding 1 to 1
adding 1 to 2
[1, 2, 3]
```
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32306>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
