Right now, the following code is valid in Python 3.9 (and infinitely loops):
```
lst = [1, 2, 3]
for i in lst:
lst.append(i)
```
IMO this is a source of bugs, and even though this behavior is technically
documented (https://bugs.python.org/issue32767), I don't know if it should
belong in the core language. For example, you cannot change the size of a set
or dict during iteration; you will get a RuntimeError. Allegedly this is due to
the underlying implementation of the dict/set iterators
(https://bugs.python.org/issue22084) but I think it would be beneficial this
was made part of the public API for list, set, and dict iterators.
Concretely, I'm proposing the following set of changes to Python:
1. If the size of a list, set, or dict changes, invalidate all existing
iterators to that containers. Document this behavior as part of the public API.
2. Attempting to call next() on an invalidated iterator raises a RuntimeError.
3. Add a key_iter function to itertools (name up for debate), which replicates
the existing functionality for lists. It could also make a copy of the keys of
a set/dict, so you could use it to mutate those as well during iteration (to
allow users to write functions that mutate an arbitrary duck-typed collection)
4. Gate (1, 2) for lists behind a __future__ flag with a long migration window
(e.g. ~3.16).
Benefits:
1. Reduce a source of bugs (deleting while iterating forward through a list
often means that your logic skips elements; appending can lead to infinite
loops or unexpected behavior).
2. Make the iterator invalidation consistent between lists, sets, and dicts.
3. More consistent behavior with collections in other languages (C++, Java).
Drawbacks:
1. Migration costs.
But mutating during iteration should be relatively rare, given that most
people discourage mutating while iterating
(https://google.github.io/styleguide/pyguide.html#284-decision,
https://stackoverflow.com/a/1637875/1625484)
__future__ feature flag plus deprecation warnings should go along way to help
people migrate their code. We would also offer an easy in-place fix (key_iter).
2. You can't write algorithms that append to a list (e.g. BFS) with the normal
list iterator.
Again, you could use key_iter (or use a deque with block size == 1; see Open
Questions).
Open Questions
Is (3) necessary? It's not hard to write an implementation yourself.
What should be done about deque for block sizes > 1? If this change gets made,
should we add way to create a deque with block size == 1 whose iterators are
never invalidated?
_______________________________________________
Python-ideas mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/G53KTBQERD2ZWMLODBRLG4CIFHILF7NJ/
Code of Conduct: http://python.org/psf/codeofconduct/
