> For `__advance__` to be an official Python protocol, it would almost
> certainly have to be of use for *general purpose iterators*, not just
> specialised ones -- and probably not *hypothetical* iterators which may
> not even exist. Do you have concrete examples of your skip list and tree
> iterators that are in wide-spread use?
"I am +0.3 on this as I don't personally have a need"
" (I'm not sure why you would ever want to `__advance__` a tree iterator)"
> What's your use case for advancing a count object, rather than just
> creating a new one?
Itertools.count was an example (hence the use of "e.g.") of an iterator
which can be efficiently
advanced without producing intermediate state. Clearly anyone can advance
it manually.
My point is that an iterator may have an efficient way to calculate its
state some point in the future
without needing to calculate the intermediate state. For example the
fibonacci sequence has a closed
form formula for the nth element and hence could be advanced efficiently.
I realize my original examples were contrived, but I have a better one.
Consider the task of collecting TCP headers from an iterator of bytes:
```
def get_tcp_headers(stream: Iterator[Byte]):
while stream:
# Move to the total length field of the IP header
stream.advance(2)
# record the total length (two bytes)
total_length = ...
# skip the rest of IP header
stream.advance(28)
# record the TCP header
header = ...
yield header
stream.advance(total_length - 32 - len(header))
```
Maybe Kevin can tell us what motivated him to post this idea but I can see
many places in parsing where you might want
to skip arbitrary portions of a stream. The beauty of iterators is you
don't need to be concerned with the underlying data
structure. Ideally I shouldn't need to write two versions of some parse
function one which operates on sequences and one
that operates on iterables, just so I can efficiently `advance` the
sequences.
-- Caleb Donovick
On Tue, Oct 6, 2020 at 6:16 PM Steven D'Aprano <[email protected]> wrote:
> On Tue, Oct 06, 2020 at 02:27:54PM -0700, Caleb Donovick wrote:
> > I am +0.3 on this as I don't personally have a need for this but do see
> the
> > utility.
> >
> > I can think of a number of examples where an `__advance__` would be
> > preferable to any of the proposed solutions:
> [...]
>
> For `__advance__` to be an official Python protocol, it would almost
> certainly have to be of use for *general purpose iterators*, not just
> specialised ones -- and probably not *hypothetical* iterators which may
> not even exist. Do you have concrete examples of your skip list and tree
> iterators that are in wide-spread use?
>
> Specialised iterators can create whatever extra APIs they want to
> support, but the official iterator protocol intentionally has a very
> basic API:
>
> - anything with an `__iter__` method which returns itself;
> - and a `__next__` method that returns the next value, raising
> StopIteration when exhausted.
>
> This is a bare minimum needed to make an iterator, and we like it that
> way. For starters, it means that generators are iterators.
>
> If people want to supply objects that support the iterator protocol
> but also offer a rich API including:
>
> - peek
> - restart
> - previous
> - jump ahead (advance)
>
> all features that have been proposed, there is nothing stopping you from
> adding those features to your iterator classes. But they all have
> problems if considered to be necessary for *all* iterators.
>
> I would expect that, given a sufficiently compelling real-world
> use-case, we would be prepared to add a jump ahead method to
> list-iterators, as a specific feature of that iterator, not of all
> iterators.
>
>
> > A skip list which doesn't support O(1) random access but can advance
> faster
> > than naively calling next repeatedly
> > A lazy infinite iterator which can efficiently calculate its state at
> some
> > future point (e.g. `itertools.count`)
>
> What's your use case for advancing a count object, rather than just
> creating a new one?
>
> it = itertools.count() # start at 0
> process(it) # process some values
> it.advance(state) # jump forward
> process(it) # process some more values
>
> as opposed to what is already possible:
>
> it = itertools.count()
> process(it)
> it = itertools.count(state)
> process(it)
>
> Real-world use-cases for this feature are far more useful than contrived
> and artifical use-cases unlikely to ever occur in real code.
>
>
> > My ladder two examples demonstrate that this could have utility outside
> of
> > sequences but for iterators in general.
>
> I'm sorry, I don't know what your ladder two examples are. Did you post
> them in another thread?
>
>
> --
> Steve
> _______________________________________________
> Python-ideas mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/I6PN73GZZ5K3L5Z4F6DQWOVHRZ2IMFMD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
_______________________________________________
Python-ideas mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/FLPQM6FJ7VV2Q6CRP56AJTNZBX7LNEZP/
Code of Conduct: http://python.org/psf/codeofconduct/
