Re: itertools.izip brokeness

Op 2006-01-10, Bengt Richter schreef <[EMAIL PROTECTED]>: > On 9 Jan 2006 08:19:21 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: > >>Op 2006-01-05, Bengt Richter schreef <[EMAIL PROTECTED]>: >>> On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: > [...] >>> But you can fix that (o

Re: itertools.izip brokeness

On 9 Jan 2006 08:19:21 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: >Op 2006-01-05, Bengt Richter schreef <[EMAIL PROTECTED]>: >> On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: [...] >> But you can fix that (only test is what you see ;-) : > >Maybe, but not with this version

Re: itertools.izip brokeness

Op 2006-01-05, Bengt Richter schreef <[EMAIL PROTECTED]>: > On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: > >>On 2006-01-04, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >>><[EMAIL PROTECTED]> wrote: But here is my real question... Why isn't something like this in i

Re: itertools.izip brokeness

Steven D'Aprano <[EMAIL PROTECTED]> writes: > > def izip5(*iterables, fill=None): > Doesn't work: keyword arguments must be listed before * and ** arguments. Eh, ok, gotta use **kw. > def function(*iterators, **kwargs): > if kwargs.keys() != ["fill"]: > raise ValueError > ...

Re: itertools.izip brokeness

On Thu, 05 Jan 2006 23:52:13 -0800, Paul Rubin wrote: > [EMAIL PROTECTED] writes: >> def izip4(*iterables, **kw): >> """kw:fill. An element that will pad the shorter iterable >> kw:infinite. Number of non-terminating iterators """ > > That's a really kludgy API. I'm not sure what to

Re: itertools.izip brokeness

On 5 Jan 2006 14:34:39 -0800, [EMAIL PROTECTED] wrote: > >Bengt Richter wrote: >> On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: >> >> >On 2006-01-04, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >> >><[EMAIL PROTECTED]> wrote: >> >>> But here is my real question... >> >>> Wh

Re: itertools.izip brokeness

Paul Rubin writes: ># quit if only discardables are left >dropwhile(lambda i,t: (not isinstance(i, Discardable)) and len(t)), > izip(t, iterables)).next() Ehh, that should say dropwhile(lambda (t,i): ...) to use tuple unpacking and

Re: itertools.izip brokeness

[EMAIL PROTECTED] writes: > def izip4(*iterables, **kw): > """kw:fill. An element that will pad the shorter iterable > kw:infinite. Number of non-terminating iterators """ That's a really kludgy API. I'm not sure what to propose instead: maybe some way of distinguishing which iterabl

Re: itertools.izip brokeness

"Michael Spencer" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > > > Bengt Richter wrote: > ... > >> >>> from itertools import repeat, chain, izip > >> >>> it = iter(lambda z=izip(chain([3,5,8],repeat("Bye")), > >> chain([11,22],repeat("Bye"))):z.next(), ("Bye","Bye")) > >> >>>

Re: itertools.izip brokeness

Paul Rubin wrote: > Michael Spencer <[EMAIL PROTECTED]> writes: >> for i in range(10): >> result = [] >> ... > > Do you mean "while True: ..."? > oops, yes! so, this should have been: from itertools import repeat def izip2(*iterables, **kw): """kw:fill. An element t

Re: itertools.izip brokeness

Michael Spencer <[EMAIL PROTECTED]> writes: > for i in range(10): > result = [] > ... Do you mean "while True: ..."? > def izip2(*iterables, **kw): > """kw:fill. An element that will pad the shorter iterable""" > fill = repeat(kw.get("fill")) Yet another attempt

Re: itertools.izip brokeness

> Bengt Richter wrote: ... >> >>> from itertools import repeat, chain, izip >> >>> it = iter(lambda z=izip(chain([3,5,8],repeat("Bye")), >> chain([11,22],repeat("Bye"))):z.next(), ("Bye","Bye")) >> >>> for t in it: print t >> ... >> (3, 11) >> (5, 22) >> (8, 'Bye') >> >> (Feel free to gene

Re: itertools.izip brokeness

On Thu, 05 Jan 2006 07:42:25 GMT, [EMAIL PROTECTED] (Bengt Richter) wrote: >On 4 Jan 2006 15:20:43 -0800, "Raymond Hettinger" <[EMAIL PROTECTED]> wrote: > [ ... 5 options enumerated ... ] >> >> >6. Could modify izip so that one could write > >from itertools import izip >zipit = izip(*seqs)

Re: itertools.izip brokeness

Bengt Richter wrote: > On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: > > >On 2006-01-04, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > >><[EMAIL PROTECTED]> wrote: > >>> But here is my real question... > >>> Why isn't something like this in itertools, or why shouldn't > >>>

Re: itertools.izip brokeness

On 5 Jan 2006 15:48:26 GMT, Antoon Pardon <[EMAIL PROTECTED]> wrote: >On 2006-01-04, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: >><[EMAIL PROTECTED]> wrote: >>> But here is my real question... >>> Why isn't something like this in itertools, or why shouldn't >>> it go into itertools? >> >> >> 4

Re: itertools.izip brokeness

On 2006-01-04, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: ><[EMAIL PROTECTED]> wrote: >> But here is my real question... >> Why isn't something like this in itertools, or why shouldn't >> it go into itertools? > > > 4) If a need does arise, it can be met by __builtins__.map() or by > writi

Re: itertools.izip brokeness

On 4 Jan 2006 15:20:43 -0800, "Raymond Hettinger" <[EMAIL PROTECTED]> wrote: >Paul Rubin wrote: >> What do you think of my suggestion of passing an optional arg to the >> StopIteration constructor, that the caller can use to resume the >> iterator or take other suitable recovery steps? Maybe this

Re: itertools.izip brokeness

Paul Rubin wrote: > What do you think of my suggestion of passing an optional arg to the > StopIteration constructor, that the caller can use to resume the > iterator or take other suitable recovery steps? Maybe this could > interact with PEP 343 in some useful way. Probably unworkable. Complex

Re: itertools.izip brokeness

> I don't understand this. Why do you need look ahead? Just before I posted, I got it (I think) but didn't want to rewrite everything. The need for unget() (or peek(), etc) is to fix the thrown-away-data problem in izip(), right? As an easier alternative, what about leaving izip() alone and si

Re: itertools.izip brokeness

"Raymond Hettinger" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > [EMAIL PROTECTED] wrote: > > izip's uses can be partitioned two ways: > > 1. All iterables have equal lengths > > 2. Iterables have different lengths. > > > > Case 1 is no problem obviously. > > In Case 2 there are t

Re: itertools.izip brokeness

On Wed, 4 Jan 2006, Raymond Hettinger wrote: > [EMAIL PROTECTED] wrote: > >> The whole point of using izip is to make the code shorter, more >> concise, and easier to write and understand. > > That should be the point of using anything in Python. The specific goal > for izip() was for an iterat

Re: itertools.izip brokeness

"Raymond Hettinger" <[EMAIL PROTECTED]> writes: > Feel free to submit a feature request to the SF tracker (surprisingly, > this behavior has not been previously reported, nor have there any > related feature requests, nor was the use case contemplated in the PEP > discussions: http://www.python.org

Re: itertools.izip brokeness

[EMAIL PROTECTED] wrote: > izip's uses can be partitioned two ways: > 1. All iterables have equal lengths > 2. Iterables have different lengths. > > Case 1 is no problem obviously. > In Case 2 there are two sub-cases: > > 2a. You don't care what values occur in the other iterators > after then en

Re: itertools.izip brokeness

[EMAIL PROTECTED] wrote: > <[EMAIL PROTECTED]> wrote: > > pissed-offedly-yr's, rurpy > > Well, i'm sorry your pissed off. I will say i believe that map(None,*sequences) mentioned above is a pretty commonly seen thing, as is padding shorter sequence for zip/izip. Also have you looked at diff

Re: itertools.izip brokeness

<[EMAIL PROTECTED]> wrote: > But here is my real question... > Why isn't something like this in itertools, or why shouldn't > it go into itertools? Never mind... I just read this in the source code for itertools.imap: 1) Itertools are designed to be easily combined and chained together. Ha

Re: itertools.izip brokeness

[EMAIL PROTECTED] wrote: > It is clear that there is a real need for iterating in parallel > over multiple iterators to the end of the longest one. Why > does something that stops at the shortest get included in > the standard library, but one that stops after the longest > doesn't? Is there any

Re: itertools.izip brokeness

"Duncan Booth" <[EMAIL PROTECTED]> wrote: > Peter Otten wrote: > > > from itertools import izip, chain, repeat > > > > def prt_files (file1, file2): > > file1 = chain(file1, repeat("")) > > file2 = chain(file2, repeat("")) > > for line1, line2 in iter(izip(file1, file2).next, ("", "")):

Re: itertools.izip brokeness

<[EMAIL PROTECTED]> wrote: > But that is exactly the behaviour of python iterator, I don't see what > is broken. > > izip/zip just read from the respectives streams and give back a tuple, > if it can get one from each, otherwise stop. And because python > iterator can only go in one direction, tho

Re: itertools.izip brokeness

On Tue, 3 Jan 2006, it was written: > [EMAIL PROTECTED] writes: > >> The problem is that sometimes, depending on which file is the shorter, >> a line ends up missing, appearing neither in the izip() output, or in >> the subsequent direct file iteration. I would guess that it was in >> izip's b

Re: itertools.izip brokeness

Paul Rubin wrote: > Any idea how Haskell would deal with this? I don't recall haskell has the map(None,...) behaviour in the standard Prelude. But then, I don't see how the iterator concept would fit into haskell as well. -- http://mail.python.org/mailman/listinfo/python-list

Re: itertools.izip brokeness

Peter Otten wrote: > from itertools import izip, chain, repeat > > def prt_files (file1, file2): > file1 = chain(file1, repeat("")) > file2 = chain(file2, repeat("")) > for line1, line2 in iter(izip(file1, file2).next, ("", "")): > print line1.rstrip(), "\t", line2.rstrip() >

Re: itertools.izip brokeness

"Diez B. Roggisch" <[EMAIL PROTECTED]> writes: > No. If you want that, use > > list(iterable) > > Then you have random access. If you _know_ there will be only so much data > needed to "unget", write yourself a buffered iterator like this: You can't use list(iterable) in general because the iter

Re: itertools.izip brokeness

> What's broken is the iterator interface is insufficient to deal with > this cleanly. I don't consider it broken. You just think too much in terms of the OPs problems or probably other fields where the actual data is available for "rewinding". But as iterators serve as abstraction for lots of th

Re: itertools.izip brokeness

Paul Rubin wrote: > [EMAIL PROTECTED] writes: > > map(None,[1,2,3],[4,5]) gives [(1,4),(2,5),(3,None)] > > I didn't know that until checking the docs just now. Oh man, what a > hack! I always thought Python should have a built-in identity > function for situations like that. I guess it does the

Re: itertools.izip brokeness

[EMAIL PROTECTED] writes: > map(None,[1,2,3],[4,5]) gives [(1,4),(2,5),(3,None)] I didn't know that until checking the docs just now. Oh man, what a hack! I always thought Python should have a built-in identity function for situations like that. I guess it does the above instead. Thanks. Jeez

Re: itertools.izip brokeness

Paul Rubin wrote: > > I think you need to use map(None,...) which would not drop anything, > > just None filled. Though you don't have a relatively lazy version as > > imap(None,...) doesn't behave like map but a bit like zip. > > I don't understand what you mean by this? None is not callable. z

Re: itertools.izip brokeness

[EMAIL PROTECTED] writes: > But that is exactly the behaviour of python iterator, I don't see what > is broken. What's broken is the iterator interface is insufficient to deal with this cleanly. > And because python iterator can only go in one direction, those > consumed do lose in the zip/izip c

Re: itertools.izip brokeness

[EMAIL PROTECTED] wrote: > The problem is that sometimes, depending on which > file is the shorter, a line ends up missing, > appearing neither in the izip() output, or in > the subsequent direct file iteration.  I would > guess that it was in izip's buffer when izip > terminates due to the except

Re: itertools.izip brokeness

[EMAIL PROTECTED] schrieb: > [izip() eats one line] as far as i can see the current implementation cannot be changed to do the Right Thing in your case. pythons iterators don't allow to "look ahead", so izip can only get the next element. if this fails for an iterator, everything up to that point

Re: itertools.izip brokeness

But that is exactly the behaviour of python iterator, I don't see what is broken. izip/zip just read from the respectives streams and give back a tuple, if it can get one from each, otherwise stop. And because python iterator can only go in one direction, those consumed do lose in the zip/izip cal

Re: itertools.izip brokeness

[EMAIL PROTECTED] writes: > The problem is that sometimes, depending on which file is the > shorter, a line ends up missing, appearing neither in the izip() > output, or in the subsequent direct file iteration. I would guess > that it was in izip's buffer when izip terminates due to the > exceptio