[Python-Dev] Bug or not? Different behaviour iterating list and collections.deque
Hello, people. I am not sure whether this is a bug or intentional, so I thought checking it with you before opening a bug. I will explain this issue, but please understand this is not a question for help to change the algorithm (this has been done already), so it's not a question of c.l.py. It's a matter of discrepancy. A list that changes while iterating produces no errors, while a deque fails. Given the following example code: #code start import itertools, collections def item_is_special(item): "Just a dummy check in this example" return item % 3 == 0 def item_products(item): "Also a dummy function for the example" return [item*20+1, item*30+1] def process_list(items, type_of_que, special_case): # we want to process some items, but some of them # produce other items to be processed by the # same algorithm products= type_of_que() if special_case: products.append(-1) for item in itertools.chain(items, products): if item_is_special(item): for product in item_products(item): products.append(product) else: print "got", item #code end we have the following cases: >>> process_list(numbers, list, False) got 1 got 2 got 61 got 91 List works as expected. >>> process_list(numbers, collections.deque, False) got 1 got 2 deque does not work, most probably because deque.__iter__ of an empty deque ignores later changes. For this reason the `special_case' parameter was inserted in the code above, so that there is at least an item when itertools.chain calls iter on the deque: >>> process_list(numbers, collections.deque, True) got 1 got 2 Traceback (most recent call last): File "", line 1, in File "testdequeiter.py", line 17, in process_list for item in itertools.chain(items, products): RuntimeError: deque mutated during iteration Is that intentional? If not, let me know so that I open a bug. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug or not? Different behaviour iterating list andcollections.deque
Forgive my piggy backing, but I forgot to include the only related post I found, which did not clear things up for me: http://groups.google.com/group/comp.lang.python/msg/e2dcb2362649a601 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug or not? Different behaviour iterating list and collections.deque
"Christos Georgiou" <[EMAIL PROTECTED]> wrote: > > Hello, people. I am not sure whether this is a bug or intentional, so I > thought checking it with you before opening a bug. I will explain this > issue, but please understand this is not a question for help to change the > algorithm (this has been done already), so it's not a question of c.l.py. > It's a matter of discrepancy. [snip] > Forgive my piggy backing, but I forgot to include the only related post I > found, which did not clear things up for me: > > http://groups.google.com/group/comp.lang.python/msg/e2dcb2362649a601 Lists are not deques, deques are not lists. As stated in the post you link to states, the behavior of lists and deques is different in the same situation, so while one may "work" for your task, the other may not. I would, generally speaking, concur with Rayomond's recommendation in the post and say that mutation of the item you are iterating over is confusing, and it certainly is likely far more tricky than you should be when writing software so that people can use it. In this case, you are making it even worse by attempting to merge a list and deque, under the presumption that they should do the same thing. They don't. They are documented not to do the same thing. And you read a post explaining that they don't do the same thing and cited it. My suggestion: don't do what you are doing. Input list, output list. Make your life simple. Appending to a list is fast. If you *need* to merge a list and deque, do to some insane requirements by an organization, write your own wrapper that does what you think is the right thing to do in this case. - Josiah ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug or not? Different behaviour iterating list andcollections.deque
"Josiah Carlson" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > > "Christos Georgiou" <[EMAIL PROTECTED]> wrote: >> [snip] >> issue, but please understand this is not a question for help to change >> the >> algorithm (this has been done already), so it's not a question of c.l.py. >> It's a matter of discrepancy. > [snip] > > [snip Josiah's wordy "it's intentional and not a bug" in the form of a > suggestion > for a change of algorithm] Like I said, this wasn't a c.l.py question, even if you thought it deserved a c.l.py answer. In any case, you answered my question, and thank you. It not being a bug suits me just fine. Allow me to make sure we talk about the same thing here, though: both the example code I provided and the original one do modify the iterable *only* between the following A and B points in time: Point A: itertools.chain calls iter() on the iterable. (Appending to the iterable (list, deque) occur here, and only here.) Point B: itertools.chain starts calling iterable.next(). This is a different case from the one mentioned in the post by Raymond, and that is why I asked. For example, if itertools.chain called iter() on its arguments when actually needing to iterate over them instead of at the beginning, the code would work. But I really, really don't mind whatever the function, as long as it's by design, and that's the reason I didn't submit a bug in the tracker. That's all. I won't (and never intended) to defend any algorithms. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] smarter temporary file object (SF #415692, #1630118)
On Tue, Jan 02, 2007 at 10:07:58PM -0800, Neal Norwitz wrote: > Thanks for your patch! With some advice from Jim Jewett, the addition of some test cases, and a paragraph of documentation, I've uploaded the corresponding patch at http://python.org/sf/1630118 if there are any other modifications or improvements anyone has in mind, please let me know. Dustin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Bug or not? Different behaviour iterating list and collections.deque
Christos Georgiou schrieb: > Is that intentional? It would have helped if you had said what "that" is you are referring to, it would also have helped if you had stated an opinion on whether you believe that to be a bug. For example, I think I would have phrased your post like this: """ If I apply .next() to an iter of a deque that has been modified, I get a RuntimeError: py> d=collections.deque() py> d.append(10) py> i=iter(d) py> d.append(10) py> i.next() Traceback (most recent call last): File "", line 1, in ? RuntimeError: deque mutated during iteration Yet when I apply .next() to an iter of an initially-empty deque, I get StopIteration: py> d=collections.deque() py> i=iter(d) py> d.append(10) py> i.next() Traceback (most recent call last): File "", line 1, in ? StopIteration Is this a bug? Shouldn't the second example also raise the RuntimeError as the deque has been modified? (also, is appending an element a modification or a mutation?) """ To this question (.next on an iterator of a modified deque), my answer would be: "yes, that is a bug". However, I feel you are referring to a different issue, unfortunately, I cannot tell from your post what that issue is. Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Pydoc Improvements / Rewrite
Neal Becker wrote: > No time to review this now, but I'd just like to say that the 1 thing I'd > like to see is support for decent mathematical markup. I think at this > point that support for latex markup is the way to achieve this. There are two separate issues related to this I'd like to point out because some of the other suggestions have indicated both of these without spelling out which they are addressing. (1.) Processing existing text markup from additional text hints that are inserted into doc strings. I think this can easily be handled with a single text output point. General *post* formatters of this type are very doable I think. We just need to document the function or method to get the *raw* plain text for that purpose. (2.) Parsing and inserting additional markup where there is none, based on what and where the information came from. This is the more difficult problem. I've tried to handle this case by creating an object for each "thing", that can be extended by adding a formatting method to it. This type of markup can be very specific and may depend on context as well as what or where the source data came from. I don't know latex markup, but it seems like mathematical latex markup might be done either way. Cheers, Ron ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
