Peter, Appreciated. I wrote something like this in another message before reading yours. Indeed one of the things I found was the deque class in the collections module.
But I was not immediately clear on whether that would be directly applicable. Their maximum sounded like if you exceeded it, it might either reject the addition or throw an error. The behavior I wanted was sort of a sliding window protocol where the oldest entry scrolled off the screen or was simply removed. Sort of like what you might do with a moving average that takes the average of just the last 20 days of a stock price. But I searched some more and stand corrected. "New in version 2.4. If maxlen is not specified or is None, deques may grow to an arbitrary length. Otherwise, the deque is bounded to the specified maximum length. Once a bounded length deque is full, when new items are added, a corresponding number of items are discarded from the opposite end. Bounded length deques provide functionality similar to the tail filter in Unix. They are also useful for tracking transactions and other pools of data where only the most recent activity is of interest." That sounds exactly like what is needed. As long as you keep adding at the end (using the append method) it should eventually remove from the beginning automatically. No need to use count and selectively remove or pop manually. And despite all the additional functionality, I suspect it is tuned and perhaps has parts written in C++ for added speed. I do note that any larger log file used in the application discussed may throw things on the deque many times but only ask it to display rarely so the former should be optimized. But one question before I go, Columbo style. The manual page (https://docs.python.org/2/library/collections.html ) suggest you call deque with an iterator. That would not necessarily meet our need as giving it the entire file as an iterator would just grind away without any logic and keep just the last N lines. We could arrange the logic in our own iterator, such as a function that reads a line at a time using its own open iterator and yields the line over but that too is problematic as to how and when you stop and print the results. But on second look, the iterator is optional and I tried creating a deque using just a maxlen=3 argument for illustration. >>> from collections import deque >>> a=deque(maxlen=3) >>> a deque([], maxlen=3) >>> a.append('line 1\n') >>> a deque(['line 1\n'], maxlen=3) >>> a.append('line 2\n') >>> a.append('line 3\n') >>> a deque(['line 1\n', 'line 2\n', 'line 3\n'], maxlen=3) >>> a.append('line N\n') >>> a deque(['line 2\n', 'line 3\n', 'line N\n'], maxlen=3) OK, that looks right so all you need to figure out is how to print it in a format you want. As it happens, deque has an str and a repr that seem the same when I try to print: >>> a.__str__() "deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)" >>> a.__repr__() "deque(['line 2\\n', 'line 3\\n', 'line N\\n'], maxlen=3)" So you either need to subclass deque to get your own printable version (or use an amazing number of other Python tricks since you can, or do something manually. >>> for line in a: print(line) line 2 line 3 line N OK, that works but my \n characters at the end of some items might suggest using end='' in the 3.X version of print for a smaller display. Summary: the method Peter mentions is a decent solution with no programming or debugging overhead. It is even flexible enough, if you choose, to store or display the lines backwards as in showing the last line that showed the error, followed by successively earlier lines. Why use a limited solution when you can play with a full deck? -----Original Message----- From: Tutor <tutor-bounces+avigross=verizon....@python.org> On Behalf Of Peter Otten Sent: Sunday, November 11, 2018 2:43 PM To: tutor@python.org Subject: Re: [Tutor] saveLine Avi Gross wrote: > Alan and others have answered the questions posed and what I am asking > now is to look at the function he proposed to keep track of the last > five lines. > > There is nothing wrong with it but I wonder what alternatives people > would prefer. His code is made for exactly 5 lines to be buffered and > is quite efficient. But what if you wanted N lines buffered, perhaps > showing a smaller number of lines on some warnings or errors and the > full N in other cases? The standard library features collections.deque. With that: buffer = collections.deque(maxlen=N) save_line = buffer.append This will start with an empty buffer. To preload the buffer: buffer = collections.deque(itertools.repeat("", N), maxlen=N) To print the buffer: print_buffer = sys.stdout.writelines or, more general: def print_buffer(items, end=""): for item in items: print(item, end=end) Also, for smallish N: def print_buffer(items, end=""): print(*items, sep=end) _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
