Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-28 Thread Steven D'Aprano
On Sun, 28 Mar 2010 06:48:21 +1100, Lie Ryan wrote: > On 03/22/2010 07:07 PM, Steven D'Aprano wrote: >> Perhaps you should have said that it was a wrapper around deque giving >> richer functionality, rather than giving the impression that it was a >> brand new data structure invented by you. Peopl

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-27 Thread Lie Ryan
On 03/22/2010 07:07 PM, Steven D'Aprano wrote: > Perhaps you should have said that it was a wrapper around deque giving > richer functionality, rather than giving the impression that it was a > brand new data structure invented by you. People are naturally going to > be more skeptical about a ne

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-23 Thread Zooko O'Whielacronx
My apologies; I left out the heading on the last of the four structures in the benchmark results. Here are those results again with the missing heading (Stringy) inserted: Regards, Zooko - Hide quoted text - On Sun, Mar 21, 2010 at 11:09 PM, Zooko O'Whielacronx wrote: > > impl: StringChain > t

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-22 Thread Zooko O'Whielacronx
On Mon, Mar 22, 2010 at 2:07 AM, Steven D'Aprano wrote: > > Perhaps you should have said that it was a wrapper around deque giving > richer functionality, rather than giving the impression that it was a > brand new data structure invented by you. People are naturally going to > be more skeptical a

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-22 Thread Steven D'Aprano
On Sun, 21 Mar 2010 23:09:46 -0600, Zooko O'Whielacronx wrote: > But the use case that I am talking about is where you need to accumulate > new incoming strings into your buffer while alternately processing > leading prefixes of the buffer. [...] > Below are the abbreviated results of the benchmar

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-21 Thread Zooko O'Whielacronx
Folks: I failed to make something sufficiently clear in my original message about StringChain. The use case that I am talking about is not simply that you need to accumulate a sequence of incoming chunks of data, concatenate them together, and then process the entire result. If that is all you nee

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-12 Thread Steven D'Aprano
On Fri, 12 Mar 2010 13:40:23 +, MRAB wrote: >> To be taken seriously, I think you need to compare stringchain to the >> list idiom. If your benchmarks favourably compare to that, then it >> might be worthwhile. >> > IIRC, someone did some work on making concatenation faster by delaying > it u

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-12 Thread MRAB
Steven D'Aprano wrote: On Fri, 12 Mar 2010 00:11:37 -0700, Zooko O'Whielacronx wrote: Folks: Every couple of years I run into a problem where some Python code that worked well at small scales starts burning up my CPU at larger scales, and the underlying issue turns out to be the idiom of accum

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-12 Thread Steven D'Aprano
On Fri, 12 Mar 2010 00:11:37 -0700, Zooko O'Whielacronx wrote: > Folks: > > Every couple of years I run into a problem where some Python code that > worked well at small scales starts burning up my CPU at larger scales, > and the underlying issue turns out to be the idiom of accumulating data > b

Re: StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-11 Thread Paul Rubin
"Zooko O'Whielacronx" writes: > Every couple of years I run into a problem where some Python code that > worked well at small scales starts burning up my CPU at larger scales, > and the underlying issue turns out to be the idiom of accumulating > data by string concatenation. I usually use Strin

StringChain -- a data structure for managing large sequences of chunks of bytes

2010-03-11 Thread Zooko O'Whielacronx
Folks: Every couple of years I run into a problem where some Python code that worked well at small scales starts burning up my CPU at larger scales, and the underlying issue turns out to be the idiom of accumulating data by string concatenation. It just happened again (http://foolscap.lothar.com/t