Re: [Haskell-cafe] Increasing memory use in stream computation

2013-10-14 Thread Arie Peterson
Hi Claude, On Thursday 10 October 2013 20:05:37 I wrote: > Although, maybe I can do all the logic of the "small" function in the list > monad, and stream the resulting list, as you do in the above. I tried a corresponding variant of my full program, but the memory use is quite a lot higher at

Re: [Haskell-cafe] Increasing memory use in stream computation

2013-10-10 Thread Arie Peterson
Hi Bertram, > Unfortunately, I don't know. I'll intersperse some remarks and > propose an alternative to stream fusion at the end, which allows > your test program to run in constant space. > > > A quicker way to spot the increased memory usage is to look at GC > statistics. I used > > > ./Tes

Re: [Haskell-cafe] Increasing memory use in stream computation

2013-10-10 Thread Arie Peterson
Hi Claude, > Looking at the heap profile graph (generated with +RTS -h, no need to > compile with profiling) I see the increasing memory use is split about > evenly between STACK and BLACKHOLE. I don't know what that means or why > it occurs, but replacing `small` solved that problem for me: >

Re: [Haskell-cafe] Increasing memory use in stream computation

2013-10-10 Thread Bertram Felgenhauer
Arie Peterson wrote: > (Sorry for the long email.) > > Summary: why does the attached program have non-constant memory use? Unfortunately, I don't know. I'll intersperse some remarks and propose an alternative to stream fusion at the end, which allows your test program to run in constant space.

Re: [Haskell-cafe] Increasing memory use in stream computation

2013-10-10 Thread Claude Heiland-Allen
Hi Arie, On 10/10/13 14:02, Arie Peterson wrote: > (Sorry for the long email.) > > Summary: why does the attached program have non-constant memory use? Looking at the heap profile graph (generated with +RTS -h, no need to compile with profiling) I see the increasing memory use is split about eve

[Haskell-cafe] Increasing memory use in stream computation

2013-10-10 Thread Arie Peterson
(Sorry for the long email.) Summary: why does the attached program have non-constant memory use? Introduction I've written a program to do a big computation. Unfortunately, the computation takes a very long time (expectedly), and the memory use increases slowly (unexpectedly), until