Benchmark attached. It just enumerates a list until EOF is reached. An interesting thing I've noticed is that IterateeMCPS performs better with no optimization, but -O2 gives IterateeM the advantage. Their relative performance depends heavily on the chunk size -- for example, CPS is much faster at chunk size 1, but slower with 100-element chunks.
On Fri, Feb 5, 2010 at 08:56, John Lato <[email protected]> wrote: > On Fri, Feb 5, 2010 at 4:31 PM, Valery V. Vorotyntsev > <[email protected]> wrote: >>> John Lato <[email protected]> wrote: >>> >>>> Both designs appear to offer similar performance in aggregate, >>>> although there are differences for particular functions. I haven't >>>> yet had a chance to test the performance of the CPS variant, although >>>> Oleg has indicated he expects it will be higher. >> >> @jwlato: >> Do you mind creating `IterateeCPS' tree in >> <http://inmachina.net/~jwlato/haskell/iteratee/src/Data/>, so we can >> start writing CPS performance testing code? > > I'm working on the CPS version and will make it public when it's done. > It may take a week or so; this term started at 90 and has picked up. > I have several benchmark sources that aren't public yet, but I can put > them online for your perusal. > >> >> AFAICS, you have benchmarks for IterateeM-driven code already: >> http://inmachina.net/~jwlato/haskell/iteratee/tests/benchmarks.hs > > Those will make more sense when I've added the context of the > codebases in use. There are several more sets of output that I simply > haven't published yet, including bytestring-based variants. > >> >> John Millikin <[email protected]> wrote: >> >>> I wrote some criterion benchmarks for IterateeM vs IterateeCPS, and >>> the CPS version was notably slower. I don't understand enough about >>> CPS to diagnose why, but the additional runtime was present in even >>> simple cases (reading from a file, writing back out). > > That's very interesting. I wonder if I'll see the same, and if I'd be > able to figure it out myself... > > Did you benchmark any cases without doing IO? Sometimes the cost of > the IO can overwhelm any other measurable differences, and also disk > caching can affect results. Criterion should highlight any major > outliers, but I still like to avoid IO when benchmarking unless > strictly necessary. > >> >> @jmillikin: >> Could you please publish those benchmarks? > > +1 > > John >
-- Benchmark for Oleg Kiselyov's iteratees. You will need: -- -- http://okmij.org/ftp/Haskell/Iteratee/LowLevelIO.hs -- http://okmij.org/ftp/Haskell/Iteratee/IterateeM.hs -- http://okmij.org/ftp/Haskell/Iteratee/IterateeMCPS.hs -- -- -- ghc --make -O2 benchmark.hs import Criterion.Main import Criterion.Config import qualified IterateeM as M import qualified IterateeMCPS as CPS import qualified Control.Monad.Identity as I config = defaultConfig { cfgPerformGC = ljust True } enumM :: Monad m => Int -> Int -> M.IterV Int m a -> M.Iteratee Int m a enumM n = M.enum_pure_nchunk [0..n] enumCPS :: Monad m => Int -> Int -> CPS.Iteratee Int m a -> m (CPS.Iteratee Int m a) enumCPS n = CPS.enum_pure_nchunk [0..n] runM :: (M.IterV el I.Identity () -> M.Iteratee el' I.Identity a) -> a runM enum = I.runIdentity . M.run $ M.skip_till_eof M.>>== enum runCPS :: Monad m => (CPS.Iteratee el m () -> CPS.Iteratee el' I.Identity a) -> a runCPS enum = I.runIdentity . CPS.run $ enum CPS.skip_till_eof main :: IO () main = defaultMainWith config (return ()) [ bgroup "IterateeM" [ bench "100/10" $ whnf runM $ enumM 100 10 , bench "200/10" $ whnf runM $ enumM 200 10 , bench "300/10" $ whnf runM $ enumM 300 10 ] , bgroup "IterateeCPS" [ bench "100/10" $ whnf runCPS $ enumCPS 100 10 , bench "200/10" $ whnf runCPS $ enumCPS 200 10 , bench "300/10" $ whnf runCPS $ enumCPS 300 10 ] ]
_______________________________________________ Haskell-Cafe mailing list [email protected] http://www.haskell.org/mailman/listinfo/haskell-cafe
