Re: [Haskell-cafe] Fast sorting with Bytestring

2008-06-18 Thread Georg Sauthoff
On Wed, Jun 18, 2008 at 08:56:43PM +0200, Ketil Malde wrote: > Stefan O'Rear <[EMAIL PROTECTED]> writes: > > GNU 'sort' uses an external sort algorithm. You can, with 200M of > > memory, give it a 50G input file, and it will work. This might explain > > the difference.. > GNU sort also handles

Re: [Haskell-cafe] Fast sorting with Bytestring

2008-06-18 Thread Ketil Malde
Stefan O'Rear <[EMAIL PROTECTED]> writes: >> Ok, strane ... Well, let's test with some 'normal' text: >> >> time ./sort < bible > /dev/null # ~ 0.4 s >> time sort < bible > /dev/null # ~ 0.56 s >> Ok, not that different. But with Haskell you often expect to get very >> slow co

Re: [Haskell-cafe] Fast sorting with Bytestring

2008-06-18 Thread Georg Sauthoff
On Wed, Jun 18, 2008 at 11:23:00AM -0700, Stefan O'Rear wrote: > GNU 'sort' uses an external sort algorithm. You can, with 200M of > memory, give it a 50G input file, and it will work. This might explain > the difference.. But the input files are both < 10 mb ... If I create a 'big_bible' file

Re: [Haskell-cafe] Fast sorting with Bytestring

2008-06-18 Thread Stefan O'Rear
On Wed, Jun 18, 2008 at 08:19:10PM +0200, Georg Sauthoff wrote: > Hi, > > I played a bit around with the nice bytestring package. At some point I > implemented a simple sorting program, because I needed line-sorting a file > with a custom line-compare function. I was a bit surprised, that the > re

[Haskell-cafe] Fast sorting with Bytestring

2008-06-18 Thread Georg Sauthoff
Hi, I played a bit around with the nice bytestring package. At some point I implemented a simple sorting program, because I needed line-sorting a file with a custom line-compare function. I was a bit surprised, that the resulting code is very fast. A lot of faster than sorting via GNU sort (with t