Re: [racket-users] Performance of the Racket reader

2017-06-01 Thread Steve Byan's Lists
> On Jun 1, 2017, at 10:52 AM, Steve Byan's Lists > wrote: > > Hi Jon, > >> On May 31, 2017, at 6:41 PM, Steve Byan's Lists >> wrote: >> >>> On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: >>> >>> >>> This way, you don't build up a list or a lazy stream; you just process >>> each datum

Re: [racket-users] Performance of the Racket reader

2017-06-01 Thread Jon Zeppieri
On Thu, Jun 1, 2017 at 11:07 AM, Jon Zeppieri wrote: > > >> On Jun 1, 2017, at 10:52 AM, Steve Byan's Lists >> wrote: >> >> Hi Jon, >> On May 31, 2017, at 6:41 PM, Steve Byan's Lists wrote: On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: So, for example: >>

Re: [racket-users] Performance of the Racket reader

2017-06-01 Thread Jon Zeppieri
> On Jun 1, 2017, at 10:52 AM, Steve Byan's Lists > wrote: > > Hi Jon, > >>> On May 31, 2017, at 6:41 PM, Steve Byan's Lists >>> wrote: >>> >>> On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: >>> >>> So, for example: >>> >>> (define (map-trace stat%-set in-port) >>> (for/fold ([sexp-co

Re: [racket-users] Performance of the Racket reader

2017-06-01 Thread Steve Byan's Lists
Hi Jon, > On May 31, 2017, at 6:41 PM, Steve Byan's Lists > wrote: > >> On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: >> >> So, for example: >> >> (define (map-trace stat%-set in-port) >> (for/fold ([sexp-count 0]) >> ([trace-record (in-port read in)]) >> (+ sexp-count 1))) >>

Re: [racket-users] Performance of the Racket reader

2017-06-01 Thread Steve Byan's Lists
> On Jun 1, 2017, at 12:25 AM, Neil Van Dyke wrote: > > Steve Byan's Lists wrote on 05/31/2017 10:05 PM: >> I'd appreciate a short example of what you mean by using `apply` and >> `lambda` to destructure the list. > > I'll babble more than you want here, in case anyone on the list is curious

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Neil Van Dyke
Steve Byan's Lists wrote on 05/31/2017 10:05 PM: I'd appreciate a short example of what you mean by using `apply` and `lambda` to destructure the list. I'll babble more than you want here, in case anyone on the list is curious in general... #lang racket/base (define (do-something-with-pmem

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Jon Zeppieri
On Wed, May 31, 2017 at 10:05 PM, Steve Byan's Lists wrote: > > I did consider using an association list representation for the attributes, > but I'm depending on s-expression pattern matching for parsing the records. > It's wonderfully convenient for this. I'm under the impression that `match`

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Steve Byan's Lists
Hi Neil, Thanks for the comments. > On May 31, 2017, at 8:21 PM, Neil Van Dyke wrote: > > In addition to what others have mentioned, at this scale, you might get > significant gains by adjusting your s-expression language. > > For example, instead of this: > > (pmem_flush > (threadId 1403390

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Neil Van Dyke
In addition to what others have mentioned, at this scale, you might get significant gains by adjusting your s-expression language. For example, instead of this: (pmem_flush (threadId 140339047277632) (startTime 923983542377819) (elapsedTime 160) (result 0) (addr 0x7fa239055954) (length 1)

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Steve Byan's Lists
> On May 31, 2017, at 6:32 PM, Matthias Felleisen wrote: > >> >> On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: >> >> >> This way, you don't build up a list or a lazy stream; you just process >> each datum as it's read. > > > Yes, that’s what I would have proposed next. Just fuse the two

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Steve Byan's Lists
Hi Jon, > On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: > > On Wed, May 31, 2017 at 5:54 PM, Steve Byan's Lists > wrote: >> So, I don't want to try to fit all the records in memory at once. I thought >> that the lazy stream would accomplish this --- am I wrong? > > You're right, but using

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Matthias Felleisen
> On May 31, 2017, at 6:14 PM, Jon Zeppieri wrote: > > On Wed, May 31, 2017 at 5:54 PM, Steve Byan's Lists > wrote: >> Hi Mathias, >> >> Thanks for taking a look. >> >>> On May 31, 2017, at 4:13 PM, Matthias Felleisen >>> wrote: >>> >>> >>> Can you explain why you create a lazy stream ins

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Jon Zeppieri
On Wed, May 31, 2017 at 5:54 PM, Steve Byan's Lists wrote: > Hi Mathias, > > Thanks for taking a look. > >> On May 31, 2017, at 4:13 PM, Matthias Felleisen wrote: >> >> >> Can you explain why you create a lazy stream instead of a plain list? > > The current size of a short binary trace file is ab

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Steve Byan's Lists
Hi Mathias, Thanks for taking a look. > On May 31, 2017, at 4:13 PM, Matthias Felleisen wrote: > > > Can you explain why you create a lazy stream instead of a plain list? The current size of a short binary trace file is about 10 GB, and I want to scale to traces many hundreds of megabytes in

Re: [racket-users] Performance of the Racket reader

2017-05-31 Thread Matthias Felleisen
Can you explain why you create a lazy stream instead of a plain list? Your code is strict in the stream so the extra memory for a stream is substantial and probably wasted. Perhaps the below is simplified and you really need only portions of the lazy stream. — Matthias > On May 31, 2017, at