Is this the sort of situation that continuation barriers were made for?
Do you have any guidance about using them?
#lang racket
(define (only-once thunk)
(dynamic-wind
(? () (displayln "pre-thunk"))
(? () (call-with-continuation-barrier thunk))
(? () (displayln "post-thunk"))))
(only-once (? () (displayln "hi")))
(let ([saved-k #f])
(only-once (? () (let/cc k (set! saved-k k)
(displayln "saving continuation"))))
(displayln "invoking continuation...")
(saved-k 11))
Thanks,
Dave
On 08/08/2013 12:10 PM, Robby Findler wrote:
As to the interactions with dynamic-wind and continuations: l did, as
you figured out, intend for you to use only the post-thunk in the
dynamic-wind (to close the pipes). In principle, you could use the
pre-thurnk to try to restore the pipe, but there really isn't enough
information to do this correctly in all cases.
I see that you've protected things a little bit by using the
port-closed? predicate as a guard, but if you did that to protect
against possible continuation re-entry, then probably you're better
off adding something to the pre-thunk that explicitly raises an error
saying that it isn't allowed to re-enter. Something like this:
#lang racket
(define (only-once thunk)
(define already-in-once? #f)
(dynamic-wind
(? ()
(when already-in-once? (error 'only-once "no no")))
(? ()
(set! already-in-once? #t)
(thunk))
void))
(only-once (? () "hi"))
(let ([saved-k #f])
(only-once (? () (let/cc k (set! saved-k k))))
(saved-k 11))
On Tue, Aug 6, 2013 at 11:16 AM, JP Verkamp <rac...@jverkamp.com
<mailto:rac...@jverkamp.com>> wrote:
I've never actually used dynamic-wind, although it does look
interesting / like what I need. A few questions / caveats though:
- Should the pipe be created in the pre-thunk or before the
dynamic-wind entirely? The thunks don't seem to share scope, so
I'm guessing the latter, but that seems a bit odd. I'm guessing
the pre-thunk is for an entirely different use case though when
you are actually dealing with closing and reopening resources are
the like as control gets passed around.
- Doesn't dynamic-wind break if the user messes with continuations
during the value-thunk? So far as I understand, when control
passes out, post-thunk is called and then pre-thunk on the way
back in, but that means that when control returns the port will be
closed. I don't know how often this will come up, but it seems to
break if I nest a thread inside of the with-gzip call. Granted, my
version did as well because of the close-input-port call. Is this
just expected behavior?
(And yes, it works fine in the more likely / sensible case of
wrapping the entire with-gzip in a thread in both cases.)
- So far as error rather than raise, raise was my original guess.
But that added another layer of indirection to the stack trace
which I didn't at first notice (I thought I wasn't even catching
the error). It makes sense to have that though in the long run.
That all being said, how does this version look?
(define (with-gunzip thunk)
(define-values (pipe-from pipe-to) (make-pipe))
(dynamic-wind
void
(? ()
(gunzip-through-ports (current-input-port) pipe-to)
(close-output-port pipe-to)
(parameterize ([current-input-port pipe-from])
(thunk)))
(? ()
(unless (port-closed? pipe-to) (close-output-port pipe-to))
(unless (port-closed? pipe-from) (close-input-port pipe-from)))))
On Tue, Aug 6, 2013 at 11:47 AM, Robby Findler
<ro...@eecs.northwestern.edu <mailto:ro...@eecs.northwestern.edu>>
wrote:
You might consider using dynamic-wind instead of that
with-handlers. Or, instead of (error 'with-gunzip ...) just do
(raise exn). That way you won't lose the stack information in
the original exception (which is likely the one a user would
want).
Robby
On Tue, Aug 6, 2013 at 10:40 AM, JP Verkamp
<rac...@jverkamp.com <mailto:rac...@jverkamp.com>> wrote:
Figured it out and cleaned it up. It turns out that I was
using with-handlers oddly, but reading further though the
documentation it works as expected. Here's a new version
(generalized to any input-port):
(define (with-gunzip thunk)
(define-values (pipe-from pipe-to) (make-pipe))
(with-handlers ([exn:fail?
(? (err)
(close-output-port pipe-to)
(close-input-port pipe-from)
(error 'with-gunzip (exn-message err)))])
(gunzip-through-ports (current-input-port) pipe-to)
(close-output-port pipe-to)
(parameterize ([current-input-port pipe-from])
(thunk))
(close-input-port pipe-from)))
If anyone's interested in a more in depth write up /
source code for this and with-gzip:
- writeup:
http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/
- source:
https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt
On Mon, Aug 5, 2013 at 5:36 PM, JP Verkamp
<rac...@jverkamp.com <mailto:rac...@jverkamp.com>> wrote:
Thanks! make-pipe isn't something that I've had to use
otherwise, so I missed the optional parameter. That
does certainly seem to help.
Here's my first take of with-input-from-gzipped-file:
(define (with-input-from-gzipped-file filename thunk
#:buffer-size [buffer-size #f])
(call-with-input-file filename
(lambda (file-from)
(define-values (pipe-from pipe-to) (make-pipe
buffer-size))
(thread
(? ()
(gunzip-through-ports file-from pipe-to)
(close-output-port pipe-to)))
(current-input-port pipe-from)
(thunk)
(close-input-port pipe-from))))
The main thing missing is that there's no error
handling (where the pipe should still be closed). At
the very least, if I try to call this on a non-gzipped
file, it breaks on the gunzip-through-ports line.
Theoretically, some variation of with-handlers should
work (error should raise an exn:fail?, yes?), but it
doesn't seem to be helping.
Any help with that?
Alternatively, I've now found this:
http://planet.racket-lang.org/display.ss?package=gzip.plt&owner=soegaard
It seems to do exactly what I need, albeit without the
call-with-* forms, but that's easy enough to wrap.
With some very basic testing, it does seem to be
buffering though, although it is a bit slower than the
above. Not enough to cause trouble though.
On Mon, Aug 5, 2013 at 4:51 PM, Ryan Culpepper
<ry...@ccs.neu.edu <mailto:ry...@ccs.neu.edu>> wrote:
On 08/05/2013 04:29 PM, JP Verkamp wrote:
Is there a nice / idiomatic way to work with
gzipped data in a streaming
manner (to avoid loading the rather large
files into memory at once). So
far as I can tell, my code isn't doing that.
It hangs for a while on the
call to gunzip-through-ports, long enough to
uncompress the entire file,
then reads are pretty quick afterwords.
Here's what I have thus far:
#lang racket
(require file/gunzip)
(define-values (pipe-from pipe-to) (make-pipe))
(with-input-from-file "test.rkt.gz"
(lambda ()
(gunzip-through-ports (current-input-port)
pipe-to)
(for ([line (in-lines pipe-from)])
(displayln line))))
You should probably 1) limit the size of the pipe
(to stop it from inflating the whole file at once)
and 2) put the gunzip-through-ports call in a
separate thread. The gunzip thread will block when
the pipe is full; when your program reads some
data out of the pipe, the gunzip thread will be
able to make some more progress. Something like this:
(define-values (pipe-from pipe-to) (make-pipe 4000))
(with-input-from-file "test.rkt.gz"
(lambda ()
(thread
(lambda ()
(gunzip-through-ports (current-input-port) pipe-to)
(close-output-port pipe-to)))
(for ([line (in-lines pipe-from)])
(displayln line))))
As an additional problem, that code doesn't
actually work.
in-lines seems to be waiting for an
eof-object? that
gunzip-through-ports isn't sending. Am I
missing something? It ends up
just hanging after reading and printing the file.
The docs don't say anything about closing the
port, so you'll probably have to do that yourself.
In the code above, I added a call to
close-output-port.
Ryan
____________________
Racket Users list:
http://lists.racket-lang.org/users
____________________
Racket Users list:
http://lists.racket-lang.org/users
____________________
Racket Users list:
http://lists.racket-lang.org/users
____________________
Racket Users list:
http://lists.racket-lang.org/users
