You might consider using dynamic-wind instead of that with-handlers. Or, instead of (error 'with-gunzip ...) just do (raise exn). That way you won't lose the stack information in the original exception (which is likely the one a user would want).
Robby On Tue, Aug 6, 2013 at 10:40 AM, JP Verkamp <rac...@jverkamp.com> wrote: > Figured it out and cleaned it up. It turns out that I was using > with-handlers oddly, but reading further though the documentation it > works as expected. Here's a new version (generalized to any input-port): > > (define (with-gunzip thunk) > (define-values (pipe-from pipe-to) (make-pipe)) > (with-handlers ([exn:fail? > (λ (err) > (close-output-port pipe-to) > (close-input-port pipe-from) > (error 'with-gunzip (exn-message err)))]) > (gunzip-through-ports (current-input-port) pipe-to) > (close-output-port pipe-to) > (parameterize ([current-input-port pipe-from]) > (thunk)) > (close-input-port pipe-from))) > > If anyone's interested in a more in depth write up / source code for this > and with-gzip: > - writeup: http://blog.jverkamp.com/2013/08/06/adventures-in-racket-gzip/ > - source: > https://github.com/jpverkamp/small-projects/tree/master/blog/with-gzip.rkt > > > On Mon, Aug 5, 2013 at 5:36 PM, JP Verkamp <rac...@jverkamp.com> wrote: > >> Thanks! make-pipe isn't something that I've had to use otherwise, so I >> missed the optional parameter. That does certainly seem to help. >> >> Here's my first take of with-input-from-gzipped-file: >> >> (define (with-input-from-gzipped-file filename thunk #:buffer-size >> [buffer-size #f]) >> (call-with-input-file filename >> (lambda (file-from) >> (define-values (pipe-from pipe-to) (make-pipe buffer-size)) >> >> (thread >> (λ () >> (gunzip-through-ports file-from pipe-to) >> (close-output-port pipe-to))) >> >> (current-input-port pipe-from) >> (thunk) >> (close-input-port pipe-from)))) >> >> The main thing missing is that there's no error handling (where the pipe >> should still be closed). At the very least, if I try to call this on a >> non-gzipped file, it breaks on the gunzip-through-ports line. >> Theoretically, some variation of with-handlers should work (error should >> raise an exn:fail?, yes?), but it doesn't seem to be helping. >> >> Any help with that? >> >> Alternatively, I've now found this: >> http://planet.racket-lang.org/display.ss?package=gzip.plt&owner=soegaard >> >> It seems to do exactly what I need, albeit without the call-with-* forms, >> but that's easy enough to wrap. With some very basic testing, it does seem >> to be buffering though, although it is a bit slower than the above. Not >> enough to cause trouble though. >> >> >> On Mon, Aug 5, 2013 at 4:51 PM, Ryan Culpepper <ry...@ccs.neu.edu> wrote: >> >>> On 08/05/2013 04:29 PM, JP Verkamp wrote: >>> >>>> Is there a nice / idiomatic way to work with gzipped data in a streaming >>>> manner (to avoid loading the rather large files into memory at once). So >>>> far as I can tell, my code isn't doing that. It hangs for a while on the >>>> call to gunzip-through-ports, long enough to uncompress the entire file, >>>> then reads are pretty quick afterwords. >>>> >>>> Here's what I have thus far: >>>> >>>> #lang racket >>>> >>>> (require file/gunzip) >>>> >>>> (define-values (pipe-from pipe-to) (make-pipe)) >>>> (with-input-from-file "test.rkt.gz" >>>> (lambda () >>>> (gunzip-through-ports (current-input-port) pipe-to) >>>> (for ([line (in-lines pipe-from)]) >>>> (displayln line)))) >>>> >>> >>> You should probably 1) limit the size of the pipe (to stop it from >>> inflating the whole file at once) and 2) put the gunzip-through-ports call >>> in a separate thread. The gunzip thread will block when the pipe is full; >>> when your program reads some data out of the pipe, the gunzip thread will >>> be able to make some more progress. Something like this: >>> >>> (define-values (pipe-from pipe-to) (make-pipe 4000)) >>> (with-input-from-file "test.rkt.gz" >>> (lambda () >>> (thread >>> >>> (lambda () >>> (gunzip-through-ports (current-input-port) pipe-to) >>> (close-output-port pipe-to))) >>> >>> (for ([line (in-lines pipe-from)]) >>> (displayln line)))) >>> >>> As an additional problem, that code doesn't actually work. >>>> in-lines seems to be waiting for an eof-object? that >>>> gunzip-through-ports isn't sending. Am I missing something? It ends up >>>> just hanging after reading and printing the file. >>>> >>> >>> The docs don't say anything about closing the port, so you'll probably >>> have to do that yourself. In the code above, I added a call to >>> close-output-port. >>> >>> Ryan >>> >>> >> > > ____________________ > Racket Users list: > http://lists.racket-lang.org/users > >
____________________ Racket Users list: http://lists.racket-lang.org/users
