Re: with-open pattern

Hey Brian, The fact that a reducible (as I have implemented it in the reducible-stream library) will fully realize itself if it is used as a sequence is a drawback that I'm not happy with. In the back of my mind I believe there may be a way around it, but I'm not sure, and it is still a topic of

Re: with-open pattern

Maybe I don't understand the problem at hand yet -- you can almost certainly do this without language help. Can you help out with a concrete example or problematic code? On Friday, May 5, 2017 at 2:18:17 PM UTC-4, Brian Craft wrote: > > This looks like a partial solution: > > https://github.co

Re: with-open pattern

This looks like a partial solution: https://github.com/pjstadig/scopes perhaps inspired by this discussion: https://dev.clojure.org/display/design/Resource+Scopes On Friday, May 5, 2017 at 5:10:27 AM UTC-7, Herwig Hochleitner wrote: > > 2017-05-04 19:35 GMT+02:00 Brian Craft >: >> >> If there

Re: with-open pattern

2017-05-04 19:35 GMT+02:00 Brian Craft : > > If there's only one with-open it can be reasonably simple to pass the > consumer into that context (though from ring it's pretty convoluted, due to > needing to pass control to ring first). > Creating streamed ring responses can be kind of tricky. In my

Re: with-open pattern

Hi Brian, Both plain seqs or reducibles can be consumed without realizing the whole thing in memory, especially important (as you note) when the whole thing is humongous and dwarfs available memory. To do it with a lazy seq, as in clojure.java.jdbc, you have to care of a few things. You must ov

Re: with-open pattern

It's definitely the same problem, but I don't think it helps me. This part, in particular: "If you treat this object like a sequence, it will fully consume the input stream and fully realize the decoded data in memory." I'm specifically trying to avoid realizing the full collection in memory,

Re: with-open pattern

I think the “reducible streams” approach described by Paul Stadig here has potential. It might not cover all of the scenarios you’re thinking of, though. On Thursday, May 4, 2017 at 1:35:48 PM UTC-4, Brian Craft wrote: > > The with-open st

with-open pattern

The with-open style is used a lot in the jdbc lib, and elsewhere. It's pretty simple when data is very small, as you can just evaluate the entire result with doall, etc. How do you deal with larger data, where you need to evaluate iteratively? If there's only one with-open it can be reasonably