Re: Stubbornly eager results in clojure.java.jdbc

> On Jun 23, 2017, at 2:26 PM, Sean Corfield wrote: > > This is excellent news as far as I’m concerned because it shows there’s no > specific bug in clojure.java.jdbc that is fundamentally causing the OOM > problem you’re seeing! I'm relieved too, given that I use clojure.java.jdbc extensivel

Re: Stubbornly eager results in clojure.java.jdbc

This is excellent news as far as I’m concerned because it shows there’s no specific bug in clojure.java.jdbc that is fundamentally causing the OOM problem you’re seeing! (that’s not to say there aren’t _other_ bugs in clojure.java.jdbc and the idea of the reducible result set definitely has

Re: Stubbornly eager results in clojure.java.jdbc

Well, shoot. I went back and revisited this because it was bugging me … I looked at the code generated with and without usage of ^:once fn* and that led me down the right path. TL;DR – the problem appears to be locals clearing always disabled in Cursive REPL. This is just my current hypothesis

Re: Stubbornly eager results in clojure.java.jdbc

Postgres, as mentioned in the mail and the linked source code. The problem at this point doesn't appear to be options given to the driver, since I show two implementations using the same driver options. One processes the results lazily, one does not. Now, I'm calling two different methods in

Re: Stubbornly eager results in clojure.java.jdbc

Hi Luke, which database are you using? I had the same issue with MySQL recently. At the end I got it working with clojure.java.jdbc. I don't have the code at hand, but according to the MySQL docs you have to set the fetch size to Integer.MIN_VALUE. https://dev.mysql.com/doc/connector-j/5.1/en/c

Re: Stubbornly eager results in clojure.java.jdbc

^{:once true} should deal with this. Try the following simplified test case with and without it: (defn query [result-fn] (let [x (for [n (range 1e6)] (make-array Object 10)) f (^:once fn* [rs] (result-fn rs))] (f x))) (defn testq [] (let [myfn (fn [rs] (doseq [r rs] nil))]

Re: Stubbornly eager results in clojure.java.jdbc

This might be a bug in java.jdbc. The code that passes the result set seq to the function is: (^{:once true} fn* [rset] ((^{:once true} fn* [rs] (result-set-fn (if as-arrays? (cons (first rs) (map row-fn (rest rs))) (ma

Re: Stubbornly eager results in clojure.java.jdbc

Anyone have any insights here? Really the most important thing I'm trying to learn is 2) how to identify when a lazy seq head is being retained, other than waiting for it to become bad enough that your program OOMs. > On Jun 16, 2017, at 6:14 PM, Luke Burton wrote: > > > Riddle me this: >

Stubbornly eager results in clojure.java.jdbc

Riddle me this: https://gist.github.com/hagmonk/a75621b143501966c22f53ed1e2bc36e Wherein I synthesize a large table in Postgres, then attempt to lazily load the table, discarding each row as I receive it. I tried *many* permuta