After patching both transaction and with-connection (which is also used by korma.db/transaction, and also creates a closure), I can pass in the seq w/o leaking.
Thanks, Christophe. On Thursday, September 12, 2013 11:20:11 AM UTC-7, Brian Craft wrote: > > or perhaps it's that macroexpand drops the ^:once > > On Thursday, September 12, 2013 10:55:28 AM UTC-7, Brian Craft wrote: >> >> I think my monkey-patch of jdbc/transaction didn't take. Doing >> user/sourcery on it shows your version, but doing a macroexpand-all shows >> the original. >> >> My code calls via korma as kdb: >> >> => (clojure.walk/macroexpand-all '(kdb/transaction nil)) >> (if (clojure.java.jdbc/find-connection) (clojure.java.jdbc/transaction* >> (fn* [] nil)) (clojure.java.jdbc/with-connection* (korma.db/get-connection >> (clojure.core/deref korma.db/_default)) (fn* ([] >> (clojure.java.jdbc/transaction* (fn* [] nil)))))) >> >> I thought I could get it to take by doing a (require 'clojure.java.jdbc) >> and applying the patch before doing a require on korma, but that doesn't >> seem to help. >> >> (require 'clojure.java.jdbc) >> ; XXX monkey-patch jdbc to avoid head retention bug >> (in-ns 'clojure.java.jdbc) >> (defmacro transaction >> [& body] >> `(transaction* (^:once fn* [] ~@body))) >> >> (ns <blahblah> (:require [korma.db as kdb])) >> >> .. and restart the repl, but macroexpand-all still gives me the result >> above. >> >> On Wednesday, September 11, 2013 1:45:41 PM UTC-7, Christophe Grand wrote: >>> >>> I don't get the same results: >>> >>> $ LEIN_JVM_OPTS=-Xmx20M lein repl >>> nREPL server started on port 61221 on host 127.0.0.1 >>> REPL-y 0.2.1 >>> Clojure 1.5.1 >>> Docs: (doc function-name-here) >>> (find-doc "part-of-name-here") >>> Source: (source function-name-here) >>> Javadoc: (javadoc java-object-or-class-here) >>> Exit: Control+D or (exit) or (quit) >>> >>> user=> (defn f [g] (g)) >>> #'user/f >>> user=> (defn t1 [n c] (f (fn [] (dorun (map identity c))))) >>> #'user/t1 >>> user=> (t1 0 (range 1000000)) >>> >>> OutOfMemoryError GC overhead limit exceeded java.lang.Long.valueOf >>> (Long.java:577) >>> user=> (defn t2 [n c] (f (fn [] (doseq [x c] (identity x))))) >>> #'user/t2 >>> user=> (defn t1 [n c] (f (^:once fn* [] (dorun (map identity c))))) >>> #'user/t1 >>> user=> (t1 0 (range 1000000)) >>> nil >>> user=> (t2 0 (range 1000000)) >>> >>> OutOfMemoryError GC overhead limit exceeded >>> clojure.lang.ChunkBuffer.chunk (ChunkBuffer.java:29) >>> >>> BUT this is because the previous OOM left the JVM in a dirty state: try >>> to reorder your expressions: >>> >>> $ LEIN_JVM_OPTS=-Xmx20M lein repl >>> nREPL server started on port 61245 on host 127.0.0.1 >>> REPL-y 0.2.1 >>> Clojure 1.5.1 >>> Docs: (doc function-name-here) >>> (find-doc "part-of-name-here") >>> Source: (source function-name-here) >>> Javadoc: (javadoc java-object-or-class-here) >>> Exit: Control+D or (exit) or (quit) >>> >>> user=> (defn f [g] (g)) >>> #'user/f >>> user=> (defn t2 [n c] (f (fn [] (doseq [x c] (identity x))))) >>> #'user/t2 >>> user=> (t2 0 (range 1000000)) >>> nil >>> >>> ^^this last one failed in the previous run. >>> >>> I'm not quite sure about why the doseq version works -- I would have to >>> research a bit. My gut feeling is that doseq is based on loop and loops are >>> lifted into ^:once fn* by the compiler in some cases. >>> https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Compiler.java#L5951 >>> >>> >>> >>> On Wed, Sep 11, 2013 at 8:20 PM, Brian Craft <craft...@gmail.com> wrote: >>> >>>> Correct, I forgot to paste that part. ;) >>>> >>>> >>>> On Wednesday, September 11, 2013 10:57:09 AM UTC-7, Sean Corfield wrote: >>>> >>>>> Just to confirm, (t2 0 (range 1000000)) -- using doseq instead of >>>>> dorun -- does NOT run out of memory, correct? >>>>> >>>>> On Wed, Sep 11, 2013 at 8:59 AM, Brian Craft <craft...@gmail.com> >>>>> wrote: >>>>> > This appears to have no effect on the problem. I tested with >>>>> > jdbc/transaction, and with Sean's simple example: >>>>> > >>>>> > user=> (defn f [g] (g)) >>>>> > #'user/f >>>>> > user=> (defn t1 [n c] (f (fn [] (dorun (map identity c))))) >>>>> > #'user/t1 >>>>> > user=> (t1 0 (range 1000000)) >>>>> > java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:0) >>>>> > user=> (defn t2 [n c] (f (fn [] (doseq [x c] (identity x))))) >>>>> > #'user/t2 >>>>> > user=> (defn t1 [n c] (f (^:once fn* [] (dorun (map identity c))))) >>>>> > #'user/t1 >>>>> > user=> (t1 0 (range 1000000)) >>>>> > java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:0) >>>>> > >>>>> > >>>>> > >>>>> > On Wednesday, September 11, 2013 7:39:48 AM UTC-7, Christophe Grand >>>>> wrote: >>>>> >> >>>>> >> ^:once on fn* (not fn, the fn macro doesn't propagate metadata) >>>>> instructs >>>>> >> the commielr to clear closed-overs ASAP. It follows that you can't >>>>> call such >>>>> >> a function twice because it forgets its closed-overs. >>>>> >> >>>>> >> >>>>> >> On Wed, Sep 11, 2013 at 4:36 PM, Brian Craft <craft...@gmail.com> >>>>> wrote: >>>>> >>> >>>>> >>> ugh. Can't find documentation for this. What does it do? >>>>> >>> >>>>> >>> On Wednesday, September 11, 2013 2:22:56 AM UTC-7, Christophe >>>>> Grand >>>>> >>> wrote: >>>>> >>>> >>>>> >>>> >>>>> >>>> On Wed, Sep 11, 2013 at 6:00 AM, Brian Craft <craft...@gmail.com> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> (defmacro transaction >>>>> >>>>> [& body] >>>>> >>>>> `(transaction* (fn [] ~@body))) >>>>> >>>>> >>>>> >>>>> I'm not sure how to avoid that. The anonymous function created >>>>> here >>>>> >>>>> doesn't take parameters. >>>>> >>>> >>>>> >>>> >>>>> >>>> The fix for this is: >>>>> >>>> (defmacro transaction >>>>> >>>> [& body] >>>>> >>>> `(transaction* (^:once fn* [] ~@body))) >>>>> >>>> >>>>> >>>> It should be the default for all one-shot fns. >>>>> >>>> >>>>> >>>> Christophe >>>>> >>>> >>>>> >>>> -- >>>>> >>>> On Clojure http://clj-me.cgrand.net/ >>>>> >>>> Clojure Programming http://clojurebook.com >>>>> >>>> Training, Consulting & Contracting http://lambdanext.eu/ >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> On Clojure http://clj-me.cgrand.net/ >>>>> >> Clojure Programming http://clojurebook.com >>>>> >> Training, Consulting & Contracting http://lambdanext.eu/ >>>>> > >>>>> > -- >>>>> > -- >>>>> > You received this message because you are subscribed to the Google >>>>> > Groups "Clojure" group. >>>>> > To post to this group, send email to clo...@googlegroups.com >>>>> > Note that posts from new members are moderated - please be patient >>>>> with your >>>>> > first post. >>>>> > To unsubscribe from this group, send email to >>>>> > clojure+u...@**googlegroups.com >>>>> > For more options, visit this group at >>>>> > http://groups.google.com/**group/clojure?hl=en<http://groups.google.com/group/clojure?hl=en> >>>>> > >>>>> > --- >>>>> > You received this message because you are subscribed to the Google >>>>> Groups >>>>> > "Clojure" group. >>>>> > To unsubscribe from this group and stop receiving emails from it, >>>>> send an >>>>> > email to clojure+u...@**googlegroups.com. >>>>> > For more options, visit >>>>> > https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>. >>>>> > >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Sean A Corfield -- (904) 302-SEAN >>>>> An Architect's View -- http://corfield.org/ >>>>> World Singles, LLC. -- http://worldsingles.com/ >>>>> >>>>> "Perfection is the enemy of the good." >>>>> -- Gustave Flaubert, French realist novelist (1821-1880) >>>>> >>>> -- >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To post to this group, send email to clo...@googlegroups.com >>>> Note that posts from new members are moderated - please be patient with >>>> your first post. >>>> To unsubscribe from this group, send email to >>>> clojure+u...@googlegroups.com >>>> For more options, visit this group at >>>> http://groups.google.com/group/clojure?hl=en >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "Clojure" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to clojure+u...@googlegroups.com. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> >>> >>> >>> -- >>> On Clojure http://clj-me.cgrand.net/ >>> Clojure Programming http://clojurebook.com >>> Training, Consulting & Contracting http://lambdanext.eu/ >>> >> -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
