On May 14, 2009, at 7:13 PM, Ian Eure wrote:
I'm trying to process mid/large result sets with Clojure, and not
having any success.
(ns foo
(:require [clojure.contrib.sql :as sql]))
(def *db* {:classname "com.mysql.jdbc.Driver"
:subprotocol "mysql"
:subname "//DSN"
:user "read"
:password "swordfish"})
(sql/with-connection *db*
(sql/with-query-results rs ["SELECT * FROM Foo;"]
(prn "Done")))
When I run this at the REPL, it dies with:
java.lang.OutOfMemoryError: Java heap space (NO_SOURCE_FILE:0)
I would expect the code above to use minimal memory, at most a tiny fraction of what it would take to cache all the results. I tried a similar experiment with Derby (based on the code in clojure.contrib.sql.test). I used visualvm to profile the memory usage of the read and in my experiment, the results fit the expectation.
I recommend you take a look at your setup using visualvm or yourkit or the like. You'll be able to get more visibility into what kinds of objects are taking up so much memory in your case.
--Steve
smime.p7s
Description: S/MIME cryptographic signature
