Re: Java out of memory problem

Thank Ken, your suggestion solved my problem with the OOM exception. I tried your suggestion to run it in parallel but I didn't see much difference. Instead I called future on the let call and that helped the performance. On Dec 17, 2:55 pm, Ken Wesson wrote: > On Fri, Dec 17, 2010 at 5:39 PM, c

Re: Java out of memory problem

On Fri, Dec 17, 2010 at 5:39 PM, clj123 wrote: > (defn persist-rows >  [headers rows id] >  (let [mrows (transform-rows rows id)] >    (with-db *db* (try >         (apply insert-into-table >                :my-table >                [:col1 :col2 :col3] >                mrows))) >     nil )) > > (d

Re: Java out of memory problem

(defn persist-rows [headers rows id] (let [mrows (transform-rows rows id)] (with-db *db* (try (apply insert-into-table :my-table [:col1 :col2 :col3] mrows))) nil )) (defn filter-data [rows item-id header id] (persist-rows

Re: Java out of memory problem

On Thu, Dec 16, 2010 at 09:19, clj123 wrote: > Hello, > > I'm trying to insert in a database large number of records, however > it's not scaling correctly. For 100 records it takes 10 seconds, for > 100 records it takes 2 min to save. But for 250 records it > throws Java Heap out of memor

Re: Java out of memory problem

You might be coming to near OOM with using in-memory processing but don't know it, and the batched (lazy) version is probably holding onto data creating the mem leak. Would you be able to post the relevant source? -- You received this message because you are subscribed to the Google Groups "Cloj

Java out of memory problem

Hello, I'm trying to insert in a database large number of records, however it's not scaling correctly. For 100 records it takes 10 seconds, for 100 records it takes 2 min to save. But for 250 records it throws Java Heap out of memory exception. I've tried separting the records processing

Re: Memory Problem

Well, that's what I am talking about. It can be controlled, but it requires good grasp of all the concepts and high concentration + there is still a chance to miss something. Now it may not be such a problem since the majority of people who use clojure are usually highly motivated enthusiasts, but

Re: Memory Problem

Passing a collection to a function that expects a lazy seq is not always an error. The seq library encourages it by calling (seq) on those collections for you. I suppose all lazy functions could emit a warning when they have to call (seq) on their arguments, based on some global variable like *war

Re: Memory Problem

Is there an automatic way to discover such error? These types of errors could be discovered by the compiler. How am I going to see if I get an ISeq in my clojure code? I would have to dig... On Jul 24, 3:37 pm, eyeris wrote: > On Jul 24, 6:17 am, Dragan Djuric wrote: > > > Sometimes (or maybe a

Re: Memory Problem

On Jul 24, 6:17 am, Dragan Djuric wrote: > Sometimes (or maybe always?) it is mentioned in the doc. In my > opinion, this is one of the cases where dynamic languages do not > excel. If we had typing, that would be solved by implementing Lazy > "interface". We do have types and we do have a lazy

Re: Memory Problem

Hi, Everything returning a seq is lazy: take, drop, iterate, map, ... Everything (maybe) returning something else is not: reduce, loop, ... Not 100% accurate, but this heuristic should get you very far. Sincerely Meikel --~--~-~--~~~---~--~~ You received this me

Re: Memory Problem

Sometimes (or maybe always?) it is mentioned in the doc. In my opinion, this is one of the cases where dynamic languages do not excel. If we had typing, that would be solved by implementing Lazy "interface". Or maybe this is an use case for metadata? Each function could have a tag :lazy? with poss

Re: Memory Problem

Oh I see, Thanks.., but how to I check if a function is lazy or not? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from

Re: Memory Problem

> ( from > http://stackoverflow.com/questions/749911/how-can-i-leak-memory-in-clojure > ) > > (drop 90 (vec (range 100))) would cause memory problem in > clojure and how to fix it? > Thanks > > Because I think my code might have si

Memory Problem

Hi : I would like to know why ( from http://stackoverflow.com/questions/749911/how-can-i-leak-memory-in-clojure ) (drop 90 (vec (range 100))) would cause memory problem in clojure and how to fix it? Thanks Because I think my code might have similar problem of memory cannot