Just a guess. If your daily data is huge you will be loading the data for only one day when using map and you will be loading the data for multiple days (equal to number of parallel threads) .. and may be this is the cause of the problem. Sunil.
On Sat, Aug 6, 2011 at 11:40 PM, Shoeb Bhinderwala < shoeb.bhinderw...@gmail.com> wrote: > Problem summary: I am running out of memory using pmap but the same code > works with regular map function. > > My problem is that I am trying to break my data into sets and process them > in parallel. My data is for an entire month and I am breaking it into 30/31 > sets - one for each day. I run a function for each daily set of data using > pmap, something like: > > (defn process-monthly-data > [grp-id month year] > (doall (pmap > #(process-daily-data grp-id % month year) > (range 31))) > > (defn process-daily-data > [grp-id day month year] > ( > ;load and process daily data … > )) > > When I run my function using regular map it works fine, but when I change > it to pmap I get an OutOfMemoryException. > > What am I doing wrong? > > -- Shoeb > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
