Hi Sunil,
Might the problem lie within the function that consumes the seq-of-maps?
There is a reduce in there that accumulates some data. I can't tell from
the code whether that would amount to much, though.
Regards,
Caspar
--
You received this message because you are subscribed to the Google
Sorry, I don't have any relevant experience to share. As an experiment, I
suggest that you try deferring the sh execution. Maybe logging the generated
commands to a script file rather than calling sh during processing. Then
execute one big script file at the end. That should make the program
It's not obvious to me from the code what the problem would be. Have
you tried using a profiler to see what kinds of objects account for
the memory? When I've run into seq issues and bugs in the past, that
was pretty helpful in figuring out the underlying problem.
On Feb 27, 8:13 pm, Sunil S Nandi
thank you ..
some times it is hard to come up with good names .. suggestions welcome ..
:)
Sunil.
On Tue, Feb 28, 2012 at 9:47 AM, Baishampayan Ghose wrote:
> > I am using lazy-seqs to join two very large csv files. I am very certain
> > that I am not holding on to any of the heads and If I did
I would like to hear what you may have to say regarding this. As always.. I
pressed send sooner than I wanted to..
On Tue, Feb 28, 2012 at 9:43 AM, Sunil S Nandihalli <
sunil.nandiha...@gmail.com> wrote:
> Hi Everybody,
> I am using lazy-seqs to join two very large csv files. I am very certain
>
> I am using lazy-seqs to join two very large csv files. I am very certain
> that I am not holding on to any of the heads and If I did .. the jvm would
> be out of memory far sooner than what I am seeing currently. The size of the
> file is something like 73 G and the Ram allocated to the jvm is a
Hi Everybody,
I am using lazy-seqs to join two very large csv files. I am very certain
that I am not holding on to any of the heads and If I did .. the jvm would
be out of memory far sooner than what I am seeing currently. The size of
the file is something like 73 G and the Ram allocated to the jv
