Yeah, CSV files can have embedded newlines, so you can't just split it up on linebreaks and expect it to work, you need to send them through a parser.
parse-csv *is* lazy, so my question is, are you doing this at the repl, exactly as you wrote? If so, it will lazily parse the file, and then print that sequence to the repl output, which will consume the whole sequence, causing it to all be in memory at once, and the exception you got. It's the same problem as if you do (repeat 10) at the repl. If you are instead consuming it lazily (by say assigning it to a variable and processing it in some way that only consumes as much as you need of it, or using a function that processes a lazy seq a piece at a time), then there is a bug in the library, and I'd appreciate it if you file an issue for me on the repo so we can get it sorted out ASAP. David On Sun, Jul 8, 2012 at 10:39 AM, Denis Labaye <denis.lab...@gmail.com> wrote: > Hi, > > I would try something like (untested): > > (map parse-csv > (line-seq (clojure.java.io/reader "/tmp/foo.csv"))) > > But it will break for CSV cells with newlines like: > a ; b > foo;"bar > baz" > x ; z > > interesting ... > > Denis > > > On Sun, Jul 8, 2012 at 6:34 PM, Timothy Washington <twash...@gmail.com> > wrote: >> >> Hi there, >> >> I'm trying out the Clojure-csv lib. But I run out of heap space when >> trying to parse a large CSV file. So this call should return a lazy >> sequence. >> >> (csv/parse-csv (io/reader "125Mfile.csv")) >> >> >> >> Instead, I get a "java.lang.OutOfMemoryError: Java heap space". Is there a >> way to get that lazy sequence before reading in the entire file? I can't see >> one, when looking at the code. >> >> >> Thanks in advance >> Tim >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en > > > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with your > first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en
