2010/6/24 William Wadsworth <will.wadsworth...@gmail.com>: > Hi! > > On Jun 24, 12:16 pm, Laurent PETIT <laurent.pe...@gmail.com> wrote: >> 2010/6/24 Laurent PETIT <laurent.pe...@gmail.com>: >> >> >> >> > Hi, >> >> > 2010/6/17 William Wadsworth <will.wadsworth...@gmail.com>: >> >> Hi. >> >> >> I have just started learning Clojure and I am really enjoying >> >> myself. However, I am still getting to grips with the workings of its >> >> concurrency model, so please excuse me if my question seems too >> >> obvious. >> >> >> I have some code that reads data from a file, parses it and generates >> >> a CSV file. The flow is as follows: >> >> >> <code> >> >> (use 'clojure.contrib.duck-streams) >> >> (use 'clojure.contrib.pprint) >> >> >> (defn parse-line [s] >> >> (cl-format nil "~{\"~A\"~^,~}" (into [] (.split s ";")))) >> >> >> (let [input-seq (for [line (read-lines "input.dat")] >> >> (parse-line line))] >> >> (write-lines "out.CSV" input-seq)) >> >> </code> >> >> >> Now the question: >> >> >> I would like to generate log entries for any malformed lines occurring >> >> in the input file. Once of the ways might be to do this in parse-line >> >> using clojure.contrib.loggingas follows: >> >> >> <code> >> >> (use 'clojure.contrib.logging) >> >> >> (defn parse-line [s] >> >> (let [cs (into [] (.split s ";"))] >> >> (if (= (count cs) 5) >> >> (cl-format nil "~{\"~A\"~^,~}" cs) >> >> (error "Incorrect number of columns in file.")))) >> >> </code> >> >> >> Since FOR returns a lazy sequence of the results, is this function >> >> safe? (Seeing that it is not side-effect free?) >> >> > You have to acknowledge that the calls to parse-line will not >> > necessarily occur during the call to for, but maybe not before calling >> > write-lines, maybe partly if chunked-seqs are in play (sequences >> > preloading items in chunks of 32 items for example). And, if there is >> > an exception that is not thrown in write-lines, your logs may miss the >> > error logs for some of the remaining items not yet consumed by >> > write-lines. >> >> s/if there is an exception that is not thrown in write-lines/if there is >> an exception that *is* thrown in write-lines/ >> >> > > That is clear. Thanks. > > While on the same topic: using read-lines on a large file (~80MB) > results > in an OutOfMemoryException. (I am running the JVM with -client rather > than > -server). To me, this is unexpected since read-lines lazily reads the > file > contents (and I also presume that the lines are not being cached in > memory). > Is there any way of being more memory efficient other than iteratively > reading > file contents using using .readLine?
You're probably holding the head. Can you share the code ? -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en