Hi!
On Jun 24, 12:16 pm, Laurent PETIT <laurent.pe...@gmail.com> wrote:
> 2010/6/24 Laurent PETIT <laurent.pe...@gmail.com>:
>
>
>
> > Hi,
>
> > 2010/6/17 William Wadsworth <will.wadsworth...@gmail.com>:
> >> Hi.
>
> >> I have just started learning Clojure and I am really enjoying
> >> myself. However, I am still getting to grips with the workings of its
> >> concurrency model, so please excuse me if my question seems too
> >> obvious.
>
> >> I have some code that reads data from a file, parses it and generates
> >> a CSV file. The flow is as follows:
>
> >> <code>
> >> (use 'clojure.contrib.duck-streams)
> >> (use 'clojure.contrib.pprint)
>
> >> (defn parse-line [s]
> >> (cl-format nil "~{\"~A\"~^,~}" (into [] (.split s ";"))))
>
> >> (let [input-seq (for [line (read-lines "input.dat")]
> >> (parse-line line))]
> >> (write-lines "out.CSV" input-seq))
> >> </code>
>
> >> Now the question:
>
> >> I would like to generate log entries for any malformed lines occurring
> >> in the input file. Once of the ways might be to do this in parse-line
> >> using clojure.contrib.loggingas follows:
>
> >> <code>
> >> (use 'clojure.contrib.logging)
>
> >> (defn parse-line [s]
> >> (let [cs (into [] (.split s ";"))]
> >> (if (= (count cs) 5)
> >> (cl-format nil "~{\"~A\"~^,~}" cs)
> >> (error "Incorrect number of columns in file."))))
> >> </code>
>
> >> Since FOR returns a lazy sequence of the results, is this function
> >> safe? (Seeing that it is not side-effect free?)
>
> > You have to acknowledge that the calls to parse-line will not
> > necessarily occur during the call to for, but maybe not before calling
> > write-lines, maybe partly if chunked-seqs are in play (sequences
> > preloading items in chunks of 32 items for example). And, if there is
> > an exception that is not thrown in write-lines, your logs may miss the
> > error logs for some of the remaining items not yet consumed by
> > write-lines.
>
> s/if there is an exception that is not thrown in write-lines/if there is
> an exception that *is* thrown in write-lines/
>
>
That is clear. Thanks.
While on the same topic: using read-lines on a large file (~80MB)
results
in an OutOfMemoryException. (I am running the JVM with -client rather
than
-server). To me, this is unexpected since read-lines lazily reads the
file
contents (and I also presume that the lines are not being cached in
memory).
Is there any way of being more memory efficient other than iteratively
reading
file contents using using .readLine?
>
> >> BTW:
> >> clojure.contrib.loggingindicates that it can use an agent for
> >> logging. Is my assumption that invoking the logger (with or without
> >> an agent) within a function would still mean that the function has
> >> side-effects correct?
>
> > Technically speaking, yes. Imagine there's an I/O exception thrown
> > from theloggingcall (disk full, no write access, network access,
> > etc.), then your function would not be "safe" anymore.
>
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
