Ooh, or maybe not. I just reread and line-seq and read-lines should implement pretty much the same thing. (In the demo code it isn't clear to me where you are using line-seq.)
Are you holding on to the head of the sequence somewhere? On Aug 31, 12:52 pm, Jonathan Smith <jonathansmith...@gmail.com> wrote: > Look at clojure-contrib/duck_streams.clj > > (specifically the read-lines function) > > I think it should be sufficiently lazy to do the job that you are > looking for. (although I don't have any 100mb txt files to test with > handy right now...) > > On Aug 31, 10:44 am, wangzx <wangzaixi...@gmail.com> wrote: > > > I just want to learn clojure by using it to parse log file and > > generate reports. and one question is: for a large text file, can we > > use it as a sequence effectively? for example, for a 100M log file, we > > need to check each line for some pattern match. > > > I just using the (line-seq rdr) but it will cause > > OutOfMemoryException. > > > demo code > > > (defn buffered-reader [file] > > (new java.io.BufferedReader > > (new java.io.InputStreamReader > > (new java.io.FileInputStream file)))) > > > (def -reader (buffered-reader "test.txt")) > > (filter #(= "some" %) -reader) > > > even there is no lines match "some", the filter operation will cause > > OutOfMemoryException. > > > Is there other APIs like the Sequence but provide stream-like API? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
