On Mon, Mar 16, 2009 at 4:29 PM, Mark Volkmann <r.mark.volkm...@gmail.com>wrote:
> > On Mon, Mar 16, 2009 at 4:56 PM, Brian Doyle <brianpdo...@gmail.com> > wrote: > > I've been using Clojure for about 6 months now and really like it. I am > > somewhat new to multi-threading > > and using any of the parallel features in Clojure though. I have a > > situation where I need to convert > > 7 files from CSV to XML. Each one of these files is about 180MB apiece > in > > size. I have dual core machine > > with 2GB of RAM and would like some advice on the best strategy for > > processing these files in a way that > > really utilizes both cores and my memory to really speed up the > > processing. I'm sure this isn't the best > > way, but I've only come up with starting up two threads at first, having > > each thread open up a file, > > call line-seq on that file, write out the XML for each line and then go > to > > the next file when it's complete. Any > > advice would be great. Thanks. > > Most libraries that write XML build a data structure to represent the > XML and then write it out. That doesn't work for writing out large XML > documents because you'll run out of memory before you finish building > the data structure. > > I don't know if there is a Clojure library that writes XML as you > specify it. If there isn't then you should consider WAX, a Java > library I wrote. You can learn more about it at > http://ociweb.com/wax/. You could use it from Clojure. Check out the > tutorial. It's a really simple library, it's really fast and it's very > memory efficient! > Mark, WAX looks really nice! I'll look around in Clojure some more about writing XML but if I don't find anything to my linking I'll give WAX a go. Thanks! > > -- > R. Mark Volkmann > Object Computing, Inc. > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
