We have several TB of indexes w/ literally billions of documents and will be using clojure w/ hadoop soon. I appreciate the hadoop mapr job examples in clojure. This is great stuff.
On Apr 7, 6:24 am, Stuart Sierra <the.stuart.sie...@gmail.com> wrote: > Hi Sean, > > It's deployed on an Ubuntu server on Amazon EC2. Just one server for > now, but designed to scale to more. No RDBMS! > > I'm not using any Amazon services besides EC2 and S3, but I'm looking > at SQS and SimpleDB. > > Deployment was pretty easy; my only problems have been non-Clojure- > related. For example, system properties set in Ant don't persist into > Log4j, so I have to use a shell script to launch the server. > Restlet's default logging doesn't match Apache's format, so making > AWStats work has been a hassle. > > If you're interested in size: > There are ~870,000 documents on the site. > The Lucene/Solr index is ~6 GB. > The HTML content is ~4 GB, compressed. > 99% of the pages are static HTML, pre-rendered in a Hadoop job. > > -Stuart Sierra > > On Apr 7, 10:47 am, Sean <francoisdev...@gmail.com> wrote: > > > Okay wow... it'll take some time to fully appreciate this. > > > Can you comment on your hardware stack? How many servers are you > > using? Is there an RDBMS in there somewhere? > > > How was deployment? > > > Looks awesome, thanks for sharing! > > > On Apr 7, 10:41 am, Stuart Sierra <the.stuart.sie...@gmail.com> wrote: > > > > Here: http://www.altlaw.org/ > > > > About 4000 lines of Clojure code, 2500 of Java, powering a web site > > > with well over a million pages, averaging around 10,000 visitors a > > > day. > > > > Some of what I'm using: > > > Restlet > > > StringTemplate > > > Solr > > > Hadoop > > > Apache Java Commons > > > markdownj > > > cpdetector > > > JRuby > > > > Most of the code is online, GPL'd, athttp://github.com/lawcommons > > > > -Stuart Sierra > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en -~----------~----~----~----~------~----~------~--~---
