On Mon, Dec 20, 2010 at 8:39 AM, Ganesh <emailg...@yahoo.co.in> wrote: > I have done some benchmarking and based on that my estimate of RAM > requirement would be 3 - 4 GB. My question is to go for 64 bit or scale out > with 3 systems? What keeps you from moving to 64bit, I mean if you have those RAM req. for JAVA HeapSpace I don't see much of a choice. You could try some address space extensions which are around but I don't think its worth it. Any reason why you hesitate?
simon > > Regards > Ganesh > > ----- Original Message ----- > From: "Toke Eskildsen" <t...@statsbiblioteket.dk> > To: <java-user@lucene.apache.org> > Sent: Thursday, December 16, 2010 4:20 PM > Subject: Re: Re: Scale up design > > >> On Thu, 2010-12-16 at 06:59 +0100, Ganesh wrote: >>> 250 GB of data, 40 GB of Index Size, 60 million records is >>> working fine with 1 GB RAM. We are storing minmal amount >>> of data in index. We are doing sorting on Date. Even in >>> single system, the database are shard. >> >> Looking back in the list, I see that you're sharding on weeks with 50+ >> weeks in the index. >> >>> build hosted solution. This stats will >>> increase by minimum 10 times in 2 - 3 years. I plan to use >>> 64 Bit, with 8 - 10 GB RAM allocated to JVM. >> >> When making a conservative estimate and multiplying with 10, you must >> remember to do the same for the system memory available for disk cache. >> >> If your shards are searched sequentially, you could measure the response >> time for a single shard (after warm up and with different queries), then >> create a test-shard by merging 10 shards and measure response-time for >> that. Subtracting the two numbers (to remove the overhead of the >> front-end layer) and multiplying with 50 should give you a rough >> estimate for the performance of an upscaled setup. >> >> Another measurement suggestion: Divide the current performance of the >> full setup with the performance of a single shard, then multiply the >> performance of a single created by merging 10 shards with that number. >> >> Regards, >> Toke Eskildsen >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download > Now! http://messenger.yahoo.com/download.php > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
