I have done some benchmarking and based on that my estimate of RAM requirement would be 3 - 4 GB. My question is to go for 64 bit or scale out with 3 systems?
Regards Ganesh ----- Original Message ----- From: "Toke Eskildsen" <t...@statsbiblioteket.dk> To: <java-user@lucene.apache.org> Sent: Thursday, December 16, 2010 4:20 PM Subject: Re: Re: Scale up design > On Thu, 2010-12-16 at 06:59 +0100, Ganesh wrote: >> 250 GB of data, 40 GB of Index Size, 60 million records is >> working fine with 1 GB RAM. We are storing minmal amount >> of data in index. We are doing sorting on Date. Even in >> single system, the database are shard. > > Looking back in the list, I see that you're sharding on weeks with 50+ > weeks in the index. > >> build hosted solution. This stats will >> increase by minimum 10 times in 2 - 3 years. I plan to use >> 64 Bit, with 8 - 10 GB RAM allocated to JVM. > > When making a conservative estimate and multiplying with 10, you must > remember to do the same for the system memory available for disk cache. > > If your shards are searched sequentially, you could measure the response > time for a single shard (after warm up and with different queries), then > create a test-shard by merging 10 shards and measure response-time for > that. Subtracting the two numbers (to remove the overhead of the > front-end layer) and multiplying with 50 should give you a rough > estimate for the performance of an upscaled setup. > > Another measurement suggestion: Divide the current performance of the > full setup with the performance of a single shard, then multiply the > performance of a single created by merging 10 shards with that number. > > Regards, > Toke Eskildsen > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > Send free SMS to your Friends on Mobile from your Yahoo! Messenger. Download Now! http://messenger.yahoo.com/download.php --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
