On Thu, 2010-12-16 at 06:59 +0100, Ganesh wrote: > 250 GB of data, 40 GB of Index Size, 60 million records is > working fine with 1 GB RAM. We are storing minmal amount > of data in index. We are doing sorting on Date. Even in > single system, the database are shard.
Looking back in the list, I see that you're sharding on weeks with 50+ weeks in the index. > build hosted solution. This stats will > increase by minimum 10 times in 2 - 3 years. I plan to use > 64 Bit, with 8 - 10 GB RAM allocated to JVM. When making a conservative estimate and multiplying with 10, you must remember to do the same for the system memory available for disk cache. If your shards are searched sequentially, you could measure the response time for a single shard (after warm up and with different queries), then create a test-shard by merging 10 shards and measure response-time for that. Subtracting the two numbers (to remove the overhead of the front-end layer) and multiplying with 50 should give you a rough estimate for the performance of an upscaled setup. Another measurement suggestion: Divide the current performance of the full setup with the performance of a single shard, then multiply the performance of a single created by merging 10 shards with that number. Regards, Toke Eskildsen --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
