Hi. I am searching for some guidance on right memory options for my Search Server application. How much memory a lucene based application should be given?
Till a few days back I was running my search server on java 1.4 with memory options "-Xmx3600m" which was running quite fine. After upgrading the JVM to *java 6* we noticed a few times that our application fell idle (perhaps hung). It was not serving the requests although the port was open. I changed the "-Xmx" from 3600m to 5000m. Currently it is running OK but this created a curiosity in my mind that how to find the right memory size for a lucene based application (or any java application). I believe it heavily depends on index size but how do I calculate it (at least approximately) without hit & trial? The details about my application and server are following *[ system configuration ]* # uname -a Linux xxx 2.6.17-13.2.xxx.finalsmp #1 SMP Wed May 9 17:27:56 IST 2007 x86_64 x86_64 x86_64 GNU/Linux *[ java version ]* # /usr/lib/jre1.6.0_20/bin/java -version java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) *[ application command line ]* # java -Xmx5000m -server -classpath lib/lucene-core-2.9.2.jar:lib/mysql-connector-j ava-3.1.11-bin.jar:lib/search.jar com.xxx.xxx.SearchServer conf/search.conf *[ memory information ]* * # cat /proc/meminfo MemTotal: 8156660 kB MemFree: 46924 kB Buffers: 53768 kB Cached: 3194224 kB SwapCached: 60 kB Active: 6086272 kB Inactive: 919276 kB SwapTotal: 2096472 kB SwapFree: 2095432 kB Dirty: 1148 kB Writeback: 0 kB AnonPages: 3756496 kB Mapped: 24800 kB Slab: 1058124 kB SReclaimable: 15144 kB SUnreclaim: 1042980 kB PageTables: 12232 kB NFS_Unstable: 0 kB Bounce: 0 kB CommitLimit: 6174800 kB Committed_AS: 4588900 kB VmallocTotal: 34359738367 kB VmallocUsed: 264904 kB VmallocChunk: 34359473387 kB More info* Search Server application is running on a few servers each of which contains same of copy of index. Total size of index: 23 GB Total Documents in index: 18,000,000 Maximum fields per index: 56 (all analyzed, 4 small fields stored, field "contents" is the largest one, created from a text of as large as 5 - 8 KB) Query frequency: 1 query per second per server (in peak hours) Queries are taking around 800 - 1000 ms per query *Other memory related things in Search application* There is really nothing else which will consume noticeable memory. No database connection is made. The application simply returns the IDs from the index based on the query. Any help on choosing right memory option is much appreciated. -- Regards, Samar
