There is no simple answer. However your app does sound to be using rather a lot of memory for what you describe as simple searches.
Are you using lucene sorting? That can use lots of memory. How are you using/reusing searchers/readers? Having multiple ones open, or failing to close old ones, will use more memory. Does memory usage grow then stabilize or keep on growing? A memory profiler/heap dump could tell you what is really using all the space. -- Ian. On Tue, Apr 27, 2010 at 1:51 PM, Samarendra Pratap <samarz...@gmail.com> wrote: > Hi. > I am searching for some guidance on right memory options for my Search > Server application. How much memory a lucene based application should be > given? > > Till a few days back I was running my search server on java 1.4 with memory > options "-Xmx3600m" which was running quite fine. After upgrading the JVM to > *java 6* we noticed a few times that our application fell idle (perhaps > hung). It was not serving the requests although the port was open. I changed > the "-Xmx" from 3600m to 5000m. > Currently it is running OK but this created a curiosity in my mind that how > to find the right memory size for a lucene based application (or any java > application). I believe it heavily depends on index size but how do I > calculate it (at least approximately) without hit & trial? > > The details about my application and server are following > > *[ system configuration ]* > # uname -a > Linux xxx 2.6.17-13.2.xxx.finalsmp #1 SMP Wed May 9 17:27:56 IST 2007 x86_64 > x86_64 x86_64 GNU/Linux > > *[ java version ]* > # /usr/lib/jre1.6.0_20/bin/java -version > java version "1.6.0_20" > Java(TM) SE Runtime Environment (build 1.6.0_20-b02) > Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) > > *[ application command line ]* > # java -Xmx5000m -server -classpath > lib/lucene-core-2.9.2.jar:lib/mysql-connector-j > ava-3.1.11-bin.jar:lib/search.jar com.xxx.xxx.SearchServer conf/search.conf > > *[ memory information ]* > * > # cat /proc/meminfo > MemTotal: 8156660 kB > MemFree: 46924 kB > Buffers: 53768 kB > Cached: 3194224 kB > SwapCached: 60 kB > Active: 6086272 kB > Inactive: 919276 kB > SwapTotal: 2096472 kB > SwapFree: 2095432 kB > Dirty: 1148 kB > Writeback: 0 kB > AnonPages: 3756496 kB > Mapped: 24800 kB > Slab: 1058124 kB > SReclaimable: 15144 kB > SUnreclaim: 1042980 kB > PageTables: 12232 kB > NFS_Unstable: 0 kB > Bounce: 0 kB > CommitLimit: 6174800 kB > Committed_AS: 4588900 kB > VmallocTotal: 34359738367 kB > VmallocUsed: 264904 kB > VmallocChunk: 34359473387 kB > > More info* > Search Server application is running on a few servers each of which contains > same of copy of index. > Total size of index: 23 GB > Total Documents in index: 18,000,000 > Maximum fields per index: 56 (all analyzed, 4 small fields stored, field > "contents" is the largest one, created from a text of as large as 5 - 8 KB) > Query frequency: 1 query per second per server (in peak hours) > Queries are taking around 800 - 1000 ms per query > > *Other memory related things in Search application* > There is really nothing else which will consume noticeable memory. No > database connection is made. The application simply returns the IDs from the > index based on the query. > > Any help on choosing right memory option is much appreciated. > > -- > Regards, > Samar > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
