Re: Memory question

2012-05-21 Thread Chris Bamford
This is a progress update on the issue: I have tried several things and they all gave improvements. In order of magnitude they are 1) Reduced heap space from 6GB to 3GB. This on it's own has so far been the biggest win as swapping almost completely stopped after this step. 2) Began limiting t

Re: Memory question

2012-05-16 Thread Chris Bamford
Thanks everyone. Looks like I have lots of reading to do :-) -Original Message- From: Nader, John P To: java-user@lucene.apache.org Sent: Wed, 16 May 2012 16:27 Subject: Re: Memory question Another good link is http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523

Re: Memory question

2012-05-16 Thread Nader, John P
Lutz > >-Original Message- >From: Chris Bamford [mailto:chris.bamf...@talktalk.net] >Sent: Dienstag, 15. Mai 2012 16:38 >To: java-user@lucene.apache.org >Subject: Re: Memory question > > > Hi John, > >Very interesting, thanks for the detailed explanation. It certainl

Re: Memory question

2012-05-16 Thread Christoph Kaser
java-user@lucene.apache.org Sent: Tue, 15 May 2012 18:10 Subject: RE: Memory question It mmaps the files into virtual memory if it runs on a 64 bit JVM. Because of that you see the mmapped CFS files. This is outside Java Heap and is all *virtual* no RAM is explicitely occupied except the O/S

Re: Memory question

2012-05-15 Thread Sean Bridges
rs and then closes them based on how full >>the heap is getting. My worry is that if the bulk of the memory is being >>allocated outside the Jvm, how can I make sensible decisions? >> >>Thanks for any pointers / info. >> >>Chris >> >> >> >>---

RE: Memory question

2012-05-15 Thread Lutz Fechner
Regards Lutz -Original Message- From: Chris Bamford [mailto:chris.bamf...@talktalk.net] Sent: Dienstag, 15. Mai 2012 16:38 To: java-user@lucene.apache.org Subject: Re: Memory question Hi John, Very interesting, thanks for the detailed explanation. It certainly sounds like the same

Re: Memory question

2012-05-15 Thread Chris Bamford
ar effect ? Thanks again, - Chris -Original Message- From: Nader, John P To: java-user@lucene.apache.org Sent: Tue, 15 May 2012 21:12 Subject: Re: Memory question We've encountered this issue and came up with a fairly good approach to address it. We are on Lucene 3.0.2 with Java 1.6.0

Re: Memory question

2012-05-15 Thread Nader, John P
ual* no RAM is explicitely occupied except the O/S cache. > >- >Uwe Schindler >H.-H.-Meier-Allee 63, D-28213 Bremen >http://www.thetaphi.de >eMail: u...@thetaphi.de > >> -Original Message- >> From: Chris Bamford [mailto:chris.bamf...@talktalk.net] >> Se

Re: RE: Memory question

2012-05-15 Thread Chris Bamford
om: u...@thetaphi.de To: java-user@lucene.apache.org Sent: Tue, 15 May 2012 18:10 Subject: RE: Memory question It mmaps the files into virtual memory if it runs on a 64 bit JVM. Because of that you see the mmapped CFS files. This is outside Java Heap and is all *virtual* no RAM is explicitely occupied excep

RE: Memory question

2012-05-15 Thread Uwe Schindler
eMail: u...@thetaphi.de > -Original Message- > From: Chris Bamford [mailto:chris.bamf...@talktalk.net] > Sent: Tuesday, May 15, 2012 4:47 PM > To: java-user@lucene.apache.org > Subject: Memory question > > Hi > > Can anyone tell me what happens to the memory when

Re: Memory question

2012-05-15 Thread Ian Lea
9:47 > To: java-user@lucene.apache.org > Subject: Memory question > > Hi > > Can anyone tell me what happens to the memory when Lucene opens an > index?  Is it loaded into the JVM's heap or is it mapped into virtual > memory outside of it? > I am running on Linux a

RE: Memory question

2012-05-15 Thread Lutz Fechner
2012 09:47 To: java-user@lucene.apache.org Subject: Memory question Hi Can anyone tell me what happens to the memory when Lucene opens an index? Is it loaded into the JVM's heap or is it mapped into virtual memory outside of it? I am running on Linux and if I use pmap on the PID of my JVM,

Memory question

2012-05-15 Thread Chris Bamford
Hi Can anyone tell me what happens to the memory when Lucene opens an index? Is it loaded into the JVM's heap or is it mapped into virtual memory outside of it? I am running on Linux and if I use pmap on the PID of my JVM, I can see lots of entries for index cfs files. Does this mean that inde