deeMail: u...@thetaphi.de> -Original
Message-> From: wangzhijiang999 [mailto:wangzhijiang...@aliyun.com]> Sent:
Tuesday, July 01, 2014 9:17 AM> To: java-user> Subject: 答复：RE: RE: About lucene
memory consumption> > My application also meet this problem last year and
> -Original Message-
> From: wangzhijiang999 [mailto:wangzhijiang...@aliyun.com]
> Sent: Tuesday, July 01, 2014 9:17 AM
> To: java-user
> Subject: 答复：RE: RE: About lucene memory consumption
>
> My application also meet this problem last year and I researched on the code
&
My application also meet this problem last year and I researched on the code
and found the reason.
The whole process is as follow:
1. When using NRTCachingDirectory, it will use RAMDirectory as cache and
MMapDirectory as delegate. The new segment will be created in the process of
flush or merge
"Uwe Schindler";;
Date: Sat, Jun 28, 2014 05:41 PM
To: "java-user";
Subject: RE: RE: About lucene memory consumption
Hi,
how does your configuration for NRTCaching directory looks like. There are 2
constructor params, one of the maxMergeSizeMB the other one is maxCac
Hi,
how does your configuration for NRTCaching directory looks like. There are 2
constructor params, one of the maxMergeSizeMB the other one is maxCachedMB. If
you correctly close (or release in case of ReaderManager/SearcherManager) all
indexes, this should limit the memory use.
There is no
use MMapDirectory instead
of NRTCachingDirectory?
Thanks & Best Regards!
-- Original --
From: "lubin";<308181...@qq.com>;
Date: Sat, Jun 28, 2014 02:03 PM
To: "java-user";
Subject: Re:RE: About lucene memory consum
quot;<308181...@qq.com>;
Subject: Re:RE: About lucene memory consumption
Could it be that you forgot to close older IndexReaders after getting a new NRT
one? This would be a huge memory leak.
I recommend to use SearcherManager to handle real time reopen correctly.
Uwe
Am 27. Juni 2014
the
>way, we commit the index for every 1000 email document.
>
>
> Could you give me kindly give me some tips to solve this problem?
>
>
>
>
>Thanks & Best Regards!
>
>
>
>
>
>
>
>
>
>
>-- Original --
you give me kindly give me some tips to solve this problem?
Thanks & Best Regards!
-- Original --
From: "Uwe Schindler";;
Date: Fri, Jun 27, 2014 08:36 PM
To: "java-user";
Subject: RE: About lucene memory consumption
age-
> From: 308181687 [mailto:308181...@qq.com]
> Sent: Friday, June 27, 2014 10:42 AM
> To: java-user
> Subject: About lucene memory consumption
>
> Hi, all
>
>
>I fould that the memory consumption of my lucene server is abnormal, and
> “jmap -histo ${pid}” show
Hi, all
I fould that the memory consumption of my lucene server is abnormal, and
“jmap -histo ${pid}” show that the class of byte[] consume almost all of the
memory. Is there memory leak in my app? Why so many byte[] instances?
The following is the top output of jmap:
num
2008/5/29 Alex <[EMAIL PROTECTED]>:
> I believe we have around 346 million documents
So that would be 346MB per indexed field that you search.
Also, if you sort on anything other than score, that will take up a
lot of memory to un-invert the field.
-Yonik
---
I believe we have around 346 million documents
Alex
> Date: Thu, 29 May 2008 18:39:31 -0400
> From: [EMAIL PROTECTED]
> To: java-user@lucene.apache.org
> Subject: Re: lucene memory consumption
>
> Alex wrote:
>> Currently
On Friday 30 May 2008 08:17:52 Alex wrote:
> Hi,
> other than the in memory terms (.tii), and the few kilobytes of opened file
> buffer, where are some other sources of significant memory consumption when
> searching on a large index ? (> 100GB). The queries are just normal term
> queries.
Norms
Alex wrote:
> Currently, searching on our index consumes around 2.5GB of ram.
> This is just a single term query, nothing that requires the in memory cache
> like in
> the FieldScoreQuery.
>
>
> Alex
>
>
>
That seems rather high. You have 10/15 million
To: java-user@lucene.apache.org
> Subject: Re: lucene memory consumption
>
> Not that I can think about. But, if you have any cached field data,
> norms array, that could be huge.
>
> Would be interested in knowing from others regarding this topic as well.
>
> Jian
>
Not that I can think about. But, if you have any cached field data,
norms array, that could be huge.
Would be interested in knowing from others regarding this topic as well.
Jian
On 5/29/08, Alex <[EMAIL PROTECTED]> wrote:
>
> Hi,
> other than the in memory terms (.tii), and the few kilobytes of
Hi,
other than the in memory terms (.tii), and the few kilobytes of opened file
buffer, where are some other sources of significant memory consumption
when searching on a large index ? (> 100GB). The queries are just normal term
queries.
___
18 matches
Mail list logo
