Re: Caching lucene index

the rigtht tool for your problem. Le mar. 12 juil. 2016 à 17:56, Abhinav Tiwari a écrit : > Any help here? > > -- Forwarded message -- > From: Abhinav Tiwari > Date: Fri, Jun 10, 2016 at 1:48 PM > Subject: Caching lucene index > To: java-user@lucene.apache

Fwd: Caching lucene index

Any help here? -- Forwarded message -- From: Abhinav Tiwari Date: Fri, Jun 10, 2016 at 1:48 PM Subject: Caching lucene index To: java-user@lucene.apache.org Hi Team, We have recently implemented a search feature in our web application using Lucene where we perform dynamic

Caching lucene index

Hi Team, We have recently implemented a search feature in our web application using Lucene where we perform dynamic indexing. We are looking for opportunities of caching the index so that we do not have to perform dynamic indexing again for the same query and query can be searched using the

RE: Lucene Searcher Caching and Performance

live [mailto:kiwi_cl...@yahoo.com.INVALID] Sent: Dienstag, 4. August 2015 17:41 To: Java-user Subject: Lucene Searcher Caching and Performance Hi Guys, We have an index/query server that contains several thousand fairly hefty indexes. Each searcher is shared between many 'user-threads'

Re: Lucene Searcher Caching and Performance

Sent: 04 August 2015 11:41 To: Java-user Subject: Lucene Searcher Caching and Performance Hi Guys, We have an index/query server that contains several thousand fairly hefty indexes. Each searcher is shared between many 'user-threads' and once opened we keep the searcher in a cache which is

Lucene Searcher Caching and Performance

strikes me that if we could cache all of our searchers on the machine (ie have all of our indexes 'open for business'), possibly having to alter kernel parameters to cater for the large number of file handles, without caching many query results, this might solve the problem, without push

RE: Lucene Java Caching Question

to the first run. So I am assuming that there is some > caching going on, but where is this stored? It is the disk cache of your operating system. It is independent of Lucene and is in-memory. Most modern operating systems uses all free memory for disk cache. Lucene uses random access all the

Lucene Java Caching Question

, the execution time is very low comparative to the first run. So I am assuming that there is some caching going on, but where is this stored? I have checked the index folder (where I have created the spatial index), and no files have been updated. I have looked on for similar question on this forum

Re: Query rewriting - caching rewritten quries

Hi again, I see I missed very important thing in your response. I thought I cannot reuse rewritten queries in different types of IndexReader but you wrote I cannot use rewritten queries even in another instance of IndexReader: "not even if it's a reopened reader against the same index". I thought

Re: Query rewriting - caching rewritten quries

Hi, Thank you for your response Chris. I see good news that I can pre-build rewritten queries for a given IndexReader and then use it in the same IndexReader. Can you tell me how I can achieve this? I see each Query has rewrite method which takes IndexReader as an argument. The only thing is just

Re: Query rewriting - caching rewritten quries

: In the system which I develop I have to store many query objects in memory. : The system also receives documents. For each document MemoryIndex is : instantiated. I execute all stored queries on this MemoryIndex. I realized : that searching over MemoryIndex takes much time for query rewriting. I'

Query rewriting - caching rewritten quries

Hi, In the system which I develop I have to store many query objects in memory. The system also receives documents. For each document MemoryIndex is instantiated. I execute all stored queries on this MemoryIndex. I realized that searching over MemoryIndex takes much time for query rewriting. I'm w

RE: Stored fields and OS file caching

Vitaly Funstein [vfunst...@gmail.com] wrote: > It's a bit of a guess on my part, but I did get better write and search > performance with size <= 2K, as opposed to the default 16K. For search that sounds plausible as that is very random access heavy and the disk cache will contain a larger amount

Re: Stored fields and OS file caching

Are there some graphic diagrams that illustrate this distinction in how stored fields and doc values are organized, including both the heap and non-heap aspects like file caching. Sometimes a picture is worth 1K words. Even if somebody could just draw it on a piece of paper and scan it

Re: Stored fields and OS file caching

Thanks for the explanation, Adrien. I do have a couple of follow-up questions. Isn't this block size used for file caching OS-dependent? And if 4K happens to be the most commonly used size, wouldn't it make more sense for the default stored fields format to have a chunk size equal to

Re: Stored fields and OS file caching

emen >> http://www.thetaphi.de >> eMail: u...@thetaphi.de >> >> >> > -Original Message- >> > From: Vitaly Funstein [mailto:vfunst...@gmail.com] >> > Sent: Friday, April 04, 2014 9:44 PM >> > To: java-user@lucene.apache.org >> &g

Re: Stored fields and OS file caching

Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > > -Original Message- > > From: Vitaly Funstein [mailto:vfunst...@gmail.com] > > Sent: Friday, April 04, 2014 9:44 PM > > To: java-user@lucene.apache.org > > Subject: Stored fields and OS file caching &g

RE: Stored fields and OS file caching

://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Vitaly Funstein [mailto:vfunst...@gmail.com] > Sent: Friday, April 04, 2014 9:44 PM > To: java-user@lucene.apache.org > Subject: Stored fields and OS file caching > > I have heard here that stored fields do

Stored fields and OS file caching

I have heard here that stored fields don't work well with OS file caching. Could someone elaborate on why that is? I am using Lucene 4.6 and we do use stored fields but not doc values; it appears most of the benefit from the latter comes as improvement in sorting performance, and I don'

Re: AutomatonQuery Caching

Jul 2011 22:42:44 +0200 > Von: Simon Willnauer > An: java-user@lucene.apache.org > Betreff: Re: AutomatonQuery Caching > Hi rene, > > On Tue, Jul 12, 2011 at 10:12 PM, Rene Hackl-Sommer > wrote: > > This sounds plausible, even if manually cleaning the Java cache has no

Re: AutomatonQuery Caching

t;>  wrote: >>> >>> Hi, >>> >>> I am running some AutomatonQueries and see caching effects happening, >>> that I can't seem to get rid of. E.g. an AutomatonQuery takes >>> >>> Run 1: 3000 ms >>> Run 2 and all subsequent runs:

Re: AutomatonQuery Caching

nning some AutomatonQueries and see caching effects happening, that I can't seem to get rid of. E.g. an AutomatonQuery takes Run 1: 3000 ms Run 2 and all subsequent runs: 250 ms The IndexSearcher is closed after every full query circle (and opened via open(Directory), so the IndexReader should be clo

Re: AutomatonQuery Caching

On Tue, Jul 12, 2011 at 10:42 AM, "René Hackl" wrote: > Hi, > > I am running some AutomatonQueries and see caching effects happening, that I > can't seem to get rid of. E.g. an AutomatonQuery takes > > Run 1: 3000 ms > Run 2 and all subsequent runs: 250 ms &

AutomatonQuery Caching

Hi, I am running some AutomatonQueries and see caching effects happening, that I can't seem to get rid of. E.g. an AutomatonQuery takes Run 1: 3000 ms Run 2 and all subsequent runs: 250 ms The IndexSearcher is closed after every full query circle (and opened via open(Directory), s

Re: NRT and Caching based on IndexReader

t what it > does, > >> its new... > >> > >> -shay.banon > >> > >> > >> On Tue, May 18, 2010 at 4:04 AM, Yonik Seeley < > yo...@lucidimagination.com>wrote: > >> > >>> On Mon, May 17, 2010 at 9:00 PM, Shay Banon

Re: NRT and Caching based on IndexReader

M, Shay Banon wrote: >>> > Great, so I am not imagining things this late into the night ... ;), not >>> so >>> > great, since using NRT with field cache (like sorting) or caching >>> filters, >>> > or anything that caches based on IndexReader not

Re: NRT and Caching based on IndexReader

On Mon, May 17, 2010 at 9:12 PM, Shay Banon wrote: > Just saw that you opened a case for that. I think that its important in your > test case to also test for object identity, not just equals. This is because > the IndexReader (or the FieldCacheKey) are used as keys in weak hash maps, > which uses

Re: NRT and Caching based on IndexReader

On Tue, May 18, 2010 at 4:04 AM, Yonik Seeley > wrote: > >> On Mon, May 17, 2010 at 9:00 PM, Shay Banon wrote: >> > Great, so I am not imagining things this late into the night ... ;), not >> so >> > great, since using NRT with field cache (like sorting) or

Re: NRT and Caching based on IndexReader

on On Tue, May 18, 2010 at 4:04 AM, Yonik Seeley wrote: > On Mon, May 17, 2010 at 9:00 PM, Shay Banon wrote: > > Great, so I am not imagining things this late into the night ... ;), not > so > > great, since using NRT with field cache (like sorting) or caching > filters, >

Re: NRT and Caching based on IndexReader

On Mon, May 17, 2010 at 9:00 PM, Shay Banon wrote: > Great, so I am not imagining things this late into the night ... ;), not so > great, since using NRT with field cache (like sorting) or caching filters, > or anything that caches based on IndexReader not really an option. This > ma

Re: NRT and Caching based on IndexReader

Great, so I am not imagining things this late into the night ... ;), not so great, since using NRT with field cache (like sorting) or caching filters, or anything that caches based on IndexReader not really an option. This makes NRT very problematic to use in a real application. -shay.banon On

Re: NRT and Caching based on IndexReader

rintln("Should not be more readers..."); >                } >            } >        } >    } > >    indexWriter.close(); > } > > > > > On Tue, May 18, 2010 at 12:30 AM, Yonik Seeley > wrote: > >> On Mon, May 17, 2010 at 5:00 PM, Shay Bano

Re: NRT and Caching based on IndexReader

riter.close(); } On Tue, May 18, 2010 at 12:30 AM, Yonik Seeley wrote: > On Mon, May 17, 2010 at 5:00 PM, Shay Banon wrote: > > I wanted to verify if my understanding is correct. Assuming that I use > > NRT, and refresh, say, every 1 second, caching based on IndexReader, such

Re: NRT and Caching based on IndexReader

On Mon, May 17, 2010 at 5:00 PM, Shay Banon wrote: >   I wanted to verify if my understanding is correct. Assuming that I use > NRT, and refresh, say, every 1 second, caching based on IndexReader, such is > what is used in the CachingWrapperFilter is basically useless No, it's fi

NRT and Caching based on IndexReader

Hi, I wanted to verify if my understanding is correct. Assuming that I use NRT, and refresh, say, every 1 second, caching based on IndexReader, such is what is used in the CachingWrapperFilter is basically useless, since, even if there is an open sub reader, it gets clones meaning there is a

Re: Doc Caching

<<>> You've hit one of my personal hot buttons . Never, never, never until 1> you're reasonably sure you actually *have* a performance issue. 2> your performance issues can't be fixed more cheaply. Memory is wy cheap compared to all the time you&#x

Re: Doc Caching

Generally best to let the OS do it. Having plenty of spare RAM is good. -- Ian. On Tue, Apr 20, 2010 at 6:16 AM, Chris B wrote: > Mike cheers for the reply. > > Is it worth setting up your own caching or letting the OS do it? I've setup > a caching system, but if the OS is

Re: Doc Caching

Mike cheers for the reply. Is it worth setting up your own caching or letting the OS do it? I've setup a caching system, but if the OS is doing it it seems pointless. Chris - Original Message - From: "Michael McCandless" To: Sent: Tuesday, April 20, 2010 2:17 AM S

Re: Doc Caching

No, Lucene doesn't. But the OS usually does (in is IO cache), assuming there is spare RAM. The "only" things that are explicitly held in memory by Lucene are the norms ("boost bytes"), terms dict index, deletions bit vector and field cache (used eg when you sort by a field), I think. Mike On Fr

Doc Caching

Hi Does Lucene cache the documents it retrieves? If so in which object? Chris GET FREE 5GB EMAIL - Check out spam free email with many cool features! Visit http://www.inbox.com/email to find out more!

Re: Caching analyzed query

ess than a millisecond per query), and so you can save this to the last minute. Doing the key/value store lookup (especially if remote!) I can see caching, but producing the Lucene query is only slow if you're doing some *really* crazy stuff. Sometimes happens, to be sure, but usually, the cra

Caching analyzed query

on of the input text? I can make a cache per JVM by holding a previously created Query object for a specific input, but in a distributed environment if I store the serialized form of Query object, the overhead of deserializing may kill all the benefits of caching here... Thanks, Erdinc

Re: 2.9 per segment searching/caching

t; > is actually faster than 2.4. > > -John > > > > On Thu, Oct 22, 2009 at 5:07 AM, Mark Miller > wrote: > > > > > >> Bill Au wrote: > >> > >>> Since Lucene 2.9 has per segment searching/caching, does query > >>> > >>

Re: 2.9 per segment searching/caching

api etc., sorting > is actually faster than 2.4. > -John > > On Thu, Oct 22, 2009 at 5:07 AM, Mark Miller wrote: > > >> Bill Au wrote: >> >>> Since Lucene 2.9 has per segment searching/caching, does query >>> >> performance >&g

Re: 2.9 per segment searching/caching

With many other coding that happened in 2.9, e.g. the PQ api etc., sorting is actually faster than 2.4. -John On Thu, Oct 22, 2009 at 5:07 AM, Mark Miller wrote: > Bill Au wrote: > > Since Lucene 2.9 has per segment searching/caching, does query > performance > > degrade les

Re: 2.9 per segment searching/caching

Bill Au wrote: > Since Lucene 2.9 has per segment searching/caching, does query performance > degrade less than before (2.9) as more segments are added to the index? > Bill > > I think non sorting cases are actually faster now over multiple segments - though you will still

Re: 2.9 per segment searching/caching

if a segment did not change since last opened. This will improve the overall performance but the general query performance will not be improved by per seg. search. simon On Thu, Oct 22, 2009 at 5:31 AM, Bill Au wrote: > Since Lucene 2.9 has per segment searching/caching, does query performa

2.9 per segment searching/caching

Since Lucene 2.9 has per segment searching/caching, does query performance degrade less than before (2.9) as more segments are added to the index? Bill

RE: caching an indexreader

Thanks for the comments. Sounds like I will probably be ok. -Original Message- From: Jason Rutherglen [mailto:jason.rutherg...@gmail.com] Sent: Friday, June 19, 2009 1:50 PM To: java-user@lucene.apache.org; java-...@lucene.apache.org Subject: Re: caching an indexreader On the topic of

Re: caching an indexreader

one solution would be to cache the Searcher for the > > duration of the user's search session so that the user's view of the > > available documents doesn't change. As I understand it, the user won't > > see any changes to the index until a new Searcher is create

Re: caching an indexreader

7;s search session so that the user's view of the > available documents doesn't change.  As I understand it, the user won't > see any changes to the index until a new Searcher is created.  However, > I'm very sensitive to the amount of session context memory that cach

Re: caching an indexreader

> As I understand it, the user won't see any changes to the index until a new Searcher is created. Correct. > How much memory will caching the searcher cost? Are there other tradeoff's I need to consider? If you're updating the index frequently (every N seconds) and t

caching an indexreader

changes to the index until a new Searcher is created. However, I'm very sensitive to the amount of session context memory that caching the searcher might take up. How much memory will caching the searcher cost? Are there other tradeoff's I need to consider? Is this a viable option or d

Re: not updating caching

What exactly are you using: Solr or some other server or straight lucene? Lucene itself doesn't do caching. When you close and start the server what exactly are you closing and starting? If by server you mean something like Tomcat then perhaps you are not reopening index readers/searchers

Re: not updating caching

HI, Thnx for the reply. But explicetly am not using or creating any caching.But why its happening Otis Gospodnetic wrote: > > > I have a feeling you are using Solr or some other server and not straight > Lucene. To turn off Solr caching, comment it out from solrconfig.xml (but &

Re: not updating caching

I have a feeling you are using Solr or some other server and not straight Lucene. To turn off Solr caching, comment it out from solrconfig.xml (but you'll need to restart after that). If this is indeed about Solr, please use solr-user list instead of this one. Otis -- Sematext --

not updating caching

Hi All, I had indexed some records and we found some searches are not found because of some reason. So again made changes and indexed it again, now we got hits for the query from cmd prompt ,but this search is not updated in caching in server. If i close and start the server it is works

Caching Filters and docIds when using MultiSearcher/IndexSearcher(MultiReader)...

Up to now I have only needed to search a single index, but now I will have many index shards to search across. My existing search mantained cached filters for the index as well as a cache of my own unique ID fields in the index, keyed by Lucene DocId. Now I need to search multiple indices, I

Re: caching fields for query performance

Yes, pull out language:ENG language:FRA language:RUS into filters, cache them, and re-use them across all queries. In Lucene, see CachingWrapperFilter() In Solr, use a separate fq (for filter query) parameter... q=+topic:m&a +topic:earn +company:MSFT&fq=language:ENG -Yonik On Fri, Jul 25, 200

caching fields for query performance

If I have a frequently queried field, which has a single value per document (such as language), how can I pre-cache all field values, such that the underlying query processing always uses memory cache (never disk i/o) for that particular field? I don't know if it is possible without some custom

search performance & caching

ing a paralle multi-searcher with a custom hit collector. 2) I'm performing some field caching during search warmup. For an index of 3.4 million doc's and 7GB, it's taking up to 30 minutes to execute the code snippet below. Most of this time is involved with the multireader.docume

need a better way of caching the value of a field ???

Hi folks, I am using a MultiSearcher object which uses 4 months indexes. I have a requirement for which i need to cache one field for documents which are less then one month old. So for that i am first creating a date query(for last one month) and using HitCollector.collect() for collecting Docume

Sorting, Caching, and garbage collection

Dear all, I have been using PyLucene for some time now (really loving it, actually), and I have now encountered an intriguing situation. I have a large inde file, 17Gb, 50M documents. I want to look for cooccurrences for terms in a certain field (a boolean query), and rank order the resul

Re: Caching in lucene

/interned by Lucene? The string values they hold are, but I'm not sure about the actual instances of Term. Warming is merely the process of avoiding latency by using a reader a bit prior to allowing consumers to access it. It does not include any extra caching layers. Solr does

Re: Caching in lucene

ailendra Mudgal: > > > Whether it caches frequently used terms ? > > I don't understand you question? > > -- > karl > > > > > > > > On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: > >> > >> > >> 18 sep 2007 k

Re: Caching in lucene

18 sep 2007 kl. 08.33 skrev Shailendra Mudgal: Whether it caches frequently used terms ? I don't understand you question? -- karl On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: 18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: What my goal is to understand the cachi

Re: Caching in lucene

Whether it caches frequently used terms ? On 9/18/07, Karl Wettin <[EMAIL PROTECTED]> wrote: > > > 18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: > > > > > What my goal is to understand the caching strategy. How well this > > work for repetitive queries. Is

Re: Caching in lucene

18 sep 2007 kl. 07.12 skrev Shailendra Mudgal: What my goal is to understand the caching strategy. How well this work for repetitive queries. Is there any room available to improve this. It is usually a loss of resources to cache results in a busy system with gaussianity distributed

Re: Caching in lucene

Hi Yonik, Thanks for your response. I'll feel great if you can explain this in more detail as i am not sure that whether i have understood this correctly or not. Or if you can direct me to some resource that will also be very good for me. What my goal is to understand the caching strategy.

Re: Caching in lucene

ons : > - is there any caching is done in lucene for search ? The following are loaded on-demand and cached: - term index - norms, per-field - FieldCache for sorting or ValueSourceQuery - CachingWrapperFilter Also, "warming" an IndexReader can even refer to getting the operating system

Caching in lucene

Hi All, One thing that i understand about IndexReader is that for subsequent queries, results come fast as the IndexReader needs to be warmed up. According to this, I am trying to find out the answers of following questions : - is there any caching is done in lucene for search ? - if yes, is it

SV: Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

Kalle and Patrick: many thanks for the suggestions! Caching the IndexSearcher in the ServletContext sounds like a very good idea. However, I have to index a number of databases, each with a different Lucene index. So keeping an IndexSearcher for each may come with a prohibitive memory cost. But

Re: Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

er, > but there's no mention of the memory cost! Since it is said to > take a long time to create, I presume that the IndexSearcher > reads the index files and keeps a lot of stuff in memory, so > the thought of caching one for each HttpSession gives me bad vibes. Why

Re: Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

29 aug 2007 kl. 14.32 skrev Per Lindberg: For each search request (it's a webapp) I currently create a new IndexSearcher, new Filter and new Sort, call searcher.search(query, filter, sorter) and later searcher.close(). You really want to reuse the IndexSearcher until new data has been added t

Caching IndexSearcher in a webapp [was: Find "latest" document (before a certain date)]

st! Since it is said to take a long time to create, I presume that the IndexSearcher reads the index files and keeps a lot of stuff in memory, so the thought of caching one for each HttpSession gives me bad vibes. (Also keeping open files; the file locking scheme in NTFS can prevent Tomcat from doing

Re: filter caching

: want to build up a set of small QueryFilters and that can be cached and : be combined in any order (using the ChainedFilter operations) while : still making use of caching. If ChainedFilter does this (or doesn't), : it would be nice to have it mentioned in the API docs. what you are a

Re: filter caching

If the Chained filter works off of the cached QueryFilters, then that is great and I don't mind not caching the ChainedFilter and doing the work to combine those bit vectors (assuming that is what it is doing). I want to build up a set of small QueryFilters and that can be cached an

Re: filter caching

: I have a question about filter caching. I have a lot of QueryFilters : that I use when searching that filter on a single field. Sometimes : alone I use them by themselves, but mostly I use them in some : combination using ChainedFilter. Does the caching take advantage of : only the final

filter caching

I have a question about filter caching. I have a lot of QueryFilters that I use when searching that filter on a single field. Sometimes alone I use them by themselves, but mostly I use them in some combination using ChainedFilter. Does the caching take advantage of only the final filter

Re: Caching of BitSets from filters and Query.equals()

Chris Hostetter wrote: : equals to get q1.equals(q2). The core Lucene Query implementations do override : equals() to satisfy that test, but some of the contrib Query implementations do : not override equals, so you would never see the same Query twice and caching : BitSets for those Query

Re: Caching of BitSets from filters and Query.equals()

: equals to get q1.equals(q2). The core Lucene Query implementations do override : equals() to satisfy that test, but some of the contrib Query implementations do : not override equals, so you would never see the same Query twice and caching : BitSets for those Query instances would be a waste

Re: Caching of BitSets from filters and Query.equals()

absolutely nothing about wether query1.equals(query2). Yes, that's not what I was after - As you say, it's not possible to implement. in general, what you describe really isn't needed for caching query result sets ... what matters is that if you've already seen the query b

Re: Caching of BitSets from filters and Query.equals()

ent results -- so how could an algorithm like you describe possibly be implemented in a Query.equals() method when the IndexReader isn't known? in general, what you describe really isn't needed for caching query result sets ... what matters is that if you've already seen the query

Re: Caching of BitSets from filters and Query.equals()

#x27;m hosting an indexing framework, so I've no idea what searches or filters a caller will want to perform. Have a look at Solr's caching mechanisms for filters, queries, and documents. Very slick and scalable stuff. Erik ---

Re: Caching of BitSets from filters and Query.equals()

Erik Hatcher wrote: Have a look at the CachingWrappingFilter: It caches filters by IndexReader instance. Doesn't that still have the same issue in terms of equality of conditions that created t

Re: Caching of BitSets from filters and Query.equals()

Have a look at the CachingWrappingFilter: It caches filters by IndexReader instance. Erik On Mar 6, 2007, at 2:03 AM, Antony Bowesman wrote: Not sure if I'm going about this the right way,

Caching of BitSets from filters and Query.equals()

Not sure if I'm going about this the right way, but I want to use Query instances as a key to a HashMap to cache BitSet instances from filtering operations. They are all for the same reader. That means equals() for any instance of the same generic Query would have to return true if the terms,

Re: possible to disable internal caching?

AIL PROTECTED]> wrote: On Wednesday 14 February 2007 17:12, jm wrote: > So my question, is it possible to disable some of the caching lucene > does so the memory consumption will be smaller (I am a bit concerned > on the memory usage side)? Or the memory savings would not pay off?

Re: possible to disable internal caching?

On Wednesday 14 February 2007 17:12, jm wrote: > So my question, is it possible to disable some of the caching lucene > does so the memory consumption will be smaller (I am a bit concerned > on the memory usage side)? Or the memory savings would not pay off? You

Re: possible to disable internal caching?

14 feb 2007 kl. 17.12 skrev jm: So my question, is it possible to disable some of the caching lucene does so the memory consumption will be smaller (I am a bit concerned on the memory usage side)? Or the memory savings would not pay off? You could try to create a new Searcher for each query

possible to disable internal caching?

Hi, That last thread about caching reminded me of something. Me need is actually the opposite... I use lucene to search in hundreds/thousands of indexes. Doing a lucene query on a set of the indexes is only one of the steps involved in my 'queries', and some of the other steps take l

Re: Caching

On 2/14/07, Mark Miller <[EMAIL PROTECTED]> wrote: Not to get off topic, but I was curious Yonik, what does solr do if many updates come in at a time opening and closing a writer each update...does the first update kick off a warm operation, then before that warm is done the second updates kicks

Re: Caching

Not to get off topic, but I was curious Yonik, what does solr do if many updates come in at a time opening and closing a writer each update...does the first update kick off a warm operation, then before that warm is done the second updates kicks off a warm operation, and then before that warm i

Re: Caching

On 2/14/07, Kainth, Sachin <[EMAIL PROTECTED]> wrote: I have an index with 5.2 million records (each record containing 3 fields) and it sometimes takes about a minute and a half for results to come back. Doe to sort fields (and other factors), the first query can be slow. Solr has built-in supp

RE: Caching

slow and that perhaps I should implement a caching strategy. -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: 14 February 2007 14:11 To: java-user@lucene.apache.org Subject: Re: Caching This is really an unanswerable question, since, to steal a phrase, "It de

Re: Caching

14 feb 2007 kl. 14.57 skrev Kainth, Sachin: I have read that Lucene performs caching of search results so that if you perform the same search in succession the second result should be returned faster. What I wanted to ask is whether this caching is any good or whether it's a good idea t

Re: Caching

This is really an unanswerable question, since, to steal a phrase, "It depends" ... Do you have any reason to believe that the current performance is inadequate for you application? Caching is notoriously difficult to get right, so I wouldn't go there unless there is a *demonst

Caching

Hi all, I have read that Lucene performs caching of search results so that if you perform the same search in succession the second result should be returned faster. What I wanted to ask is whether this caching is any good or whether it's a good idea to add some sort of caching layer on t

RE: Method to speed up caching for faceted navigation

> I don't think it really matters wether you do deletes on the same > IndexReader -- what matters is if there has been any deletes > done to the > index prior to opening the reader since it was last > optimized. The reason > being that deleting a document just causes a record of the > deletion

Re: Method to speed up caching for faceted navigation

: I looked at the implementation of 'read(int[], int[])' in : 'SegmentTermDocs' and saw that it did the following things: : - check if the document has a frequency higher than 1, and if so read : it; : - check if the document has been deleted, and if so don't add it to the : result; : - store the

  1   2   >