Re: How do I build a DocValuesIterator similar to the way FieldCache can be used?

Answering my own question in case someone fails to find the answer easily. As pointed out in this really helpful discussion , MultiDocValues can be used. I t

How do I build a DocValuesIterator similar to the way FieldCache can be used?

Greetings, I'm trying to use BinaryDocValues but I cannot find a way to create an instance of this type in the way FieldCache can be used: that is, provide an IndexReader and get back the field cache. DocValues.getBinary is expecting a LeafReader and I cannot use it by simply providi

Re: what's replacement of FieldCache in Lucene 7

:27, Yonghui Zhao a > > > écrit : > > > > > > > My case is when I get some docs from lucene, I need also get some > field > > > > value of the retrieved docs. > > > > > > > > For example in lucene 4, I use FieldCache like this.

Re: what's replacement of FieldCache in Lucene 7

; processing, not retrieving stored fieds. > > > > Le ven. 13 avr. 2018 à 05:27, Yonghui Zhao a > > écrit : > > > > > My case is when I get some docs from lucene, I need also get some field > > > value of the retrieved docs. > > > > > >

Re: what's replacement of FieldCache in Lucene 7

neck should be query > processing, not retrieving stored fieds. > > Le ven. 13 avr. 2018 à 05:27, Yonghui Zhao a > écrit : > > > My case is when I get some docs from lucene, I need also get some field > > value of the retrieved docs. > > > >

Re: what's replacement of FieldCache in Lucene 7

a écrit : > My case is when I get some docs from lucene, I need also get some field > value of the retrieved docs. > > For example in lucene 4, I use FieldCache like this. > > FieldCache.DEFAULT.getTerms(reader, name, > false).get(locDocId).utf8ToString(); > > FieldC

Re: what's replacement of FieldCache in Lucene 7

My case is when I get some docs from lucene, I need also get some field value of the retrieved docs. For example in lucene 4, I use FieldCache like this. FieldCache.DEFAULT.getTerms(reader, name, false).get(locDocId).utf8ToString(); FieldCache.DEFAULT.getInts(reader, name, false).get(locDocId

Re: what's replacement of FieldCache in Lucene 7

never call it on decreasing doc IDs. If that doesn't work for you, can you describe you use-case, maybe there are better ways to implement what you need. Le jeu. 12 avr. 2018 à 13:54, Yonghui Zhao a écrit : > Hi, > > I am upgrading my project from Lucene 4 to 7. > > FieldCache

what's replacement of FieldCache in Lucene 7

Hi, I am upgrading my project from Lucene 4 to 7. FieldCache is removed in lucene 7, DocValue is replacement? But seems DocValue doesn't support random access. I need random access to get some specified field value quickly. So how to solve it?

Re: How to get document effectively. or FieldCache example

> > > : > > doc.add(new > > > : > > Field("title",section,Field.Store.YES,Field.Index.NOT_ > ANALYZED)); > > > : > > String newRem = new String(rem); > > > : > > > > > : >

Re: How to get document effectively. or FieldCache example

> "").replaceAll(" ", ""),Field.Store.YES,Field.Index.ANALYZED)); > > : > > > > : > > doc.add(new > > : > > Field("field1",Author,Field.Store.YES,Field.Index.NOT_ANALYZED)); > > : > >

Re: How to get document effectively. or FieldCache example

> : > > Field("field2",Book,Field.Store.YES,Field.Index.NOT_ANALYZED)); > : > > doc.add(new > : > > Field("field3",sec,Field.Store.YES,Field.Index.NOT_ANALYZED)); > : > > > : > > writer.addDocument(doc); > : >

Re: How to get document effectively. or FieldCache example

> {/code} : > > : > > On Thu, Apr 20, 2017 at 5:57 PM, Adrien Grand wrote: : > > : > > > IndexSearcher.doc is the right way to retrieve documents. If this is : > > > slowing things down for you, I'm wondering that you might be fetching : > too : > > >

RE: How to get document effectively. or FieldCache example

www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: neeraj shah [mailto:neerajsha...@gmail.com] > Sent: Friday, April 21, 2017 1:22 PM > To: java-user@lucene.apache.org > Subject: Re: How to get document effectively. or FieldCache example > > t

Re: How to get document effectively. or FieldCache example

> IndexSearcher.doc is the right way to retrieve documents. If this is > > > slowing things down for you, I'm wondering that you might be fetching > too > > > many results? > > > > > > Le jeu. 20 avr. 2017 à 14:16, neeraj shah a > > > é

Re: How to get document effectively. or FieldCache example

ing things down for you, I'm wondering that you might be fetching too > > many results? > > > > Le jeu. 20 avr. 2017 à 14:16, neeraj shah a > > écrit : > > > > > Hello Everyone, > > > > > > I am using Lucene 3.6. I have to index around 60k docuem

Re: How to get document effectively. or FieldCache example

g the search when i try to reterive documents from seacher using > > searcher.doc(docid) it slows down the search . > > Please is there any other way to get the document. > > > > Also if anyone can give me an end-to-end example for working FieldCache. > > While implementing the ca

Re: How to get document effectively. or FieldCache example

60k docuemnts. After > performing the search when i try to reterive documents from seacher using > searcher.doc(docid) it slows down the search . > Please is there any other way to get the document. > > Also if anyone can give me an end-to-end example for working FieldCache. > Whi

How to get document effectively. or FieldCache example

example for working FieldCache. While implementing the cache i have : int[] fieldIds = FieldCache.DEFAULT.getInts(indexMultiReader, "id"); now i dont know how to further use the fieldIds for improving search. Please give me an end-to-end example. Thanks Neeraj

How to use FieldCache and Custom Collector to improve search speed

I am using Lucene 3.6 and i am trying to implement FieldCache. I have seen some posts but did not get any clear idea. Can anyone please suggest me any link where i can find proper example of FieldCache and how to use it while searching.

FieldCache missing values?

In an attempt to make some backwards compatibilities (to a lucene 4.7 java 6) we would like to create a custom comparator that will parse a term field and make the sort based on some inner properties of that field. Started out with an implementation for FieldComparator the loaded BinaryDocValues u

Aw: Re: trouble with Collector and FieldCache

13:12 Uhr Von: "Ian Lea" An: java-user@lucene.apache.org Betreff: Re: trouble with Collector and FieldCache How are you storing the id field? A wild guess might be that this error might be caused by having some documents with id stored, perhaps, as a StringField or TextField and some

Re: trouble with Collector and FieldCache

How are you storing the id field? A wild guess might be that this error might be caused by having some documents with id stored, perhaps, as a StringField or TextField and some as an IntField. -- Ian. On Wed, Jan 14, 2015 at 2:07 PM, Sascha Janz wrote: > > hello, > > i am using lucene 4.6. i

trouble with Collector and FieldCache

hello,   i am using lucene 4.6.  in my query i use a collector to get field values.   setNextReader is implemented as below.   public void setNextReader(AtomicReaderContext context) throws IOException { cacheIDs = FieldCache.DEFAULT.getInts(context.reader(), "id", true); } and collect public

Re: FieldCache & DocValues Filter

mal Fields the filter will create a > DocValues for that field using FieldCache. > > Arun > > > On Thu, Jun 6, 2013 at 3:49 PM, Michael McCandless < > luc...@mikemccandless.com> wrote: > > > On Thu, Jun 6, 2013 at 5:35 AM, Robert Muir wrote: > >

Re: FieldCache & DocValues Filter

Hi, Thanks Robert ! This info is exactly what i need. Just for getting myself clear. If the field is a DocValue field the FieldCacheTermsFilter will use the existing DocValues Field. For Normal Fields the filter will create a DocValues for that field using FieldCache. Arun On Thu, Jun 6, 2013

Re: FieldCache & DocValues Filter

On Thu, Jun 6, 2013 at 5:35 AM, Robert Muir wrote: > Its confusing: I think we should rename FieldCacheXXX to DocValuesXXX. +1 Mike McCandless http://blog.mikemccandless.com - To unsubscribe, e-mail: java-user-unsubscr...@luc

Re: FieldCache & DocValues Filter

filters like FieldCacheTermsFilter, TermsFilter i > see that the first filtering take up enough time may be for building the > FieldCache. Subsequent filters are fast enough. > Currently, I am using CachingWrapperFilter over TermsFilter over String > Fields. > > a) I want to reduc

FieldCache & DocValues Filter

Hi Guys, I was trying to better the filtering mechanism for my use case. When i use the existing filters like FieldCacheTermsFilter, TermsFilter i see that the first filtering take up enough time may be for building the FieldCache. Subsequent filters are fast enough. Currently, I am using

Sorting with FieldCache throughout all segments

Hi all! I need to get top document IDs for a query by a custom sort method (e.g. "created" field, or some multi-field condition), I use FieldCache for that purpose, as community members suggested me. With the latest Lucene Atomic* trends, I make per-segment top document lists. It is f

Re: Migrating from using doc IDs to using application IDs from the FieldCache

Unfortunately, t's not possible/easy to just add one new field to all existing docs ... there are several issues open to do this, eg see https://issues.apache.org/jira/browse/LUCENE-4258 and LUCENE-3837 and LUCENE-4272. Mike McCandless http://blog.mikemccandless.com On Thu, Jan 31, 2013 at 8:00

Migrating from using doc IDs to using application IDs from the FieldCache

Hi all. We have an application which has been around for so long that it's still using doc IDs to key to an external database. Obviously this won't work forever (even in Lucene 3.x we had to use a custom merge policy to keep it working) so we want to introduce application IDs eventually. We have

Re: FieldCache

rticle.getCategories()){ >        doc.add(new Field("categories", c.toLowerCase(), >        Field.Store.YES, Field.Index.ANALYZED)); > } > > I have a search space of 2 million docs and I need to access the category > field of each hitdoc. I would like to use FieldCache but since I

FieldCache

hitdoc. I would like to use FieldCache but since I am indexing the field as mutlifield this is a problem. Is there a recommend solution to this problem? thank you Peyman

Re: questions about fieldCache

OK, after some sorting fieldCache has some entries and also all other caches. Next I called optimize which started a new searcher. All caches a cleared, _except_ fieldCache. I then started a GC with jconsole and the logfile reported "Full GC". The heap reduced its size but the fie

Re: questions about fieldCache

> So action that starts a new searcher and closes the old one (like > replication) > should release cache from fieldCache through garbage collection? Absolutely. It won't be immediate, because the JVM has some heuristics it uses to initiate garbage collection. You could try attachi

Re: questions about fieldCache

Currently I'm using version 3.2. I used already 4.x some month ago but there was to much change to that time so I decided to go with 3.0.x and updated to 3.1 and now to 3.2. I'm still dealing with my fieldCache OOM issue and want to understand why things are as they are. I have alrea

Re: questions about fieldCache

g wrote: > I'm trying to understand the logic of/behind fieldCache. > > Who has written this peace of code or has good knowledge about it? > > Why is it under the hood of jetty? > > I see FieldCache$StringIndex with > - f_dccollection > - f_dcyear > - f_dctype &g

questions about fieldCache

I'm trying to understand the logic of/behind fieldCache. Who has written this peace of code or has good knowledge about it? Why is it under the hood of jetty? I see FieldCache$StringIndex with - f_dccollection - f_dcyear - f_dctype but also - dctitle --> f_dctitle --> f_dccrea

best way to update custom fieldcache after index commit?

Hi, We use Solr, but this seems more like a lucene related issue. So we use lucene fieldcache to like this static DocTerms myfieldvalues = org.apache.lucene.search.FieldCache.DEFAULT.getTerms(reader, "myField"); which is initialized at first use and will stay in memory for fast re

RE: FieldCache and 2.9

Carl -Original Message- From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of Yonik Seeley Sent: 11 May 2010 14:41 To: java-user@lucene.apache.org Subject: Re: FieldCache and 2.9 You are requesting the FieldCache entry from the top-level reader and hence a whole new FieldCache

Re: FieldCache and 2.9

You are requesting the FieldCache entry from the top-level reader and hence a whole new FieldCache entry must be created. Lucene 2.9 sorting requests FieldCache entries at the segment level and hence reuses entries for those segments that haven't changed. -Yonik Apache Lucene Eurocon 2010

FieldCache and 2.9

Hi, I have been using the FieldCache in lucene version 2.9 compared to that in 2.4. The load time is massively decreased, however I am not seeing any benefit in getting a field cache after re-open of an index reader when I have only added a few extra documents. A small test class is included

Re: FieldCache memory estimation - term values are interned?

Yonik Seeley wrote: On Sat, May 1, 2010 at 8:23 PM, Koji Sekiguchi wrote: Yonik Seeley wrote: Values are not interned, but in a single field cache entry (String[]) the same String object is used for all docs with that same value. Yeah, you are right. Because I could see the arb

Re: FieldCache memory estimation - term values are interned?

On Sat, May 1, 2010 at 8:23 PM, Koji Sekiguchi wrote: > Yonik Seeley wrote: >> Values are not interned, but in a single field cache entry (String[]) >> the same String object is used for all docs with that same value. > > Yeah, you are right. Because I could see the arbitrary two Strings that have

Re: FieldCache memory estimation - term values are interned?

Yonik Seeley wrote: 2010/4/30 Koji Sekiguchi : Are Strings that are got via FieldCache.DEFAULT.getStrings( reader, field ) interned? Since I have a requirement for having FieldCaches of some fields in 250M docs index, I'd like to estimate memory consumed by FieldCache. By looki

Re: FieldCache memory estimation - term values are interned?

2010/4/30 Koji Sekiguchi : > Are Strings that are got via FieldCache.DEFAULT.getStrings( reader, > field ) interned? > > Since I have a requirement for having FieldCaches of some > fields in 250M docs index, I'd like to estimate memory > consumed by FieldCache. > >

FieldCache memory estimation - term values are interned?

Hello, Are Strings that are got via FieldCache.DEFAULT.getStrings( reader, field ) interned? Since I have a requirement for having FieldCaches of some fields in 250M docs index, I'd like to estimate memory consumed by FieldCache. By looking at FieldCacheImpl source code, it seems that

Re: custom FieldCache cost too much time. how can I preload the the custom fieldCache when new segment exits!

If you look at the javadocs for IndexWriter it explains how to do it. You just provide a class that implements the warm method, and inside that method you do whatever app specific things you need to do to the provided IndexReader to warm it. Note that the SearcherManager class from LIA2 handles se

Re: FieldCache cost too much time. how can I preload the the custom fieldCache when new segment exits!

Sounds like you should simply open & warm the reader in a background thread... You might want to use the SearcherManager class from upcoming Lucene in Action 2nd edition (NOTE: I'm a co-author). You can download the source code @ http://manning.com/hatcher3. Mike ---

答复: custom FieldCache cost too m uch time. how can I preload the the custom fieldCache when new segment exits!

@lucene.apache.org 主题: 答复: custom FieldCache cost too much time. how can I preload the the custom fieldCache when new segment exits! I set merge factor 4, every five minute I reopen the reader. yes most of the time is very fast. But sometimes it is very slow. For example,when start the program,the

答复: custom FieldCache cost too m uch time. how can I preload the the custom fieldCache when new segment exits!

,my English is not good! I hope I can preload custom fieldCache in a another thread,not the query thread. So I will not have performace issue. -邮件原件- 发件人: Michael McCandless [mailto:luc...@mikemccandless.com] 发送时间: 2010年2月27日 18:37 收件人: java-user@lucene.apache.org 主题: Re: custom FieldCache

Re: custom FieldCache cost too much time. how can I preload the the custom fieldCache when new segment exits!

d its still not fast enough, then you should just warm the reader before using it (additionally, for a near-real-time reader you should warm newly merged segments by installing a mergedSegmentWarmer on the writer). Mike On Sat, Feb 27, 2010 at 3:35 AM, luocanrao wrote: > custom FieldCache cost

custom FieldCache cost too much time. how can I preload the the custom fieldCache when new segment exits!

custom FieldCache cost too much time. So every first time,reopen the new reader ,it interfere the performance of search I hope someone can tell me,how can I preload the the custom fieldCache when new segment exits! Thanks again! here is source , In FieldComparator.setNextReader method

Re: How to limit fields being loaded into the FieldCache ?

Sounds like you need a better search engine ;) Mike On Sun, Nov 15, 2009 at 10:21 PM, Wenbo Zhao wrote: > Sorry, all folks, please ignore this thread. > I found the section in doc, just start to read that. > I just used wrong term to search before :-) > I searched for 'Field

Re: How to limit fields being loaded into the FieldCache ?

Sorry, all folks, please ignore this thread. I found the section in doc, just start to read that. I just used wrong term to search before :-) I searched for 'FieldCache' but in the book it's 'Field cache' Anyway just ignore this. 2009/11/15 Wenbo Zhao : > Hi all, >

How to limit fields being loaded into the FieldCache ?

Hi all, In 'Lucene in Action', I read "Limit how many fields you directly load into the FieldCache" to reduce RAM usage during searching. But I can't find how. In javadoc, I can't find any note about how FieldCache is used in IndexReader or IndexSearcher. Can s

Re: Extending Sort/FieldCache

I think CSF hasn't been implemented because it's only marginally useful yet requires fairly significant rewrites of core code (i.e. SegmentMerger) so no one's picked it up including myself. An interim solution that fulfills the same function (quickly loading field cache values) using what works rel

Re: Extending Sort/FieldCache

I didn't say we won't need CSF, but that at least conceptually, CSF and my sort-by-payload are the same. If however it turns out that CSF performs better, then I'll definitely switch my sort-by-payload package to use it. I thought that CSF is going to be implemented using payloads, but perhaps I'm

Re: Extending Sort/FieldCache

On Sun, Sep 6, 2009 at 4:42 AM, Shai Erera wrote: >> I've resisted using payloads for this purpose in Solr because it felt >> like an interim hack until CSF is implemented. > > I don't see it as a hack, but as a proper use of a great feature in Lucene. It's proper use for an application perhaps, b

Re: Extending Sort/FieldCache

: Thanks Mike. I did not phrase well my understanding of Cache reload. I : didn't mean literally as part of the reopen, but *because* of the reopen. : Because FieldCache is tied to an IndexReader instance, after reopen it gets : refreshed. If I keep my own Cache, I'll need to code that

Re: Extending Sort/FieldCache

> > I've resisted using payloads for this purpose in Solr because it felt > like an interim hack until CSF is implemented. > I don't see it as a hack, but as a proper use of a great feature in Lucene. CSF and this are essentially the same. Adding document-level metadata, or adding a term which put

Re: Extending Sort/FieldCache

On Fri, Sep 4, 2009 at 12:33 AM, Shai Erera wrote: > 2) Contribute my payload-based sorting package. Currently it only reads from > disk during searches, and I'd like to enhance it to use in-memory cache as > well. It's a moderate-size package, so this one will need to wait until (1) > is done, and

Re: Extending Sort/FieldCache

rote: > Thanks Mike. I did not phrase well my understanding of Cache reload. I > didn't mean literally as part of the reopen, but *because* of the reopen. > Because FieldCache is tied to an IndexReader instance, after reopen it gets > refreshed. If I keep my own Cache, I'll ne

Re: Extending Sort/FieldCache

Thanks Mike. I did not phrase well my understanding of Cache reload. I didn't mean literally as part of the reopen, but *because* of the reopen. Because FieldCache is tied to an IndexReader instance, after reopen it gets refreshed. If I keep my own Cache, I'll need to code that logic, an

Re: Extending Sort/FieldCache

On Fri, Sep 4, 2009 at 12:33 AM, Shai Erera wrote: > 1) Refactor the FieldCache API (and TopFieldCollector) such that one can > provide its own Cache of native values. I'd hate to rewrite the > FieldComparators logic just because the current API is not extendable. That > I agre

Re: Extending Sort/FieldCache

Thanks I plan to look into two things, and then probably create two separate issues: 1) Refactor the FieldCache API (and TopFieldCollector) such that one can provide its own Cache of native values. I'd hate to rewrite the FieldComparators logic just because the current API is not exten

Re: Extending Sort/FieldCache

: I wanted to avoid two things: : * Writing the logic that invokes cache-refresh upon IndexReader reload. Uh... i don't think there is any code that FieldCache refreshing on reload (yet), so you wouldn't be missing out on anything. (as long as your custom cache works at the Seg

Re: Extending Sort/FieldCache

stomizable" such that someone can pass in his own FieldCachePQ. I'll start w/ my own version and if possible suggest an extension to TFC. Thanks again, Shai On Tue, Aug 25, 2009 at 8:36 PM, Chris Hostetter wrote: > > : I'd like to extend Lucene's FieldCache su

Re: Extending Sort/FieldCache

: I'd like to extend Lucene's FieldCache such that it will read native values ... : payload value to long, and store it in the cache. The reason I want to : extend Lucene's FieldCache is because I'd like Lucene to take care of : updating this cache when necessary (such

Extending Sort/FieldCache

Hi I'd like to extend Lucene's FieldCache such that it will read native values from a different place (in my case, payloads). That is, instead of iterating on a field's terms and parsing each String to long (for example), I'd like to iterate over one term (sort:long, aga

Re: question about (problem with?) use of FieldCache$StringIndex

Otis Gospodnetic wrote: FieldCache class is used for sorting. Are you sorting by a few different fields by any chance? Yes, we're sorting for one or two fields, depending on user settings. Uwe Schindler wrote: This class is used, when you sort your result against a field, which con

Re: question about (problem with?) use of FieldCache$StringIndex

Ah, the trusted LIA... :) FieldCache class is used for sorting. Are you sorting by a few different fields by any chance? Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message > From: Ulf Dittmer > To: java-user@lucene.apache.org > Sent:

RE: question about (problem with?) use of FieldCache$StringIndex

...@thetaphi.de > -Original Message- > From: Ulf Dittmer [mailto:u...@ulfdittmer.com] > Sent: Thursday, June 25, 2009 11:14 PM > To: java-user@lucene.apache.org > Subject: question about (problem with?) use of FieldCache$StringIndex > > Hello- > > We're looki

question about (problem with?) use of FieldCache$StringIndex

Hello- We're looking at memory issues we're having with a fair-sized web app that uses Lucene for search. While looking at heap dumps, we discovered that there were 3 instances of org.apache.lucene.search.FieldCache$StringIndex, each about 110MB in size (out of a total of 1 GB). Looking

Re: FieldCache Question

th String sorts. > > That is, unless you are creating your own separate FieldCaches on > multisegmentreaders - then you would double everything. > > >> Yes - we're runnning about 80G in the indices so there's not enough RAM >> for >> all the data in the fi

Re: FieldCache Question

0G in the indices so there's not enough RAM for all the data in the fieldcache. That is a large index. Can you share how many documents? - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional comm

Re: FieldCache Question

On Wed, Feb 4, 2009 at 10:01 AM, Mark Miller wrote: > Todd Benge wrote: >> >> Hi, >> >> I've been looking into the FieldCache API because of memory problems >> we've been seeing in our production environment. >> >> We use various differen

Re: FieldCache Question

Todd Benge wrote: Hi, I've been looking into the FieldCache API because of memory problems we've been seeing in our production environment. We use various different sorts so over time the cache builds up and servers stop responding. I decided to apply the patch for JIRA 8

FieldCache Question

Hi, I've been looking into the FieldCache API because of memory problems we've been seeing in our production environment. We use various different sorts so over time the cache builds up and servers stop responding. I decided to apply the patch for JIRA 831: https://issues.apach

Re: how to cache multivalued field using fieldcache.

No. FieldCache is only for single-valued field. You would need to use your own data structure to cache multi-valued field. Or leave the index on disk and use Solid State Disk to read for faster access. -- Chris Lu - Instant Scalable Full-Text Search On Any Database

Re: Document ids collected from HitCollector.collect and used in FieldCache..

On Thu, Apr 17, 2008 at 1:45 PM, Shailendra Mudgal < > > > [EMAIL PROTECTED]> wrote: > > > > > > > Hi All, > > > > > > > > I have a small confusion regarding the document ids which we collect > > > using > > > > HitCollector.collect() met

Re: Document ids collected from HitCollector.collect and used in FieldCache..

, Shailendra Mudgal < > > [EMAIL PROTECTED]> wrote: > > > > > Hi All, > > > > > > I have a small confusion regarding the document ids which we collect > > using > > > HitCollector.collect() method. Here is the description of the > confusion &g

Re: Document ids collected from HitCollector.collect and used in FieldCache..

wrote: > > > Hi All, > > > > I have a small confusion regarding the document ids which we collect > using > > HitCollector.collect() method. Here is the description of the confusion > : > > > > First i created a FieldCache of type > using > a &g

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

Chris Hostetter wrote: : But then the FieldCache is just starting to feel alot like column-stride : fields : (LUCENE-1231). that's what i've been thinking ... my goal with LUCENE-831 was to make it easier to manage FieldCache and hopefully the norms[] as well particularly in t

how to cache multivalued field using fieldcache.

Hi All, I have a multivalued field in the index. Can i use FieldCache for caching that field. Thanks, Vipin

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

: But then the FieldCache is just starting to feel alot like column-stride : fields : (LUCENE-1231). that's what i've been thinking ... my goal with LUCENE-831 was to make it easier to manage FieldCache and hopefully the norms[] as well particularly in the case of reopen ... but w

Re: Document ids collected from HitCollector.collect and used in FieldCache..

ve a small confusion regarding the document ids which we collect using > HitCollector.collect() method. Here is the description of the confusion : > > First i created a FieldCache of type > using a > query which collects all the articles which are only a month old. I am > storing th

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

e and see about possibly keying on the directory or something? I think, with how IndexReader.reopen() now works, we should switch to somehow having the FieldCache "attached" to each SegmentReader instead of stored globally keyed by the top MultiSegmentReader. This way if we do a reopen and s

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

how IndexReader.reopen() now works, we should switch to > somehow having the FieldCache "attached" to each SegmentReader instead > of stored globally keyed by the top MultiSegmentReader. > > This way if we do a reopen and say the only change to the index was 10 > added

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

h how IndexReader.reopen() now works, we should switch to somehow having the FieldCache "attached" to each SegmentReader instead of stored globally keyed by the top MultiSegmentReader. This way if we do a reopen and say the only change to the index was 10 added docs then the only new Field

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

ing 2 reader open at > the same time (I am using solr searchers btw, but I guess these use the same > underlying lucene-code, I'll have to check) can these 2 readers shares the > same fieldcache and thus eliminate the required double memory while > warming? > > Thanks. &

Document ids collected from HitCollector.collect and used in FieldCache..

Hi All, I have a small confusion regarding the document ids which we collect using HitCollector.collect() method. Here is the description of the confusion : First i created a FieldCache of type > using a query which collects all the articles which are only a month old. I am storing them int

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

f for performance of course) Moreover, when warming readers with the patch, thus having 2 reader open at the same time (I am using solr searchers btw, but I guess these use the same underlying lucene-code, I'll have to check) can these 2 readers shares the same fieldcache and thus eliminate th

Re: Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

've seen some recent activity on LUCENE-831 "Complete overhaul of FieldCache > API" and read that it must be able to cleanly patch to trunk (haven't tried > yet). > > What I'd like to know from people involved is if this patch incorporates > offloading of fiel

Does LUCENE-831) "Complete overhaul of FieldCache API" provide fieldcache offloading to disk?

I've seen some recent activity on LUCENE-831 "Complete overhaul of FieldCache API" and read that it must be able to cleanly patch to trunk (haven't tried yet). What I'd like to know from people involved is if this patch incorporates offloading of fieldcache to disk, o

Re: FieldCache performance

-- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Angel Faus <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Tuesday, April 8, 2008 8:15:11 PM Subject: FieldCache performance Hi, I have a puzzling question about FieldCache; we use FieldCache to sp

FieldCache performance

Hi, I have a puzzling question about FieldCache; we use FieldCache to speed up faceted search [just like solr does] and also to avoid creating Document instances when listing results. It works great, but for one field where performance is awful: on the order of 15-30 seconds for one index, and

Re: payload performance wrt fieldcache

. Removed call to > tp.nextPosition, performance improved by a factor of multiple digits. > > I would think this call should be free. > > > > Thanks > > -John > > On Thu, Apr 3, 2008 at 8:16 AM, Chris Lu <[EMAIL PROTECTED]> wrote: > > > If your index si

Re: payload performance wrt fieldcache

; If your index size grows larger, payload method would be more slower. > It's because Payload are read from hard disk. Fieldcache is in the > memory, which is much faster. > > Unless you are going with Solid State Disk, you'd better go with > Fieldca

  1   2   >