RE: splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-11-04 Thread Stephen GRAY
dless [mailto:luc...@mikemccandless.com] Sent: Monday, 4 November 2013 9:49 PM To: Lucene Users Subject: Re: splitting docIds from a search by segment [SEC=UNOFFICIAL] On Sun, Nov 3, 2013 at 7:59 PM, Stephen GRAY wrote: > UNOFFICIAL > > Hi Mike, > > I ran it again and this time the

RE: splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-11-03 Thread Stephen GRAY
) to get the sub-readers. The >>docIDs are just "appended" from these sub-readers, so you'd walk your >>docIDs and also walk you sub-readers, moving to the next sub-reader >>once you have a docID that's beyond its end. Each sub-reader spans >>Atomi

RE: splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-11-03 Thread Stephen GRAY
UNOFFICIAL That's what I did. You just pass searcher.search a very large value for max docs so you get them all, then iterate through the ScoreDoc[] array - the docId is in scoreDoc.doc. Regards, Steve Stephen Gray Java Developer Border Midrange Systems Support Department of Immigratio

RE: splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-11-03 Thread Stephen GRAY
UNOFFICIAL Hi Kyle, Actually the MultiDocValues method turned out to be about twice as fast as walking the docIds and sub-readers, which I wasn't expecting. I'm happy to post the code I used if you want to try both methods. Regards, Steve Stephen Gray Java Developer Border Midran

RE: splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-10-30 Thread Stephen GRAY
ond its end. Each sub-reader spans AtomicReaderContext.docBase to docBase + AtomicReaderContext.reader.maxDoc(). Mike McCandless http://blog.mikemccandless.com On Wed, Oct 30, 2013 at 2:21 AM, Stephen GRAY wrote: > UNOFFICIAL > Hi everyone, > > I am trying to write an applicat

splitting docIds from a search by segment [SEC=UNOFFICIAL]

2013-10-29 Thread Stephen GRAY
UNOFFICIAL Hi everyone, I am trying to write an application that loops through 500,000 - 1,000,000 documents returned by a search and calculates some statistics using the value in a stored field. Obviously this needs to be as fast as possible so I am using a NumericDocValues field to store the

RE: Retrieving values for a NumericDocValuesField [SEC=UNOFFICIAL]

2013-10-24 Thread Stephen GRAY
UNOFFICIAL Hi Adrien, Thanks for your help, I'll try that. Regards, Steve Stephen Gray Java Developer Border Midrange Systems Support Department of Immigration and Border Protection Phone: (02) 6223 9207 Mobile: 0419 885 959 -Original Message- From: Adrien Grand [mailto

RE: Retrieving values for a NumericDocValuesField [SEC=UNOFFICIAL]

2013-10-23 Thread Stephen GRAY
you can get similar performance measurement. Best regards, Duke If not now, when? If not me, who? On Thu, Oct 24, 2013 at 7:18 AM, Stephen GRAY wrote: > UNOFFICIAL > > Thanks to Mike and Adrien for the helpful replies. > > I actually need to loop through a large number of do

RE: Retrieving values for a NumericDocValuesField [SEC=UNOFFICIAL]

2013-10-23 Thread Stephen GRAY
emccandless.com On Wed, Oct 23, 2013 at 3:46 AM, Adrien Grand wrote: > Hi Stephen, > > On Wed, Oct 23, 2013 at 9:29 AM, Stephen GRAY > wrote: >> UNOFFICIAL >> Hi everyone, >> >> I have a question about how to retrieve the values in a >> NumericDocVa

Retrieving values for a NumericDocValuesField [SEC=UNOFFICIAL]

2013-10-23 Thread Stephen GRAY
UNOFFICIAL Hi everyone, I have a question about how to retrieve the values in a NumericDocValuesField. I understand how to do this in situations where you have an AtomicReaderContext available (context.reader().getNumericDocValues(field)). However in a situation where I have just done a search

Calculating min, max and sum of a field in docs returned by search [SEC=UNOFFICIAL]

2013-10-13 Thread Stephen GRAY
UNOFFICIAL Hi everyone, I'd appreciate some help with a problem I'm having. I have a collection of documents in my index. Each doc contains an IntField with a value in it. What I want is to find out the minimum, maximum and sum of this field for all documents returned by a search. I was thinki

Re: How Can I let my many application can Know the Index change,but Not need re-open Index.

2007-05-24 Thread Stephen Gray
n > I use RMI or ICE ? > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- Stephen Gray Archive IT Officer Australian Social Science Data Archive 18 Balmain Crescent (Building #66) The Australian National University

Re: Memory leak (JVM 1.6 only)

2007-05-20 Thread Stephen Gray
Thanks, the link was helpful. I'll let you know if I find anything. Thanks for all the replies to this. Steve Doron Cohen wrote: Stephen Gray wrote: Thanks. If the extra memory allocated is native memory I don't think jconsole includes it in "non-heap" as it

Re: Memory leak (JVM 1.6 only)

2007-05-17 Thread Stephen Gray
e Doron Cohen wrote: Stephen Gray <[EMAIL PROTECTED]> wrote on 17/05/2007 22:40:01: One interesting thing is that although the memory allocated as reported by the processes tab of Windows Task Manager goes up and up, and the JVM eventually crashes with an OutOfMemory error, the total size of h

Re: Memory leak (JVM 1.6 only)

2007-05-17 Thread Stephen Gray
indexing that are the source of OOM and not something else, such as a bug in Tomcat? Otis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simpy -- http://www.simpy.com/ - Tag - Search - Share - Original Message From: Stephen Gray <[EMAIL PROTECTED]> To: java-user

Re: Memory leak (JVM 1.6 only)

2007-05-15 Thread Stephen Gray
leak. Keep playing with the Netbeans profiler...if it really is a leak, you aught to be able to find it. - Mark Stephen Gray wrote: Hi everyone, I have an application that indexes/searches xml documents using Lucene. I'm having a problem with what looks like a memory leak, which o

Memory leak (JVM 1.6 only)

2007-05-14 Thread Stephen Gray
Hi everyone, I have an application that indexes/searches xml documents using Lucene. I'm having a problem with what looks like a memory leak, which occurs when indexing a large number of documents, but only when the application is running under JVM 1.6. Under JVM 1.5 there is no problem. What

Re: Updating index if there is a database changes

2006-05-03 Thread Stephen Gray
ta will be huge... 4 million records something.. Thanks in advance, --Kiran - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] Stephen Gray Archive Research Officer Australian Social Sci

Re-creating IndexSearcher after update

2006-03-20 Thread Stephen Gray
Hi everyone, I have an application that allows both indexing and searching. I'm using a single IndexSearcher shared among all sessions. I want to re-create the IndexSearcher whenever the index changes, but I'm assuming that if I do this bad things will happen to people who may be currently r

Re: ArrayIndexOutOfBounds being thrown ...

2006-02-23 Thread Stephen Gray
Hi everyone, Sorry for not replying to original post (from Muffadal Khumri, 22/2) - I'm new to the list. I also had this problem, but it seems not to be in the source - downloading and building the1.9-rc1 source fixed the problem for me. Steve Stephen Gray Archive Research Of