Re: DefaultIndexAccessor

2008-02-05 Thread Jay
Great effort for much improved indexaccessor, Mark! A couple questions and observations: 1. In release(Searcher), you removed a check if the given searcher is the cached one from an earlier version. This could potentially cause problems for some people. 2. The createdSearchers variable is not

Re: Concurrent Indexing + Searching

2008-02-05 Thread ajay_garg
Thanks a ton Mark. I am really obliged to interact with you, who is never hesistant to reply on the slightest of queries. Thanks again. Ajay Garg markrmiller wrote: > > >> Again, if the >> indexerThreads are bombarding the writer continuously, then the moment, >> when >> no indexer is accessi

Re: Extracting terms from a query splitting a phrase.

2008-02-05 Thread Spencer Tickner
I guess to be move concise I'm looking to get all the terms that were searched for so I can highlight them in the original document. After looking through the highlighter contrib class I figure I had found my solution with query.extractTerms. Works great for searches like: genera* -> generally, ge

Re: Extracting terms from a query splitting a phrase.

2008-02-05 Thread Spencer Tickner
Hi Erick, Thanks for your response. I think you're right about the Whitespace anlayzer. I was actually useing the StandardAnalyzer before and tried the Whitespace analyzer to see if the StandardAnalyzer was pulling off the quotes. I guess what I'm trying to mimic is the information found: http://

Re: Extracting terms from a query splitting a phrase.

2008-02-05 Thread Erick Erickson
I don't think WhitespaceAnalyzer is doing what you think it is. From the Javadoc... public class *WhitespaceTokenizer*extends CharTokenizer A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Extracting terms from a query splitting a phrase.

2008-02-05 Thread Spencer Tickner
Hi List, Thanks in advance for the help. I'm trying to extract terms from a query. From the reading I've done a phrase such as "General Act" is considered a term. http://lucene.apache.org/java/docs/queryparsersyntax.html#Terms . However when I'm doing testing to get the extractTerms of my query it

RE: lucene 2.3 in production

2008-02-05 Thread Steven A Rowe
Hi GokulAnand, On 02/05/2008 at 12:33 AM, GokulAnand wrote: > Can some one get me the link to get lucene 2.3 jars. It is considered bad form on this list to reply to an existing thread with a message on a different topic than the one already being discussed - this is called "thread hijacking".

Re: outof memory error

2008-02-05 Thread Erick Erickson
See below: On Feb 5, 2008 9:41 AM, SK R <[EMAIL PROTECTED]> wrote: > Hi, > Thanks for your help Erick. > > I changed my code to flush writer before document add which helps to > reduce memory usage. > Also reducing mergefactor and max buffered docs to some level help me to > avoid this OOM

Re: Lucene Search with relational operators

2008-02-05 Thread Erick Erickson
Did you see this page on the Wiki? http://wiki.apache.org/lucene-java/BooleanQuerySyntax Best Erick On Feb 5, 2008 12:23 AM, GokulAnand <[EMAIL PROTECTED]> wrote: > > Hi all, > > When i do a search ( lucene ) with the combination of relational > operators, > it does not get the required result.

Re: SnapshotDeletionPolicy usage

2008-02-05 Thread Michael McCandless
jm wrote: Thanks for the reply Mike. You can also wrap any other deletion policy (it doesn't have to be KeepOnlyLastCommit). When you want to do a backup, make sure to do try/finally, ie: IndexCommitPoint cp = dp.snapshot(); try { Collection files = cp.getFileNames(); }

Re: SnapshotDeletionPolicy usage

2008-02-05 Thread jm
Thanks for the reply Mike. > You can also wrap any other deletion policy (it doesn't have to be > KeepOnlyLastCommit). > > When you want to do a backup, make sure to do try/finally, ie: > >IndexCommitPoint cp = dp.snapshot(); >try { > Collection files = cp.getFileNames(); > >

Re: Concurrent Indexing + Searching

2008-02-05 Thread Mark Miller
Again, if the indexerThreads are bombarding the writer continuously, then the moment, when no indexer is accessing the writer, may never come. Thus, I invested some of my time, and wrote my own code, to control the sleeping of indexerThreads. I don't know how much of a concern this is. All y

Re: outof memory error

2008-02-05 Thread SK R
Hi, Thanks for your help Erick. I changed my code to flush writer before document add which helps to reduce memory usage. Also reducing mergefactor and max buffered docs to some level help me to avoid this OOM error (eventhough index size is ~1GB). But please clarify below doubts Make s

Re: Concurrent Indexing + Searching

2008-02-05 Thread ajay_garg
Thanks Mark. Just one last thing, this issue seems to be similar to the case, where the Lucene source code says, that if an explicit "flush" method is called on an IndexWriter instance, then again, it will wait for all the indexerThreads to release the writer, and only then will the flush happen

Re: Concurrent Indexing + Searching

2008-02-05 Thread Mark Miller
P.S. About that write bombardment...its still very difficult for that to be a problem. Take a look at the tests. I start a bunch of threads searching as fast as they can, and a bunch of threads writing as fast as they can - nonstop. And still there are plenty of moments where the references h

Re: Concurrent Indexing + Searching

2008-02-05 Thread Mark Miller
ajay_garg wrote: Thanks Mark. Ok, I got your point. So it happens like this : a) If it is me, who is re-opening an IndxReader, at any time, but "manually-programmatically". That is, I don't want a-sort-of-automatic-reopening-of-IndexWriter, then I am fine. Sure...your kind of doing what In

Re: Concurrent Indexing + Searching

2008-02-05 Thread ajay_garg
Thanks Mark. Ok, I got your point. So it happens like this : a) If it is me, who is re-opening an IndxReader, at any time, but "manually-programmatically". That is, I don't want a-sort-of-automatic-reopening-of-IndexWriter, then I am fine. b) If I do wish this automatic-reopening of index (usin

Re: SnapshotDeletionPolicy usage

2008-02-05 Thread Michael McCandless
jm wrote: Hi guys, I want to make use of the possibylity of hot backups in 2.3. If i understand correctly, the only thing i need to do is to open the writers with SnapshotDeletionPolicy, is that correct? Right. SnapshotDeletionPolicy dp = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDe

RE: Luke error browsing to a lucene index

2008-02-05 Thread Mitchell, Erica
Hi Erick The version of Lucene is 2.3.0 and Luke is 0.7.1 from the web site that I was using. I'll give the 0.8 luke version a shot. Thanks erica -Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: 04 February 2008 18:01 To: java-user@lucene.apache.org Subject: Re:

SnapshotDeletionPolicy usage

2008-02-05 Thread jm
Hi guys, I want to make use of the possibylity of hot backups in 2.3. If i understand correctly, the only thing i need to do is to open the writers with SnapshotDeletionPolicy, is that correct? SnapshotDeletionPolicy dp = new SnapshotDeletionPolicy(new KeepOnlyLastCommitDeletionPolicy()); final I