Re: IndexSearcher in servlet containers

- Original Message - From: "Erik Hatcher" <[EMAIL PROTECTED]> To: Sent: Thursday, October 06, 2005 2:10 AM Subject: Re: IndexSearcher in servlet containers > > On Oct 5, 2005, at 9:03 PM, Cyril Barlow wrote: > > > > > > >>> I'm getting : > >>> > >>> java.io.IOException: The handle is i

Re: IndexSearcher in servlet containers

On Oct 5, 2005, at 9:03 PM, Cyril Barlow wrote: I'm getting : java.io.IOException: The handle is invalid at java.io.RandomAccessFile.seek(Native Method) at Did you perhaps close the IndexSearcher somewhere along the way? Erik No, but I'm now creating new IndexSearchers every s

Re: IndexSearcher in servlet containers

> > I'm getting : > > > > java.io.IOException: The handle is invalid at > > java.io.RandomAccessFile.seek(Native Method) at > > Did you perhaps close the IndexSearcher somewhere along the way? > > Erik No, but I'm now creating new IndexSearchers every servlet doPost method and not closing

Re: RemoteSearchable and sorting

Thanks Rasik. If this is the case, why is this exposed in the API? Should the overloaded search method on ParallelMultiSearcher that takes a Sort object be removed? I'm using the 1.4.3 codebase. -j On 10/5/05, Rasik Pandey <[EMAIL PROTECTED]> wrote: > > Hi Jeff, > > Sorting needs access to an I

Re: IndexSearcher in servlet containers

On Oct 5, 2005, at 8:15 PM, Cyril Barlow wrote: Using one IndexSearcher across the whole application doesn't seem to work. It works for lucenebook.com :) I'm getting : java.io.IOException: The handle is invalid at java.io.RandomAccessFile.seek(Native Method) at Did you perhaps close the

Re: RemoteSearchable and sorting

Hi Jeff, Sorting needs access to an IndexReader so it can do Term lookups, and I don't think there is a remote impl of IndexReader probably because, among other reasons, not all objects related to Terms are Serializable. IMHO, it would be NICE to have a RemoteReader and a ParallelMultiReader to ro

Re: IndexSearcher in servlet containers

- Original Message - From: "Erik Hatcher" <[EMAIL PROTECTED]> To: Sent: Wednesday, October 05, 2005 8:20 PM Subject: Re: IndexSearcher in servlet containers > > On Oct 5, 2005, at 2:09 PM, Cyril Barlow wrote: > >> I'm really confused on the dilemma here. > >> > >> You can create a start

RemoteSearchable and sorting

Are there known limitations or issues with sorting and RemoteSearchable? I'm encountering problems attempting to sort through a MultiSearcher (ParallelMultiSearcher, actually). I'm using an array of RemoteSearchable objects as the Searchable[] source. If I change the source indexes to be local Inde

Re: Query to return all documents in the index

: method to gather results. But as it turns out, sometimes we just want all of : the documents that match with the filter, sorted by the sort field. Does : anyone know a query that returns all the documents in the index, so that i : could use that in this case? use a ConstantScoreQuery wrapped ar

Re: Regarding Lucene and LSI

gossamerthreadsshows an exchange about this. It seems not yet. BTW, meant to post to the list so forward my last email. Adam On 10/5/05, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > > On 10/5/05, adasal <[EMAIL PROTECTED]

Re: What is a Hits object?

A Hits object is essentially a cache on query results. It caches in 2 ways: 1. When a query returning Hits is requested, only the top 100 document IDs and scores are requested from the scoring system, and the ID/Score pairs are stored in a list in the Hits object. Whenever a document ID, score

Query to return all documents in the index

Hi, In my project we've been using the Searcher.search(query, filter, sort) method to gather results. But as it turns out, sometimes we just want all of the documents that match with the filter, sorted by the sort field. Does anyone know a query that returns all the documents in the index, so that

Query returns an empty result set; but it is not empty actually!

Hi, I have a dictionary. It is indexed as follows: private Document buildDocument(SozlukBirimi birim){ Document doc = new Document(); doc.add(Field.Keyword("soz", birim.getSoz())); doc.add(Field.Text("soz1", birim.getSoz())); doc.add(Field.Text("anlam", birim.getAnl

RE: What is a Hits object?

Hits is a list of reference points to Documents, it does not contain the entire document, only when you ask for a document it goes and read the document from the index Aviran http://www.aviransplace.com -Original Message- From: Cyril Barlow [mailto:[EMAIL PROTECTED] Sent: Wednesday, Octo

What is a Hits object?

Is it an actual array of full Documents or a list of reference points to Documents? And what's the typical size in memory of a Hits object with say 1000 avg size docs? ___ Yahoo! Messenger - NEW crystal cl

Re: IndexSearcher in servlet containers

On Oct 5, 2005, at 2:09 PM, Cyril Barlow wrote: I'm really confused on the dilemma here. You can create a startup hook using one of the Servlet specification listeners, create an IndexSearcher there, stuff it into application scope (context.setAttribute()). There is no digging into Jetty's gut

RE: Lucene Security Advice

The simple solution is to put each section in a separate field and query the appropriate fields according to the user group. Aviran http://www.aviransplace.com -Original Message- From: Steven Thompson [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 05, 2005 2:04 PM To: java-user@luce

RE: Lucene Security Advice

I'm sorry, the moment I pressed the send-button I realized that this is not a solution for your problem. A solution might be to index split up the sections into different documents with a field 'section' on which you can filter. Regards, Daan -Original Message- From: Daan de Wit [mailto:

RE: Lucene Security Advice

Hi Steve, I think you can best split up the document into separate fields, and then use FilteredQueries to search. The QueryFilter (which does not effect scoring) can be built using a BooleanQuery with the sections the user can search on as optional Terms Regards, Daan -Original Message-

Lucene Security Advice

The simple scenario is this: One large text document with three sections Section A Section B Section C Three user groups: User Group X User Group Y User Group Z I want to provide security to the various sections of the document to certain groups of users. For example, User Group X can only sea

Re: IndexSearcher in servlet containers

- Original Message - From: "Erik Hatcher" <[EMAIL PROTECTED]> To: Sent: Wednesday, October 05, 2005 6:34 PM Subject: Re: IndexSearcher in servlet containers > > On Oct 5, 2005, at 9:54 AM, Cyril Barlow wrote: > > Thanks all for your feedback. I'm going to look at building a > > dedicate

Re: IndexWriter.optimize() need to much time.

Eric Louvard wrote: my problem is that IndexWriter.optimize() take 20 minutes. OK it is not a lot of time, but I can't allow me to block the system such a long time :-(. If you're worried about blocking, queue changes to the index and have a separate thread which processes the queue, adding a

Re: IndexSearcher in servlet containers

On Oct 5, 2005, at 9:54 AM, Cyril Barlow wrote: Thanks all for your feedback. I'm going to look at building a dedicated server that keeps one IndexSearcher open and each servlet would use that. Either that or look deeper into the Jetty framework to see if I can simply invoke a global IndexS

RE: partial reindex

Hi Erik Practically it is a big performance issue by access external security data through filter. And it could take even longer than the end user could wait. It is true that it is an extra cost to rebuild the security index but it seems there are no other better options. Regards, hui -Origi

Re: IndexWriter.optimize() need to much time.

this approach has one pitfall: merging indexes call optimize() two times (before and after merge) Mordo, Aviran (EXP N-NANNATEK) wrote: While optimizing you can create a new temp index (you can even use a RAMDirectory - since it'll be fairly small), and index all the new documents there. W

RE: IndexWriter.optimize() need to much time.

While optimizing you can create a new temp index (you can even use a RAMDirectory - since it'll be fairly small), and index all the new documents there. When you are finished with the optimizing just merge the temp into the main index. HTH Aviran http://www.aviransplace.com -Original Mes

Re: IndexWriter.optimize() need to much time.

you can do indexing in separate thread keeping waiting documents in some kind of queue waiting to be indexed. Eric Louvard wrote: Of course I can acces IndexReader, but I need to acces IndexWriter during optimization. If I am using a 'temp-index' I need to merge it with the optimized index ho

Re: IndexWriter.optimize() need to much time.

Of course I can acces IndexReader, but I need to acces IndexWriter during optimization. If I am using a 'temp-index' I need to merge it with the optimized index how can I do it ? Thanks. Éric Mordo, Aviran (EXP N-NANNATEK) wrote: The index is available for search even during optimization,

Re: IndexWriter.optimize() need to much time.

Just use 2 different indices? One temp one that is being optimized and one current being used. Then switch the paths on your application when the temp one is ready. - Original Message - From: "Eric Louvard" <[EMAIL PROTECTED]> To: Sent: Wednesday, October 05, 2005 3:10 PM Subject: IndexWr

RE: IndexWriter.optimize() need to much time.

The index is available for search even during optimization, you should not have any problem with that. Aviran http://www.aviransplace.com -Original Message- From: Eric Louvard [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 05, 2005 10:10 AM To: java-user@lucene.apache.org Subject: I

IndexWriter.optimize() need to much time.

Hello, my problem is that IndexWriter.optimize() take 20 minutes. OK it is not a lot of time, but I can't allow me to block the system such a long time :-(. I don't know when I can take the time for optimize(). Has someone allready find a workarround for this problem ? Is it possible to make

Re: IndexSearcher in servlet containers

Thanks all for your feedback. I'm going to look at building a dedicated server that keeps one IndexSearcher open and each servlet would use that. Either that or look deeper into the Jetty framework to see if I can simply invoke a global IndexSearcher when the server starts and use that. Not quite s

RE: IndexSearcher in servlet containers

Take a look at the DelayCloseIndexSearcher I contributed yesterday. http://issues.apache.org/jira/browse/LUCENE-445 You should set up a SearcherFactory in an object that implements ServletContextListener that receives webapp startup/shutdown events and your servlets should get an IndexSearcher fr

Re: IndexSearcher in servlet containers

Cache IndexSearcher and only use *one* instance for all requests. Application scope works well for this in a servlet environment. Erik On Oct 5, 2005, at 9:15 AM, Cyril Barlow wrote: Has anyone got experience of using the IndexSearcher in a servlet? I'm having caching problems when the

Re: Optimization

On Oct 5, 2005, at 9:05 AM, Aigner, Thomas wrote: Have a question.. Is there any obvious things that can be done to help speed up query lookups especially wildcard searches (i.e. *lamps). Obvious? Sort of. *lamps needs to scan through _every_ single term in the index (for the specified

RE: IndexSearcher in servlet containers

There where no problems for me. Do you use the same IndexReader for all your searchers ? Aviran http://www.aviransplace.com -Original Message- From: Cyril Barlow [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 05, 2005 9:15 AM To: java-user@lucene.apache.org Subject: IndexSearcher in

Re: partial reindex

On Oct 5, 2005, at 8:56 AM, Eugeny N Dzhurinsky wrote: On Wed, Oct 05, 2005 at 08:38:21AM -0400, Erik Hatcher wrote: But could Lucene mix up 2 indexes in single query? Using ParallelReader - yes. Read the javadocs to learn more. May be MultiReader? I didn't find ParallelReader in my API

Re: Regarding Lucene and LSI

There are LSI and LSI like implementations in open source, but I am unsure what the state of play is in Java. The most interesting work I know of is in Perl available (cvs permitting) from NITLE. What I would like to see is an implementation of Magnus

Re: partial reindex

> May be MultiReader? I didn't find ParallelReader in my API docs for Lucene 1.4.3. It's not in 1.4.3... you need to check out the latest 1.9 development version from Subversion (the source code repository used now). -Yonik Now hiring -- http://tinyurl.com/7m67g On 10/5/05, Eugeny N Dzhurinsky

IndexSearcher in servlet containers

Has anyone got experience of using the IndexSearcher in a servlet? I'm having caching problems when there's a lot of different concurrent users with the current setup. Currently the setup is to create a IndexSearcher per servlet doPost call and close it after. But with concurrent requests - espe

Optimization

Howdy all, Have a question.. Is there any obvious things that can be done to help speed up query lookups especially wildcard searches (i.e. *lamps). We have created a server application on a linux box that listens to a socket and processes searches as they come in. We thought th

Regarding Lucene and LSI

Hi all, I am looking for LSI implementation i lucene. Is it available. I couldnt find it in the website. I searched in the archives but no help. could some one tell me if it is available or not. Could you tell me where can i see to find if there are any Language processing tools for Indexing and

Re: partial reindex

On Wed, Oct 05, 2005 at 08:38:21AM -0400, Erik Hatcher wrote: > > But could Lucene mix up 2 indexes in single query? > Using ParallelReader - yes. Read the javadocs to learn more. May be MultiReader? I didn't find ParallelReader in my API docs for Lucene 1.4.3. Im'm trying to think in this way:

Re: partial reindex

On Oct 5, 2005, at 7:38 AM, Eugeny N Dzhurinsky wrote: On Wed, Oct 05, 2005 at 07:03:45AM -0400, Erik Hatcher wrote: On Oct 5, 2005, at 4:01 AM, Eugeny N Dzhurinsky wrote: Is it possible somehow to change some partial fields in indexed documents without reindexing all documents? No, not wit

Multiple Index in other machine

Hi, I want to know about your experience about indexs used by others machines. I have a multiple index in a Machine, and I search in these indexes with a Multisearcher. Now I new to add another computer to the platform and I don't know well how can I do to have this multiple index accessible f

Re: partial reindex

On Wed, Oct 05, 2005 at 07:03:45AM -0400, Erik Hatcher wrote: > On Oct 5, 2005, at 4:01 AM, Eugeny N Dzhurinsky wrote: > >Is it possible somehow to change some partial fields in indexed > >documents without reindexing all documents? > No, not with Lucene 1.4.3. But the Subversion trunk has a fea

Re: partial reindex

On Oct 5, 2005, at 4:01 AM, Eugeny N Dzhurinsky wrote: Is it possible somehow to change some partial fields in indexed documents without reindexing all documents? No, not with Lucene 1.4.3. But the Subversion trunk has a feature that can facilitate this sort of thing by building two index

partial reindex

Is it possible somehow to change some partial fields in indexed documents without reindexing all documents? The thing is we have set of "searchable" documents and set of access privileges (which builds the tree-like structure, i'e access privileges could be inherited from parent node) for these

Re: Renewing IndexSearcher on index change.

J.J. Larrea wrote: At 6:39 PM +0200 10/4/05, Olivier Jaquemet wrote: In every case I think I will use this to prevent any problem but why nobody uses finalize methods? is it somehow bad to try to close things correctly that way? Because they are not run under "brutal termination" co