Re: Faceted search using Lucene 4.3

2013-05-20 Thread raj
Shai, Looks like this is what I was looking for. Will try out Thansk a lot! Rajesh On Mon, May 20, 2013 at 7:27 PM, Shai Erera wrote: > Raj, > > If I understand your question, you can integrate regular and faceted search > as follows: > > Query q = new TermQuery("f", "word"); > TopDocsCollec

Re: Faceted search using Lucene 4.3

2013-05-20 Thread Shai Erera
Raj, If I understand your question, you can integrate regular and faceted search as follows: Query q = new TermQuery("f", "word"); TopDocsCollector tdc = TopScoreDocsCollector.create(); FacetsCollector fc = FacetsCollector.create(); searcher.search(q, MultiCollector.wrap(tdc, fc)); TopDocs topDoc

Re: Faceted search using Lucene 4.3

2013-05-20 Thread raj
Is it possible to combine normal Lucene search and Facet Search? I have seperately implemented basic search and also Faceted Search in my project based on the sample provided. But a bit confused on how to query the document content which also has associated Facets. On Thu, May 16, 2013 at 11:3

Re: Faceted search using Lucene 4.3

2013-05-15 Thread raj
Thanks a lot Shai! That was really quick response. On Thu, May 16, 2013 at 11:32 AM, Shai Erera wrote: > Hi Raj, > > Unfortunately the userguide is outdated after refactorings made to the > package. We have an issue open to fix that. > Until then, you can find an example code here: > > https://

Re: Faceted search using Lucene 4.3

2013-05-15 Thread Shai Erera
Hi Raj, Unfortunately the userguide is outdated after refactorings made to the package. We have an issue open to fix that. Until then, you can find an example code here: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_4_3/lucene/demo/src/java/org/apache/lucene/demo/facet/SimpleFac

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
So then all is good. We were only pursuing this to explain it. Now that we know your directories are empty, that explains it. So you should call maybeReopen() inside get(), as long as it does not slow queries down. Mike Amin Mohammed-Coleman wrote: I think that is the case. When my

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
I think that is the case. When my SearchManager is initialised the directories are empty so when I do a get() nothing is present. Subsequent calls seem to work. Is there something I can do? or do I accept this or just do a maybeReopen and do a get(). As you mentioned it depends on timiing but I

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
Well the code looks fine. I can't explain why you see no search results if you don't call maybeReopen() in get, unless at the time you first create SearcherManager the Directories each have an empty index in them. Mike Amin Mohammed-Coleman wrote: Hi Here is the code that I am using, I'

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
Hi Here is the code that I am using, I've modified the get() method to include the maybeReopen() call. Again I'm not sure if this is a good idea. public Summary[] search(final SearchRequest searchRequest) throwsSearchExecutionException { final String searchTerm = searchRequest.getSearchTerm();

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
It makes perfect sense to call maybeReopen() followed by get(), as long as maybeReopen() is never slow enough to be noticeable to an end user (because you are making random queries pay the reopen/warming cost). If you call maybeReopen() after get(), then that search will not see the new

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
Hi Just out of curiosity does it not make sense to call maybeReopen and then call get()? If I call get() then I have a new mulitsearcher, so a call to maybeopen won't reinitialise the multi searcher. Unless I pass the multi searcher into the maybereopen method. But somehow that doesn't m

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
In my test case I have a set up method that should populate the indexes before I start using the document searcher. I will start adding some more debug statements. So basically I should be able to do: get() followed by maybeReopen. I will let you know what the outcome is. Cheers Amin On Mon,

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
Is it possible that when you first create the SearcherManager, there is no index in each Directory? If not... you better start adding diagnostics. EG inside your get(), print out the numDocs() of each IndexReader you get from the SearcherManager? Something is wrong and it's best to exp

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
Nope. If i remove the maybeReopen the search doesn't work. It only works when i cal maybeReopen followed by get(). Cheers Amin On Mon, Mar 2, 2009 at 12:56 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > > That's not right; something must be wrong. > > get() before maybeReopen() sh

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
That's not right; something must be wrong. get() before maybeReopen() should simply let you search based on the searcher before reopening. If you just do get() and don't call maybeReopen() does it work? Mike Amin Mohammed-Coleman wrote: I noticed that if i do the get() before the maybeRe

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
I noticed that if i do the get() before the maybeReopen then I get no results. But otherwise I can change it further. On Mon, Mar 2, 2009 at 11:46 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > > There is no such thing as final code -- code is alive and is always > changing ;) > >

Re: Faceted Search using Lucene

2009-03-02 Thread Michael McCandless
There is no such thing as final code -- code is alive and is always changing ;) It looks good to me. Though one trivial thing is: I would move the code in the try clause up to and including the multiSearcher=get() out above the try. I always attempt to "shrink wrap" what's inside a try

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
Hi there Good morning! Here is the final search code: public Summary[] search(final SearchRequest searchRequest) throwsSearchExecutionException { final String searchTerm = searchRequest.getSearchTerm(); if (StringUtils.isBlank(searchTerm)) { throw new SearchExecutionException("Search string ca

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
You're calling get() too many times. For every call to get() you must match with a call to release(). So, once at the front of your search method you should: MultiSearcher searcher = get(); then use that searcher to do searching, retrieve docs, etc. Then in the finally clause, pass that

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Hi The searchers are injected into the class via Spring. So when a client calls the class it is fully configured with a list of index searchers. However I have removed this list and instead injecting a list of directories which are passed to the DocumentSearchManager. DocumentSearchManager is Se

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
I don't understand where searchers comes from, prior to initializeDocumentSearcher? You should, instead, simply create the SearcherManager (from your Directory instances). You don't need any searchers during initialize. Is DocumentSearcherManager the same as SearcherManager (just renamed)? Th

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Sorry...i'm getting slightly confused. I have a PostConstruct which is where I should create an array of SearchManagers (per indexSeacher). From there I initialise the multisearcher using the get(). After which I need to call maybeReopen for each IndexSearcher. So I'll do the following: @PostCo

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
This is not quite right -- you should only create SearcherManager once (per Direcotry) at startup/app load, not with every search request. And I don't see release -- it must call SearcherManager.release of each of the IndexSearchers previously returned from get(). Mike Amin Mohammed-Coleman wr

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Hi Thanks again for helping on a Sunday! I have now modified my maybeOpen() to do the following: private void maybeReopen() throws Exception { LOGGER.debug("Initiating reopening of index readers..."); IndexSearcher[] indexSearchers = (IndexSearcher[]) multiSearcher .getSearchables(); for (

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Hi again... Thanks for your patience, I modified the code to do the following: private void maybeReopen() throws Exception { startReopen(); try { MultiSearcher newMultiSeacher = get(); boolean refreshMultiSeacher = false; List indexSearchers = new ArrayList(); synchronized (searchers)

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
Your maybeReopen has an excess incRef(). I'm not sure how you open the searchers in the first place? The list starts as empty, and nothing populates it? When you do the initial population, you need an incRef. I think you're hitting IllegalStateException because maybeReopen is closing a reader

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
OK new version of SearcherManager, that fixes maybeReopen() so that it can be called from multiple threads. NOTE: it's still untested! Mike package lia.admin; import java.io.IOException; import java.util.HashMap; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.inde

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
sorrry I added release(multiSearcher); instead of multiSearcher.close(); On Sun, Mar 1, 2009 at 2:17 PM, Amin Mohammed-Coleman wrote: > Hi > I've now done the following: > > public Summary[] search(final SearchRequest searchRequest) > throwsSearchExecutionException { > > final String searchT

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Hi I've now done the following: public Summary[] search(final SearchRequest searchRequest) throwsSearchExecutionException { final String searchTerm = searchRequest.getSearchTerm(); if (StringUtils.isBlank(searchTerm)) { throw new SearchExecutionException("Search string cannot be empty. There wi

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
I was wondering the same thing ;) It's best to call this method from a single BG "warming" thread, in which case it would not need its own synchronization. But, to be safe, I'll add internal synchronization to it. You can't simply put synchronized in front of the method, since you don't w

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
just a quick point: public void maybeReopen() throws IOException { //D long currentVersion = currentSearcher.getIndexReader().getVersion(); if (IndexReader.getCurrentVersion(dir) != currentVersion) { IndexReader newReader = currentSearcher.getIndexReader().reopen();

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
thanks. i will rewrite..in between giving my baby her feed and playing with the other child and my wife who wants me to do several other things! On Sun, Mar 1, 2009 at 1:20 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > > Amin Mohammed-Coleman wrote: > > Hi >> Thanks for your inp

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
Amin Mohammed-Coleman wrote: Hi Thanks for your input. I would like to have a go at doing this myself first, Solr may be an option. * You are creating a new Analyzer & QueryParser every time, also creating unnecessary garbage; instead, they should be created once & reused. -- I can moved

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
Hi Thanks for your input. I would like to have a go at doing this myself first, Solr may be an option. * You are creating a new Analyzer & QueryParser every time, also creating unnecessary garbage; instead, they should be created once & reused. -- I can moved the code out so that it is onl

Re: Faceted Search using Lucene

2009-03-01 Thread Michael McCandless
On a quick look, I think there are a few problems with the code: * I don't see any synchronization -- it looks like two search requests are allowed into this method at the same time? Which is dangerous... eg both (or, more) will wastefully reopen the readers. * You are over-inc

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Forgot to mention that the previous code that i sent was related to facet search. This is a general search method I have implemented (they can probably be combined...). On Thu, Feb 26, 2009 at 8:21 PM, Amin Mohammed-Coleman wrote: > Hi > I have modified my search code. Here is the following: >

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Hi I have modified my search code. Here is the following: [code] public Summary[] search(SearchRequest searchRequest) throwsSearchExecutionException { String searchTerm = searchRequest.getSearchTerm(); if (StringUtils.isBlank(searchTerm)) { throw new SearchExecutionException("Search string ca

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Hi Thanks for your help. I will modify my facet search and my other code to use the recommendations. Would it be ok to get a review of the completed code? I just want to make sure that I'm not doing anything that may cause any problems (threading, memory). Cheers On Thu, Feb 26, 2009 at 1:10

Re: Faceted Search using Lucene

2009-02-26 Thread Michael McCandless
See below -- this is an excerpt from the upcoming Lucene in Action revision (chapter 10). It's a simple class. Use it like this for searching: IndexSearcher searcher = manager.get(); try { searcher.search(...). ...render results... } finally { manager.release(searcher); s

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Hi Thanks for your reply. Without sound completely ...silly...how do i go abouts using the methods you mentioned... Cheers Amin On Thu, Feb 26, 2009 at 10:24 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > > Actually, it's best to use IndexReader.incRef/decRef to track the > Index

Re: Faceted Search using Lucene

2009-02-26 Thread Michael McCandless
Actually, it's best to use IndexReader.incRef/decRef to track the IndexReader. You should not rely on GC to close your IndexReader since this can easily tie up resources (eg open file descriptors) for too long. Mike Michael Stoppelman wrote: If another thread is executing a query with t

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Hi Thanks for your reply. I have modified the code to the following: public Map getFacetHitCount(String searchTerm) { QueryParser queryParser = newMultiFieldQueryParser(FieldNameEnum.fieldNameDescriptions(), analyzer); Query baseQuery = null; try { if (!StringUtils.isBlank(searchTerm)) { ba

Re: Faceted Search using Lucene

2009-02-25 Thread Michael Stoppelman
If another thread is executing a query with the handle to one of readers[i] you're going to kill it since the IndexReader is now closed. Just don't call the IndexReader#close() method. If nothing is pointing at the readers they should be garbage collected. Also, you might want to warm up your new I

Re: Faceted Search using Lucene

2009-02-24 Thread Amin Mohammed-Coleman
The reason for the indexreader.reopen is because I have a webapp which enables users to upload files and then search for the documents. If I don't reopen i'm concerned that the facet hit counter won't be updated. On Tue, Feb 24, 2009 at 8:32 PM, Amin Mohammed-Coleman wrote: > Hi > I have been ab

Re: Faceted Search using Lucene

2009-02-24 Thread Amin Mohammed-Coleman
Hi I have been able to get the code working for my scenario, however I have a question and I was wondering if I could get some help. I have a list of IndexSearchers which are used in a MultiSearcher class. I use the indexsearchers to get each indexreader and put them into a MultiIndexReader. Ind

Re: Faceted Search using Lucene

2009-02-22 Thread Amin Mohammed-Coleman
Hi Thanks just what I needed! Cheers Amin On 22 Feb 2009, at 16:11, Marcelo Ochoa wrote: Hi Amin: Please take a look a this blog post: http://sujitpal.blogspot.com/2007/04/lucene-search-within-search-with.html Best regards, Marcelo. On Sun, Feb 22, 2009 at 1:18 PM, Amin Mohammed-Coleman >

Re: Faceted Search using Lucene

2009-02-22 Thread Marcelo Ochoa
Hi Amin: Please take a look a this blog post: http://sujitpal.blogspot.com/2007/04/lucene-search-within-search-with.html Best regards, Marcelo. On Sun, Feb 22, 2009 at 1:18 PM, Amin Mohammed-Coleman wrote: > Hi > > Sorry to re send this email but I was wondering if I could get some advice > o

Re: Faceted Search using Lucene

2009-02-22 Thread Amin Mohammed-Coleman
Hi Sorry to re send this email but I was wondering if I could get some advice on this. Cheers Amin On 16 Feb 2009, at 20:37, Amin Mohammed-Coleman wrote: Hi I am looking at building a faceted search using Lucene. I know that Solr comes with this built in, however I would like to t