Wildcard Case Sensitivity

2011-01-20 Thread Amin Mohammed-Coleman
Hi Apologies up front if this question has been asked before. I have a document which contains a field that stores an untokenized value such as TEST_TYPE. The analyser used is StandardAnalyzer and I pass the same analyzer into the query. I perform the following query : fieldName:TEST_*, howe

Re: TermRangeQuery

2010-11-28 Thread Amin Mohammed-Coleman
, Ian Lea wrote: >> >>> What sort of ranges are you trying to use? Maybe you could store a >>> separate field, just for these queries, with some normalized form of >>> the ids, with all numbers padded out to the same length etc. >>> >>> -- >>> Ian

Re: TermRangeQuery

2010-11-26 Thread Amin Mohammed-Coleman
as you expect. >>> >>> I'm not clear what you mean by XDF-123 but if you've got >>> >>> AAA-123 >>> AAA-124 >>> ... >>> ABC-123 >>> ABC-234 >>> etc. >>> >>> then you'll be fine. If

Re: TermRangeQuery

2010-11-26 Thread Amin Mohammed-Coleman
es a Collator but note the > performance warning in the javadocs. > > > -- > Ian. > > > On Fri, Nov 26, 2010 at 2:18 PM, Amin Mohammed-Coleman > wrote: >> Hi All >> >> I was wondering whether I can use TermRangeQuery for my use case. I have a >> colle

Re: TermRangeQuery

2010-11-26 Thread Amin Mohammed-Coleman
note the > performance warning in the javadocs. > > > -- > Ian. > > > On Fri, Nov 26, 2010 at 2:18 PM, Amin Mohammed-Coleman > wrote: >> Hi All >> >> I was wondering whether I can use TermRangeQuery for my use case. I have a >> collection of ids (r

TermRangeQuery

2010-11-26 Thread Amin Mohammed-Coleman
Hi All I was wondering whether I can use TermRangeQuery for my use case. I have a collection of ids (represented as XDF-123) and I would like to do a search for all the ids (might be in the range of 1) and for each matching id I want to get the corresponding data that is stored in the inde

Re: Batch Operation and Commit

2010-08-26 Thread Amin Mohammed-Coleman
indexwriter OR commit the changes > before you can see your changes, see IndexWriter.close/commit > > Best > Erick > > > > On Thu, Aug 26, 2010 at 10:42 AM, Amin Mohammed-Coleman > wrote: > >> Hi >> >> >> I have a list of batch tasks that need t

Batch Operation and Commit

2010-08-26 Thread Amin Mohammed-Coleman
Hi I have a list of batch tasks that need to be executed. Each batch contains 1000 documents and basically I use a RAMDirectory based index writer, and at the end of adding 1000 documents to the memory i perform the following: ramWriter.commit(); indexWriter.addIndexesNoOptimize(ramW

Re: hit exception flushing segment _0 - IndexWriter configuration

2010-08-03 Thread Amin Mohammed-Coleman
s wrote: > Can you post the full exception? And also the log output from > IndexWriter.setInfoStream. > > Mike > > On Tue, Aug 3, 2010 at 5:28 PM, Amin Mohammed-Coleman > wrote: >> Hi >> >> Apologies for re sending this email but I was just wondering if

Re: hit exception flushing segment _0 - IndexWriter configuration

2010-08-03 Thread Amin Mohammed-Coleman
Hi Apologies for re sending this email but I was just wondering if any one might be able to advise on the below. I'm not sure if I've provided enough info. Again any help would be appreciated. Amin Sent from my iPhone On 1 Aug 2010, at 20:00, Amin Mohammed-Coleman wrote: >

hit exception flushing segment _0 - IndexWriter configuration

2010-08-01 Thread Amin Mohammed-Coleman
Hi I am currently building an application whereby there is a remote index server (yes it probably does sound like Solr :)) and users use my API to send documents to the indexing server for indexing. The 2 methods primarily used is add and commit. So the user can send requests for documents to

Special Characters and QueryParser

2009-10-22 Thread Amin Mohammed-Coleman
Hi I am looking at handling special characters in the query as using certain characters cause an exception. I looked at QueryParser.escape(..) to handle this. It works to a certain extent for example using '!' doesn't cause an exception. However when I use a wildcard then the wildcard is ignored.

Re: Filtering question/advice

2009-09-24 Thread Amin Mohammed-Coleman
Hi Sorry for not getting back to you. Been swamped with stuff and work and home. Just managed to check my lucene emails! You are right i made some silly mistakes with the testcase and have updated accordingly. The test is still failing but the properties are set correctly: public class Underwr

Re: Filtering question/advice

2009-09-09 Thread Amin Mohammed-Coleman
Hi Thanks for your reponse. Here is the following testcase: public class UnderwriterReferenceTest { private Directory directory; private Analyzer analyzer; private IndexSearcher indexSearcher; private IndexWriter indexWriter; private Document layerDocumentA; @Before

Re: Filtering question/advice

2009-09-04 Thread Amin Mohammed-Coleman
-Coleman wrote: > Hi > I include a testcase to show what I am trying to do. Testcase number 3 > fails. > > Thanks > Amin > > > On Fri, Sep 4, 2009 at 10:17 AM, Amin Mohammed-Coleman > wrote: > >> Hi, >> >> I am looking at applying a security fi

Re: Filtering question/advice

2009-09-04 Thread Amin Mohammed-Coleman
Hi I include a testcase to show what I am trying to do. Testcase number 3 fails. Thanks Amin On Fri, Sep 4, 2009 at 10:17 AM, Amin Mohammed-Coleman wrote: > Hi, > > I am looking at applying a security filter for our lucene document and I > was wondering if I could get feedback on

Filtering question/advice

2009-09-04 Thread Amin Mohammed-Coleman
Hi, I am looking at applying a security filter for our lucene document and I was wondering if I could get feedback on whether the solution I have come up with. Firstly I will explain the scenario and followed by the proposed solution: We have a concept of a Layer which is a project whereby a br

Re: [EASY]How to change the demo of lucene143 into a multithread one?

2009-08-13 Thread Amin Mohammed-Coleman
Hi I have recently created an indexing reference project using Spring Integration. May not help you with what you're doing but it might be interesting for creating asynchronous indexing using JMS. http://code.google.com/p/lucene-indexing-with-si/ Cheers Amin On Thu, Aug 13, 2009 at 11:53 AM,

Re: Contribute to Lucene

2009-08-13 Thread Amin Mohammed-Coleman
ouple of weeks but i > haven't seen a lot of test cases so far. > > Maybe the contribs are a good place to start from. > > simon > > > On Thu, Aug 13, 2009 at 9:09 AM, Amin Mohammed-Coleman > wrote: > > Thanks for your replies. I have checked out trunk and have st

Re: Contribute to Lucene

2009-08-13 Thread Amin Mohammed-Coleman
Thanks for your replies. I have checked out trunk and have started looking at what I can do. Any more suggestions as usual always welcome. Thanks all! Amin On Wed, Aug 12, 2009 at 10:28 PM, Chris Hostetter wrote: > > : that you use. Also, we are nearing 2.9 release, so it would > : be great t

Fwd: Contribute to Lucene

2009-08-12 Thread Amin Mohammed-Coleman
gs or anything. I am currently developing a reference project for using Spring Integration for Lucene indexing and I have done some other projects one of them being an Integration between GigaSpaces and Hibernate Search. Kind Regards Amin Mohammed-Coleman

Re: Slightly Off-topic: How to decide whether or not to add a document?

2009-08-04 Thread Amin Mohammed-Coleman
I've been working on a indexing solution using Spring integration and lucene. the example project uses jms to create work items (index add or update) and then a service that polls for work to do. I should have this complete soon and will be putting it on google code. Not much of help right now but

Re: Storing a serialized object ?

2009-07-04 Thread Amin Mohammed-Coleman
Hi I think you might want to look at Hibernate Search. You can use projections which basically store instance fields in the index. It does not store the object in a serialised form in the index. It holds a reference (id) to the persistant entity. Cheers Amin On Sat, Jul 4, 2009 at 2:39 AM, Er

Re: IndexWriter

2009-07-02 Thread Amin Mohammed-Coleman
will throw LockObtainFailedException, In this case, Unlock it and Open the > writer again. > > > > Regards > > Ganesh > > > > - Original Message - > > From: "Amin Mohammed-Coleman" > > To: ; > > Sent: Wednesday, July 01, 2009 5:3

Re: IndexWriter

2009-07-01 Thread Amin Mohammed-Coleman
9 PM, Ganesh wrote: > > Yes. Single IndexWriter could be maintained in a App and it could be > closed when the App is shutdown. > > > > Regards > > Ganesh > > > > - Original Message - > > From: "Amin Mohammed-Coleman" > >

IndexWriter

2009-07-01 Thread Amin Mohammed-Coleman
Hi This question has probably been asked before so apologies for asking it again. Just to confirm that it is ok to use a single index writer in a web application and only close that single instance on application shutdown? As the indexwriter is thread safe there is no need for any external synch

Re: Lucene directory copy - master copy to local index

2009-06-23 Thread Amin Mohammed-Coleman
Hi Sorry for sending the below..what I meant to say was is there any documentation that I can be pointed to with using lucene and rsync? I been up since 2am so brain slowing down really quickly... Cheers Amin On Tue, Jun 23, 2009 at 10:13 AM, Amin Mohammed-Coleman wrote: > Hi > > T

Re: Lucene directory copy - master copy to local index

2009-06-23 Thread Amin Mohammed-Coleman
Amin, > >> > >> Have a look at Solr, it may be what you are after: > >> http://lucene.apache.org/solr/ > >> > >> Otis > >> -- > >> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > >> > >> > >> > &g

Lucene directory copy - master copy to local index

2009-06-20 Thread Amin Mohammed-Coleman
Hi I am prototyping the following situation: 1) Multiple nodes in a cluster 2) Each node has a local index 3) Search requests are maded against the local index 4) Index updates are sent to a JMS where a master process adds document to index 5) Each node is configured to check whether the local ind

Re: Using lucene in a clustered app server

2009-06-15 Thread Amin Mohammed-Coleman
Hi I'm looking at Hadoop and Katta and I was wondering if some may be able clarify the following: 1) Is Katta replacing the Hadoop Lucene contribution 2) Are people still using Hadoop Lucene to perform indexing Cheers Amin On Sat, Jun 13, 2009 at 7:46 AM, Amin Mohammed-Coleman wrote:

Re: Using lucene in a clustered app server

2009-06-12 Thread Amin Mohammed-Coleman
rge.net > > -Tarandeep > > On Thu, Jun 11, 2009 at 1:04 AM, Amin Mohammed-Coleman >wrote: > > > Hi > > > > we are thinking of implementing a solution like this. All nodes on the > > cluster has a local index from which searches are performed. Once an >

Re: Using lucene in a clustered app server

2009-06-11 Thread Amin Mohammed-Coleman
Hi we are thinking of implementing a solution like this. All nodes on the cluster has a local index from which searches are performed. Once an update on the index needs to happen the work is sent to a queue where there is a master who is responsible for updating the central index. The master is

Performing Asynchronous Search

2009-05-20 Thread Amin Mohammed-Coleman
Hi All This may not be a question for this mailing list but i wasn't sure where to start. Please accept my apologies if anyone thinks that this is not the appropriate place for this question. I am currently working on building a proof of concept search solution for my company using Lucene and Hi

Re: Seattle / PNW Hadoop + Lucene User Group?

2009-05-19 Thread Amin Mohammed-Coleman
Will there any videocasts/presentation notes? I would so so so so like to turn up for that! Hope it's great! Cheers Amin On Tue, May 19, 2009 at 6:52 PM, Bradford Stephens < bradfordsteph...@gmail.com> wrote: > Hello everyone! We (finally) have space secured (it's a tough task!): > University o

Re: Using Luke on a Lucene Index in a Database

2009-05-19 Thread Amin Mohammed-Coleman
Are you using an object relational mapping tool like Hibernate? if you are you could use hibernate search to index your persistent entities and then use luke to inspect the indexes. There may other ways of doing it I guess. Just a thought. Cheers Amin On Tue, May 19, 2009 at 9:23 AM, Christo

Re: Seattle / PNW Hadoop + Lucene User Group?

2009-04-17 Thread Amin Mohammed-Coleman
I would love to come but I'm afraid I'm stuck in rainy old England :( Amin On 18 Apr 2009, at 01:08, Bradford Stephens wrote: OK, we've got 3 people... that's enough for a party? :) Surely there must be dozens more of you guys out there... c'mon, accelerate your knowledge! Join us in Seat

Re: SpellChecker in use with composite query

2009-04-14 Thread Amin Mohammed-Coleman
he general approach is. Also whenever the indexes change should I clear the spell index and start again? Once again apologies for bringing this up. Cheers Amin On Sat, Apr 11, 2009 at 9:59 PM, Amin Mohammed-Coleman wrote: > Hi > Another thing that I was wondering is how to apply the con

Re: SpellChecker in use with composite query

2009-04-11 Thread Amin Mohammed-Coleman
Hi Another thing that I was wondering is how to apply the construction of the spell index. Where is the most appropriate place to create the spell index? For example: IndexReader spellReader = IndexReader.open(fsDirectory1); IndexReader spellReader2 = IndexReader.open(fsDirectory2); MultiRead

SpellChecker in use with composite query

2009-04-10 Thread Amin Mohammed-Coleman
Hi I have been playing around with the SpellChecker class and so far it looks really good. While developing a testcase to show it working I came across a couple of issues which I have resolved but I'm not certain if this is the correct approach. I would therefore be grateful if anyone could tell

Re: Syncing lucene index with a database

2009-03-27 Thread Amin Mohammed-Coleman
Hi I was going to suggest looking at hibernate search. It comes with event listeners that modify your indexes when the persistent entity changes. It use lucene under the hood so if you need to access lucene the you can. Indexing can be done sync or async and the documentation shows how to

Re: question about grouping text

2009-03-26 Thread Amin Mohammed-Coleman
Hi I was wondering if soemthing like LingPipe or Gate (for text extraction) might be an idea? I've started looking at it and I'm just thinking it may be applicable (I maybe wrong). Cheers Amin On Wed, Mar 25, 2009 at 4:18 PM, Grant Ingersoll wrote: > Hi MFM, > > This comes down to a preprocess

Re: Performance tips on searching

2009-03-20 Thread Amin Mohammed-Coleman
s worked. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Amin Mohammed-Coleman [mailto:ami...@gmail.com] Sent: Friday, March 20, 2009 6:43 PM To: java-user@lucene.apache.org Cc: ; Subject: Re: Performance t

Re: Performance tips on searching

2009-03-20 Thread Amin Mohammed-Coleman
Hi How do you expose a pagination without a customized hit collector. The multi searcher does not expose a method for hit collector and sort. Maybe this is not an issue for people ... Cheers Amin On 20 Mar 2009, at 17:25, "Uwe Schindler" wrote: Why not use a MultiSearcher an all single

Re: Similarity and Lucene

2009-03-20 Thread Amin Mohammed-Coleman
e. Cheers Amin On Fri, Mar 20, 2009 at 4:20 PM, Amin Mohammed-Coleman wrote: > Hi > > If I choose to subclass the default similarity, do I need to apply the > same subclassed Similarity to IndexReader, IndexWriter and IndexSearcher? > > I am interested in doing the bel

Similarity and Lucene

2009-03-20 Thread Amin Mohammed-Coleman
Hi If I choose to subclass the default similarity, do I need to apply the same subclassed Similarity to IndexReader, IndexWriter and IndexSearcher? I am interested in doing the below: Similarity sim = new DefaultSimilarity() { public float lengthNorm(String field, int numTerms) { if(field

Re: Pagination with MultiSearcher

2009-03-19 Thread Amin Mohammed-Coleman
Hi Please ignore the problem I raised. User error ! Sorry Amin On 19 Mar 2009, at 09:41, Amin Mohammed-Coleman wrote: Hi I've implemented the solution using the PageHitCounter from the link and I have noticed that in certain instances I get a 0 score for queries like "d

Re: Pagination with MultiSearcher

2009-03-19 Thread Amin Mohammed-Coleman
Hi I've implemented the solution using the PageHitCounter from the link and I have noticed that in certain instances I get a 0 score for queries like "document OR aspectj". has anyone else experienced this? Cheers Amin On Mon, Mar 16, 2009 at 8:07 PM, Amin Mohammed-Coleman wrot

Re: Pagination with MultiSearcher

2009-03-16 Thread Amin Mohammed-Coleman
ery,filter,pageHitCollector) I intend to use comparators to do the sorting and use collections.sort(). I would be grateful for any feedback on whether this is a good approach. Cheers Amin On Mon, Mar 16, 2009 at 8:03 AM, Amin Mohammed-Coleman wrote: > Hi Erick > > I&#x

Re: Pagination with MultiSearcher

2009-03-16 Thread Amin Mohammed-Coleman
d something off > on Sunday that I don't really understand well enough > > Sorry 'bout that > Erick > > On Sun, Mar 15, 2009 at 9:15 AM, Amin Mohammed-Coleman >wrote: > > > HI Erick > > Thanks for your reply, glad to see I'm not the only person >

Re: how to index keyword and value

2009-03-15 Thread Amin Mohammed-Coleman
Mar 15, 2009 at 8:49 AM, Seid Mohammed > wrote: > > > >> that is exactly my question > >> how can I do that? > >> > >> thanks a lot > >> Seid M > >> > >> On 3/15/09, Amin Mohammed-Coleman wrote: > >> > Why don't you c

Re: Pagination with MultiSearcher

2009-03-15 Thread Amin Mohammed-Coleman
/search/Filter.html > > > filter, > int n, > Sort > < > http://lucene.apache.org/java/2_4_1/api/org/apache/lucene/search/Sort.html > > > sort) >throws IOException > <http://java.sun.com/j2se/1

Re: how to index keyword and value

2009-03-15 Thread Amin Mohammed-Coleman
Why don't you create a Lucene document that represents a Person and then index the fields name, age, phone number, etc. Search on the name and then get the corresponding phone number from the search. Cheers Amin On Sun, Mar 15, 2009 at 10:56 AM, Seid Mohammed wrote: > I want to Index Person_Nam

Pagination with MultiSearcher

2009-03-14 Thread Amin Mohammed-Coleman
Hi I'm looking at trying to implement pagination for my search project. I've been google-ing for a solution. So far no luck. I've seen implementations of HitCollector which looks promising, however my search method has to completely change. For example I'm currently using the following:

Re: Lucene Highlighting and Dynamic Summaries

2009-03-13 Thread Amin Mohammed-Coleman
Ok. I tried to apply the patch(s) and completely messed it up (user error). Is there a full example of the highlighter that is available that I can apply and test? Cheers Amin On Fri, Mar 13, 2009 at 12:09 PM, Amin Mohammed-Coleman wrote: > Absolutely! I have received considerable help f

Re: Lucene Highlighting and Dynamic Summaries

2009-03-13 Thread Amin Mohammed-Coleman
ow by pulling the patch attached to the issue & testing it > yourself. If you do so, please report back! This is how Lucene improves. > > I'm hoping we can include it in 2.9... > > Mike > > > On Mar 13, 2009, at 6:35 AM, Amin Mohammed-Coleman wrote: > > Sw

Re: Lucene Highlighting and Dynamic Summaries

2009-03-13 Thread Amin Mohammed-Coleman
Sweet! When will this highlighter be available? Can I use this now? Cheers! On Fri, Mar 13, 2009 at 10:10 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > > Amin Mohammed-Coleman wrote: > > I think that would be good. >> > > I'll open an issue.

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
ighlighted terms, etc.). Maybe we should do the same for highlighter? Mike Amin Mohammed-Coleman wrote: I did the following: highlighter.setMaxDocCharsToAnalyze(Integer.MAX_VALUE); which works. On Thu, Mar 12, 2009 at 6:41 PM, Amin Mohammed-Coleman >wrote: JIRA updated. Includes new t

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
I did the following: highlighter.setMaxDocCharsToAnalyze(Integer.MAX_VALUE); which works. On Thu, Mar 12, 2009 at 6:41 PM, Amin Mohammed-Coleman wrote: > JIRA updated. Includes new testcase which shows highlighter not working as > expected. > > > On Thu, Mar 12, 2009 a

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
JIRA updated. Includes new testcase which shows highlighter not working as expected. On Thu, Mar 12, 2009 at 5:56 PM, Amin Mohammed-Coleman wrote: > Hi > > I have found that it is not issue with POI. I extracted text using PoI but > differenlty and the term is extracted properly.

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
le on JIRA. Currently on a cramped train! Cheers On 11 Mar 2009, at 18:11, markharw00d wrote: If you can supply a Junit test that recreates the problem I think we can start to make progress on this. Amin Mohammed-Coleman wrote: Hi Apologies for re sending this mail. Just wondering if a

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
JIRA raised: https://issues.apache.org/jira/browse/LUCENE-1559 Thanks On Thu, Mar 12, 2009 at 11:29 AM, Amin Mohammed-Coleman wrote: > Hi > > Did both attachments not come through? > > Cheers > Amin > > > On Thu, Mar 12, 2009 at 9:52 AM, mark harwood wrote: > &g

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
sometimes for other files. > > > Cheers > Amin > > > On Wed, Mar 11, 2009 at 6:11 PM, markharw00d > wrote: > > If you can supply a Junit test that recreates the problem I think we can > start to make progress on this. > > > > Amin Mohammed-Coleman wrote:

Re: Lucene Highlighting and Dynamic Summaries

2009-03-12 Thread Amin Mohammed-Coleman
n this. > > > > Amin Mohammed-Coleman wrote: > >> Hi >> >> Apologies for re sending this mail. Just wondering if anyone has >> experienced the below. I'm not sure if this could happen due nature of >> document. It does seem strange one term search returns summar

Re: Lucene Highlighting and Dynamic Summaries

2009-03-11 Thread Amin Mohammed-Coleman
ng this so I can code around this if is normal. Apologies again for re sending this mail Cheers Amin Sent from my iPhone On 9 Mar 2009, at 07:50, Amin Mohammed-Coleman wrote: Hi I am seeing some strange behaviour with the highlighter and I'm wondering if anyone else is experie

Re: Lucene Highlighting and Dynamic Summaries

2009-03-09 Thread Amin Mohammed-Coleman
but no highlighted summary. However if I search using "aspectj" I get the same doucment with highlighted summary. Just to mentioned I do rewrite the original query before performing the highlighting. I'm not sure what i'm missing here. Any help would be appreciated.

Re: Lucene Highlighting and Dynamic Summaries

2009-03-07 Thread Amin Mohammed-Coleman
Hi Got it working! Thanks again for your help! Amin On Sat, Mar 7, 2009 at 12:25 PM, Amin Mohammed-Coleman wrote: > Thanks! The final piece that I needed to do for the project! > Cheers > > Amin > > On Sat, Mar 7, 2009 at 12:21 PM, Uwe Schindler wrote: > >> >

Re: Lucene Highlighting and Dynamic Summaries

2009-03-07 Thread Amin Mohammed-Coleman
Thanks! The final piece that I needed to do for the project! Cheers Amin On Sat, Mar 7, 2009 at 12:21 PM, Uwe Schindler wrote: > > cool. i will use compression and store in index. is there anything > > special > > i need to for decompressing the text? i presume i can just do > > doc.get("cont

Re: Lucene Highlighting and Dynamic Summaries

2009-03-07 Thread Amin Mohammed-Coleman
turday, March 07, 2009 12:46 PM > > To: java-user@lucene.apache.org > > Subject: Re: Lucene Highlighting and Dynamic Summaries > > > > It depends :) > > > > It's a trade-off. If storing is not prohibitive, I recommend that as > > it makes life easier

Re: Lucene Highlighting and Dynamic Summaries

2009-03-07 Thread Amin Mohammed-Coleman
ael McCandless wrote: > > >> You should look at contrib/highlighter, which does exactly this. >> >> Mike >> >> Amin Mohammed-Coleman wrote: >> >> Hi >>> I am currently indexing documents (pdf, ms word, etc) that are uploaded, >>> these

Lucene Highlighting and Dynamic Summaries

2009-03-07 Thread Amin Mohammed-Coleman
Hi I am currently indexing documents (pdf, ms word, etc) that are uploaded, these documents can be searched and what the search returns to the user are summaries of the documents. Currently the summaries are extracted when indexing the file (summary constructed by taking the first 10 lines of the

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
call > maybeReopen() in get, unless at the time you first create SearcherManager > the Directories each have an empty index in them. > > Mike > > Amin Mohammed-Coleman wrote: > > Hi >> Here is the code that I am using, I've modified the get() method to >> inc

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
that search will not see the > newly opened readers, but the next search will. > > I'm just thinking that since you see no results with get() alone, debug > that case first. Then put back the maybeReopen(). > > Can you post your full code at this point? > > > Mike > >

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
esn't make sense. I maybe missing something here. Cheers Amin On 2 Mar 2009, at 15:48, Amin Mohammed-Coleman wrote: I'm seeing some interesting behviour when i do get() first followed by maybeReopen then there are no documents in the directory (directory that i am interested in.

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
ut the numDocs() of each IndexReader you get from the SearcherManager? > > Something is wrong and it's best to explain it... > > > Mike > > Amin Mohammed-Coleman wrote: > > Nope. If i remove the maybeReopen the search doesn't work. It only works >> when i c

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
gt; get() before maybeReopen() should simply let you search based on the > searcher before reopening. > > If you just do get() and don't call maybeReopen() does it work? > > > Mike > > Amin Mohammed-Coleman wrote: > > I noticed that if i do the get() before the maybe

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
" + >> stopWatch.getTotalTimeMillis() + " ms"); >> >> return summaryList.toArray(new Summary[] {}); >> >> } >> >> >> >> I hope this makes sense...thanks again! >> >> >> Cheers >> >> Amin >> >&g

Re: Faceted Search using Lucene

2009-03-02 Thread Amin Mohammed-Coleman
t; LOGGER.debug("Initialising multi searcher "); >> >> documentSearcherManagers = new >> DocumentSearcherManager[directories.size()]; >> >> for (int i = 0; i < directories.size() ;i++) { >> >> Directory directory = directories.get(i); &

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
> SearcherManager (from your Directory instances). You don't need any > searchers during initialize. > > Is DocumentSearcherManager the same as SearcherManager (just renamed)? > > The release method is wrong -- you're calling .get() and then > immediately release

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
your help! On Sun, Mar 1, 2009 at 4:18 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > > This is not quite right -- you should only create SearcherManager once > (per Direcotry) at startup/app load, not with every search request. > > And I don't see releas

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
d call maybeReopen(), and then call get() and gather each > IndexSearcher instance into a new array. Then, make a new > MultiSearcher (opposite of what I said before): while that creates a > small amount of garbage, it'll keep your code simpler (good > tradeoff). > > Mike > &

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
o other thread is reopening > #E Finish reopen and notify other threads > #F Reopen searcher if there are changes > #G Check index version and reopen, warm, swap if needed > #H Returns current searcher > #I Release searcher > #J Swaps currentSearcher to new searcher >

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
sorrry I added release(multiSearcher); instead of multiSearcher.close(); On Sun, Mar 1, 2009 at 2:17 PM, Amin Mohammed-Coleman wrote: > Hi > I've now done the following: > > public Summary[] search(final SearchRequest searchRequest) > throwsSearchExecutionExceptio

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
> > It's best to call this method from a single BG "warming" thread, in which > case it would not need its own synchronization. > > But, to be safe, I'll add internal synchronization to it. You can't simply > put synchronized in front of the method, since

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
(); assert newReader != currentSearcher.getIndexReader(); IndexSearcher newSearcher = new IndexSearcher(newReader); warm(newSearcher); swapSearcher(newSearcher); } } should the above be synchronised? On Sun, Mar 1, 2009 at 1:25 PM, Amin Mohammed-Coleman wrote: > thanks. i w

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
thanks. i will rewrite..in between giving my baby her feed and playing with the other child and my wife who wants me to do several other things! On Sun, Mar 1, 2009 at 1:20 PM, Michael McCandless < luc...@mikemccandless.com> wrote: > > Amin Mohammed-Coleman wrote: > > Hi

Re: Faceted Search using Lucene

2009-03-01 Thread Amin Mohammed-Coleman
so >creating unnecessary garbage; instead, they should be created once >& reused. > > You should consider simply using Solr -- it handles all this logic for > you and has been well debugged with time... > > Mike > > Amin Mohammed-Coleman wrote: > > The reaso

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
Forgot to mention that the previous code that i sent was related to facet search. This is a general search method I have implemented (they can probably be combined...). On Thu, Feb 26, 2009 at 8:21 PM, Amin Mohammed-Coleman wrote: > Hi > I have modified my search code. Here is the fol

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
dexreader is not up to date. When this is set to true the indexsearchers are refreshed. I would be grateful on your thoughts. On Thu, Feb 26, 2009 at 1:35 PM, Amin Mohammed-Coleman wrote: > Hi > > Thanks for your help. I will modify my facet search and my other code to > u

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
release(currentSearcher); >currentSearcher = newSearcher; > } > } > > /* > #A Current IndexSearcher > #B Create initial searcher > #C Implement in subclass to warm new searcher > #D Call this to reopen searcher if index changed > #E Returns current searcher

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
dexSearcher before you switch to it, meaning >> run >> a few queries on it before you swap the old one out. >> >> M >> >> >> >> On Tue, Feb 24, 2009 at 12:48 PM, Amin Mohammed-Coleman > >wrote: >> >> The reason for the indexreader.re

Re: Faceted Search using Lucene

2009-02-26 Thread Amin Mohammed-Coleman
all the IndexReader#close() method. If nothing is pointing at > the readers they should be garbage collected. Also, you might > want to warm up your new IndexSearcher before you switch to it, meaning run > a few queries on it before you swap the old one out. > > M > > > > O

Re: Faceted Search using Lucene

2009-02-24 Thread Amin Mohammed-Coleman
The reason for the indexreader.reopen is because I have a webapp which enables users to upload files and then search for the documents. If I don't reopen i'm concerned that the facet hit counter won't be updated. On Tue, Feb 24, 2009 at 8:32 PM, Amin Mohammed-Coleman wrote: >

Re: Faceted Search using Lucene

2009-02-24 Thread Amin Mohammed-Coleman
ope that made sense...! On Mon, Feb 23, 2009 at 7:20 AM, Amin Mohammed-Coleman wrote: > Hi > > Thanks just what I needed! > > Cheers > Amin > > > On 22 Feb 2009, at 16:11, Marcelo Ochoa wrote: > > Hi Amin: >> Please take a look a this blog post: >> ht

Re: Faceted Search using Lucene

2009-02-22 Thread Amin Mohammed-Coleman
Hi Thanks just what I needed! Cheers Amin On 22 Feb 2009, at 16:11, Marcelo Ochoa wrote: Hi Amin: Please take a look a this blog post: http://sujitpal.blogspot.com/2007/04/lucene-search-within-search-with.html Best regards, Marcelo. On Sun, Feb 22, 2009 at 1:18 PM, Amin Mohammed-Coleman

Re: Faceted Search using Lucene

2009-02-22 Thread Amin Mohammed-Coleman
Hi Sorry to re send this email but I was wondering if I could get some advice on this. Cheers Amin On 16 Feb 2009, at 20:37, Amin Mohammed-Coleman wrote: Hi I am looking at building a faceted search using Lucene. I know that Solr comes with this built in, however I would like to

Faceted Search using Lucene

2009-02-16 Thread Amin Mohammed-Coleman
Hi I am looking at building a faceted search using Lucene. I know that Solr comes with this built in, however I would like to try this by myself (something to add to my CV!). I have been looking around and I found that you can use the IndexReader and use TermVectors. This looks ok but I'm not su

Re: Field.Store.YES Question

2009-02-05 Thread Amin Mohammed-Coleman
enized content. > > Regards > Ganesh > > - Original Message - From: "Amin Mohammed-Coleman" < > ami...@gmail.com> > To: > Sent: Thursday, February 05, 2009 2:00 PM > Subject: Field.Store.YES Question > > > > Hi >> >> I'm

Field.Store.YES Question

2009-02-05 Thread Amin Mohammed-Coleman
Hi I'm probably going to get shot down for asking this simple question. Although I think I understand the basic concept of Field I feel there is something that I am missing and I was wondering if someone might help to clarify. You can store a field value in an index using Field.Store.YES or if th

Re: Indexing and Searching Web Application

2009-01-21 Thread Amin Mohammed-Coleman
al (closed) one is still there. indexSearchers.add(indexSearch); } [EOE] So if you use searchers anywhere from here on, it's got closed readers in it if you closed any of them. Best Erick On Wed, Jan 21, 2009 at 4:19 PM, Amin Mohammed-Coleman >wrote: Hi I am trying to get an understa

Re: Indexing and Searching Web Application

2009-01-21 Thread Amin Mohammed-Coleman
adds the newly opened searcher to the end of your array. The original (closed) one is still there. indexSearchers.add(indexSearch); } [EOE] So if you use searchers anywhere from here on, it's got closed readers in it if you closed any of them. Best Erick On Wed, Jan 21, 2009 at 4:19 PM,

  1   2   >