Re: Databases

2010-07-27 Thread manjula wijewickrema
Hi, Thanks a lot for your information. Regards, Manjula. On Fri, Jul 23, 2010 at 12:48 PM, tarun sapra wrote: > You can use HibernateSearch to maintain the synchronization between Lucene > index and Mysql RDBMS. > > On Fri, Jul 23, 2010 at 11:16 AM, manjula wijewickrema > wrote: > > > Hi, > >

Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt

2010-07-27 Thread David Sitsky
Incidentally.. this is what TerminateProcess does say from MSDN: TerminateProcess initiates termination and returns immediately. This stops execution of all threads within the process and requests cancellation of all pending I/O. The terminated process cannot exit until all pending I/O has been co

Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt

2010-07-27 Thread David Sitsky
Hi Mike, Definitely no OOME (or other critical errors), and if there were, we would have terminated the program straight away. Cheers, David On 28 July 2010 01:44, Michael McCandless wrote: > Were there any exceptions during indexing, before the > TerminateProcess() call?  EG OOME? > > Mike > >

Re: understanding lucene

2010-07-27 Thread skant
Get "Lucene In Action." http://www.manning.com/hatcher2/ On Wed, Jul 28, 2010 at 12:06 AM, Yakob wrote: > hello everyone, > I am starting to understand lucene in java and I am having a hard time > in implementing it. > I am trying to develop a java application that can do indexing, > searching

Re: understanding lucene

2010-07-27 Thread Li Li
lucene in action 2nd ed. is a good book 2010/7/28 Yakob : > hello everyone, > I am starting to understand lucene in java and I am having a hard time > in implementing it. > I am trying to develop a java application that can do indexing, > searching and whatnot. and using lucene framework is one of

understanding lucene

2010-07-27 Thread Yakob
hello everyone, I am starting to understand lucene in java and I am having a hard time in implementing it. I am trying to develop a java application that can do indexing, searching and whatnot. and using lucene framework is one of the best solution for me. only the problem was I am a bit new in thi

Re: Using lucene for substring matching

2010-07-27 Thread William Newport
Ramdirectorys seem useful but as the index gets larger, java heap sizes can become a problem in terms of garbage collection pauses. Some customers are looking to use data grid products such as IBM websphere extreme scale or oracle coherence to act as the directory for the index. This stores the ind

Re: No hits when querying multiple fields

2010-07-27 Thread Erick Erickson
H, what analyzers are you using at index and query time? Are they identical? But I think your basic problem is phrases. Parsing text:"hello world" expects the words "hello" and "world" to appear sequentially in the text field. Try something like title:(+hello +world). But depending upon how yo

Re: No hits when querying multiple fields

2010-07-27 Thread Geir Gullestad Pettersen
Just to clarify some things that could be misunderstood. First, I meant that I added two fields to a document which was then indexed, not two separate documents. Second, I noticed in the lucene mail archive that some additional charactes, especially "*", had sneaked into my query examples. This w

Re: Using lucene for substring matching

2010-07-27 Thread Geir Gullestad Pettersen
Thanks for your feedback, Ian. I have written a first implementation of this service that works well. You mentioned something about technologies for speeding up lucene, something I am interested in knowing more about. Would you, or anyone, please mind elaborating a bit, or giving me some pointers?

No hits when querying multiple fields

2010-07-27 Thread Geir Gullestad Pettersen
Consider the following two documents which I have added to my index: doc.add( new Field("text", "hello world", Field.Store.YES, > Field.Index.ANALYZED)); > doc.add( new Field("id", "1", Field.Store.YES, Field.Index.ANALYZED)); > Using the StandardQueryParser I can retrieve my document with eithe

Re: Structure of .tii-file

2010-07-27 Thread Michael McCandless
On Tue, Jul 27, 2010 at 12:06 PM, Alexander vom Berg wrote: >> However, this may not be a fair performance test since with 'cp' >> presumably the IO system may have optimized how the files are >> allocated to blocks on disk. Ie, you'll get a different allocation >> than had Lucene directly opened

Re: Structure of .tii-file

2010-07-27 Thread Alexander vom Berg
Hello Mike, Am 27.07.2010 14:38, schrieb Michael McCandless: On Tue, Jul 27, 2010 at 7:58 AM, Alexander vom Berg wrote: Hello Mike, thanks for your answer! I am currently working with Lucene 3.0.1 and except the .tii - file all other descriptions are comprehensible. The idea behind the ti

Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt

2010-07-27 Thread Michael McCandless
Were there any exceptions during indexing, before the TerminateProcess() call? EG OOME? Mike On Tue, Jul 27, 2010 at 8:43 AM, Michael McCandless wrote: > On Mon, Jul 26, 2010 at 7:10 PM, David Sitsky wrote: >> During processing.. there might be a number of reasons why we need to >> shutdown th

Re: Different ranking results

2010-07-27 Thread Grant Ingersoll
Can you post a full example as a Unit test? On Jul 21, 2010, at 5:25 PM, Philippe wrote: > Hi, > > I just performed two queries which, in my opinion, should lead to the same > document rankings. However, the document ranking differ between these two > queries. For better understanding I prepar

Re: Get all terms of a specific field

2010-07-27 Thread Grant Ingersoll
On Jul 27, 2010, at 8:50 AM, Philippe wrote: > Hi, > > what would be the fastest way to get all terms for all documents matching a > specific query? > > Sofar I: > > 1.) Query the index > 2.) Retrieve all scoreDocs > 3.) Iterate the scoreDocs and retrieve all terms using the getValues method

Get all terms of a specific field

2010-07-27 Thread Philippe
Hi, what would be the fastest way to get all terms for all documents matching a specific query? Sofar I: 1.) Query the index 2.) Retrieve all scoreDocs 3.) Iterate the scoreDocs and retrieve all terms using the getValues method and a customised "FieldSelector" However, retrieving and itera

Re: java.io.IOException: background merge hit exception: Caused by: java.io.FileNotFoundException: _1vw.fdt

2010-07-27 Thread Michael McCandless
On Mon, Jul 26, 2010 at 7:10 PM, David Sitsky wrote: > During processing.. there might be a number of reasons why we need to > shutdown the indexing process, but perhaps what is unusual is we call > the win32 API TerminateProcess() call rather than System.exit(), for > slightly obscure reasons.  W

Re: Structure of .tii-file

2010-07-27 Thread Michael McCandless
On Tue, Jul 27, 2010 at 7:58 AM, Alexander vom Berg wrote: > Hello Mike, > > thanks for your answer! > I am currently working with Lucene 3.0.1 and except the .tii - file all > other descriptions are comprehensible. > The idea behind the tii/tis file structure is for faster retrieving the > correc

Re: Structure of .tii-file

2010-07-27 Thread Alexander vom Berg
Hello Mike, thanks for your answer! I am currently working with Lucene 3.0.1 and except the .tii - file all other descriptions are comprehensible. The idea behind the tii/tis file structure is for faster retrieving the correct terms. At first I lookup in memory (tii-file) and take the most near

Re: Lucene Index Vs Database Index

2010-07-27 Thread Ian Lea
http://lucene.apache.org/java/3_0_2/fileformats.html#Inverted%20Indexing -- Ian. On Tue, Jul 27, 2010 at 3:22 AM, shravan wrote: > > Hi, > > Can any one clarify me difference between lucene index and database index? > > I am just trying to understand how lucene stores index, like databases store