Re: Creating Indexes when data inside the file is being written.

2013-08-13 Thread Ian Lea
I'm not sure what you're getting at. If you've got one job reading data, writing to an output file and indexing as you go, it should work. If you've got multiple jobs trying to write to the same output file and lucene index you'll need some external synchronisation. -- Ian. On Tue, Aug 13, 20

Re: Creating Indexes when data inside the file is being written.

2013-08-13 Thread Jugal Kolariya
Probably, Last doubt: The data in my application is coming from a stream after performing some functionality. This stream is getting continously written in the file. So , effectively, if I open a lucene index and create indexes using this file, I would be able to create the indexes ..??? Wo

Re: Creating Indexes when data inside the file is being written.

2013-08-13 Thread Ian Lea
If I've understood your question correctly, the answer is yes. Assuming the input data is coming from another file the flow will be along the lines of . Open input file for reading . Open output file for writing . Open (or create) lucene index . For each input record - write to output file

Re: Creating Indexes when data inside the file is being written.

2013-08-13 Thread Jugal Kolariya
That only answer my 2nd part. My most important question still remains. " In my code case, I am creating a new file and writing data to that file. Now, when the file writing is in progress, I would like to create Lucene Indexes. Once indexes are created, I can then perform operation on the ind

Re: Creating Indexes when data inside the file is being written.

2013-08-12 Thread Michael McCandless
You'll have to periodically re-index that document, if it's content is constantly changing. Alternatively, it's possible to index sub-documents so that each new "chunk" of content added because a new document, and then you join or group the results back into a single document ... Mike McCandless

Creating Indexes when data inside the file is being written.

2013-08-12 Thread Jugal Kolariya
Hello, I have a potential usecase for which I am not sure whether using lucene will help me or not. In my code case, I am creating a new file and writing data to that file. Now, when the file writing is in progress, I would like to create Lucene Indexes. Once indexes are created, I