Re: appending field to an existing index

2008-02-04 Thread Cam Bazz
Hello, I have read the parallel reader doc. It says it must have the same number of documents as the other index. When we are using a writer - searcher combination, how can we integrate this parallel reader into game. Simply, I have some documents, and I just like to mark them, in an efficient wa

Re: appending field to an existing index

2008-02-04 Thread Andrzej Bialecki
Erick Erickson wrote: As always, "it depends". You can try to reconstruct the doc from an index, see Luke. But depending upon you you indexed things, it may be more or less lossy. I remember this was discussed recently, you might have some luck if you search the archive. But it may be very, very

Re: appending field to an existing index

2008-02-02 Thread Erik Hatcher
One option is to index the new field in sync (same index order) into a new index, and search using a ParallelIndexReader. Erik On Jan 30, 2008, at 7:42 PM, John Wang wrote: Hi all: We have a large index and it is difficult to reindex. We want to add another field to the in

Re: appending field to an existing index

2008-02-01 Thread Michael McCandless
John Wang wrote: Thanks Chris! This works with lucene2.1 and greater (breaks with lucene2.0) A follow up question, anyway to rename and to delete a field. Rename field: I tried to use a hex editor to edit the .fnm file (screwed up my index, but I was careful and followed the index format

Re: appending field to an existing index

2008-02-01 Thread Michael McCandless
Woops -- disregard this! Chris' approach will work. And, we don't need addParallelIndices! Mike Michael McCandless wrote: I don't think this will do the right thing in this case, because addIndexes "appends" the documents from all indices together. Whereas John wants to "merge in" a

Re: appending field to an existing index

2008-02-01 Thread Michael McCandless
I don't think this will do the right thing in this case, because addIndexes "appends" the documents from all indices together. Whereas John wants to "merge in" a new field into all docs in an existing index. Really what we need is a new "addParallellIndices" method. I think, conceptual

Re: appending field to an existing index

2008-02-01 Thread John Wang
Thanks Chris! This works with lucene2.1 and greater (breaks with lucene2.0) A follow up question, anyway to rename and to delete a field. Rename field: I tried to use a hex editor to edit the .fnm file (screwed up my index, but I was careful and followed the index format) Deleting a field: ideas?

Re: appending field to an existing index

2008-01-31 Thread Chris Hostetter
: I have to keep one index though. Is there a way to reproduce an index from : an indexReader? asuming you have indexes that work in conjunction with eachther they way you want when using ParallelReader, you should (in theory) be able to use... ParallelReader r = ...; IndexWriter w = new

Re: appending field to an existing index

2008-01-31 Thread John Wang
I was actually thinking of creating a separate index with only the extra field and them modify the other index (change some files etc.) Sounds hacky. Dunno if its possible. Thanks -john On Jan 31, 2008 9:36 AM, Erick Erickson <[EMAIL PROTECTED]> wrote: > As always, "it depends". You can try to

Re: appending field to an existing index

2008-01-31 Thread Erick Erickson
As always, "it depends". You can try to reconstruct the doc from an index, see Luke. But depending upon you you indexed things, it may be more or less lossy. I remember this was discussed recently, you might have some luck if you search the archive. But it may be very, very expensive to reconstruc

Re: appending field to an existing index

2008-01-31 Thread John Wang
I have to keep one index though. Is there a way to reproduce an index from an indexReader? -John On Jan 31, 2008 1:30 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Just beware, though, that with ParallelReader you must ensure that > the internal docIDs of both indices remain "aligned" ov

Re: appending field to an existing index

2008-01-31 Thread Michael McCandless
Just beware, though, that with ParallelReader you must ensure that the internal docIDs of both indices remain "aligned" over time. If you never do deletions, then that happens for free. If you do deletions, then, you must change IndexWriter to buffer by doc count (same doc count for all wr

Re: appending field to an existing index

2008-01-31 Thread Doron Cohen
This may help: http://www.nabble.com/Updating-Lucene-Index-with-Unstored-fields-tt15188818.html#a15188818 Doron On Thu, Jan 31, 2008 at 2:42 AM, John Wang <[EMAIL PROTECTED]> wrote: > Hi all: > >We have a large index and it is difficult to reindex. > >We want to add another field to the

appending field to an existing index

2008-01-30 Thread John Wang
Hi all: We have a large index and it is difficult to reindex. We want to add another field to the index without reindexing, e.g. just create a new inverted index, dictionary files etc. How feasible is it to add this to lucene? Thanks -John