lucene scalability questions

2007-01-04 Thread Mark Mei
So this question has two parts: 1. How does Lucene scale, exactly? Do we distribute the index to multiple servers somehow? Or is it one index, sitting on some sort of a shared filesystem, shared by all Lucene servers? If it's the latter, the bottleneck will be I/O ... anyway, elaborate on scalabi

digester/lucene runtime problems

2007-01-03 Thread Mark Mei
Hi, We have 500k xml documents in a file. We ran the digest/lucene and get the following error messages. We had ran a smaller size file with 20k xml documents without any problems. Can anyone help us to resolve this problem? Thank you very much. Regards, Mark Jan 3, 2007 10:17:37 AM org.apach

lucene functionality

2006-12-13 Thread Mark Mei
At the bottom of this email is the sample xml file that we are using today. We have about 10 million of these. We need to know whether Lucene can support the following functionalities. (1) Each field is searchable and indexable. (2) Fields such as STARTTIME and ENDTIME need to be treated as a pai