Re: Lucene as a primary datastore

2010-01-22 Thread Aaron McCurry
>> To: java-user@lucene.apache.org >> Sent: Wed, January 20, 2010 4:25:09 PM >> Subject: Re: Lucene as a primary datastore >> >> Thanks for the response. I understand all of what you wrote, but what >> I care about and what I had a little trouble describing exactly in m

Re: Lucene as a primary datastore

2010-01-20 Thread Otis Gospodnetic
: java-user@lucene.apache.org > Sent: Wed, January 20, 2010 4:25:09 PM > Subject: Re: Lucene as a primary datastore > > Thanks for the response. I understand all of what you wrote, but what > I care about and what I had a little trouble describing exactly in my > previous question is:

Re: Lucene as a primary datastore

2010-01-20 Thread Erick Erickson
It depends (tm). From what I've seen on this list, *if* the index gets corrupted, you'll see some exceptions somewhere. They may be head-scratchers, but you'll get exceptions. But when I've seen this kind of thing reported, it's been because of coding errors. Manually unlocking the IndexWriter and

Re: Lucene as a primary datastore

2010-01-20 Thread Jacob Rhoden
In the same way that you should take regular exports/dumps of your mysql databases, you could have the same strategy with lucene. As long as you have code that can export your data that runs daily, and code that can rebuild your index from that data, In the event of a problem the most you will

Re: Lucene as a primary datastore

2010-01-20 Thread Guido Bartolucci
Thanks for the response. I understand all of what you wrote, but what I care about and what I had a little trouble describing exactly in my previous question is: - Are all problems with Lucene obvious (e.g., you get an exception and you know your data is now bad) or are there subtle corruptions th

Re: Lucene as a primary datastore

2010-01-20 Thread Chris Lu
I have 3 concerns of making Lucene as a primary database. 1) Lucene is stable when it's stable. But you will have java exceptions. What would you do when FileNotFoundException or "Lucene 2.9.1 'read past EOF' IOException under system load" happens? For me, I don't the data is safe this way. Or, y

Re: Lucene as a primary datastore

2010-01-20 Thread Karl Wettin
20 jan 2010 kl. 04.58 skrev Guido Bartolucci: Am I just ignorant and scared of Lucene and too trusting of Oracle and MySQL? Since all your comparations is with relational databases I feel obligated to say what has been said so many times on this list: Lucene is an index and not a relatio

RE: Lucene as a primary datastore

2010-01-20 Thread Darren Hartford
To: java-user@lucene.apache.org Subject: Re: Lucene as a primary datastore My preference is to put the effort into preserving the original source on the theory that I'm sure no information is lost that way. So the suitability of Lucene to store it varies depending upon the source IMO. If it

Re: Lucene as a primary datastore

2010-01-20 Thread Erick Erickson
My preference is to put the effort into preserving the original source on the theory that I'm sure no information is lost that way. So the suitability of Lucene to store it varies depending upon the source IMO. If it's raw text, then storing all the raw text in an un-indexed field in Lucene might

Re: Lucene as a primary datastore

2010-01-20 Thread Chris Harris
I don't do a lot of work with straight Lucene right now, but I do use Solr, and from time to time the Lucene index inside my master Solr server gets corrupted; in particular, some of the Lucene segment files that are still in use somehow get deleted, resulting in Lucene throwing FileNotFoundExcepti

Re: Lucene as a primary datastore

2010-01-20 Thread Király Péter
Hi, I am using Lucene for the same purpose since years. I import an XML files with records, and in Lucene there is a special field, which stores the original XML (this used for displaying with XSLT), the other fields are for searching. There is a webform, where the users can modify the data. If us

Re: Lucene as a primary datastore

2010-01-19 Thread Ganesh
om: Ganesh >> To: java-user@lucene.apache.org >> Sent: Wed, January 20, 2010 1:13:21 AM >> Subject: Re: Lucene as a primary datastore >> >> We have data in compound files and we use Lucene as primary database. Its >> working great and much faster with millions of

Re: Lucene as a primary datastore

2010-01-19 Thread Otis Gospodnetic
20, 2010 1:13:21 AM > Subject: Re: Lucene as a primary datastore > > We have data in compound files and we use Lucene as primary database. Its > working great and much faster with millions of records. The only issue, I > face > is with sorting. Lucene sorting consumes good amount of m

Re: Lucene as a primary datastore

2010-01-19 Thread Ganesh
c" To: ; Sent: Wednesday, January 20, 2010 10:45 AM Subject: Re: Lucene as a primary datastore > You are not alone, Guido. It's a good question. In my experience, Lucene is > as stable as MySQL/PostgreSQL in terms of its ability to hold your data and > not corrupt it. Of co

Re: Lucene as a primary datastore

2010-01-19 Thread Otis Gospodnetic
You are not alone, Guido. It's a good question. In my experience, Lucene is as stable as MySQL/PostgreSQL in terms of its ability to hold your data and not corrupt it. Of course, even with the most expensive databases, you'd want to make backups. The same goes with Lucene. Nowadays, one way