RE: lucense index/document architecture

2007-01-27 Thread Joost Schouten
e.org Subject: Re: lucense index/document architecture 100TB? Ouch. Yes, most certainly very different. Again, how to split the index and design the whole system depends on how this is going to be used, how it's going to be changed, if it's going to be changed, how it's going to gr

Re: lucense index/document architecture

2007-01-27 Thread Erick Erickson
0 PM To: java-user@lucene.apache.org Subject: Re: lucense index/document architecture To steal a phrase from Mr. Hatcher... it depends . I'd try keeping it all in one index at the start until you get some clue how big the index will eventually grow to and whether your searching is acceptable.

Re: lucense index/document architecture

2007-01-27 Thread Otis Gospodnetic
u start working with 100TB of data, and I'm sure others would, too. Otis - Original Message From: Joost Schouten <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Saturday, January 27, 2007 6:08:49 PM Subject: RE: lucense index/document architecture Erick, Otis, Thank

RE: lucense index/document architecture

2007-01-27 Thread Joost Schouten
--Original Message- From: Erick Erickson [mailto:[EMAIL PROTECTED] Sent: Saturday, January 27, 2007 1:30 PM To: java-user@lucene.apache.org Subject: Re: lucense index/document architecture To steal a phrase from Mr. Hatcher... it depends . I'd try keeping it all in one index at the star

Re: lucense index/document architecture

2007-01-26 Thread Otis Gospodnetic
A single index with an id field sounds like a fine approach here. Otis - Original Message From: Joost Schouten <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Saturday, January 27, 2007 6:40:51 AM Subject: lucense index/document architecture Hi, I'm setting up lucene to work

Re: lucense index/document architecture

2007-01-26 Thread Erick Erickson
To steal a phrase from Mr. Hatcher... it depends . I'd try keeping it all in one index at the start until you get some clue how big the index will eventually grow to and whether your searching is acceptable. Do you have any idea how big the raw data you're going to ask the index to hold? 1M? 1G?,