Re: Design question

2007-04-13 Thread Peter W .
Dan, I agree Lucene users are repeatedly solving the same problems of reading, writing and creating indexes, building queries, scaling, parsing docs etc. There's a 'HowTo' section on the Wiki made for sharing tips and best practices: http://wiki.apache.org/lucene-java/HowTo but few new addition

Re: Design question [too many fields?]

2005-07-02 Thread Chris Hostetter
: My head was thinking to find a generic solution to Lucene's : limitation: The TooManyClauses problem when using RangeQuery and there : are more than 1024 values. It should be another thread. It's been discussed in several threads, and i can think of 2 good solutions at this point... Using a Ra

Re: Vedr. Re: Design question [too many fields?]

2005-07-01 Thread markharw00d
about 4900 room units which I think is OK as far as Still we have optimization work to do. Assuming your availability is a year in advance and yours is a reputable chain of hotels that books rooms by the day, (not the hour!) You only need: 4900 * 365 bits of true/false info to cache all the ava

Vedr. Re: Design question [too many fields?]

2005-07-01 Thread Naimdjon Takhirov
Guys, thanks for your inputs. I think the solution Mark has suggested does solves the problem in an acceptable way. Its actually gonna be a little better than the solution the customer is has right now. Apart from the availability we have to also check if there is any price for room units saved in

Re: Design question [too many fields?]

2005-07-01 Thread Chris Lu
Erik, Mark and Naimdjon, Sorry I totally misunderstood the question, of multiple dates for a Document. I came to agree with Erik and Mark on this problem. My head was thinking to find a generic solution to Lucene's limitation: The TooManyClauses problem when using RangeQuery and there are more th

Re: Design question [too many fields?]

2005-07-01 Thread Erik Hatcher
On Jun 30, 2005, at 11:27 PM, Chris Lu wrote: Mark, your suggestion will incur another trip to the database. And if the search results is large, filtering in DB by pk is not really good. Chris - I disagree with that last comment. It can be a great solution when the filter is cached. Cer

Re: Vedr. Re: Design question [too many fields?]

2005-07-01 Thread Chris Lu
> It is anyway going to be too many fields then? Days of > year for the whole year ahead? Since the fromDate and > toDate can be across two months and the customer wants > the data be available for one year. It won't have too many fields. > > My suggestion is, use "year" + "month" + "day" three >

Vedr. Re: Design question [too many fields?]

2005-06-30 Thread Naimdjon Takhirov
Hi Chris, It is anyway going to be too many fields then? Days of year for the whole year ahead? Since the fromDate and toDate can be across two months and the customer wants the data be available for one year. Naimdjon --- Chris Lu <[EMAIL PROTECTED]> skrev: > Mark, your suggestion will incur a

Re: Design question [too many fields?]

2005-06-30 Thread Chris Lu
Mark, your suggestion will incur another trip to the database. And if the search results is large, filtering in DB by pk is not really good. Erik, your original "date" field is good when there is not many dates(<1024) in the database. Otherwise, Range Query can not handle it. My suggestion is

Re: Vedr. Re: Design question [too many fields?]

2005-06-29 Thread markharw00d
I suspect the most performant is as follows (but could require bags of RAM) : Heres the pseudo code . [on IndexReader open, initialize map] int []luceneDocIdsByDbKey=new int [largestDbKey]; //could be large array! for (int i=0;i;Should be super-quick but requires (int size* num db records) m

Vedr. Re: Design question [too many fields?]

2005-06-29 Thread Naimdjon Takhirov
Hi Jian, Thanks for your inputs. The (DB) datamodel is quite complex with rooms and room units (I skipped it to make the case easier to understand), so I guess the easiest and actually best way to do it is with the filter. Mark: yes, there are a lot of text fields the user should be able to searc

Re: Design question [too many fields?]

2005-06-29 Thread jian chen
Hi, Naimdjon, I have some suggestions as well along the lines of Mark Harwood. As an example, suppose for each hotel room there is a description, and you want the user to do free text search on the description field. You could do the following: 1) store hotel room reservation info as rows in a

Re: Design question [too many fields?]

2005-06-29 Thread Erik Hatcher
I second Mark's suggestion over the alternative I posted. My alternative was merely to invert the field structure originally described, but using a Filter for the volatile information is wiser. Erik On Jun 29, 2005, at 9:58 AM, mark harwood wrote: Presumably there is also a free-text

Re: Design question [too many fields?]

2005-06-29 Thread Erik Hatcher
On Jun 29, 2005, at 9:18 AM, Naimdjon Takhirov wrote: Hi, We are porting our search functionality over to lucene in our hotel solution which is java based. Today search is done directly against the database. There is a date search, i.e tourist would like to search for free rooms fromDate and to

Re: Design question [too many fields?]

2005-06-29 Thread mark harwood
Presumably there is also a free-text element to the search or you wouldn't be using Lucene. Multiple fields is not the way to go. A single Lucene field could contain multiple terms ( the available dates) but I still don't think that's the best solution. The availability info is likely to be pretty