Re: Breaking up a query results based upon ROWNUM or something similar?

2012-03-21 Thread Erick Erickson
Sure, in Solr you can specify start/rows parameters on queries like: &start=0&rows=1 &start=1&rows=1 &start=2&rows=1 You'll hit the "deep paging" problem, however. Briefly as you page deeper and deeper you're response time will drop, see: https://issues.apache.org/jira/browse/S

Re: A key value field storing

2012-03-21 Thread Ian Lea
Ah, I see. More complicated than I realized. How about using two sorts of documents. Type 1, one lucene doc for your example textid: 1234 text: some text about something Type 2, 3 lucene docs for your example First textid: 1234 company: IBM score: 0.6 Second textid: 1234 company:

RE: A key value field storing

2012-03-21 Thread Uwe Schindler
You can use a CustomScoreQuery wrapping your scored query to multiply the "confidence level" (as a DocValues field in Lucene trunk, or an indexed NumericField with precisionStep=Integer.MAX_VALUE using FieldCache) into the score. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.

Re: A key value field storing

2012-03-21 Thread Deb Lucene
Hi Ian, Thanks for the reply. I am not sure if the bq solution will b able to solve the problem. Let me explain with an example - document 1 - (some text) IBM - 0.6 Google - 0.1 Apple - 0.4 Now suppose I index the document based on the "company name" and "confidence scores" separately and search

Re: A key value field storing

2012-03-21 Thread Ian Lea
Why do you want to link name and confidence in one field? Store confidence as a NumericField and search something like BooleanQuery bq = new BooleanQuery(); Query nameq = parser.parse(...) or whatever Query confq = NumericRangeQuery.newXxx(...); bq.add(nameq, ...); bq,add(confq, ...); and search

A key value field storing

2012-03-21 Thread Deb Lucene
Hi Group, Sorry for cross posting! We need to index a document corpus (news articles) with some meta data features. The meta data are actually company names with some scoring (a double, between 0 to 1). For example, two documents can be - document 1 (some text - say a technical article from NY t

Breaking up a query results based upon ROWNUM or something similar?

2012-03-21 Thread Doug Kunzman
I'm new to lucene and have searched the archives and FAQ for an answer on how to do something and have found no information. Our server is running SOLR and is running out of memory when returning large result sets. I was curious if anyone there would know how to break up a query so that retur