Question on Lucene hot-backup functionality.

2013-07-23 Thread Marcos Juarez Lopez
I'm trying to get Lucene's hot backup functionality to work. I posted the question in detail over at StackOverflow, but it seems there's very little Lucene knowledge over there. Basically, I think I have setup everything correctly, but I can't get a valid snapshot when trying to do a backup. I'm

Re: Question on Lucene hot-backup functionality.

2013-07-25 Thread Marcos Juarez Lopez
tLiveConfig().getIndexDeletionPolicy(), > > rather than IndexWriterConfig.getIndexDeletionPolicy(). I'm not sure what > > Indexer.getSnapshotter() does, but I'd make sure that it uses IW. > > > > Shai > > > > > > On Wed, Jul 24, 2013 at 7:34 AM,

Custom FieldInfo.IndexOptions

2013-09-19 Thread Marcos Juarez Lopez
Hi, I'm trying to optimize an index we have, and one thing that has come up recently is that we're not really using term frequencies, and we don't need any scoring. We noticed that the term frequencies (.doc files) are a significant chunk of the total index size, and we'd like to reduce those, or

Re: Custom FieldInfo.IndexOptions

2013-09-20 Thread Marcos Juarez Lopez
rcos Juarez On Fri, Sep 20, 2013 at 5:02 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > On Thu, Sep 19, 2013 at 7:18 PM, Marcos Juarez Lopez > wrote: > > Hi, > > > > I'm trying to optimize an index we have, and one thing that has come up > > rec

Re: Custom FieldInfo.IndexOptions

2013-09-26 Thread Marcos Juarez Lopez
Thanks Mike. Yeah, I guess it wouldn't work for us in that case. Marcos On Mon, Sep 23, 2013 at 6:00 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > On Fri, Sep 20, 2013 at 11:59 AM, Marcos Juarez Lopez > wrote: > > > Thanks for your quick response

Bit-set implementation for a low cardinality custom field

2013-09-27 Thread Marcos Juarez Lopez
I've been looking at multiple optimizations we could do in our Lucene indexes (currently around ~8B total spread out in indexes of ~250M documents each) for querying fields that have very low cardinality (usually true/false, or in some cases, less than 10 categories). I would have thought Lucene o

Extending Query class for custom query evaluation.

2013-10-16 Thread Marcos Juarez Lopez
Posted something similar some time ago, but didn't get any responses, so I thought I'd try again with more details. We allow end-user queries that have our own proprietary query language, which we then translate to a Lucene Query* AST. This has worked well for us. However, a few of the operators