Re: Ways to store and search tens of billions of text document content in one lucene index

2017-06-23 Thread Trejkaz
On Fri, Jun 23, 2017 at 4:24 PM, Ranganath B N wrote: > Hi, [cutting X-Y problem stuff] > What strategies do you recommend for this task "Ways to store and search > tens of billions > of text document content in one lucene index"? so that I can accomplish > this in optimal time. Split i

RE: Extending Analyzer at runtime

2017-06-23 Thread Allison, Timothy B.
Head meet brick. Thank you, Uwe! -Original Message- From: Uwe Schindler [mailto:u...@thetaphi.de] Sent: Friday, June 23, 2017 11:23 AM To: java-user@lucene.apache.org Subject: RE: Extending Analyzer at runtime Hi, Or just use CustomAnalyzer, shipped with Lucene since version 5.0. No ne

Re: Updating the DocValues field doesn't seem to update its associated StoredField value

2017-06-23 Thread Joe Ye
Thanks very much Mike! That's very helpful! I got MultiDocValues.getNumericValues to work. A follow up question: what's the best way/how do I retrieve binaryDocValues? Regards, Joe On Fri, Jun 23, 2017 at 11:00 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > Try subscribing to the

RE: Extending Analyzer at runtime

2017-06-23 Thread Uwe Schindler
Hi, Or just use CustomAnalyzer, shipped with Lucene since version 5.0. No need to write your own one. Uwe - Uwe Schindler Achterdiek 19, D-28357 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Allison, Timothy B. [mailto:talli...@mitre.org] > Sent

Re: Extending Analyzer at runtime

2017-06-23 Thread nb...@ebi.ac.uk
Thanks Alan, I will take a look at it. Nicola -- Original message--From: Alan WoodwardDate: Fri, 23 Jun 2017 14:55To: java-user@lucene.apache.org;nb...@ebi.ac.uk;Cc: Subject:Re: Extending Analyzer at runtime Hi, You should be able to use AnalyzerWrapper for this, adding your TokenFilters

Re: Extending Analyzer at runtime

2017-06-23 Thread Alan Woodward
Hi, You should be able to use AnalyzerWrapper for this, adding your TokenFilters in wrapComponents(). Alan Woodward www.flax.co.uk > On 23 Jun 2017, at 14:33, Nicola Buso wrote: > > Hi, > > maybe it's a known question but I could not find and answer. > I need to base an Analyzer on another

RE: Extending Analyzer at runtime

2017-06-23 Thread Allison, Timothy B.
I plagiarized Solr's org.apache.solr.analysis.TokenizerChain to read the configuration from a json file: https://github.com/tballison/lucene-addons/blob/6.x/gramreaper/src/main/java/org/tallison/gramreaper/ingest/schema/MyTokenizerChain.java I wouldn't recommend using anything in gramreaper just

Extending Analyzer at runtime

2017-06-23 Thread Nicola Buso
Hi, maybe it's a known question but I could not find and answer. I need to base an Analyzer on another Analyzer at runtime. I know that the Analyzer is a factory and I should really look at combine the Filters. I'm looking for a way to get the TokenStreamComponents from an analyzer at runtime and

Re: Updating the DocValues field doesn't seem to update its associated StoredField value

2017-06-23 Thread Michael McCandless
Try subscribing to the mailing list again? Just send an email to java-user-subscr...@lucene.apache.org, then follow the instructions of the email it replies with. You shouldn't have to open a new DirectoryReader; instead, use the one you just searched (where you got your ScoreDocs from); use Inde