Re: reset versus setReader on TokenStream

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 4:18 PM, Benson Margulies wrote: > If I'm following, you've created a division of labor between setReader and > reset. Thats not true. setReader shouldnt be doing any labor. its really only a setter! One possibility here is to make it final (though its not obvious to me t

RE: reset versus setReader on TokenStream

2012-08-29 Thread Uwe Schindler
Hi, > To me, it seemed natural to overload #setReader so that our tokenizer was in a > consistent state once it was called. It occurs to me to wonder about > order: if #reset is called before #setReader, I'm up creek unless I copy my > reset > implementation into a local override of #setReader.

Re: reset versus setReader on TokenStream

2012-08-29 Thread Benson Margulies
If I'm following, you've created a division of labor between setReader and reset. We have a tokenizer that has a good deal of state, since it has to split the input into chunks. If I'm following here, you'd recommend that we do nothing special in setReader, but have #reset fix up all the state on

RE: reset versus setReader on TokenStream

2012-08-29 Thread Uwe Schindler
Yeah, make setReader() final in TokenStream. CharTokenizer must do this resetting in reset(). This is a similar problem like in good old StandardTokenizer which called reset() inside setReader(Reader) [at that time reset(Reader)]. But the latter was fixed already. - Uwe Schindler H.-H.-Meie

Re: reset versus setReader on TokenStream

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 3:58 PM, Benson Margulies wrote: > I think I'm beginning to get the idea. Is the following plausible? > > At the bottom of the stack, there's an actual source of data -- like a > tokenizer. For one of those, reset() is a bit silly, and something like > setReader is the brai

Re: reset versus setReader on TokenStream

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 3:54 PM, Benson Margulies wrote: > Some interlinear commentary on the doc. > > * Resets this stream to the beginning. > > To me this implies a rewind. As previously noted, I don't see how this > works for the existing implementations. its not a rewind. the javadocs here

Re: reset versus setReader on TokenStream

2012-08-29 Thread Benson Margulies
I think I'm beginning to get the idea. Is the following plausible? At the bottom of the stack, there's an actual source of data -- like a tokenizer. For one of those, reset() is a bit silly, and something like setReader is the brains of the operation. Some number of other components may be stacke

Re: reset versus setReader on TokenStream

2012-08-29 Thread Benson Margulies
Some interlinear commentary on the doc. * Resets this stream to the beginning. To me this implies a rewind. As previously noted, I don't see how this works for the existing implementations. * As all TokenStreams must be reusable, * any implementations which have state that needs to be re

Re: reset versus setReader on TokenStream

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 3:45 PM, Benson Margulies wrote: > On Wed, Aug 29, 2012 at 3:37 PM, Robert Muir wrote: > >> ok, lets help improve it: I think these have likely always been confusing. >> >> before they were both reset: reset() and reset(Reader), even though >> they are unrelated. I thought

Re: reset versus setReader on TokenStream

2012-08-29 Thread Benson Margulies
On Wed, Aug 29, 2012 at 3:37 PM, Robert Muir wrote: > ok, lets help improve it: I think these have likely always been confusing. > > before they were both reset: reset() and reset(Reader), even though > they are unrelated. I thought the rename would help this :) > > Does the TokenStream workfloat

Re: reset versus setReader on TokenStream

2012-08-29 Thread Robert Muir
ok, lets help improve it: I think these have likely always been confusing. before they were both reset: reset() and reset(Reader), even though they are unrelated. I thought the rename would help this :) Does the TokenStream workfloat here help? http://lucene.apache.org/core/4_0_0-BETA/core/org/ap

reset versus setReader on TokenStream

2012-08-29 Thread Benson Margulies
I've read the javadoc through a few times, but I confess that I'm still feeling dense. Are all tokenizers responsible for implementing some way of retaining the contents of their reader, so that a call to reset without a call to setReader rewinds? I note that CharTokenizer doesn't implement #reset

Re: Using coord of one BooleanQuery as a multiplier for its siblings

2012-08-29 Thread Robert Muir
On Mon, Aug 27, 2012 at 8:31 PM, pranshu sharma wrote: > Hi there, > > I had a question about migrating the coord value one level up. My current > query structure has a root BooleanQuery with a bunch of nested BooleanQuery > children: one of these looks for all terms in the query issued, and I wan

Re: Using a char filter in solr createComponents

2012-08-29 Thread Robert Muir
http://svn.apache.org/viewvc/lucene/dev/branches/branch_4x/lucene/core/src/java/org/apache/lucene/analysis/package.html?r1=1378591&r2=1378590&pathrev=1378591 On Wed, Aug 29, 2012 at 10:35 AM, Robert Muir wrote: > sure. you need to wrap the reader in Analyzer.initReader() to do this: > > /** >

Re: ResourceLoader?

2012-08-29 Thread Benson Margulies
On Wed, Aug 29, 2012 at 10:42 AM, Robert Muir wrote: > Right and what does the @deprecated message say :) > Yes, indeed, sorry. I got caught in a maze of twisty passages and my brain turned off. I'm better now. > > On Wed, Aug 29, 2012 at 10:40 AM, Benson Margulies > wrote: > > Hang on: > > >

Re: ResourceLoader?

2012-08-29 Thread Chris Male
Yeah I deprecated that originally when I first started the work on migrating the Analysis Factories, to reduce the upgrade load. On Thu, Aug 30, 2012 at 2:40 AM, Benson Margulies wrote: > Hang on: > > [deprecation] org.apache.solr.util.plugin.ResourceLoaderAware in > org.apache.solr.util.plugin h

Re: ResourceLoader?

2012-08-29 Thread Robert Muir
Right and what does the @deprecated message say :) On Wed, Aug 29, 2012 at 10:40 AM, Benson Margulies wrote: > Hang on: > > [deprecation] org.apache.solr.util.plugin.ResourceLoaderAware in > org.apache.solr.util.plugin has been deprecated > > > > On Wed, Aug 29, 2012 at 10:30 AM, Robert Muir wro

Re: ResourceLoader?

2012-08-29 Thread Benson Margulies
Hang on: [deprecation] org.apache.solr.util.plugin.ResourceLoaderAware in org.apache.solr.util.plugin has been deprecated On Wed, Aug 29, 2012 at 10:30 AM, Robert Muir wrote: > On Wed, Aug 29, 2012 at 10:27 AM, Benson Margulies > wrote: > > I'm confused. Isn't inform/ResourceLoader deprecate

Re: Using a char filter in solr createComponents

2012-08-29 Thread Robert Muir
sure. you need to wrap the reader in Analyzer.initReader() to do this: /** * Override this if you want to add a CharFilter chain. */ protected Reader initReader(String fieldName, Reader reader) { return reader; } by default it returns the original Reader unchanged. There are exam

Re: ResourceLoader?

2012-08-29 Thread Benson Margulies
On Wed, Aug 29, 2012 at 10:30 AM, Robert Muir wrote: > On Wed, Aug 29, 2012 at 10:27 AM, Benson Margulies > wrote: > > I'm confused. Isn't inform/ResourceLoader deprecated? But your example > use > > it? > > > > Where is it deprecated? What does the deprecation message say? > I see. It moved fr

Re: ResourceLoader?

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 10:27 AM, Benson Margulies wrote: > I'm confused. Isn't inform/ResourceLoader deprecated? But your example use > it? > Where is it deprecated? What does the deprecation message say? -- lucidworks.com -

Using a char filter in solr createComponents

2012-08-29 Thread Benson Margulies
I'm close to the bottom of my list here. I've got an Analyzer that, in 3.1, set up a CharFilter in the tokenStream method. So now I have to migrate that to createComponents. Can someone give me a shove in the right direction?

Re: ResourceLoader?

2012-08-29 Thread Benson Margulies
I'm confused. Isn't inform/ResourceLoader deprecated? But your example use it? On Wed, Aug 29, 2012 at 10:20 AM, Robert Muir wrote: > On Wed, Aug 29, 2012 at 10:10 AM, Benson Margulies > wrote: > > Our Solr 3.x code used init(ResourceLoader) and then called the loader to > > read a file. > > >

Re: ResourceLoader?

2012-08-29 Thread Benson Margulies
That's what I meant, thanks. On Wed, Aug 29, 2012 at 10:20 AM, Robert Muir wrote: > On Wed, Aug 29, 2012 at 10:10 AM, Benson Margulies > wrote: > > Our Solr 3.x code used init(ResourceLoader) and then called the loader to > > read a file. > > > > What's the new approach to reading content from

Re: ResourceLoader?

2012-08-29 Thread Robert Muir
On Wed, Aug 29, 2012 at 10:10 AM, Benson Margulies wrote: > Our Solr 3.x code used init(ResourceLoader) and then called the loader to > read a file. > > What's the new approach to reading content from files in the 'usual place'? I'm not aware of init(ResourceLoader), only inform(ResourceLoader).

ResourceLoader?

2012-08-29 Thread Benson Margulies
Our Solr 3.x code used init(ResourceLoader) and then called the loader to read a file. What's the new approach to reading content from files in the 'usual place'?

Re: Payload class

2012-08-29 Thread Robert Muir
Replace with BytesRef (https://issues.apache.org/jira/browse/LUCENE-4122) I'll add a note to MIGRATE.txt about this. On Wed, Aug 29, 2012 at 9:46 AM, Benson Margulies wrote: > I'm failing to find advice in MIGRATE.txt on how to replace 'new > Payload(...)' in migrating to 4.0. What am I missing

Payload class

2012-08-29 Thread Benson Margulies
I'm failing to find advice in MIGRATE.txt on how to replace 'new Payload(...)' in migrating to 4.0. What am I missing?