Re: Unique tokens analyzer

2008-10-14 Thread markharw00d
Related: https://issues.apache.org/jira/browse/LUCENE-725 - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]

Unique tokens analyzer

2008-10-14 Thread Rafael C. de Almeida
Hello, Is there a analyzer that will tokenize the stream such that there's no repeated tokens in the stream? I have a keyword-field on my document, so if one keyword already appears on the list there's no point in having it shown again. Does it make sense having that analyzer? Or indexing