On 6/28/05, Aigner, Thomas <[EMAIL PROTECTED]> wrote:
> Hello all,
>
> I am VERY new to Lucene and we are trying out Lucene to see if
> it will accomplish the vast majority of our search functions.
>
> I have a question about a good way to index some of our product
> description codes. We have description codes like 21-MA-GAB and other
> punctuation. Our users need to be able to search for "21 MA GAB" or
> "21-MA_GAB" or "21MAGAB". Is the best way to accomplish this by
> creating synonyms for the 3 different ways when punctuation is in parts
> to search for? I know I can stop punctuation in the index but what about
> grouping the information together or with spaces?
>
> Thanks all in advance,
> Tom
There is a couple ways to do this, and I'm not sure which would be
best. (I'm also fairly new to lucene)
You can create a grammar that recognizes your product codes (see
StandardAnalyzer code for examples on how to do that) then use a
custom filter to normalize everything.
Forgive my poor lex but general idea
| <CODE: <NUM><NUM> ("-"|"_"|""|" ") <ALPHA>+ ("-"|"_"|""|" ") <ALPHA>+ >
Then in the filter, normalize to strip out all of the punctuation.
This can be done with a regex or something faster but just for
reference.
if (type == CODE_TYPE) {
return new org.apache.lucene.analysis.Token(text.replaceAll("-",
""), t.startOffset(), t.endOffset(), type);
} ...
See StandardAnalyzer, it has a lot of code that would do what you need
and you can copy, paste and edit.
You could also do synonyms but that seems like it would be more overhead.
If you think of a better way, let me know, I have to do something similar.
Cheers,
Chris
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
