Re: tokenizer to strip a set of characters

2013-11-21 Thread Jack Krupansky
the start or end. -- Jack Krupansky -Original Message- From: Stephane Nicoll Sent: Thursday, November 21, 2013 9:42 AM To: java-user@lucene.apache.org Subject: tokenizer to strip a set of characters Hi, I am using lucene 3.6 and I am looking to a tokenized that would remove certain

tokenizer to strip a set of characters

2013-11-21 Thread Stephane Nicoll
Hi, I am using lucene 3.6 and I am looking to a tokenized that would remove certain characters when they are present at the beginning or at the end of a token. I initially used the StandardAnalyzer and switched to the WhitespaceAnalyser because it was too agressive for my use case. A few example