Thanks mark.
Take the comma out of: | <#P: ("_"|"-"|"/"|"."|",") > in the .jj file
Its working for me...
- Bhavin pandya
- Original Message -
From: "Mark Miller" <[EMAIL PROTECTED]>
To:
Sent: Monday, Septem
Take the comma out of: | <#P: ("_"|"-"|"/"|"."|",") > in the .jj file
(around line 92). Keep in mind that this will affect being able to find
tokens that where previously indexed with the comma there (obviously). I
believe the javacc target in the build file will rebuild...you need to
get javac
Hi,
Standard tokenizer works pretty well for me... but i found one problem with my
usage...
I want to tokenize..."TheRing6,Proposal6,GuyandGirl6" as a three saparate
tokens.. while standard analyzer considering it as a one word because it has
one digit in token.
Expected three tokens:
1. ther
