Re: How to tokenize with comma in standard tokenizer

2007-09-18 Thread Bhavin Pandya
Thanks mark. Take the comma out of: | <#P: ("_"|"-"|"/"|"."|",") > in the .jj file Its working for me... - Bhavin pandya - Original Message - From: "Mark Miller" <[EMAIL PROTECTED]> To: Sent: Monday, Septem

Re: How to tokenize with comma in standard tokenizer

2007-09-17 Thread Mark Miller
Take the comma out of: | <#P: ("_"|"-"|"/"|"."|",") > in the .jj file (around line 92). Keep in mind that this will affect being able to find tokens that where previously indexed with the comma there (obviously). I believe the javacc target in the build file will rebuild...you need to get javac