> I think IIRC, I actually tried a simple test case and it would work.
I meant wouldn't work, I could not get it to work ever.
Mike
On Thu, Jun 4, 2015 at 2:36 PM, Michael Schmalle <teotigraphix...@gmail.com>
wrote:
> BTW, the loop always happens at the VERY end of the comment, so when you
> get to the end the(the last call of next() that should return null);
>
> tok = tokenizer.next();
>
> never returns, it gets stuck trying to exit.
>
> Mike
>
> On Thu, Jun 4, 2015 at 2:34 PM, Michael Schmalle <
> teotigraphix...@gmail.com> wrote:
>
>> I posted about this a couple weeks ago and I tried recompiling with JFlex
>> 1.5 I think, the older version and still had the problem.
>>
>> Maybe I messed up something but I tried with my same asdoc code when I
>> fixed the build for the FlexJS asdocs. I wanted to see it work with my
>> version of a documentor.
>>
>> I think IIRC, I actually tried a simple test case and it would work.
>>
>> I have code that uses the tokenizer;
>>
>>
>> public void compile()
>> {
>> if (token == null)
>> return;
>>
>> String data = token.getText();
>> ASDocTokenizer tokenizer = new ASDocTokenizer(false);
>> tokenizer.setReader(new StringReader(data));
>> ASDocToken tok = tokenizer.next();
>> boolean foundDescription = false;
>> DocTag pendingTag = null;
>>
>> try
>> {
>> while (tok != null)
>> {
>> if (!foundDescription
>> && tok.getType() == ASTokenTypes.TOKEN_ASDOC_TEXT)
>> {
>> description = tok.getText();
>> }
>> else
>> {
>> // do tags
>> if (tok.getType() == ASTokenTypes.TOKEN_ASDOC_TAG)
>> {
>> if (pendingTag != null)
>> {
>> addTag(pendingTag);
>> pendingTag = null;
>> }
>> pendingTag = new
>> DocTag(tok.getText().substring(1));
>> }
>> else if (tok.getType() ==
>> ASTokenTypes.TOKEN_ASDOC_TEXT)
>> {
>> pendingTag.setDescription(tok.getText());
>> addTag(pendingTag);
>> pendingTag = null;
>> }
>> }
>>
>> foundDescription = true;
>>
>> tok = tokenizer.next();
>> }
>> }
>> catch (Exception e)
>> {
>> e.printStackTrace();
>> }
>> }
>>
>> Mike
>>
>>
>> On Thu, Jun 4, 2015 at 2:30 PM, Alex Harui <aha...@adobe.com> wrote:
>>
>>>
>>>
>>> On 6/4/15, 11:23 AM, "Michael Schmalle" <teotigraphix...@gmail.com>
>>> wrote:
>>> >>Hmm. Maybe I should spend some time looking into fixing
>>> ASDocTokenizer?
>>> >> Was the problem that it didn’t work on every AS file we current have?
>>> >>
>>> >
>>> >
>>> >It doesn't work on anything, there is an infinite loop in the scanner
>>> that
>>> >is created by JFlex, the RawASDocTokenizer is broken.
>>> >
>>> >What is weird is I was using the SAME code base when I wrote the asdoc
>>> >documenter I have 2 years ago and it worked fine.
>>>
>>> We upgraded the version of JFlex, IIRC. I’ll take a look. What setup
>>> did
>>> you have for trying it? Did you run it on the Flex SDK or FlexJS SDK or
>>> did it even loop on a simple test case?
>>>
>>> -Alex
>>>
>>>
>>
>
