I just pushed a change that seems to get ASDoc working. There was a rule
that looked like it tried to eliminate double-spaces and somehow the lexer
ended up there at the end of string instead of thinking it was done.
I have no idea what that was for since the output currently seems to
capture line-feeds in the asdoc so whitespace in general probably needs to
be trimmed before turning it into its final form.
Let me know if you think that lexer rule was important and we can revisit
why we get stuck there.
-Alex
On 6/4/15, 12:13 PM, "Michael Schmalle" <teotigraphix...@gmail.com> wrote:
>Yeah sorry to confuse you, the Velocity stuff doesn't matter, for that
>matter my asdoc framework I wrote doesn't(it was just showing I had all
>this working in the context of Falcon), we just need the ASDocTokenizer to
>tokenize the comment data given to the ASDocDelegate.
>
>Mike
>
>On Thu, Jun 4, 2015 at 3:12 PM, Michael Schmalle
><teotigraphix...@gmail.com>
>wrote:
>
>> The way I did it was, I did exactly what you did, implemented the
>> ASDocDelegate and saved the tokes as it parsed all the files.
>>
>> Then I used the token String like you in the ASDocTokenizer to parse the
>> loop I showed you above.
>>
>> I would add the DocTag and stuff if you can get the ASDocTokenizer
>>working
>> like how I have it in the code above. I already wrote an API for easy
>> access to the tags and comment in an ASDocCOmment class that has a list
>>of
>> DocTags.
>>
>> Mike
>>
>> On Thu, Jun 4, 2015 at 3:09 PM, Alex Harui <aha...@adobe.com> wrote:
>>
>>> For this exercise though, we don’t care about the output as Velocity or
>>> XSL right? All you want is ASDocTokens in the AST? IIRC, in Falcon
>>>you
>>> retrieve ASDoc comments via node.getASDocComment() and get an
>>>ASDocComment
>>> instance. Do you want the Token to be the root of a mini-tree of
>>>parsed
>>> nodes?
>>>
>>> -Alex
>>>
>>>
>>> On 6/4/15, 11:47 AM, "Michael Schmalle" <teotigraphix...@gmail.com>
>>> wrote:
>>>
>>> >I actually wrote a WHOLE NEW asdoc program that uses Apache Velocity
>>> >templates instead of XSL.
>>> >
>>> >That DocTag is my class.
>>> >
>>> >Mike
>>> >
>>> >On Thu, Jun 4, 2015 at 2:45 PM, Alex Harui <aha...@adobe.com> wrote:
>>> >
>>> >> I don’t see any signs of ASDoc support in flex-falcon. I see
>>> >> ASDocTokenizer and ASDocToken, but no ASDOC.java that would be
>>> >>equivalent
>>> >> to MXMLC.java and have a main() method. The current Flex SDK has an
>>> >> ASDoc.jar. Shouldn’t we have these pieces? Do you have them around
>>> >> somewhere? Otherwise I will try to quickly create them.
>>> >>
>>> >> -Alex
>>> >>
>>> >> On 6/4/15, 11:36 AM, "Michael Schmalle" <teotigraphix...@gmail.com>
>>> >>wrote:
>>> >>
>>> >> >BTW, the loop always happens at the VERY end of the comment, so
>>>when
>>> >>you
>>> >> >get to the end the(the last call of next() that should return
>>>null);
>>> >> >
>>> >> >tok = tokenizer.next();
>>> >> >
>>> >> >never returns, it gets stuck trying to exit.
>>> >> >
>>> >> >Mike
>>> >> >
>>> >> >On Thu, Jun 4, 2015 at 2:34 PM, Michael Schmalle
>>> >> ><teotigraphix...@gmail.com>
>>> >> >wrote:
>>> >> >
>>> >> >> I posted about this a couple weeks ago and I tried recompiling
>>>with
>>> >> >>JFlex
>>> >> >> 1.5 I think, the older version and still had the problem.
>>> >> >>
>>> >> >> Maybe I messed up something but I tried with my same asdoc code
>>>when
>>> >>I
>>> >> >> fixed the build for the FlexJS asdocs. I wanted to see it work
>>>with
>>> >>my
>>> >> >> version of a documentor.
>>> >> >>
>>> >> >> I think IIRC, I actually tried a simple test case and it would
>>>work.
>>> >> >>
>>> >> >> I have code that uses the tokenizer;
>>> >> >>
>>> >> >>
>>> >> >> public void compile()
>>> >> >> {
>>> >> >> if (token == null)
>>> >> >> return;
>>> >> >>
>>> >> >> String data = token.getText();
>>> >> >> ASDocTokenizer tokenizer = new ASDocTokenizer(false);
>>> >> >> tokenizer.setReader(new StringReader(data));
>>> >> >> ASDocToken tok = tokenizer.next();
>>> >> >> boolean foundDescription = false;
>>> >> >> DocTag pendingTag = null;
>>> >> >>
>>> >> >> try
>>> >> >> {
>>> >> >> while (tok != null)
>>> >> >> {
>>> >> >> if (!foundDescription
>>> >> >> && tok.getType() ==
>>> >> >>ASTokenTypes.TOKEN_ASDOC_TEXT)
>>> >> >> {
>>> >> >> description = tok.getText();
>>> >> >> }
>>> >> >> else
>>> >> >> {
>>> >> >> // do tags
>>> >> >> if (tok.getType() ==
>>> >>ASTokenTypes.TOKEN_ASDOC_TAG)
>>> >> >> {
>>> >> >> if (pendingTag != null)
>>> >> >> {
>>> >> >> addTag(pendingTag);
>>> >> >> pendingTag = null;
>>> >> >> }
>>> >> >> pendingTag = new
>>> >> >> DocTag(tok.getText().substring(1));
>>> >> >> }
>>> >> >> else if (tok.getType() ==
>>> >> >> ASTokenTypes.TOKEN_ASDOC_TEXT)
>>> >> >> {
>>> >> >> pendingTag.setDescription(tok.getText());
>>> >> >> addTag(pendingTag);
>>> >> >> pendingTag = null;
>>> >> >> }
>>> >> >> }
>>> >> >>
>>> >> >> foundDescription = true;
>>> >> >>
>>> >> >> tok = tokenizer.next();
>>> >> >> }
>>> >> >> }
>>> >> >> catch (Exception e)
>>> >> >> {
>>> >> >> e.printStackTrace();
>>> >> >> }
>>> >> >> }
>>> >> >>
>>> >> >> Mike
>>> >> >>
>>> >> >>
>>> >> >> On Thu, Jun 4, 2015 at 2:30 PM, Alex Harui <aha...@adobe.com>
>>> wrote:
>>> >> >>
>>> >> >>>
>>> >> >>>
>>> >> >>> On 6/4/15, 11:23 AM, "Michael Schmalle"
>>><teotigraphix...@gmail.com
>>> >
>>> >> >>> wrote:
>>> >> >>> >>Hmm. Maybe I should spend some time looking into fixing
>>> >> >>>ASDocTokenizer?
>>> >> >>> >> Was the problem that it didn’t work on every AS file we
>>>current
>>> >> >>>have?
>>> >> >>> >>
>>> >> >>> >
>>> >> >>> >
>>> >> >>> >It doesn't work on anything, there is an infinite loop in the
>>> >>scanner
>>> >> >>> that
>>> >> >>> >is created by JFlex, the RawASDocTokenizer is broken.
>>> >> >>> >
>>> >> >>> >What is weird is I was using the SAME code base when I wrote
>>>the
>>> >>asdoc
>>> >> >>> >documenter I have 2 years ago and it worked fine.
>>> >> >>>
>>> >> >>> We upgraded the version of JFlex, IIRC. I’ll take a look. What
>>> >>setup
>>> >> >>>did
>>> >> >>> you have for trying it? Did you run it on the Flex SDK or
>>>FlexJS
>>> >>SDK
>>> >> >>>or
>>> >> >>> did it even loop on a simple test case?
>>> >> >>>
>>> >> >>> -Alex
>>> >> >>>
>>> >> >>>
>>> >> >>
>>> >>
>>> >>
>>>
>>>
>>
