Re: Which lexer do people use?

2020-07-06 Thread John P. Hartmann
On 7/6/20 06:36, Akim Demaille wrote: I don't think you ever posted a snippet of that grammar of yours. I suppose there are IP issues, but maybe you could share just a small bit, or a fake example, so that we can have a better idea of it? I did send you one years ago, but here goes again. You

Re: Which lexer do people use?

2020-07-06 Thread Maury Markowitz
> On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck > wrote: > The only thing that people are missing once in a while on scanner side is > unicode support I am missing the capability to support grammars with optional spaces between elements. In BASIC: FORI=1TO5 is equivalent to: FOR I

Re: Which lexer do people use?

2020-07-06 Thread Christian Schoenebeck
On Montag, 6. Juli 2020 14:00:17 CEST Maury Markowitz wrote: > > On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck > > wrote: The only thing that people are missing > > once in a while on scanner side is unicode support > > I am missing the capability to support grammars with optional spaces betw

Re: Which lexer do people use?

2020-07-06 Thread uxio prego
This is elegant to me. I think it can be put to work, but am not sure. for_construct : TOKEN_FOR optional_whitespace identifier TOKEN_EQ number optional_whitespace TOKEN_TO optional_whitespace number { /* nodes with values at indexes $3, $5, $9 */ } > On

Re: Which lexer do people use?

2020-07-06 Thread Akim Demaille
> Le 6 juil. 2020 à 14:00, Maury Markowitz a écrit : > >> On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck >> wrote: >> The only thing that people are missing once in a while on scanner side is >> unicode support > > I am missing the capability to support grammars with optional spaces bet

Parsing a language with optional spaces

2020-07-06 Thread Maury Markowitz
Moving to a new thread - I was surprised I could even post, previous efforts were bounced from the list server for no obvious reason. Someone helpfully posted for me in the past. And now everything is magically working, so I hope you don't all mind the duplicate. > On Jul 6, 2020, at 9:04 AM, C

Re: Parsing a language with optional spaces

2020-07-06 Thread Christian Schoenebeck
On Montag, 6. Juli 2020 18:25:56 CEST Maury Markowitz wrote: > Moving to a new thread - I was surprised I could even post, previous efforts > were bounced from the list server for no obvious reason. Someone helpfully > posted for me in the past. And now everything is magically working, so I > hope

Re: Parsing a language with optional spaces

2020-07-06 Thread Akim Demaille
Hi Maury, > Le 6 juil. 2020 à 18:25, Maury Markowitz a écrit : > > Moving to a new thread - I was surprised I could even post, previous efforts > were bounced from the list server for no obvious reason. That's really weird. >> On Jul 6, 2020, at 9:04 AM, Christian Schoenebeck >> wrote: >>

Re: Parsing a language with optional spaces

2020-07-06 Thread Maury Markowitz
> On Jul 6, 2020, at 1:30 PM, Christian Schoenebeck > wrote: > Which would resolve that ambiguity of your example, if it actually exists, as > I have never seen "FOREX" as valid BASIC loop definition before, if there is, > sources for that specification appreciated. BASIC Computer Games, Super S

Re: Parsing a language with optional spaces

2020-07-06 Thread Maury Markowitz
> On Jul 6, 2020, at 3:23 PM, Akim Demaille wrote: > > FOR/{sp}{id}{sp}={sp}{num}{sp}TO{sp}{num} { printf("for: %s\n", yytext); } This is a very different style than what I have seen in the past. In the past, most examples of flex tend to match against the tokens as individual items and retur

Re: Parsing a language with optional spaces

2020-07-06 Thread Christian Schoenebeck
On Montag, 6. Juli 2020 21:48:19 CEST Maury Markowitz wrote: > > On Jul 6, 2020, at 1:30 PM, Christian Schoenebeck > > wrote: Which would resolve that ambiguity of > > your example, if it actually exists, as I have never seen "FOREX" as > > valid BASIC loop definition before, if there is, sources

Re: Parsing a language with optional spaces

2020-07-06 Thread billfen
I remember many years ago reading through the IBM 360 (Mainframe) assembler code for the Fortran 66 H compiler to understand how the Hollerith constants were parsed. I suspect finding an "H" after an integer (and skipping that many following characters) might be problematic for today's scanners. T

Re: Parsing a language with optional spaces

2020-07-06 Thread Akim Demaille
> Le 6 juil. 2020 à 22:01, Maury Markowitz a écrit : > >> On Jul 6, 2020, at 3:23 PM, Akim Demaille wrote: >> >> FOR/{sp}{id}{sp}={sp}{num}{sp}TO{sp}{num} { printf("for: %s\n", yytext); } > > This is a very different style than what I have seen in the past. In the > past, most examples of