Re: Unquoted strings in BASIC

2024-12-04 Thread Maury Markowitz
To top this thread off, I went with this: /* DATA statements convert everything to a string */ { [^,:\n]* { // eat any leading and trailing whitespace yytext = str_trim(yytext); // and quotes yytext = str_unquote(yytext);

Re: Unquoted strings in BASIC

2024-12-04 Thread Hans Åberg
> On 3 Dec 2024, at 21:24, James K. Lowden wrote: > > On Tue, 3 Dec 2024 17:57:13 +0100 > Hans Åberg wrote: > >> In general, one tries to exploit tokens to do special things, which >> is also useful in a Bison parser when adding mid-rule actions. > > That's good advice, hard-won. > >>> i

Re: Unquoted strings in BASIC

2024-12-04 Thread James K. Lowden
On Tue, 3 Dec 2024 17:57:13 +0100 Hans Åberg wrote: > In general, one tries to exploit tokens to do special things, which > is also useful in a Bison parser when adding mid-rule actions. That's good advice, hard-won. > > it seems it is much easier to parse everything in the data section > >

Re: Unquoted strings in BASIC

2024-12-03 Thread Hans Åberg
> On 3 Dec 2024, at 17:44, Maury Markowitz wrote: > >> On Dec 1, 2024, at 2:07 PM, James K. Lowden wrote: >> >> require some real thinking, never easy. For example, is >> >> 10100 DATA "one",2B,"three" >> >> valid? If it is, you'll need one pattern for "leading alpha" that might >> be jus

Re: Unquoted strings in BASIC

2024-12-03 Thread Maury Markowitz
> On Dec 1, 2024, at 2:07 PM, James K. Lowden wrote: > > require some real thinking, never easy. For example, is > > 10100 DATA "one",2B,"three" > > valid? If it is, you'll need one pattern for "leading alpha" that might > be just one character, and another for "leading digit" that requ

Re: Unquoted strings in BASIC

2024-12-02 Thread Hans Åberg
> On 2 Dec 2024, at 03:10, Maury Markowitz wrote: > > I had managed to get to this point for testing the conditional states: > > [^0-9][A-Za-z]*[,:\n] { > yytext[strlen(yytext) - 1] = '\0'; > yylval.s = str_new(yytext + 1); > return STRING; > } Notice tha

Re: Unquoted strings in BASIC

2024-12-02 Thread Hans Åberg
> On 1 Dec 2024, at 20:07, James K. Lowden wrote: > > On Sun, 1 Dec 2024 21:10:31 -0500 > Maury Markowitz wrote: > >>> DATA { yy_push_state(data); } >> >> Is there a difference between yy_push_state(data) and BEGIN(data)? > > Yes. As stated in the manual, BEGIN simply changes the start > c

Re: Unquoted strings in BASIC

2024-12-02 Thread James K. Lowden
On Sun, 1 Dec 2024 21:10:31 -0500 Maury Markowitz wrote: > > DATA { yy_push_state(data); } > > Is there a difference between yy_push_state(data) and BEGIN(data)? Yes. As stated in the manual, BEGIN simply changes the start condition. yy_push_state and yy_pop_state use a stack, so you can retur

Re: Unquoted strings in BASIC

2024-12-01 Thread Maury Markowitz
> On Nov 30, 2024, at 8:42 PM, James K. Lowden wrote: > I'm just a bit skeptical. If the rule is that > >> DATA 10,20,"HELLO,WORLD!" > and >> DATA 10,20,HELLO,WORLD! > Or is it the case that the quoted equivalent would be > >> DATA 10,20,"HELLO","WORLD!" This. It's basically CSV, and given

Re: Unquoted strings in BASIC

2024-12-01 Thread James K. Lowden
On Sun, 1 Dec 2024 10:07:45 -0500 Maury Markowitz wrote: > DATA 10,20,HELLO,WORLD! > > I am looking for ways to attack this. To add to what Hans said, you want to extend what the scanner accepts in a DATA statement. In your case, DATA has a slightly more expansive view of what an expression is

Re: Unquoted strings in BASIC

2024-12-01 Thread Hans Åberg
> On 1 Dec 2024, at 19:08, Maury Markowitz wrote: > >> This will probably require you to take into account the current context; see >> Hans's reply. > > Yeah, I am completely new to that side of things. I'm in the bison dox now > and I can't say I'm understanding much yet. Check the Flex ma

Re: Unquoted strings in BASIC

2024-12-01 Thread Maury Markowitz
> On Dec 1, 2024, at 12:31 PM, EML wrote: > > This matches, among other things, a string which starts with a double quote, > terminated by a newline, with no closing quote. Not even Basic can be that > bad. Sorry to be the bearer of bad news, but BASIC is precisely that bad. Since many PRINTs

Re: Unquoted strings in BASIC

2024-12-01 Thread EML
Basic... wow. Start by fixing your regexes: [0-9]*[0-9.][0-9]*([Ee][-+]?[0-9]+)? { yylval.d = strtod(yytext, NULL); return NUMBER; } This matches a single '.', '.E0', and so on. Presumably you want something which looks more like dec_digit [0-9] suffix 

Re: Unquoted strings in BASIC

2024-12-01 Thread Hans Åberg
> On 1 Dec 2024, at 16:07, Maury Markowitz wrote: > > But many dialects allow strings to be unquoted as long as they do not contain > a line end, colon or comma: > > DATA 10,20,HELLO,WORLD! One way is to use context switches; see the Flex and Bison manuals. In the .l file one has say: %x D

Unquoted strings in BASIC

2024-12-01 Thread Maury Markowitz
I seem to have programmed myself into a corner, and I'm hoping someone can offer some suggestions. Source code here: https://github.com/maurymarkowitz/RetroBASIC/tree/master/src The BASIC language has two constant types, numbers and strings. The ~300 line scanner is basically a list of the keyw