Error with grammar arguments
Hello,
I have a (large) grammar where I have code similar to that below:
| identifier ':' array_spec_init {
log_debug( "%p", $identifier );
log_debug( "%p", $array_spec_init );
}
When this grammar is run, the following is shown in the output:
Stack now 0 1 3 15 83
Entering state 194
Reducing stack by rule 154 (line 2062):
$1 = token identifier (1.6-25: )
$2 = token ':' (1.27: )
$3 = nterm array_spec_init (1.29-56: )
debug: *0x255b930*
debug: *0x255b930*
Note that the same pointer address is returned for $identifier and
$array_spec_init.
Before asking anyone to look at my grammar, I thought I would ask as to
whether anyone has seen an error such as this before?
Thanks in advance.
Rob
___
help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Re: Error with grammar arguments
Rob,
To ask a sanity question, you do strdup (or otherwise allocate fresh memory
for) yytext on the Flex side when returning tokens, yes?
-Chris
On Tue, Dec 18, 2018 at 9:15 AM Rob Casey wrote:
> Hello,
>
> I have a (large) grammar where I have code similar to that below:
>
> | identifier ':' array_spec_init {
> log_debug( "%p", $identifier );
> log_debug( "%p", $array_spec_init );
> }
>
> When this grammar is run, the following is shown in the output:
>
> Stack now 0 1 3 15 83
> Entering state 194
> Reducing stack by rule 154 (line 2062):
>$1 = token identifier (1.6-25: )
>$2 = token ':' (1.27: )
>$3 = nterm array_spec_init (1.29-56: )
> debug: *0x255b930*
> debug: *0x255b930*
>
> Note that the same pointer address is returned for $identifier and
> $array_spec_init.
>
> Before asking anyone to look at my grammar, I thought I would ask as to
> whether anyone has seen an error such as this before?
>
> Thanks in advance.
>
> Rob
> ___
> help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
>
___
help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Re: Error with grammar arguments
Yes - That is correct Chris. I am allocating memory within Flex for tokens.
This issue first manifest itself when I started to add code for the proper
freeing on this memory (where no longer required) within grammar actions.
On Wed, 19 Dec. 2018, 04:40 Chris verBurg Rob,
>
> To ask a sanity question, you do strdup (or otherwise allocate fresh
> memory for) yytext on the Flex side when returning tokens, yes?
>
> -Chris
>
>
> On Tue, Dec 18, 2018 at 9:15 AM Rob Casey wrote:
>
>> Hello,
>>
>> I have a (large) grammar where I have code similar to that below:
>>
>> | identifier ':' array_spec_init {
>> log_debug( "%p", $identifier );
>> log_debug( "%p", $array_spec_init );
>> }
>>
>> When this grammar is run, the following is shown in the output:
>>
>> Stack now 0 1 3 15 83
>> Entering state 194
>> Reducing stack by rule 154 (line 2062):
>>$1 = token identifier (1.6-25: )
>>$2 = token ':' (1.27: )
>>$3 = nterm array_spec_init (1.29-56: )
>> debug: *0x255b930*
>> debug: *0x255b930*
>>
>> Note that the same pointer address is returned for $identifier and
>> $array_spec_init.
>>
>> Before asking anyone to look at my grammar, I thought I would ask as to
>> whether anyone has seen an error such as this before?
>>
>> Thanks in advance.
>>
>> Rob
>> ___
>> help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
>>
>
___
help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Re: Error with grammar arguments
> On 18 Dec 2018, at 20:06, Rob Casey wrote: > > Yes - That is correct Chris. I am allocating memory within Flex for tokens. > This issue first manifest itself when I started to add code for the proper > freeing on this memory (where no longer required) within grammar actions. It looks like a lexer issue, same pointer passed to the Bison parser. You might write out the strings to see whether it is overwritten or not written the second time. > On Wed, 19 Dec. 2018, 04:40 Chris verBurg >> Rob, >> >> To ask a sanity question, you do strdup (or otherwise allocate fresh >> memory for) yytext on the Flex side when returning tokens, yes? >> >> -Chris ___ help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Re: Error with grammar arguments
Thanks for the tip Chris, Hans. I have a straight-forward unit test where this issue is manifesting, so I will explore this with respect to lexer token allocation. On Wed, Dec 19, 2018 at 8:21 AM Hans Åberg wrote: > > > On 18 Dec 2018, at 20:06, Rob Casey wrote: > > > > Yes - That is correct Chris. I am allocating memory within Flex for > tokens. > > This issue first manifest itself when I started to add code for the > proper > > freeing on this memory (where no longer required) within grammar actions. > > It looks like a lexer issue, same pointer passed to the Bison parser. You > might write out the strings to see whether it is overwritten or not written > the second time. > > > On Wed, 19 Dec. 2018, 04:40 Chris verBurg wrote: > > > >> Rob, > >> > >> To ask a sanity question, you do strdup (or otherwise allocate fresh > >> memory for) yytext on the Flex side when returning tokens, yes? > >> > >> -Chris > > > ___ help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Re: Error with grammar arguments
Thanks again for the assistance Chris and Hans. I found a path through my grammar where $$ was not explicitly set within a rule action which would explain identifier and array_spec_init having the same pointer value. On Wed, Dec 19, 2018 at 8:42 AM Rob Casey wrote: > Thanks for the tip Chris, Hans. I have a straight-forward unit test where > this issue is manifesting, so I will explore this with respect to lexer > token allocation. > > > On Wed, Dec 19, 2018 at 8:21 AM Hans Åberg wrote: > >> >> > On 18 Dec 2018, at 20:06, Rob Casey wrote: >> > >> > Yes - That is correct Chris. I am allocating memory within Flex for >> tokens. >> > This issue first manifest itself when I started to add code for the >> proper >> > freeing on this memory (where no longer required) within grammar >> actions. >> >> It looks like a lexer issue, same pointer passed to the Bison parser. You >> might write out the strings to see whether it is overwritten or not written >> the second time. >> >> > On Wed, 19 Dec. 2018, 04:40 Chris verBurg > wrote: >> > >> >> Rob, >> >> >> >> To ask a sanity question, you do strdup (or otherwise allocate fresh >> >> memory for) yytext on the Flex side when returning tokens, yes? >> >> >> >> -Chris >> >> >> ___ help-bison@gnu.org https://lists.gnu.org/mailman/listinfo/help-bison
Bison 3.2.3 released [stable]
Bison 3.2.3 is a bug fix release of Bison 3.2.
Bison 3.2 brought massive improvements to the deterministic C++ skeleton,
lalr1.cc. When variants are enabled and the compiler supports C++11 or
better, move-only types can now be used for semantic values. C++98 support
is not deprecated. Please see the NEWS below for more details.
Many thanks to Frank Heckenbach for paving the way for this release with his
implementation of a skeleton in C++17, and to Nelson H. F. Beebe for testing
exhaustively portability issues.
==
Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables. Bison can also generate IELR(1) or
canonical LR(1) parser tables. Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.
Bison is upward compatible with Yacc: all properly-written Yacc grammars
ought to work with Bison with no change. Anyone familiar with Yacc should be
able to use Bison with little trouble. You need to be fluent in C or C++
programming in order to use Bison. Java is also supported.
Here is the GNU Bison home page:
https://gnu.org/software/bison/
==
Here are the compressed sources:
https://ftp.gnu.org/gnu/bison/bison-3.2.3.tar.gz (4.1MB)
https://ftp.gnu.org/gnu/bison/bison-3.2.3.tar.xz (2.1MB)
Here are the GPG detached signatures[*]:
https://ftp.gnu.org/gnu/bison/bison-3.2.3.tar.gz.sig
https://ftp.gnu.org/gnu/bison/bison-3.2.3.tar.xz.sig
Use a mirror for higher download bandwidth:
https://www.gnu.org/order/ftp.html
[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:
gpg --verify bison-3.2.3.tar.gz.sig
If that command fails because you don't have the required public key,
then run this command to import it:
gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E
and rerun the 'gpg --verify' command.
This release was bootstrapped with the following tools:
Autoconf 2.69
Automake 1.16.1
Flex 2.6.4
Gettext 0.19.8.1
Gnulib v0.1--g48a6c46b0
NEWS
* Noteworthy changes in release 3.2.3 (2018-12-18) [stable]
** Bug fixes
Properly support token constructors in C++ with types that include commas
(e.g., std::pair). A regression introduced in Bison 3.2.
* Noteworthy changes in release 3.2.2 (2018-11-21) [stable]
** Bug fixes
C++ portability issues.
* Noteworthy changes in release 3.2.1 (2018-11-09) [stable]
** Bug fixes
Several portability issues have been fixed in the build system, in the
test suite, and in the generated parsers in C++.
* Noteworthy changes in release 3.2 (2018-10-29) [stable]
** Backward incompatible changes
Support for DJGPP, which has been unmaintained and untested for years, is
obsolete. Unless there is activity to revive it, it will be removed.
** Changes
%printers should use yyo rather than yyoutput to denote the output stream.
Variant-based symbols in C++ should use emplace() rather than build().
In C++ parsers, parser::operator() is now a synonym for the parser::parse.
** Documentation
A new section, "A Simple C++ Example", is a tutorial for parsers in C++.
A comment in the generated code now emphasizes that users should not
depend upon non-documented implementation details, such as macros starting
with YY_.
** New features
*** C++: Support for move semantics (lalr1.cc)
The lalr1.cc skeleton now fully supports C++ move semantics, while
maintaining compatibility with C++98. You may now store move-only types
when using Bison's variants. For instance:
%code {
#include
#include
}
%skeleton "lalr1.cc"
%define api.value.type variant
%%
%token INT "int";
%type > int;
%type >> list;
list:
%empty{}
| list int { $$ = std::move($1); $$.emplace_back(std::move($2)); }
int: "int" { $$ = std::make_unique($1); }
*** C++: Implicit move of right-hand side values (lalr1.cc)
In modern C++ (C++11 and later), you should always use 'std::move' with
the values of the right-hand side symbols ($1, $2, etc.), as they will be
popped from the stack anyway. Using 'std::move' is mandatory for
move-only types such as unique_ptr, and it provides a significant speedup
for large types such as std::string, or std::vector, etc.
If '%define api.value.automove' is set, every occurrence '$n' is replaced
by 'std::move ($n)'. The second rule in the previous grammar can be
simplified to:
list: list int { $$ = $1; $$.emplace_back($2); }
With automove enabled, the semantic values are no longer lvalues, so do
not use the s
