Re: Further C++ operators for position

[Hans Åberg]

> On 5 Nov 2019, at 07:51, Akim Demaille  wrote:
> 
>> Le 4 nov. 2019 à 21:16, Hans Åberg  a écrit :
>> 
>> 
>>> On 4 Nov 2019, at 18:12, Akim Demaille  wrote:
>>> 
 Le 4 nov. 2019 à 17:03, Matthew Fernandez  a 
 écrit :
 
 The std::less implementation you suggest is to also lexicographically 
 compare the filenames themselves? I’m not sure this makes sense, because 
 source positions from two different files aren’t really orderable at all.
>>> 
>>> The point of defining std::less is to have an easy means to insert 
>>> positions in a sorted container, say std::map.  Now, the order in itself is 
>>> well defined, but my not reflect the order the user would like to see.
>>> 
>>> To be clear: I don't have a problem with std::less which I see as an 
>>> implementation detail, but operators such as <= and the like are different: 
>>> they express a total
> 
> (I meant "natural" here).
> 
>>> order that we can't implement easily.  
>> 
>> The total order is expressed via std::less in containers such as std::map, 
>> with undefined results if not fulfilling the specs for that.
> 
> Yes, but that's not my point.  I mean: it is not important std::less "means" 
> something natural, what matters is only that it's total and well-defined 
> (unless, of course, you make this order visible to the user).  So I wouldn't 
> mind defining std::less for position and locations.

The point of implementing it would in use of containers like std::map, which 
will assume that std::less can be to define a total order.

> But operator<= is expected to mean something natural (in addition to well 
> defined and total).  So I would not define such an operator (except with a 
> global offset/counter).

It is only in C++ they are expected to relate to a total order, common use in 
partially ordered sets [1]. In fact, I defined a type “order” with values 
unordered, less, equal, greater, and it can be used to define a partial order. 

1. https://en.wikipedia.org/wiki/Partially_ordered_set


>>> In addition, think of C where you also have main.c that #include "foo.h" 
>>> somewhere, which results in main.c:1 (i.e., line 1) < foo.h:1 < ... < 
>>> foo.h:42 < ... < main.c:3.
>> 
>> Here the files are stacked, and if the nested files are closed after being 
>> read, the location pointers are dead.
> 
> W00t?  Typical parsers generate ASTs and typical ASTs are decorated with 
> locations.

Only that when open an included file, one may use yyin = new 
std::ifstream(str), where all data, buffers and locations are stacked. Then 
after the file has been read, it is closed and the yyin pointer is deallocated.

Re: Further C++ operators for position

[Matthew Fernandez]

> On Nov 4, 2019, at 22:58, Akim Demaille  wrote:
> 
> hi Matthew,
> 
>> Le 5 nov. 2019 à 02:10, Matthew Fernandez  a 
>> écrit :
>> 
>>> On Nov 4, 2019, at 12:16, Hans Åberg  wrote:
>>> 
 On 4 Nov 2019, at 18:12, Akim Demaille  wrote:
 
 If we want a total order here, it's actually easy: positions should have a 
 counter somewhere which is the *total* "offset" since the first byte of 
 the first file.  Or something like that.
>> 
>> Is it? What if you parsed two separate input sources? It makes no sense to 
>> order a position in one file with respect to a position in the other.
> 
> Maybe I should have used the word "offset" here, which was to read together 
> with the "*total*" before it.
> 
> What I mean is a unique counter you increment each time you read a char, 
> whatever the file.  Printed after @ below.
> 
> main.c:1@1 < foo.h:1@16 < ... < foo.h:42@1010 < ... < main.c:3@2020.

No, I meant that you could instantiate the parser twice and parse two unrelated 
files. Each of the two resulting ASTs will have attached locations and 
positions. It seems to me that it doesn’t make sense to compare a position from 
one AST to a position in the other.

Re: Bison 3.4.90 released [beta]

[Adrian Vogelsgesang]

Hi Akim,

I quickly upgraded Hyper (having a SQL grammar, similar to Postgres’ grammar)
to use Bison 3.4.90.
Things are looking good, our internal test cases are successful.
I did not go through all our tests, though, e.g. I did not check address 
sanitizer and friends.

I also tested LAC, and it works :)

> positions moved from unsigned (for line and column numbers) to int.
Yes, I ran into this. Our build was failing at first due to this.

…/Debug/codegen/hyper/cts/parser/combinedsql.ypp: In function ‘int 
yylex(hyper::astql::AST**, sqlparser::SQLParser::location_type*, 
hyper::SQLLexer&, hyper::ASTContainer&)’:
…/Debug/codegen/hyper/cts/parser/combinedsql.ypp:2504:79: error: narrowing 
conversion of ‘(unsigned int)(long int)token.hyper::SQLLexer::Token::begin) 
- ((long int)scannerBegin)) (ceiling /) 1) + 1)’ from ‘unsigned int’ to ‘int’ 
inside { } [-Werror=narrowing]

(*yyloc)=sqlparser::SQLParser::location_type{sqlparser::position{nullptr,1,static_cast(token.begin-scannerBegin+1)},sqlparser::position{nullptr,1,static_cast(token.end-scannerBegin+1)}};
   
^

Fortunately, it was easy to fix by changing the static_cast to 
static_cast.
We are abusing the bison-provided location class to store byte-offsets here, so 
I guess this falls into the category of “breakage due to API abuse”.

Cheers,
Adrian



From: bison-patches  
on behalf of Akim Demaille 
Reply-To: Bison Bugs 
Date: Wednesday, 30 October 2019 at 07:30
To: "bison-annou...@gnu.org" 
Cc: Benno Schulenberg , Bison Help 
, Bison Bugs , Bison Patches 

Subject: Bison 3.4.90 released [beta]

Dear Bison riders,

There is already quite a large number of patches since Bison 3.4:
265 so far (3.0: 1225, 3.1: 252, 3.2: 297, 3.3: 255, 3.4: 209). Many
of them are "invisible" and mostly related to refactoring and
preparing forthcoming changes. But some of them have a very visible
impact.

In particular the use of integral types has been revised in both the
generator and the generated parsers (thanks to Paul Eggert). As a
consequence small parsers have a smaller footprint, and very large
automata are now possible with the default backend (yacc.c).

There is one breaking change that somewhat worries me: in C++
positions moved from unsigned (for line and column numbers) to int.
It might break some existing projects, and maybe not. We *need* to
know! If it does break your project, then we will design some
migration path. Otherwise, let's keep things simple.

Please, torture this beta, and report your mileage!

Cheers!

==

Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables. Bison can also generate IELR(1) or
canonical LR(1) parser tables. Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change. Anyone familiar with Yacc should be able to
use Bison with little trouble. You need to be fluent in C, C++ or Java
programming in order to use Bison.

Here is the GNU Bison home page:
https://gnu.org/software/bison/

==

Here are the compressed sources:
https://alpha.gnu.org/gnu/bison/bison-3.4.90.tar.gz
 (4.1MB)
https://alpha.gnu.org/gnu/bison/bison-3.4.90.tar.xz
 (3.1MB)

Here are the GPG detached signatures[*]:
https://alpha.gnu.org/gnu/bison/bison-3.4.90.tar.gz.sig
https://alpha.gnu.org/gnu/bison/bison-3.4.90.tar.xz.sig

Use a mirror for higher download bandwidth:
https://www.gnu.org/order/ftp.html

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact. First, be sure to download both the .sig file
and the corresponding tarball. Then, run a command like this:

gpg --verify bison-3.4.90.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

gpg --keyserver keys.gnupg.net --recv-keys 
0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
Autoconf 2.69
Automake 1.16.1
Flex 2.6.4
Gettext 0.19.8.1
Gnulib v0.1-2899-g56ca994d4

==

NEWS

* Noteworthy changes in release 3.4.90 (2019-10-29) [beta]

** Backward incompatible changes

L

Re: Further C++ operators for position

[Akim Demaille]

Hi Matthew,

> Le 5 nov. 2019 à 16:04, Matthew Fernandez  a 
> écrit :
> 
> No, I meant that you could instantiate the parser twice and parse two 
> unrelated files. Each of the two resulting ASTs will have attached locations 
> and positions. It seems to me that it doesn’t make sense to compare a 
> position from one AST to a position in the other.

Indeed, it should not happen.