Re: Writing a really fast lexer

2020-12-12 Thread Sebastiaan Koppe via Digitalmars-d-learn
On Saturday, 12 December 2020 at 18:15:11 UTC, vnr wrote: Yes, I know Pegged, it's a really interesting parser generator engine, nevertheless, the grammar of what I would like to analyse is not a PEG. But I am also curious to know the performances of this tool for very large inputs. The perfo

Re: Writing a really fast lexer

2020-12-12 Thread Bastiaan Veelo via Digitalmars-d-learn
On Saturday, 12 December 2020 at 18:15:11 UTC, vnr wrote: On Saturday, 12 December 2020 at 16:43:43 UTC, Bastiaan Veelo wrote: Have you looked at Pegged [1]? It will give you the lexer and parser in one go. I'd be very interested to see how it performs on that kind of input. -- Bastiaan. [1]

Re: Writing a really fast lexer

2020-12-12 Thread vnr via Digitalmars-d-learn
On Saturday, 12 December 2020 at 16:43:43 UTC, Bastiaan Veelo wrote: On Friday, 11 December 2020 at 19:49:12 UTC, vnr wrote: For a project with good performance, I would need to be able to analyse text. To do so, I would write a parser by hand using the recursive descent algorithm, based on a s

Re: Writing a really fast lexer

2020-12-12 Thread vnr via Digitalmars-d-learn
On Friday, 11 December 2020 at 20:19:49 UTC, H. S. Teoh wrote: On Fri, Dec 11, 2020 at 07:49:12PM +, vnr via Digitalmars-d-learn wrote: [...] If you want a *really* fast lexer, I recommend using GNU Flex (https://github.com/westes/flex/). Unfortunately, AFAIK it does not support D direc

Re: Writing a really fast lexer

2020-12-12 Thread Bastiaan Veelo via Digitalmars-d-learn
On Friday, 11 December 2020 at 19:49:12 UTC, vnr wrote: For a project with good performance, I would need to be able to analyse text. To do so, I would write a parser by hand using the recursive descent algorithm, based on a stream of tokens. I started writing a lexer with the d-lex package (h

Re: Writing a really fast lexer

2020-12-11 Thread H. S. Teoh via Digitalmars-d-learn
On Fri, Dec 11, 2020 at 07:49:12PM +, vnr via Digitalmars-d-learn wrote: > For a project with good performance, I would need to be able to > analyse text. To do so, I would write a parser by hand using the > recursive descent algorithm, based on a stream of tokens. I started > writing a lexer w