Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On 4/26/2016 10:38 AM, Alexander Lisachenko wrote: > Hello, Nikita! > > It's a great news! Having this stuff in the 7.1 core is awesome! > > But I have several petitions for this API: > 1) Could it avoid using of raw functions for parsing API, like > ast\parse_file or ast\parse_code > 2) Could yo

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2016-04-26 11:15 GMT+03:00 Nikita Popov : > As an update here, I plan to create an RFC for bundling the php-ast > extension (maybe with minor modifications) with php-src. This was planned > for 7.1 anyway, and with the annotations RFC under discussion, this seems > like a good time. > ... > I'll t

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Tue, Apr 26, 2016 at 9:42 AM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > Hello, internals! > > I'd like to bring this topic to the discussion again, because now we have > a shiny PHP7 engine and keep moving. PHP grammar becomes more complex and > internal AST representation isn't

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Hello, internals! I'd like to bring this topic to the discussion again, because now we have a shiny PHP7 engine and keep moving. PHP grammar becomes more complex and internal AST representation isn't available for the userland developers, so all static analysis tools suffering from the lack of nat

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On 3 March 2015 at 11:56, Alexander Lisachenko wrote: > Good morning! > > I have cleaned https://wiki.php.net/rfc/parser-extension-api and restricted > it's scope only to the parsing API. Extension API can be implemented later +1 > on top of > https://github.com/php/php-src/commit/1010b0ea4f4b9f

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Good morning! I have cleaned https://wiki.php.net/rfc/parser-extension-api and restricted it's scope only to the parsing API. Extension API can be implemented later on top of https://github.com/php/php-src/commit/1010b0ea4f4b9f96ae744f04c1191ac228580e48 and current implementation, because it requi

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Thu, Feb 19, 2015 at 8:42 AM, Sara Golemon wrote: > On Wed, Feb 18, 2015 at 11:22 AM, Dmitry Stogov wrote: > > I think the AST API shouldn't use "public" properties. > > Using it, we will have to construct the whole tree of objects, > duplicating > > information from AST. > > I would propose

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 11:22 AM, Dmitry Stogov wrote: > I think the AST API shouldn't use "public" properties. > Using it, we will have to construct the whole tree of objects, duplicating > information from AST. > I would propose SimpleXML approach instead - construct object only for > node(s) we

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 11:00 PM, Nikita Popov wrote: > On Wed, Feb 18, 2015 at 8:22 PM, Dmitry Stogov wrote: > >> I think the AST API shouldn't use "public" properties. >> Using it, we will have to construct the whole tree of objects, >> duplicating information from AST. >> I would propose Simp

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 10:34 PM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > > 2015-02-18 22:22 GMT+03:00 Dmitry Stogov : > >> I think the AST API shouldn't use "public" properties. >> Using it, we will have to construct the whole tree of objects, >> duplicating information from AST.

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 8:22 PM, Dmitry Stogov wrote: > I think the AST API shouldn't use "public" properties. > Using it, we will have to construct the whole tree of objects, duplicating > information from AST. > I would propose SimpleXML approach instead - construct object only for > node(s) we

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

I want to attach a link to the Python AST documentation: https://docs.python.org/3.2/library/ast.html What I like there is iteration of nodes with generators (yields). NodeVisitor that can be extended and used as coroutine to simplify logic of traversal. I want to keep Node class as simple DTO wit

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-18 22:22 GMT+03:00 Dmitry Stogov : > I think the AST API shouldn't use "public" properties. > Using it, we will have to construct the whole tree of objects, duplicating > information from AST. > I would propose SimpleXML approach instead - construct object only for > node(s) we currently a

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

> I'm okay with having stuff like ->getKindName() on the nodes, however I'd > still keep around the free-standing functions, because they can be used > without a node (i.e. you only need the kind AST_FOO and not an instantiated > node). I think that getting name of kind is useless without value o

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

I think the AST API shouldn't use "public" properties. Using it, we will have to construct the whole tree of objects, duplicating information from AST. I would propose SimpleXML approach instead - construct object only for node(s) we currently access. So at first we will construct just single obje

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 4:22 PM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > > 2015-02-18 17:59 GMT+03:00 Nikita Popov : > >> Alexander, I would recommend you to split this into two RFCs, one dealing >> only with AST export (and maybe pretty printing) and the second one with >> the co

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, 2015-02-18 at 16:17 +0100, Nikita Popov wrote: > On Wed, Feb 18, 2015 at 4:06 PM, Cesar Rodas wrote: > > > > > On 18/02/15 15:59, Nikita Popov wrote: > > > On Wed, Feb 18, 2015 at 7:22 AM, Dmitry Stogov wrote: > > > > > >> Hi, > > >> > > >> On Tue, Feb 17, 2015 at 2:46 PM, Alexander Lisa

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-18 17:59 GMT+03:00 Nikita Popov : > Alexander, I would recommend you to split this into two RFCs, one dealing > only with AST export (and maybe pretty printing) and the second one with > the compilation hooks. There's probably a few questions about the export > API that should be discussed

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 4:06 PM, Cesar Rodas wrote: > > On 18/02/15 15:59, Nikita Popov wrote: > > On Wed, Feb 18, 2015 at 7:22 AM, Dmitry Stogov wrote: > > > >> Hi, > >> > >> On Tue, Feb 17, 2015 at 2:46 PM, Alexander Lisachenko < > >> lisachenko...@gmail.com> wrote: > >> > >>> Hello, internals

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On 18/02/15 15:59, Nikita Popov wrote: > On Wed, Feb 18, 2015 at 7:22 AM, Dmitry Stogov wrote: > >> Hi, >> >> On Tue, Feb 17, 2015 at 2:46 PM, Alexander Lisachenko < >> lisachenko...@gmail.com> wrote: >> >>> Hello, internals! >>> >>> I want to introduce a RFC for providing a userland API for acce

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Wed, Feb 18, 2015 at 7:22 AM, Dmitry Stogov wrote: > Hi, > > On Tue, Feb 17, 2015 at 2:46 PM, Alexander Lisachenko < > lisachenko...@gmail.com> wrote: > >> Hello, internals! >> >> I want to introduce a RFC for providing a userland API for accessing an >> Abstract Syntax Tree of the source code

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Hi, On Tue, Feb 17, 2015 at 2:46 PM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > Hello, internals! > > I want to introduce a RFC for providing a userland API for accessing an > Abstract Syntax Tree of the source code and to provide userland parser > hooks for source code modification

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Hello, Lars! 2015-02-17 22:09 GMT+03:00 Lars Strojny : > Looks cool and I could see a couple of interesting possibilities arising. > One thing: any particular reason ExtensionInterface is static? I could see > a couple of benefits having extensions carry state and registerExtension() > taking an

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Hi Alexander, > On 17 Feb 2015, at 12:46, Alexander Lisachenko > wrote: > > Hello, internals! > > I want to introduce a RFC for providing a userland API for accessing an > Abstract Syntax Tree of the source code and to provide userland parser > hooks for source code modification: > https://wi

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Alexander Lisachenko wrote on 17/02/2015 14:21: 2015-02-17 16:35 GMT+03:00 Rowan Collins >: The other alternative is to go to the other extreme, and have the extensions scoped to a particular file, more like Perl pragmas. Dare I say we could use the de

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-17 16:35 GMT+03:00 Rowan Collins : > I think "should" is OK for extensions, but not for userland hooks. There > needs to be a very firm definition of the right and wrong way to implement > these hooks, which can at the very least warn the user when they are > slipping into undefined behavi

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

Alexander Lisachenko wrote on 17/02/2015 12:41: 2015-02-17 15:29 GMT+03:00 Benjamin Eberlei : Well not really, depending on the path towards a require a parser extension is already registered or not. Or you have conditional registration of extensions. So you could load a file with an extension

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-17 15:48 GMT+03:00 Leigh : > So: Internal Parse AST -> Extension Parse AST -> Generate OpCodes -> > OpCache? > Yes, it's correct flow. 2015-02-17 15:48 GMT+03:00 Leigh : > So if the parser extension does anything dynamic, then OpCache will > have the wrong version cached. > This restri

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On 17 February 2015 at 12:22, Alexander Lisachenko wrote: > Yes, parser extensions will be called for all require/include/evals after > registration. This part is transparent for opcache, because opcache just > stores an opcodes for the file. AST is parsed only once for each file, then > hooks can

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-17 15:29 GMT+03:00 Benjamin Eberlei : > Well not really, depending on the path towards a require a parser > extension is already registered or not. Or you have conditional > registration of extensions. So you could load a file with an extension > registered, but it still serves the old opc

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Tue, Feb 17, 2015 at 1:22 PM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > > 2015-02-17 15:09 GMT+03:00 Benjamin Eberlei : > >> The visitor API is the essential part here and you left it out. > > > Yes, I decided not to put Visitor in the RFC (this was added as open > question to th

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-17 15:09 GMT+03:00 Benjamin Eberlei : > The visitor API is the essential part here and you left it out. Yes, I decided not to put Visitor in the RFC (this was added as open question to the RFC). But this part can be discussed in the case of general acceptance. 2015-02-17 15:09 GMT+03:0

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

2015-02-17 15:06 GMT+03:00 Leigh : > Does the AST provided by the internal parser provide all of the > information required to be able to turn it back into source code? > > Yes, this should be possible, via zend_emit_op*() and zend_compile*() 2015-02-17 15:06 GMT+03:00 Leigh : > Re: Extending t

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On Tue, Feb 17, 2015 at 12:46 PM, Alexander Lisachenko < lisachenko...@gmail.com> wrote: > Hello, internals! > > I want to introduce a RFC for providing a userland API for accessing an > Abstract Syntax Tree of the source code and to provide userland parser > hooks for source code modification: >

Re: [PHP-DEV] [RFC][Discussion] Parser extension API

On 17 February 2015 at 11:46, Alexander Lisachenko wrote: > Hello, internals! > > I want to introduce a RFC for providing a userland API for accessing an > Abstract Syntax Tree of the source code and to provide userland parser > hooks for source code modification: > https://wiki.php.net/rfc/parser

[PHP-DEV] [RFC][Discussion] Parser extension API

Hello, internals! I want to introduce a RFC for providing a userland API for accessing an Abstract Syntax Tree of the source code and to provide userland parser hooks for source code modification: https://wiki.php.net/rfc/parser-extension-api Thanks!