On Thu, 23 Mar 2017 18:16:31 +0100, Nikita Popov <nikita....@gmail.com>
> I'd like to add a new TOKEN_AS_OBJECT flag to token_get_all(), which
> returns an array of PhpToken objects, rather than the mix of plain strings
> and arrays we currently have. The PhpToken class is defined as:
>
> class PhpToken {
> public $type;
> public $text;
> public $line;
> }
>
> This has been previously suggested and implemented by Rouven Weßling [1],
> I've just ported this feature to master and optimized the implementation
> [2].
>
>
Yeah, IIRC the more recent discussion ended at "Oh, and this! And
this! And this!" which eventually went nowhere (largely my ADD). IMO
there's no harm in adding this, and the class format seems entirely
reasonable.
If I may bikeshed a TINY bit, I'd ask that all tokens return as
objects, rather than char|PhpToken similar to the current char|array
format we have. (Maybe that's in the PR, I haven't looked at either)
On Thu, Mar 23, 2017 at 5:25 PM, Jan Tvrdík <j....@centrum.cz> wrote:
> Regarding memory - would it be possible to return iterator instead of array?
>
That has some hairy edges to it since the lexer isn't actually
reentrant within a given thread. Image the following:
foreach (token_get_all($code, TOKEN_AS_ITERATOR) as $token) {
$moretokens = token_get_all($morecode, $whateverflags); // <--- Here
be broken dreams and crying unicorns
}
Worse still if you have parallel iterators.
That's probably fixable, but it's a much heavier refactor and one that
should probably have an RFC.
-Sara
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
