[PHP-DEV] TOKEN_AS_OBJECT for token_get_all()

[Nikita Popov]

Hi internals,

I'd like to add a new TOKEN_AS_OBJECT flag to token_get_all(), which
returns an array of PhpToken objects, rather than the mix of plain strings
and arrays we currently have. The PhpToken class is defined as:

class PhpToken {
    public $type;
    public $text;
    public $line;
}

This has been previously suggested and implemented by Rouven Weßling [1],
I've just ported this feature to master and optimized the implementation
[2].

Apart from being a nicer interface than the current format, an additional
advantage is that the  TOKEN_AS_OBJECT mode uses about 1.8x less memory and
is 1.4x times faster.

Are there any objections to this change?

Regards,
Nikita

[1]: https://github.com/php/php-src/pull/1727
[2]: https://github.com/php/php-src/pull/2430