On Fri, Sep 9, 2011 at 9:15 AM, Nikita Popov <nikita....@googlemail.com> wrote:
> In Bug #54089 [1] a patch was applied that cuts of token_get_all()
> output after a T_HALT_COMPILER token. This was done because otherwise
> PHP would keep on lexing after that and would generate errors because
> of binary data (which is not valid PHP, mostly.)
>
> The problem with the patch is, that there are some tokens after
> T_HALT_COMPILER that are of interest, namely the '(' ')' ';'. After
> the patch it is impossible to get those tokens, without either
> relexing the code after T_HALT_COMPILER (that way you get the binary
> data problem back, just with much more complex code) or writing a
> regular expression to match it (which is really hard, as there may be
> any token dropped by the PHP parser in there, i.e. whitespace,
> comments, PHP tags).
>
> This issue was pointed out by the creator of the bug report, but was
> those comments were ignored for some reason.
>
> I would like this patch to be reverted on the 5.4 and trunk branches.
> I assume it's too late to revert it on 5.3, as it has been there for
> some time already. It is just counterproductive. (Alternatively one
> could fix token_get_all to return the (); tokens after
> __halt_compiler, too, but that would be hard, probably.)I think that it wouldn't be too hard. >From a quick glance on the code, we should introduce a new local variable, set that to true where we break now ( http://svn.php.net/viewvc/php/php-src/trunk/ext/tokenizer/tokenizer.c?view=markup#l155 ) and don't break there but for the next ';'. another maybe less confusing solution would be to explicitly add '(', ')' and ';' to the result in the T_HALT_COMPILER condition before breking out of the loop. I will create a patch for this afternoon. or could there be other important tokens after the __halt_compiler() which should be present in the token_get_all() result? -- Ferenc Kovács @Tyr43l - http://tyrael.hu -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
