Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Rowan Tommins [IMSoP]
On 21 July 2024 19:33:11 BST, Juliette Reinders Folmer wrote: >The crux - to me - is that it is an undocumented breaking change, which by >definition is a bug. There are two parts of this which are bugs, in my opinion: - That it wasn't documented, e.g. with a line in UPGRADING listing the a

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Juliette Reinders Folmer
On 21-7-2024 17:11, Rowan Tommins [IMSoP] wrote: Oddly, the crux of this debate seems to be less that all options have major impact, and more that none of them do. If the implementation had caused a serious security or performance regression, I don't think we'd have any hesitation in backing it

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Rowan Tommins [IMSoP]
On 21 July 2024 17:45:27 BST, Deleu wrote: >I don't agree with the sentiment of "given the 2 options >available, we prefer the option where we know we are negatively impacting >community members consuming token_get_all() in favor of protecting >imaginary PHP users that would have been adventuro

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Deleu
On Sun, Jul 21, 2024 at 12:14 PM Rowan Tommins [IMSoP] wrote: > > Oddly, the crux of this debate seems to be less that all options have > major impact, and more that none of them do. > My only disagreement with your statement (and Tim's messages) is that one option *MAY* impact some imaginary PH

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Juliette Reinders Folmer
On 21-7-2024 13:28, Ilija Tovilo wrote: Hi Ilja, I agree that it's not very likely that many people started placing comments between yield and from, but certainly not impossible. The main problem with reverting this change is that such code will completely stop working. Patch updates are routi

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Rowan Tommins [IMSoP]
On 21 July 2024 12:25:27 BST, "Tim Düsterhus" wrote: >The change has also been explicitly acked by Gina and Christoph before Ilija >committed it and Bob also participated in the PR without raising concerns, so >it's also not an unapproved change. The fact that none of them anticipated >this sid

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Tim Düsterhus
Hi On 7/21/24 00:16, Larry Garfield wrote: Yes, any syntax change means tools need to adapt, but that doesn't mean tokenization can change randomly and accidentally. Syntax changes require an RFC. If an RFC passes that necessitates SA tools update, so be it. (That happens almost every vers

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Ilija Tovilo
Hi Juliette On Sat, Jul 20, 2024 at 6:40 PM Juliette Reinders Folmer wrote: > > On 20-7-2024 18:04, Tim Düsterhus wrote: > > As I've said: I agree that the current situation is unfortunate. But the > correct solution is not "disallow comments", but "split the T_YIELD_FROM into > T_YIELD T_WHITE

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-21 Thread Tim Düsterhus
Hi On 7/20/24 20:14, Deleu wrote: Is there any evidence that PHP users are relying on code that: - Was released just 7 months ago - Was not documented - Nobody knew about it until very recently That is the wrong question to ask (see at the bottom). And furthermore, why should undocumented,

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Larry Garfield
On Sat, Jul 20, 2024, at 11:51 AM, Tim Düsterhus wrote: > Hi > > On 7/20/24 18:40, Juliette Reinders Folmer wrote: >> Tim, you're making my point for me. This is *exactly* why the current >> change should be reverted. > > I am not sure how you read "PHP users have no idea what a token is" as > an

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Deleu
On Sat, Jul 20, 2024 at 1:53 PM Tim Düsterhus wrote: > Hi > > On 7/20/24 18:40, Juliette Reinders Folmer wrote: > > Tim, you're making my point for me. This is *exactly* why the current > > change should be reverted. > > I am not sure how you read "PHP users have no idea what a token is" as > an

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Tim Düsterhus
Hi On 7/20/24 18:40, Juliette Reinders Folmer wrote: Tim, you're making my point for me. This is *exactly* why the current change should be reverted. I am not sure how you read "PHP users have no idea what a token is" as an argument in favor of reverting the change, because reverting the cha

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Juliette Reinders Folmer
On 20-7-2024 18:04, Tim Düsterhus wrote: PHP users have no idea what a token is internally. I'm looking at this from a PHP user perspective. It looks like two keywords, it walks like two keywords and it quacks like two keywords. I find it reasonable for users to consider this as two keywords an

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Tim Düsterhus
Hi On 7/20/24 17:42, Juliette Reinders Folmer wrote: I already mentioned cast tokens before. Whitespace is perfectly acceptable within the parentheses of these. Comments are not: https://3v4l.org/A6Sgj and https://3v4l.org/nE9H8 Now you may argue that cast tokens "feel like" a single operator,

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Matthew Weier O'Phinney
On Sat, Jul 20, 2024, 9:31 AM Tim Düsterhus wrote: > Hi > > On 7/19/24 00:51, Christoph M. Becker wrote: > > And frankly, how much code would be affected? I mean, does anybody > > actually put a comment between `yield` and `from`? Is there a case > > where this may make sense? "Because we can"

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Tim Düsterhus
Hi On 7/19/24 07:22, Juliette Reinders Folmer wrote: More than anything, I find it concerning that this change sets a precedent for tokens to include comments. Just as an example: what does this mean for the PHP 8.0 nullsafe object operator ? Should we now suddenly allow that to be written as `

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Rob Landers
On Sat, Jul 20, 2024, at 16:35, Marco Aurélio Deleu wrote: > > > On 20 Jul 2024, at 11:30, Tim Düsterhus wrote: > > > > Hi > > > >> On 7/19/24 00:51, Christoph M. Becker wrote: > >> And frankly, how much code would be affected? I mean, does anybody > >> actually put a comment between `yield

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Marco Aurélio Deleu
> On 20 Jul 2024, at 11:30, Tim Düsterhus wrote: > > Hi > >> On 7/19/24 00:51, Christoph M. Becker wrote: >> And frankly, how much code would be affected? I mean, does anybody >> actually put a comment between `yield` and `from`? Is there a case >> where this may make sense? "Because we ca

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-20 Thread Tim Düsterhus
Hi On 7/19/24 00:51, Christoph M. Becker wrote: And frankly, how much code would be affected? I mean, does anybody actually put a comment between `yield` and `from`? Is there a case where this may make sense? "Because we can" isn't a strong argument, in my opinion. I don't really follow thi

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-18 Thread Juliette Reinders Folmer
On 19-7-2024 1:09, Bob Weinand wrote: Hey Christoph, Am 19.07.2024 um 00:51 schrieb Christoph M. Becker : Hi Bob! On 18.07.2024 at 15:41, Bob Weinand wrote: Moreover, it can - at least - be worked around in tooling by special casing the T_YIELD_FROM token and extracting the comment from th

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-18 Thread Bob Weinand
Hey Christoph, Am 19.07.2024 um 00:51 schrieb Christoph M. Becker : Hi Bob! On 18.07.2024 at 15:41, Bob Weinand wrote: Moreover, it can - at least - be worked around in tooling by special casing the T_YIELD_FROM token and extracting the comment from the raw parsed string: var_dump(token_get_

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-18 Thread Christoph M. Becker
Hi Tim! On 18.07.2024 at 21:05, Tim Düsterhus wrote: > On 7/18/24 19:48, Marco Aurélio Deleu wrote: > >> Forcing all tooling that uses token_get_all() to handle this >> unintentional change seems to generate more unnecessary and real >> busywork for something only theoretical possible to break. >

Re: [PHP-DEV] Request for opinions: bug vs feature - change intokenization of yield from

2024-07-18 Thread Christoph M. Becker
Hi Bob! On 18.07.2024 at 15:41, Bob Weinand wrote: > Moreover, it can - at least - be worked around in tooling by special casing > the T_YIELD_FROM token and extracting the comment from the raw parsed string: > > var_dump(token_get_all(' > will contain: > > [1]=> array(3) { [0]=> int(270) [1]=>