Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Tom Lane
Greg Stark writes: > On Sun, Apr 19, 2009 at 5:42 PM, Tom Lane wrote: >> * Nested slash-star comments are now handled properly. > as opposed to? They nest, as required by the SQL spec and implemented by our core lexer. plpgsql didn't use to get this right. regards, tom

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Grzegorz Jaskiewicz
On 19 Apr 2009, at 18:28, Greg Stark wrote: On Sun, Apr 19, 2009 at 6:24 PM, Grzegorz Jaskiewicz wrote: Will it also mean, that queries are going to be analyzed deeper ? Ie, afaik I am able now to create plpgsql function, that tries to run query accessing non existent table, or columns. Or

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Greg Stark
On Sun, Apr 19, 2009 at 6:24 PM, Grzegorz Jaskiewicz wrote: > Will it also mean, that queries are going to be analyzed deeper ? > Ie, afaik I am able now to create plpgsql function, that tries to run query > accessing non existent table, or columns. > Or, if I rename column/table/relation now, vie

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Grzegorz Jaskiewicz
On 19 Apr 2009, at 17:42, Tom Lane wrote: The attached proposed patch rips out plpgsql's handling of comments and string literals, and puts in scanner rules that are extracted from the core lexer (but simplified in a few places where we don't need all the complexity). The net user-visible e

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Greg Stark
On Sun, Apr 19, 2009 at 5:42 PM, Tom Lane wrote: > * Nested slash-star comments are now handled properly. as opposed to? -- greg -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-19 Thread Tom Lane
I wrote: > So I think we are down to a choice of doing nothing for 8.4, or teaching > the existing plpgsql lexer about standard_conforming_strings. Assuming > the current proposal for U& literals holds up, it should not be > necessary for plpgsql to know about those explicitly as long as it obeys

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-18 Thread Stephen Cook
++ David Fetter wrote: On Fri, Apr 17, 2009 at 01:01:39PM -0400, Alvaro Herrera wrote: David Fetter wrote: Speaking of standard_conforming_strings, I know it's late, but if we make it a requirement now, a lot of problems just go away. Yes, it's inconvenient, but we're making lots of big chan

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread Tom Lane
David Fetter writes: >> It's hardly a "show stopping bug", considering it's been there since >> standard_conforming_strings was invented. > A known sploit would be a show-stopper. We're not turning on standard_conforming_strings right now. We are *certainly* not forcing it on without recourse i

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread David Fetter
On Fri, Apr 17, 2009 at 02:07:55PM -0400, Tom Lane wrote: > David Fetter writes: > > Depends. If we've found show-stopping bugs, as it appears we may > > have done, in not requiring standards_conforming_strings, we can't > > just pull a MySQL and ship anyhow. > > It's hardly a "show stopping bug

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread David Fetter
On Fri, Apr 17, 2009 at 02:03:45PM -0400, Andrew Dunstan wrote: > Alvaro Herrera wrote: >> David Fetter wrote: >> >>> Speaking of standard_conforming_strings, I know it's late, but if >>> we make it a requirement now, a lot of problems just go away. >>> Yes, it's inconvenient, but we're making l

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread Tom Lane
David Fetter writes: > Depends. If we've found show-stopping bugs, as it appears we may have > done, in not requiring standards_conforming_strings, we can't just > pull a MySQL and ship anyhow. It's hardly a "show stopping bug", considering it's been there since standard_conforming_strings was i

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread Andrew Dunstan
Alvaro Herrera wrote: David Fetter wrote: Speaking of standard_conforming_strings, I know it's late, but if we make it a requirement now, a lot of problems just go away. Yes, it's inconvenient, but we're making lots of big changes, so one more shouldn't halt adoption. 16 days too l

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread David Fetter
On Fri, Apr 17, 2009 at 01:01:39PM -0400, Alvaro Herrera wrote: > David Fetter wrote: > > > Speaking of standard_conforming_strings, I know it's late, but if > > we make it a requirement now, a lot of problems just go away. > > Yes, it's inconvenient, but we're making lots of big changes, so > > o

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread Alvaro Herrera
David Fetter wrote: > Speaking of standard_conforming_strings, I know it's late, but if we > make it a requirement now, a lot of problems just go away. Yes, it's > inconvenient, but we're making lots of big changes, so one more > shouldn't halt adoption. 16 days too late ... -- Alvaro Herrera

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread David Fetter
On Fri, Apr 17, 2009 at 12:12:12PM -0400, Tom Lane wrote: > I wrote: > > I had earlier speculated semi-facetiously about ripping out the > > plpgsql lexer altogether, but the more I think about it the less > > silly the idea looks. > > This little project crashed and burned upon remembering that p

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-17 Thread Tom Lane
I wrote: > I had earlier speculated semi-facetiously about ripping out the plpgsql > lexer altogether, but the more I think about it the less silly the idea > looks. This little project crashed and burned upon remembering that plpgsql invokes raw_parser() to syntax-check each chunk of SQL that it

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Joshua D. Drake
On Wed, 2009-04-15 at 11:45 -0400, Tom Lane wrote: > Andrew Dunstan writes: > > We support back branches for a long time for a reason. > > I think that's really the bottom line here. If we insist on new major > releases always being bug-compatible with prior releases, our ability to > improve th

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Tom Lane
Andrew Dunstan writes: > We support back branches for a long time for a reason. I think that's really the bottom line here. If we insist on new major releases always being bug-compatible with prior releases, our ability to improve the software will go to zero. The solution we have opted for ins

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Robert Haas
On Wed, Apr 15, 2009 at 5:56 AM, Greg Stark wrote: > On Wed, Apr 15, 2009 at 10:12 AM, Simon Riggs wrote: >> >> How do you know which is the offending function? If we force a full >> application retest we put in place a significant barrier to upgrade. >> That isn't useful for us as developers, no

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Robert Haas
On Wed, Apr 15, 2009 at 5:56 AM, Greg Stark wrote: > On Wed, Apr 15, 2009 at 10:12 AM, Simon Riggs wrote: >> >> How do you know which is the offending function? If we force a full >> application retest we put in place a significant barrier to upgrade. >> That isn't useful for us as developers, no

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Andrew Dunstan
Simon Riggs wrote: How do you know which is the offending function? If we force a full application retest we put in place a significant barrier to upgrade. That isn't useful for us as developers, nor is it useful for users. We support back branches for a long time for a reason. Nobody in

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Heikki Linnakangas
Simon Riggs wrote: On Tue, 2009-04-14 at 18:29 -0400, Tom Lane wrote: Simon Riggs writes: Potentially changing the behaviour of thousands of functions just to fix a rare bug will not endear us to our users. The bug may be something that people are relying on in some subtle way, ugly as that so

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Greg Stark
On Wed, Apr 15, 2009 at 11:33 AM, Simon Riggs wrote: > >> This is a fundamental conflict, not one that has a single simple answer. >> >> However this seems like a strange place to pick your battle. > > I think you are right that you perceive a fundamental conflict and most > things I say become ba

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Simon Riggs
On Wed, 2009-04-15 at 10:56 +0100, Greg Stark wrote: > On Wed, Apr 15, 2009 at 10:12 AM, Simon Riggs wrote: > > > > How do you know which is the offending function? If we force a full > > application retest we put in place a significant barrier to upgrade. > > That isn't useful for us as develope

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Greg Stark
On Wed, Apr 15, 2009 at 10:12 AM, Simon Riggs wrote: > > How do you know which is the offending function? If we force a full > application retest we put in place a significant barrier to upgrade. > That isn't useful for us as developers, nor is it useful for users. This is a fundamental conflict,

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Heikki Linnakangas
Simon Riggs wrote: On Wed, 2009-04-15 at 11:36 +0300, Heikki Linnakangas wrote: Extract the source of the offending plpgsql function using e.g pg_dump, modify it so that it works again, and restore the function. There's your workaround. Forcing manual re-editing of an unknown number of lines

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-15 Thread Simon Riggs
On Wed, 2009-04-15 at 11:36 +0300, Heikki Linnakangas wrote: > Extract the source of the offending plpgsql function using e.g > pg_dump, modify it so that it works again, and restore the function. > There's your workaround. Forcing manual re-editing of an unknown number of lines of code is not a

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Simon Riggs
On Tue, 2009-04-14 at 18:29 -0400, Tom Lane wrote: > Simon Riggs writes: > > On Tue, 2009-04-14 at 16:37 -0400, Tom Lane wrote: > >> Comments, objections, better ideas? > > > Please, if you do this, make it optional. > > I don't think making the plpgsql lexer pluggable is realistic. Doesn't so

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Stephen Frost
* Bruce Momjian (br...@momjian.us) wrote: > Well, this bug has existed long before 8.4 so we could just leave it for > 8.5, and it is not like we have had tons of complaints; the only > complaint I saw was one from March, 2008. I think it's a good thing to do in general. I'm also concerned about

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Tom Lane
Bruce Momjian writes: > Well, this bug has existed long before 8.4 so we could just leave it for > 8.5, and it is not like we have had tons of complaints; the only > complaint I saw was one from March, 2008. We had one last week, which is what prompted me to start looking at the plpgsql lexer si

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Bruce Momjian
Tom Lane wrote: > Simon Riggs writes: > > On Tue, 2009-04-14 at 16:37 -0400, Tom Lane wrote: > >> Comments, objections, better ideas? > > > Please, if you do this, make it optional. > > I don't think making the plpgsql lexer pluggable is realistic. > > > Potentially changing the behaviour of th

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Tom Lane
Simon Riggs writes: > On Tue, 2009-04-14 at 16:37 -0400, Tom Lane wrote: >> Comments, objections, better ideas? > Please, if you do this, make it optional. I don't think making the plpgsql lexer pluggable is realistic. > Potentially changing the behaviour of thousands of functions just to fix >

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Tom Lane
Andrew Dunstan writes: > Robert Haas wrote: >> All this sounds good. As for how to handle := and .., I think making >> them lex the same way in PL/pgsql and core SQL would be a good thing. > They don't have any significance in core SQL. What would we do with the > lexeme? It would just fail --

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Andrew Dunstan
Robert Haas wrote: All this sounds good. As for how to handle := and .., I think making them lex the same way in PL/pgsql and core SQL would be a good thing. They don't have any significance in core SQL. What would we do with the lexeme? ISTR we've used some hacks in the past to spli

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Simon Riggs
On Tue, 2009-04-14 at 16:37 -0400, Tom Lane wrote: > Comments, objections, better ideas? Please, if you do this, make it optional. Potentially changing the behaviour of thousands of functions just to fix a rare bug will not endear us to our users. The bug may be something that people are relying

Re: [HACKERS] Replacing plpgsql's lexer

2009-04-14 Thread Robert Haas
On Tue, Apr 14, 2009 at 4:37 PM, Tom Lane wrote: > Whichever way the current discussion about Unicode literals turns out, > it's clear that plpgsql is not up to speed on matching the core lexer's > behavior --- it's wrong anyway with respect to > standard_conforming_strings. > > I had earlier spec