Re: our checks for read-only queries are not great

2020-03-05 Thread Bruce Momjian
On Mon, Jan 27, 2020 at 02:26:42PM -0500, Stephen Frost wrote: > Greetings, > > * Bruce Momjian (br...@momjian.us) wrote: > > On Mon, Jan 13, 2020 at 01:56:30PM -0500, Stephen Frost wrote: > > > > I think that having ALTER SYSTEM commands in pg_dumpall output > > > > would be a problem. It would

Re: our checks for read-only queries are not great

2020-01-27 Thread Stephen Frost
Greetings, * Bruce Momjian (br...@momjian.us) wrote: > On Mon, Jan 13, 2020 at 01:56:30PM -0500, Stephen Frost wrote: > > > I think that having ALTER SYSTEM commands in pg_dumpall output > > > would be a problem. It would cause all kinds of problems whenever > > > parameters change. Thinking of

Re: our checks for read-only queries are not great

2020-01-18 Thread Bruce Momjian
On Mon, Jan 13, 2020 at 01:56:30PM -0500, Stephen Frost wrote: > > I think that having ALTER SYSTEM commands in pg_dumpall output > > would be a problem. It would cause all kinds of problems whenever > > parameters change. Thinking of the transition "checkpoint_segments" > > -> "max_wal_size", yo

Re: our checks for read-only queries are not great

2020-01-16 Thread Robert Haas
On Thu, Jan 16, 2020 at 12:22 PM Stephen Frost wrote: > I think I agree with you regarding the original intent, though even > there, as discussed elsewhere, it seems like there's perhaps either a > bug or a disagreement about the specifics of what that means when it > relates to committing a 2-pha

Re: our checks for read-only queries are not great

2020-01-16 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Tue, Jan 14, 2020 at 1:46 PM Stephen Frost wrote: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > > Robert Haas writes: > > > > Speaking of sensible progress, I think we've drifted off on a tangent > > > > here about ALTER SYSTEM. > > >

Re: our checks for read-only queries are not great

2020-01-16 Thread Robert Haas
On Tue, Jan 14, 2020 at 1:46 PM Stephen Frost wrote: > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > Robert Haas writes: > > > Speaking of sensible progress, I think we've drifted off on a tangent > > > here about ALTER SYSTEM. > > > > Agreed, that's not terribly relevant for the proposed patch. > >

Re: our checks for read-only queries are not great

2020-01-15 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 15, 2020 at 10:25 AM Peter Eisentraut > wrote: > > Well, if the transaction was declared read-only, then committing it > > (directly or 2PC) shouldn't change anything. This appears to be a > > circular argument. > > I don't t

Re: our checks for read-only queries are not great

2020-01-15 Thread Robert Haas
On Wed, Jan 15, 2020 at 10:25 AM Peter Eisentraut wrote: > Well, if the transaction was declared read-only, then committing it > (directly or 2PC) shouldn't change anything. This appears to be a > circular argument. I don't think it's a circular argument. Suppose that someone decrees that, as of

Re: our checks for read-only queries are not great

2020-01-15 Thread Peter Eisentraut
On 2020-01-13 20:14, Robert Haas wrote: On Mon, Jan 13, 2020 at 5:57 AM Peter Eisentraut wrote: On 2020-01-10 14:41, Robert Haas wrote: This rule very nearly matches the current behavior: it explains why temp table operations are allowed, and why ALTER SYSTEM is allowed, and why REINDEX etc. a

Re: our checks for read-only queries are not great

2020-01-14 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > Speaking of sensible progress, I think we've drifted off on a tangent > > here about ALTER SYSTEM. > > Agreed, that's not terribly relevant for the proposed patch. I agree that the proposed patch seems alright by itself

Re: our checks for read-only queries are not great

2020-01-14 Thread Tom Lane
Robert Haas writes: > Speaking of sensible progress, I think we've drifted off on a tangent > here about ALTER SYSTEM. Agreed, that's not terribly relevant for the proposed patch. regards, tom lane

Re: our checks for read-only queries are not great

2020-01-14 Thread Robert Haas
On Mon, Jan 13, 2020 at 3:00 PM Stephen Frost wrote: > I've never heard that and I don't agree with it being a justification > for blocking sensible progress. Speaking of sensible progress, I think we've drifted off on a tangent here about ALTER SYSTEM. As I understand it, nobody's opposed to the

Re: our checks for read-only queries are not great

2020-01-13 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Mon, 2020-01-13 at 13:56 -0500, Stephen Frost wrote: > > > I think that having ALTER SYSTEM commands in pg_dumpall output > > > would be a problem. It would cause all kinds of problems whenever > > > parameters change. Thinking of

Re: our checks for read-only queries are not great

2020-01-13 Thread Laurenz Albe
On Mon, 2020-01-13 at 13:56 -0500, Stephen Frost wrote: > > I think that having ALTER SYSTEM commands in pg_dumpall output > > would be a problem. It would cause all kinds of problems whenever > > parameters change. Thinking of the transition "checkpoint_segments" > > -> "max_wal_size", you'd hav

Re: our checks for read-only queries are not great

2020-01-13 Thread Robert Haas
On Mon, Jan 13, 2020 at 5:57 AM Peter Eisentraut wrote: > On 2020-01-10 14:41, Robert Haas wrote: > > This rule very nearly matches the current behavior: it explains why > > temp table operations are allowed, and why ALTER SYSTEM is allowed, > > and why REINDEX etc. are allowed. However, there's a

Re: our checks for read-only queries are not great

2020-01-13 Thread Stephen Frost
Greetings, * Laurenz Albe (laurenz.a...@cybertec.at) wrote: > On Fri, 2020-01-10 at 09:29 -0500, Tom Lane wrote: > > > ALTER SYSTEM is read only in my mind. > > > > I'm still having trouble with this conclusion. I think it can only > > be justified by a very narrow reading of "reflected in pg_du

Re: our checks for read-only queries are not great

2020-01-13 Thread Peter Eisentraut
On 2020-01-10 14:41, Robert Haas wrote: This rule very nearly matches the current behavior: it explains why temp table operations are allowed, and why ALTER SYSTEM is allowed, and why REINDEX etc. are allowed. However, there's a notable exception: PREPARE, COMMIT PREPARED, and ROLLBACK PREPARED a

Re: our checks for read-only queries are not great

2020-01-12 Thread Tom Lane
Laurenz Albe writes: > Perhaps it would be good to consider this question: > Do we call something "read-only" if it changes nothing, or do we call it > "read-only" if it is allowed on a streaming replication standby? > The first would be more correct, but the second may be more convenient. Yeah,

Re: our checks for read-only queries are not great

2020-01-12 Thread Laurenz Albe
On Fri, 2020-01-10 at 09:29 -0500, Tom Lane wrote: > > ALTER SYSTEM is read only in my mind. > > I'm still having trouble with this conclusion. I think it can only > be justified by a very narrow reading of "reflected in pg_dump" > that relies on the specific factorization we've chosen for upgrad

Re: our checks for read-only queries are not great

2020-01-10 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Fri, Jan 10, 2020 at 9:30 AM Tom Lane wrote: > > If somebody comes up with a patch > > that causes "pg_dumpall -g" to include ALTER SYSTEM SET commands for > > whatever is in postgresql.auto.conf (not an unreasonable idea BTW), > > will

Re: our checks for read-only queries are not great

2020-01-10 Thread Robert Haas
On Fri, Jan 10, 2020 at 9:30 AM Tom Lane wrote: > If somebody comes up with a patch > that causes "pg_dumpall -g" to include ALTER SYSTEM SET commands for > whatever is in postgresql.auto.conf (not an unreasonable idea BTW), > will you then decide that ALTER SYSTEM SET is no longer read-only? > Or

Re: our checks for read-only queries are not great

2020-01-10 Thread Tom Lane
Peter Eisentraut writes: > I don't really remember, but that was basically the opinion I had > arrived at as I was reading through this current thread. Roughly > speaking, anything that changes the database state (data or schema) in a > way that would be reflected in a pg_dump output is not re

Re: our checks for read-only queries are not great

2020-01-10 Thread Robert Haas
On Fri, Jan 10, 2020 at 7:23 AM Peter Eisentraut wrote: > I don't really remember, but that was basically the opinion I had > arrived at as I was reading through this current thread. Roughly > speaking, anything that changes the database state (data or schema) in a > way that would be reflected i

Re: our checks for read-only queries are not great

2020-01-10 Thread Peter Eisentraut
On 2020-01-09 21:52, Tom Lane wrote: Peter might remember more clearly, but I have a feeling that we concluded that the intent of the spec was for read-only-ness to disallow globally-visible changes in the visible database contents. I don't really remember, but that was basically the opinion I

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > On Thu, Jan 9, 2020 at 3:07 PM Tom Lane wrote: >> You could argue about exactly how to extend that to non-spec >> utility commands, but for the most part allowing them seems >> to make sense if DML is allowed. > But I think we allow them on all tables, not just temp tables,

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:37 PM Robert Haas wrote: > > You could argue about exactly how to extend that to non-spec > > utility commands, but for the most part allowing them seems > > to make sense if DML is allowed. > > But I think we allow them on all tables, not just temp tables, so I > don't th

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 3:07 PM Tom Lane wrote: > Robert Haas writes: > > Maybe the SQL standard has something to say about this? > > [ pokes around ... ] Yeah, it does, and I'd say it's pretty clearly > in agreement with what Peter did, so far as DML ops go. For instance, > this bit from SQL99'

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > Maybe the SQL standard has something to say about this? [ pokes around ... ] Yeah, it does, and I'd say it's pretty clearly in agreement with what Peter did, so far as DML ops go. For instance, this bit from SQL99's description of DELETE: 1) If the access mode of

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Thu, Jan 9, 2020 at 2:24 PM Tom Lane wrote: > Robert Haas writes: > > I'd be really interested to hear if anyone knows the history behind > > allowing CLUSTER, REINDEX, VACUUM, and some operations on temp tables. > > It seems to have been that way for a long time. I wonder if it was a > > deli

Re: our checks for read-only queries are not great

2020-01-09 Thread Tom Lane
Robert Haas writes: > I'd be really interested to hear if anyone knows the history behind > allowing CLUSTER, REINDEX, VACUUM, and some operations on temp tables. > It seems to have been that way for a long time. I wonder if it was a > deliberate choice or something that just happened semi-acciden

Re: our checks for read-only queries are not great

2020-01-09 Thread Robert Haas
On Wed, Jan 8, 2020 at 5:57 PM Stephen Frost wrote: > Yeah, I don't like the WEAKLY_READ_ONLY idea either- explicitly having > COMMAND_OK_IN_X seems cleaner. Done that way. v2 attached. > Would we be able to make a rule of "can't change on-disk stuff, except > for things in temporary schemas" an

Re: our checks for read-only queries are not great

2020-01-08 Thread Stephen Frost
Greetings, (I'm also overall in favor of the direction this is going, so general +1 from me, and I took a quick look through the patch and didn't particularly see anything I didn't like besides what's commented on below.) * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Jan 8, 2020 at 3:26

Re: our checks for read-only queries are not great

2020-01-08 Thread Robert Haas
On Wed, Jan 8, 2020 at 3:26 PM Tom Lane wrote: > Robert Haas writes: > > On Wed, Jan 8, 2020 at 2:57 PM Tom Lane wrote: > >> * I find COMMAND_IS_WEAKLY_READ_ONLY to be a more confusing concept > >> than what it replaces. The test for LockStmt is an example --- the > >> comment talks about restr

Re: our checks for read-only queries are not great

2020-01-08 Thread Tom Lane
Robert Haas writes: > On Wed, Jan 8, 2020 at 2:57 PM Tom Lane wrote: >> * I find COMMAND_IS_WEAKLY_READ_ONLY to be a more confusing concept >> than what it replaces. The test for LockStmt is an example --- the >> comment talks about restricting locks during recovery, which is fine and >> underst

Re: our checks for read-only queries are not great

2020-01-08 Thread Robert Haas
On Wed, Jan 8, 2020 at 2:57 PM Tom Lane wrote: > Hmm. I like the idea of deciding this in one place and insisting that > that one place have a switch case for every statement type. That'll > address the root issue that people fail to think about this when adding > new statements. Right. Assumin

Re: our checks for read-only queries are not great

2020-01-08 Thread Tom Lane
Robert Haas writes: > I spent some time studying the question of how we classify queries as > either read-only or not, and our various definitions of read-only, and > found some bugs. ... > However, I think that all of them can be tracked back to a more > fundamental underlying cause, which is tha

our checks for read-only queries are not great

2020-01-08 Thread Robert Haas
I spent some time studying the question of how we classify queries as either read-only or not, and our various definitions of read-only, and found some bugs. Specifically: 1. check_xact_readonly() prohibits most kinds of DDL in read-only transactions, but a bunch of recently-added commands were no