Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Simon Riggs
On 12 November 2012 16:22, Robert Haas wrote: > But I guess that raises the question - should COPY (FREEZE) silently > ignore the option for not-new relfilenodes, or should it error out? > Simon proposed the former, but I'm wondering if the latter would be > better. It's got some complex pre-con

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Robert Haas
On Mon, Nov 12, 2012 at 11:20 AM, Tom Lane wrote: > Robert Haas writes: >> On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: >>> So what we're talking about here is a new mode for COPY, that when >>> requested will pre-freeze tuples when loading into a newly >>> created/truncated table. If the

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Tom Lane
Robert Haas writes: > On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: >> So what we're talking about here is a new mode for COPY, that when >> requested will pre-freeze tuples when loading into a newly >> created/truncated table. If the table isn't newly created/truncated >> then we'll just ig

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-12 Thread Robert Haas
On Fri, Nov 9, 2012 at 3:31 PM, Simon Riggs wrote: > So what we're talking about here is a new mode for COPY, that when > requested will pre-freeze tuples when loading into a newly > created/truncated table. If the table isn't newly created/truncated > then we'll just ignore it and continue. I see

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 16:27, Tom Lane wrote: > Simon Riggs writes: >> On 9 November 2012 15:34, Kevin Grittner wrote: >>> If we're not talking about making conflicts with other transactions >>> behave just the same as an unqualified DELETE from a user >>> perspective, I'm not sure what the goal is,

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Tom Lane
Robert Haas writes: > Just having an option to preload frozen tuples dodges all of these > issues by throwing our hands up in the air, but it does have the > advantage of being more general. Even if we do that I'm not sure it > would be a bad thing to try to solve this issue in a somewhat more >

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Robert Haas
On Fri, Nov 9, 2012 at 11:27 AM, Tom Lane wrote: >> My goal is to allow COPY to load frozen tuples without causing MVCC >> violations. > > If that's the goal, I question why you're insisting on touching > TRUNCATE's behavior. We already have the principle that "TRUNCATE is > like DELETE except n

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Robert Haas
On Fri, Nov 9, 2012 at 9:22 AM, Kevin Grittner wrote: > (1) Any attempt to read from the truncated table would not block. The > pg_class version included in the transaction's snapshot would > determine which heap and indexes were accessed. Well, the thing is, you can't actually do this. When the

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Tom Lane
Simon Riggs writes: > On 9 November 2012 15:34, Kevin Grittner wrote: >> If we're not talking about making conflicts with other transactions >> behave just the same as an unqualified DELETE from a user >> perspective, I'm not sure what the goal is, exactly. > Reasonable question. > My goal is t

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Kevin Grittner
Simon Riggs wrote: > My goal is to allow COPY to load frozen tuples without causing MVCC > violations. OK. That wasn't initially clear to me. > Forcing a tightly scoped proposal into a much wider one will just > kill this and leave it blocked. The goal is important enough and narrow enough to m

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Kevin Grittner
Simon Riggs wrote: >> This seems like a subset of the issues which one might want to >> address by making DDL statement behave in a more strictly MVCC >> fashion. Does it make sense to pick those off one at a time, or >> should something like this be done only in the context of an >> overall plan

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 15:46, Simon Riggs wrote: > Altering TRUNCATE so it behaves perfectly from an MVCC/Serializable > perspective is a much bigger, and completely different goal, as well > as something I don't see as desirable anyway for at least 2 good > reasons, as explained. IMHO if people want

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 15:34, Kevin Grittner wrote: > If we're not talking about making conflicts with other transactions > behave just the same as an unqualified DELETE from a user > perspective, I'm not sure what the goal is, exactly. Reasonable question. My goal is to allow COPY to load frozen t

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Kevin Grittner
Merlin Moncure wrote: > Kevin Grittner wrote: >> Robert Haas wrote: >> It seems to me that the goal would be to make this semantically >> idential to the behavior users would see if an unqualified DELETE >> were run against the table rather than a TRUNCATE. > > but, triggers would not fire, righ

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Merlin Moncure
On Fri, Nov 9, 2012 at 8:22 AM, Kevin Grittner wrote: > Robert Haas wrote: > >> What I've been wondering since this last came up is whether we >> could use some variant of the SIREAD locks Kevin introduced for SSI >> to handle this case - essentially have the transaction doing the >> TRUNCATE make

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 14:22, Kevin Grittner wrote: > Robert Haas wrote: > >> What I've been wondering since this last came up is whether we >> could use some variant of the SIREAD locks Kevin introduced for SSI >> to handle this case - essentially have the transaction doing the >> TRUNCATE make an en

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 14:55, Marti Raudsepp wrote: > On Fri, Nov 9, 2012 at 4:24 PM, Simon Riggs wrote: >> We need a fast lookup structure that is expandable to accommodate >> arbitrary numbers of truncates. Shared hash table, with some form of >> overflow mechanism. > > Surely you only need to reme

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Kevin Grittner
Robert Haas wrote: > What I've been wondering since this last came up is whether we > could use some variant of the SIREAD locks Kevin introduced for SSI > to handle this case - essentially have the transaction doing the > TRUNCATE make an entry in the lock table that will force a > serialization

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Marti Raudsepp
On Fri, Nov 9, 2012 at 4:24 PM, Simon Riggs wrote: > We need a fast lookup structure that is expandable to accommodate > arbitrary numbers of truncates. Shared hash table, with some form of > overflow mechanism. Surely you only need to remember the last completed truncate for each relation? The l

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 14:01, Robert Haas wrote: > I think the question that hasn't really been adequately answered is: > where and how are we going to track conflicts? Your previous patch > involved storing an XID in pg_class, but I think we both found that a > bit grotty - it'd probably need speci

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Robert Haas
On Fri, Nov 9, 2012 at 8:22 AM, Simon Riggs wrote: >> Personally I think the behavior should be dictated by the *reader*. >> The one doing the truncation may not know about the consistency >> requirements of particular readers. Especially when you do the >> truncate via pg_restore or some other ge

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 9 November 2012 10:28, Marti Raudsepp wrote: > On Wed, Nov 7, 2012 at 5:34 PM, Simon Riggs wrote: >> I was unhappy with changing the behaviour of TRUNCATE, and still am. >> So the proposal here is to have a specific modifier on TRUNCATE >> command that makes it MVCC safe by throwing a serializ

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Marti Raudsepp
On Wed, Nov 7, 2012 at 5:34 PM, Simon Riggs wrote: > I was unhappy with changing the behaviour of TRUNCATE, and still am. > So the proposal here is to have a specific modifier on TRUNCATE > command that makes it MVCC safe by throwing a serialization error. > That new behaviour should be requestabl

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Hannu Krosing
On 11/09/2012 09:34 AM, Simon Riggs wrote: On 8 November 2012 23:20, Hannu Krosing wrote: On 11/08/2012 08:51 PM, Simon Riggs wrote: On 8 November 2012 17:07, Robert Haas wrote: On Wed, Nov 7, 2012 at 10:34 AM, Simon Riggs wrote: For 9.2 we discussed having COPY setting tuples as frozen. V

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-09 Thread Simon Riggs
On 8 November 2012 23:20, Hannu Krosing wrote: > On 11/08/2012 08:51 PM, Simon Riggs wrote: >> >> On 8 November 2012 17:07, Robert Haas wrote: >>> >>> On Wed, Nov 7, 2012 at 10:34 AM, Simon Riggs >>> wrote: For 9.2 we discussed having COPY setting tuples as frozen. Various details

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-08 Thread Hannu Krosing
On 11/08/2012 08:51 PM, Simon Riggs wrote: On 8 November 2012 17:07, Robert Haas wrote: On Wed, Nov 7, 2012 at 10:34 AM, Simon Riggs wrote: For 9.2 we discussed having COPY setting tuples as frozen. Various details apply. Earlier threads: "RFC: Making TRUNCATE more "MVCC-safe" "COPY wit

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-08 Thread Simon Riggs
On 8 November 2012 17:07, Robert Haas wrote: > On Wed, Nov 7, 2012 at 10:34 AM, Simon Riggs wrote: >> For 9.2 we discussed having COPY setting tuples as frozen. Various >> details apply. >> Earlier threads: >> "RFC: Making TRUNCATE more "MVCC-safe" >> "COPY with hints, rebirth" >> >> I was un

Re: [HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-08 Thread Robert Haas
On Wed, Nov 7, 2012 at 10:34 AM, Simon Riggs wrote: > For 9.2 we discussed having COPY setting tuples as frozen. Various > details apply. > Earlier threads: > "RFC: Making TRUNCATE more "MVCC-safe" > "COPY with hints, rebirth" > > I was unhappy with changing the behaviour of TRUNCATE, and stil

[HACKERS] TRUNCATE SERIALIZABLE and frozen COPY

2012-11-07 Thread Simon Riggs
For 9.2 we discussed having COPY setting tuples as frozen. Various details apply. Earlier threads: "RFC: Making TRUNCATE more "MVCC-safe" "COPY with hints, rebirth" I was unhappy with changing the behaviour of TRUNCATE, and still am. So the proposal here is to have a specific modifier on TRUNC