Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-07 Thread Florian Pflug
On Jan7, 2014, at 09:45 , Heikki Linnakangas wrote: > Overall, I'm leaning towards biting the bullet and always detoasting > everything in master. Probably best to just leave the stable branches alone. +1 The fact that de-TOAST-ing can happen lazily is, at least to me, an implementation detail

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-07 Thread Andres Freund
On 2014-01-07 10:45:24 +0200, Heikki Linnakangas wrote: > Overall, I'm leaning towards biting the bullet and always detoasting > everything in master. Probably best to just leave the stable branches alone. If we're doing something coarse grained as this, I agree, it should be master only. I perso

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-07 Thread Heikki Linnakangas
On 01/06/2014 08:29 PM, Andres Freund wrote: On 2014-01-06 12:40:25 -0500, Robert Haas wrote: On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund wrote: On 2014-01-06 11:08:41 -0500, Robert Haas wrote: Yea. But at least it would fail reliably instead of just under concurrency and other strange circ

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 8:02 PM, Jim Nasby wrote: > If concurrent TRUNCATE isn't safe outside of this case then why do we allow > it? IE: why doesn't TRUNCATE exclusive lock the relation? It *does*. The problem is that the *other* transaction that's reading the relation can still retain a TOAST p

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Jim Nasby
On 1/6/14, 2:21 PM, Tom Lane wrote: Andres Freund writes: On 2014-01-06 12:40:25 -0500, Robert Haas wrote: Is "forcibly detoast everything" a complete no-go? I realize there are performance concerns with that approach, but I'm not sure how realistic a worry it actually is. The scenario I a

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Jim Nasby
On 1/2/14, 1:32 PM, Tom Lane wrote: Heikki Linnakangas writes: The simplest fix would be to just detoast everything on assignment but that was rejected on performance grounds in that previous thread. I don't see any other realistic way to fix this, however, so maybe we should just bite the bull

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Tom Lane
Andres Freund writes: > On 2014-01-06 12:40:25 -0500, Robert Haas wrote: >> Is "forcibly detoast everything" a complete no-go? I realize there >> are performance concerns with that approach, but I'm not sure how >> realistic a worry it actually is. > The scenario I am primarily worried about is

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 12:40:25 -0500, Robert Haas wrote: > On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund wrote: > > On 2014-01-06 11:08:41 -0500, Robert Haas wrote: > > Yea. But at least it would fail reliably instead of just under > > concurrency and other strange circumstances - and there'd be a safe w

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Tom Lane
Robert Haas writes: > Is "forcibly detoast everything" a complete no-go? I realize there > are performance concerns with that approach, but I'm not sure how > realistic a worry it actually is. It's certainly possible to think of scenarios under which it'd be painful, eg, you fetch all columns in

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 11:47 AM, Andres Freund wrote: > On 2014-01-06 11:08:41 -0500, Robert Haas wrote: >> On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund wrote: >> > On 2014-01-06 09:43:45 -0500, Robert Haas wrote: >> >> > I actually vote for not allowing doing so at all by erroring out when >> >

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 11:08:41 -0500, Robert Haas wrote: > On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund wrote: > > On 2014-01-06 09:43:45 -0500, Robert Haas wrote: > >> > I actually vote for not allowing doing so at all by erroring out when > >> > accessing a plpgsql variable created in an aborted subxac

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:47 AM, Andres Freund wrote: > On 2014-01-06 09:43:45 -0500, Robert Haas wrote: >> > I actually vote for not allowing doing so at all by erroring out when >> > accessing a plpgsql variable created in an aborted subxact, unless you >> > explicitly signal that you want to do

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Mon, Jan 6, 2014 at 9:19 AM, Andres Freund wrote: > On 2014-01-06 09:10:48 -0500, Robert Haas wrote: >> On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund wrote: >> >> I think the only principled fixes are to either retain the lock or >> >> forcibly detoast before releasing it. >> > >> > I don't th

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 09:43:45 -0500, Robert Haas wrote: > > I actually vote for not allowing doing so at all by erroring out when > > accessing a plpgsql variable created in an aborted subxact, unless you > > explicitly signal that you want to do do so by calling some function > > deleting the information

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Andres Freund
On 2014-01-06 09:10:48 -0500, Robert Haas wrote: > On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund wrote: > >> I think the only principled fixes are to either retain the lock or > >> forcibly detoast before releasing it. > > > > I don't think that's sufficient. Unless I miss something the problem >

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-06 Thread Robert Haas
On Thu, Jan 2, 2014 at 4:15 PM, Andres Freund wrote: > On 2014-01-02 16:05:09 -0500, Robert Haas wrote: >> On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund wrote: >> >> I was wondering if we could somehow arrange to not >> >> release the subtransaction's AccessShareLock on the table, as long as it >

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-04 Thread Amit Kapila
On Fri, Jan 3, 2014 at 9:05 AM, Amit Kapila wrote: > On Fri, Jan 3, 2014 at 12:51 AM, Heikki Linnakangas > wrote: >> On 01/02/2014 02:24 PM, Rushabh Lathia wrote: >>> Do you think we should detoast the local variable before >>> RollbackAndReleaseCurrentSubTransaction ? Or any other options ? >>

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Amit Kapila
On Fri, Jan 3, 2014 at 12:51 AM, Heikki Linnakangas wrote: > On 01/02/2014 02:24 PM, Rushabh Lathia wrote: >> >> Hi All, >> >> Test case: >> >> drop table if exists t; >> create table t(c text); >> insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); >> select pg_column_size(c), p

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 16:05:09 -0500, Robert Haas wrote: > On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund wrote: > >> I was wondering if we could somehow arrange to not > >> release the subtransaction's AccessShareLock on the table, as long as it > >> was protecting toasted references someplace. > > > > So

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Robert Haas
On Thu, Jan 2, 2014 at 3:19 PM, Andres Freund wrote: >> I was wondering if we could somehow arrange to not >> release the subtransaction's AccessShareLock on the table, as long as it >> was protecting toasted references someplace. > > Sounds fairly ugly... I think the only principled fixes are to

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 15:00:58 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: > >> I don't see any other realistic way to fix this, however, so maybe we > >> should just bite the bullet and do it anyway. > > > We could remember the subtransactio

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Tom Lane
Andres Freund writes: > On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: >> I don't see any other realistic way to fix this, however, so maybe we >> should just bite the bullet and do it anyway. > We could remember the subtransaction a variable was created in and error > out if it the crea

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Andres Freund
On 2014-01-02 21:21:15 +0200, Heikki Linnakangas wrote: > I don't see any other realistic way to fix this, however, so maybe we > should just bite the bullet and do it anyway. We could remember the subtransaction a variable was created in and error out if it the creating subtransaction aborted and

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Tom Lane
Heikki Linnakangas writes: > The simplest fix would be to just detoast everything on assignment but > that was rejected on performance grounds in that previous thread. I > don't see any other realistic way to fix this, however, so maybe we > should just bite the bullet and do it anyway. Or jus

Re: [HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Heikki Linnakangas
On 01/02/2014 02:24 PM, Rushabh Lathia wrote: Hi All, Test case: drop table if exists t; create table t(c text); insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); select pg_column_size(c), pg_column_size(c || '') FROM t; CREATE OR REPLACE FUNCTION copy_toast_out() RETURNS V

[HACKERS] ERROR: missing chunk number 0 for toast value

2014-01-02 Thread Rushabh Lathia
Hi All, Test case: drop table if exists t; create table t(c text); insert into t values ('x'), (repeat(md5('abcdefghijklmnop'), 1)); select pg_column_size(c), pg_column_size(c || '') FROM t; CREATE OR REPLACE FUNCTION copy_toast_out() RETURNS VOID AS $$ declare v text; BEGIN