Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Tom Lane wrote: I don't think this has been adequately thought through at all ... but at least make it ExclusiveLock. What is the use-case for allowing SELECT FOR UPDATE in parallel with this? Ok, patch applied -- I adjusted it to use ExclusiveLock, and fleshed out some of the comments. -Neil In

[HACKERS] \x in psql

2005-03-22 Thread Christopher Kings-Lynne
When you turn on \x mode for query output in psql, it wrecks the output of \d , etc. Should we change it so that the \d is unaffected by \x? What about for other \d commands? Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet

Re: [HACKERS] "they only drink coffee at dec"

2005-03-22 Thread Christopher Kings-Lynne
From src/backend/tcop/postgres.c: appendStringInfo(&str, "!\t%ld/%ld [%ld/%ld] filesystem blocks in/out\n", r.ru_inblock - Save_r.ru_inblock, /* they only drink coffee at dec */ r.ru_oublock - Save_r.ru_oublock,

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Tom Lane wrote: This is presuming that we abandon the notion that system catalog access use SnapshotNow. Which opens the question of what they should use instead ... to which "transaction snapshot" isn't the answer, because we have to be able to do system catalog accesses before we've set the snap

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Tom Lane wrote: It isn't 100% MVCC, I agree. But it works because system catalog lookups are SnapshotNow, and so when another session comes and wants to look at the table it will see the committed new version of the pg_class row pointing at the new relfilenode file. If by "works", you mean "provid

Re: [HACKERS] WAL: O_DIRECT and multipage-writer

2005-03-22 Thread ITAGAKI Takahiro
Hi, Mark. Mark Wong <[EMAIL PROTECTED]> wrote: > In light of this thread, have you compared the performance on > Linux-2.4? No, but I'm just testing my patch on Linux-2.4 with a middle-range server. I will report the results sometime soon. By the way, I found the debug option (XLOG_MULTIPAGE_W

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > - if we make these changes, we will need some way to delete a > no-longer-visible relfilenode. This is presuming that we abandon the notion that system catalog access use SnapshotNow. Which opens the question of what they should use instead ... to which

Re: [HACKERS] swapping relfilenodes (was: Re: locks in CREATE TRIGGER,

2005-03-22 Thread Tom Lane
Bruce Momjian writes: > In fact, would a truncate during a backup cause the backup to be > inconsistent because it wouldn't be a true snapshot of the database at > backup start time? Seems so. No, because pg_dump holds AccessShareLock on every table that it intends to dump, thereby ensuring that

Re: [HACKERS] swapping relfilenodes (was: Re: locks in CREATE TRIGGER,

2005-03-22 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > In fact, would a truncate during a backup cause the backup to be > > inconsistent because it wouldn't be a true snapshot of the database at > > backup start time? Seems so. > > No, because pg_dump holds AccessShareLock on every table that it intends >

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > How can we drop the file at commit, given that a serializable > transaction's snapshot should still be able to see old relfilenode's > content? It isn't 100% MVCC, I agree. But it works because system catalog lookups are SnapshotNow, and so when another

Re: [HACKERS] swapping relfilenodes (was: Re: locks in CREATE TRIGGER,

2005-03-22 Thread Bruce Momjian
Andrew - Supernews wrote: > On 2005-03-23, Neil Conway <[EMAIL PROTECTED]> wrote: > > - swap the relfilenodes of the old and temporary heap relations > > While discussing this one further on IRC, I noticed the following: > > Everywhere I could find that currently replaces the relfilenode of a > r

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Bruce Momjian
Neil Conway wrote: > Tom Lane wrote: > > Utterly wrong. When you commit you will physically drop the old table. > > If there is a SELECT running against the old table it will be quite > > unhappy after that. > > How can we drop the file at commit, given that a serializable > transaction's snapsh

[HACKERS] swapping relfilenodes (was: Re: locks in CREATE TRIGGER, ADD FK)

2005-03-22 Thread Andrew - Supernews
On 2005-03-23, Neil Conway <[EMAIL PROTECTED]> wrote: > - swap the relfilenodes of the old and temporary heap relations While discussing this one further on IRC, I noticed the following: Everywhere I could find that currently replaces the relfilenode of a relation does so while holding an AccessE

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Tom Lane wrote: Utterly wrong. When you commit you will physically drop the old table. If there is a SELECT running against the old table it will be quite unhappy after that. How can we drop the file at commit, given that a serializable transaction's snapshot should still be able to see old relfi

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > Well, we will be holding an ExclusiveLock on the heap relation > regardless. We "replace" the heap table by swapping its relfilenode, so > ISTM we needn't hold an AccessExclusiveLock. Utterly wrong. When you commit you will physically drop the old table

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Neil Conway wrote: ... except that when we rebuild the relation's indexes, we acquire an AccessExclusiveLock on the index. This would introduce the risk of deadlock. It seems necessary to acquire an AccessExclusiveLock when rebuilding shared indexes, since we do the index build in-place, but I

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Christopher Kings-Lynne <[EMAIL PROTECTED]> writes: > It would keep the old table around while building the new, then grab > an exclusive lock to swap the two. Lock upgrading is right out. regards, tom lane ---(end of broadcast)-

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > /* > ! * Grab an exclusive lock on the pk table, so that someone doesn't > ! * delete rows out from under us. (Although a lesser lock would do for > ! * that purpose, we'll need exclusive lock anyway to add triggers to > ! * the

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: >> AndrewSN pointed out on IRC that ALTER TABLE ... ADD FOREIGN KEY and >> CREATE TRIGGER both acquire AccessExclusiveLocks on the table they are >> adding triggers to (the PK table, in the case of ALTER TABLE). Is this >> necessary? I don't see why we can

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Bruce Momjian wrote: Certainly we need to upgrade to an exclusive table lock to replace the heap table. Well, we will be holding an ExclusiveLock on the heap relation regardless. We "replace" the heap table by swapping its relfilenode, so ISTM we needn't hold an AccessExclusiveLock. Do we want t

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Bruce Momjian
Neil Conway wrote: > So I think it should be possible to lock both the heap relation and the > index with ExclusiveLock, which would allow SELECTs on them. This would > apply to both the single relation and multiple relation variants of > CLUSTER (since we do each individual clustering in its ow

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Christopher Kings-Lynne
Huh, cluster already does that. It does and it doesn't. Something like the first thing it does is muck with the old table's filenode IIRC, meaning that immediately the old table will no longer work. Chris ---(end of broadcast)--- TIP 8: explain an

Re: [HACKERS] Prevent conflicting SET options from being set

2005-03-22 Thread Bruce Momjian
Thanks, fixed. --- Qingqing Zhou wrote: > > "Bruce Momjian" writes > > Removed. I think we have all the conflicting options fixed already: > > > > One more thing, there is a small typo in TODO list: duplidated "Allow a w

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Christopher Kings-Lynne wrote: If you want to be my friend forever, then fix CLUSTER so that it uses sharerowexclusive as well :D Hmm, this might be possible as well. During a CLUSTER, we currently - lock the heap relation with AccessExclusiveLock - lock the index we're clustering on with AccessEx

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Alvaro Herrera
On Wed, Mar 23, 2005 at 10:42:01AM +0800, Christopher Kings-Lynne wrote: > >>If you want to be my friend forever, then fix CLUSTER so that it uses > >>sharerowexclusive as well :D > > > >I don't think it's as easy as that, because you have to move tuples > >around in the cluster operation. Same s

Re: [HACKERS] odd problem !

2005-03-22 Thread Bruce Momjian
Tom Lane wrote: > Oleg Bartunov writes: > >> This isn't a problem in normal use of course, but it'd be a serious > >> issue for someone engaging in WAL-shipping, if their backup postmaster > >> were living at a different absolute path. We probably need to think > > > right, this is normal situat

Re: [HACKERS] psql and pg_dump using obselete copy commands

2005-03-22 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > > Oh, if we do that, do we disallow connecting to older servers? > > Not at all, since the logic would be like this: > > if we have new copy functions >and we have protocol version function > and protocol version >= 3 >then use new copy functions

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Christopher Kings-Lynne
If you want to be my friend forever, then fix CLUSTER so that it uses sharerowexclusive as well :D I don't think it's as easy as that, because you have to move tuples around in the cluster operation. Same sort of issue as vacuum full I would suggest. Cluster doesn't move rows... I didn't say it

Re: [HACKERS] Using new copy libpq functions on a v2 protocol backend

2005-03-22 Thread Christopher Kings-Lynne
If I use PQgetCopyData, PQputCopyData and PQputCopyEnd against a v2 protocol backend, will it work? I see no mention of it in the docs... OK, my testing proves that they work just fine against an older server, so no problem. Chris ---(end of broadcast)---

Re: [HACKERS] Using new copy libpq functions on a v2 protocol backend

2005-03-22 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > >> If I use PQgetCopyData, PQputCopyData and PQputCopyEnd against a v2 > >> protocol backend, will it work? > >> > >> I see no mention of it in the docs... > > OK, my testing proves that they work just fine against an older server, > so no problem. > OK, what i

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Russell Smith
On Wed, 23 Mar 2005 12:40 pm, Christopher Kings-Lynne wrote: > If you want to be my friend forever, then fix CLUSTER so that it uses > sharerowexclusive as well :D > I don't think it's as easy as that, because you have to move tuples around in the cluster operation. Same sort of issue as vacuum

Re: [HACKERS] psql and pg_dump using obselete copy commands

2005-03-22 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > > We have been telling people to use newer pg_dump's on older servers, but > > we only support reloading into the current PostgreSQL version, so I see > > no reason not to updated it to the current syntax. > > > > We added the new syntax in 7.3. > > > > Added to T

Re: [HACKERS] psql and pg_dump using obselete copy commands

2005-03-22 Thread Christopher Kings-Lynne
Oh, if we do that, do we disallow connecting to older servers? Not at all, since the logic would be like this: if we have new copy functions and we have protocol version function and protocol version >= 3 then use new copy functions else use old copy functions That would be even simpl

Re: [HACKERS] psql and pg_dump using obselete copy commands

2005-03-22 Thread Christopher Kings-Lynne
We have been telling people to use newer pg_dump's on older servers, but we only support reloading into the current PostgreSQL version, so I see no reason not to updated it to the current syntax. We added the new syntax in 7.3. Added to TODO: o Update pg_dump to use the newer COPY syntax I think y

Re: [HACKERS] psql and pg_dump using obselete copy commands

2005-03-22 Thread Bruce Momjian
Christopher Kings-Lynne wrote: > Should psql and pg_dump be upgraded to use the new v3 protocol copy > functions if they are available, as they are currently using the > deprecated API. We have been telling people to use newer pg_dump's on older servers, but we only support reloading into the cu

Re: [HACKERS] Prevent conflicting SET options from being set

2005-03-22 Thread Qingqing Zhou
"Bruce Momjian" writes > Removed. I think we have all the conflicting options fixed already: > One more thing, there is a small typo in TODO list: duplidated "Allow a warm standby system to also allow read-only queries ". Regards, Qingqing ---(end of broadcast)--

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Christopher Kings-Lynne
If you want to be my friend forever, then fix CLUSTER so that it uses sharerowexclusive as well :D Chris Neil Conway wrote: Neil Conway wrote: AndrewSN pointed out on IRC that ALTER TABLE ... ADD FOREIGN KEY and CREATE TRIGGER both acquire AccessExclusiveLocks on the table they are adding trigg

Re: [HACKERS] locks in CREATE TRIGGER, ADD FK

2005-03-22 Thread Neil Conway
Neil Conway wrote: AndrewSN pointed out on IRC that ALTER TABLE ... ADD FOREIGN KEY and CREATE TRIGGER both acquire AccessExclusiveLocks on the table they are adding triggers to (the PK table, in the case of ALTER TABLE). Is this necessary? I don't see why we can't allow SELECT queries on the ta

Re: [HACKERS] Using new copy libpq functions on a v2 protocol backend

2005-03-22 Thread Christopher Kings-Lynne
Hey guys, I really need answer to this one, for the PHP code I just committed :P Chris Christopher Kings-Lynne wrote: Hi, If I use PQgetCopyData, PQputCopyData and PQputCopyEnd against a v2 protocol backend, will it work? I see no mention of it in the docs... Chris ---(end

Re: [HACKERS] Prevent conflicting SET options from being set

2005-03-22 Thread Bruce Momjian
Qingqing Zhou wrote: > > "Tom Lane" <[EMAIL PROTECTED]> writes > > > > We already have the ability to issue custom messages in assign_hooks, > > and I think that's sufficient in practice. > > Yes, I agree this is already sufficient - seems we need to remove that TODO > item in the list. Removed.

Re: [HACKERS] odd problem !

2005-03-22 Thread Tom Lane
Oleg Bartunov writes: >> This isn't a problem in normal use of course, but it'd be a serious >> issue for someone engaging in WAL-shipping, if their backup postmaster >> were living at a different absolute path. We probably need to think > right, this is normal situation if you backup to the sam

Re: [HACKERS] odd problem !

2005-03-22 Thread Oleg Bartunov
On Tue, 22 Mar 2005, Tom Lane wrote: Oleg Bartunov writes: What PG version is this exactly? REL8_0_STABLE, I believe. I posted another problem, now "cut and pasted". I've been able to duplicate this here. What is happening is that the damage to ./t1 is being done when you start the postmaster in

Re: [HACKERS] odd problem !

2005-03-22 Thread Tom Lane
Oleg Bartunov writes: >> What PG version is this exactly? > REL8_0_STABLE, I believe. I posted another problem, now "cut and pasted". I've been able to duplicate this here. What is happening is that the damage to ./t1 is being done when you start the postmaster in ./t2. It looks to me like the

Re: [HACKERS] Another history question

2005-03-22 Thread Tom Lane
Juan Pablo Espino <[EMAIL PROTECTED]> writes: > My question is if the architecture of postgreSQL were inherited of > postgres original project or postgreSQL were developed completely with > a new concept. Thanks in advance. There hasn't been any fundamental rearchitecting since Berkeley days. For

Re: [HACKERS] odd problem !

2005-03-22 Thread Oleg Bartunov
On Tue, 22 Mar 2005, Tom Lane wrote: template1 | postgres | KOI8 test | postgres | KOI8 (3 rows) 11. [EMAIL PROTECTED]:~/test$ psql test FATAL: database "test" does not exist psql: FATAL: database "test" does not exist What PG version is this exactly? I suppose that you're seeing one o

Re: [HACKERS] odd problem !

2005-03-22 Thread Tom Lane
Oleg Bartunov writes: > below is the problem I just bitten when play with toy db. I did: > 1.initdb -D ./t1 > 2. pg_ctl -D ./t1 start > 3. createdb test > 4. psql test -c "create table a (f integer);" > 5. run script which populates table a in background > perl bgupdate.pl & > 6. cp -a ./t1 .

Re: [HACKERS] WAL: O_DIRECT and multipage-writer

2005-03-22 Thread Mark Wong
On Tue, Jan 25, 2005 at 06:06:23PM +0900, ITAGAKI Takahiro wrote: > Environment: > OS : Linux kernel 2.6.9 > CPU: Pentium 4 3GHz > disk : ATA 5400rpm (Data and WAL are placed on same partition.) > memory : 1GB > config : shared_buffers=1, wal_buffers=256, >XLOG_S

Re: [HACKERS] odd problem !

2005-03-22 Thread Oleg Bartunov
OK, here is more cleaner cut and paste from my notebook: [EMAIL PROTECTED]:~/test$ initdb -D ./t1 [EMAIL PROTECTED]:~/test$ pg_ctl -D ./t1 start postmaster starting [EMAIL PROTECTED]:~/test$ LOG: database system was shut down at 2005-03-23 01:09:34 MSK LOG: checkpoint record is at 0/A2C844 LOG:

[HACKERS] Another history question

2005-03-22 Thread Juan Pablo Espino
I know that postgres was a project directed by Michael Stonebraker in Berkeley (1986-1994) and that soon Jolly Chen and Andrew Yu did postgres95. I understand that the main change in postgres95 was to implement SQL instead of POSTQUEL. Then after the appearance of postgres95 postgreSQL 6.0 arises

[HACKERS] odd problem !

2005-03-22 Thread Oleg Bartunov
Hi there, below is the problem I just bitten when play with toy db. I did: 1.initdb -D ./t1 2. pg_ctl -D ./t1 start 3. createdb test 4. psql test -c "create table a (f integer);" 5. run script which populates table a in background perl bgupdate.pl & 6. cp -a ./t1 ./t2 6. pg_ctl -D ./t1 stop i

Re: [HACKERS] Proposal: OUT parameters for plpgsql

2005-03-22 Thread Tom Lane
Pavel Stehule <[EMAIL PROTECTED]> writes: > On Mon, 21 Mar 2005, Tom Lane wrote: >> So typical call style would be >> SELECT * FROM foo(1,2,'xyzzy'); > I am not sure so this syntax is readable. I'm sure, so this solution is > possible and usefull, but you mix SRF style of calling and normal styl

[HACKERS] Permissions on tables and views

2005-03-22 Thread subhash
Hi, I have a requirement where I have a table and a view on top of it. I want to make some changes so that a user in a given group would be able to insert/update only on the view (ofcourse i have a rule which inserts/updates the table) but not on the underlying table. I tried revoke the insert/upda

Re: [HACKERS] Failed to re-find parent key

2005-03-22 Thread Peter Eisentraut
Am Dienstag, 22. März 2005 15:54 schrieb Tom Lane: > Any chance of going in with a debugger, or capturing a > tarball image of the database for someone else to look at? Unfortunately, this database is restricted and I don't have access myself. I will tell the customer that they should provide a

Re: [HACKERS] Failed to re-find parent key

2005-03-22 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > I think it takes a lot of concurrency for > the situation to arise. Maybe. Since Peter can reproduce the error, there's not any concurrency misbehavior involved in VACUUM itself; what we are dealing with is probably corruption in the on-disk state of t

Re: [HACKERS] Failed to re-find parent key

2005-03-22 Thread Alvaro Herrera
On Tue, Mar 22, 2005 at 12:31:55PM +0100, Peter Eisentraut wrote: > What does the error message > > failed to re-find parent key in "tablename_pkey" > > mean? This happens reproducibly during VACUUM on a certain table. This has been reported before, but no one has been able to reproduce it (not

Re: [HACKERS] Failed to re-find parent key

2005-03-22 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > What does the error message > failed to re-find parent key in "tablename_pkey" > mean? This happens reproducibly during VACUUM on a certain table. If it happens during vacuum (not vacuum full) then it must be coming from _bt_pagedel, and it means tha

[HACKERS] Failed to re-find parent key

2005-03-22 Thread Peter Eisentraut
What does the error message failed to re-find parent key in "tablename_pkey" mean? This happens reproducibly during VACUUM on a certain table. Would REINDEX fix it? Anything else we should check? This is PostgreSQL 7.4.2. Are there relevant fixes later in the 7.4 series? -- Peter Eisentrau

Re: [HACKERS] what to do with backend flowchart

2005-03-22 Thread Dave Page
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Robert Treat > Sent: 21 March 2005 19:05 > To: Tom Lane > Cc: pgsql-hackers@postgresql.org; [EMAIL PROTECTED] > Subject: Re: [HACKERS] what to do with backend flowchart > > My thoughts were that it

Re: [HACKERS] caches lifetime with SQL vs PL/PGSQL procs

2005-03-22 Thread strk
It is embarassing for me, but I could not reproduce the bug. :( Maybe I just ended up with a corrupted database (or I was just too tired). Behaviour seems to be the same for both SQL and pl/pgsql functions on a new database (and I got rid of the old one). Sorry. --strk; On Thu, Mar 17, 2005 at