Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tatsuo Ishii
> On 12/07/2010 01:22 AM, Tom Lane wrote: >> Josh Berkus writes: However, if you were doing something like parallel pg_dump you could just run the parent and child instances all against the slave, so the pg_dump scenario doesn't seem to offer much of a supporting use-case for w

[HACKERS] To Signal The postmaster

2010-12-06 Thread aaliya zarrin
Hi, I want to signal the postmaster (User defined signal) once I created the trigger file (for switch over). I want to reduce my switch over time further by signaling the postmaster once the trigger file is created. Is this provision already there in the code? There ate already SIGUSR1 and SIGUSR

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Stefan Kaltenbrunner
On 12/07/2010 01:22 AM, Tom Lane wrote: > Josh Berkus writes: >>> However, if you were doing something like parallel pg_dump you could >>> just run the parent and child instances all against the slave, so the >>> pg_dump scenario doesn't seem to offer much of a supporting use-case for >>> worrying

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Craig Ringer
On 07/12/10 10:48, Josh Berkus wrote: > >> It seems plausible to fix the first one, but how would you fix the >> second one? You either allow SET ROLE (which you need, to support the >> pooler changing authorization), or you don't. There doesn't seem to be >> a usable middleground. > > Well, th

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Tom Lane
Alvaro Herrera writes: > Maybe we should have a single tunable for processes that just sleep > waiting for events or postmaster death. For example pgstats has a > hardcoded 2 seconds, and the archiver process has a hardcoded value too > AFAICS. That would make sense once we get to the point wher

Re: [HACKERS] Re: Rethinking hint bits WAS: Protecting against unexpected zero-pages: proposal

2010-12-06 Thread Jim Nasby
On Nov 14, 2010, at 3:40 PM, Greg Stark wrote:On Sun, Nov 14, 2010 at 8:52 PM, Josh Berkus wrote:For example, imagine if the hint bits were moved to a separate per-tablebitmap outside the table instead of being stored with each row, as thecurrent FSM is.How many times do we have to keep going arou

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Fujii Masao
On Tue, Dec 7, 2010 at 12:22 PM, Robert Haas wrote: >> Fair enough. How about increasing the default to 10 seconds? >> Since bgwriter has already using 10s as a nap time if there is no >> configured activity, I think that 10s is non-nonsense default value. > > What do we get out of making this non

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 10:07 PM, Fujii Masao wrote: > On Tue, Dec 7, 2010 at 11:49 AM, Tom Lane wrote: >> Fujii Masao writes: >>> One problem with the patch is that it takes longer (at most 10s) to >>> detect the unexpected death of postmaster (by calling PostmasterIsAlive()). >>> This is OK for

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Koichi Suzuki
We may need other means to ensure that the snapshot is available on the slave. It could be a bit too early to use the snapshot on the slave depending upon the delay of WAL replay. -- Koichi Suzuki 2010/12/7 Tom Lane : > marcin mank writes: >> On Sun, Dec 5, 2010 at 7:28 PM, Tom Lane w

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Fujii Masao
On Tue, Dec 7, 2010 at 11:49 AM, Tom Lane wrote: > Fujii Masao writes: >> One problem with the patch is that it takes longer (at most 10s) to >> detect the unexpected death of postmaster (by calling PostmasterIsAlive()). >> This is OK for me. But does anyone want to specify the delay to detect >>

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Alvaro Herrera
Excerpts from Tom Lane's message of lun dic 06 23:49:52 -0300 2010: > Fujii Masao writes: > > One problem with the patch is that it takes longer (at most 10s) to > > detect the unexpected death of postmaster (by calling PostmasterIsAlive()). > > This is OK for me. But does anyone want to specify t

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 9:37 PM, Alvaro Herrera wrote: > Excerpts from Robert Haas's message of lun dic 06 23:09:56 -0300 2010: >> On Mon, Dec 6, 2010 at 2:47 PM, Josh Berkus wrote: >> > >> >> Please explain more precisely what is wrong with SET SESSION >> >> AUTHORIZATION / SET ROLE. >> > >> > 1)

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Tom Lane
Fujii Masao writes: > One problem with the patch is that it takes longer (at most 10s) to > detect the unexpected death of postmaster (by calling PostmasterIsAlive()). > This is OK for me. But does anyone want to specify the delay to detect > that within a short time? Oh. Hm. I'm hesitant to re

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Josh Berkus
> It seems plausible to fix the first one, but how would you fix the > second one? You either allow SET ROLE (which you need, to support the > pooler changing authorization), or you don't. There doesn't seem to be > a usable middleground. Well, this is why such a pooler would *have* to be built

Re: [HACKERS] Per-column collation

2010-12-06 Thread Itagaki Takahiro
On Sun, Dec 5, 2010 at 01:04, Peter Eisentraut wrote: > Here is an updated patch to address the issues discussed during this > commitfest. I found another issue in the patch; ILIKE in WHERE clause doesn't work. It was surprising because LIKE in WHERE clause and ILIKE in SELECT list works expected

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Fujii Masao
On Tue, Dec 7, 2010 at 12:08 AM, Tom Lane wrote: > Fujii Masao writes: >> Walsender doesn't need the periodic wakeups anymore, thanks to >> the latch feature. So wal_sender_delay is basically useless now. >> How about dropping wal_sender_delay or increasing the default >> value? > > If we don't n

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Alvaro Herrera
Excerpts from Robert Haas's message of lun dic 06 23:09:56 -0300 2010: > On Mon, Dec 6, 2010 at 2:47 PM, Josh Berkus wrote: > > > >> Please explain more precisely what is wrong with SET SESSION > >> AUTHORIZATION / SET ROLE. > > > > 1) Session GUCS do not change with a SET ROLE (this is a TODO I h

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Josh Berkus
All, Geirth's results from his FreeBSD 7.1 server using 8.4's test_fsync: Simple write timing: write0.007081 Compare fsync times on write() and non-write() descriptor: If the times are similar, fsync() can sync data written on a different descriptor. write, fs

Re: [HACKERS] Spread checkpoint sync

2010-12-06 Thread Alvaro Herrera
Excerpts from Greg Smith's message of dom dic 05 20:02:48 -0300 2010: > When ends up happening if you push toward fully sync I/O is the design > you see in some other databases, where you need multiple writer > processes. Then requests for new pages can continue to allocate as > needed, while

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Tom Lane
Josh Berkus writes: > OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. > What should I have it do instead? Report that it fails, and keep testing the other methods. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Josh Berkus
On 12/6/10 6:10 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Dec 6, 2010 at 9:04 PM, Josh Berkus wrote: >>> Actually, on OSX 10.5.8, o_dsync and fdatasync aren't even available. >>> From my run, it looks like even so regular fsync might be better than >>> open_sync. > >> But I think you

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Tom Lane
Robert Haas writes: > On Mon, Dec 6, 2010 at 9:04 PM, Josh Berkus wrote: >> Actually, on OSX 10.5.8, o_dsync and fdatasync aren't even available. >> From my run, it looks like even so regular fsync might be better than >> open_sync. > But I think you need to use fsync_writethrough if you actuall

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 2:47 PM, Josh Berkus wrote: > >> Please explain more precisely what is wrong with SET SESSION >> AUTHORIZATION / SET ROLE. > > 1) Session GUCS do not change with a SET ROLE (this is a TODO I haven't > had any time to work on) > > 2) Users can always issue their own SET ROLE

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 9:04 PM, Josh Berkus wrote: > >> Mac OS X:  Like Solaris, there's a similar mechanism but it's not >> O_DIRECT; see >> http://stackoverflow.com/questions/2299402/how-does-one-do-raw-io-on-mac-os-x-ie-equivalent-to-linuxs-o-direct-flag >> for notes about the F_NOCACHE  featur

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Josh Berkus
> Whether it's complex or not isn't really the issue. The issue is that > what test_fsync is testing had better match what the backend does, or > people will be making choices based on not-comparable test results. > I think we should have test_fsync just automatically fold in O_DIRECT > the same

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Josh Berkus
> Mac OS X: Like Solaris, there's a similar mechanism but it's not > O_DIRECT; see > http://stackoverflow.com/questions/2299402/how-does-one-do-raw-io-on-mac-os-x-ie-equivalent-to-linuxs-o-direct-flag > for notes about the F_NOCACHE feature used. Same basic situation as > Solaris; there's an AP

Re: [HACKERS] Timeout for asynchronous replication Re: Timeout and wait-forever in sync rep

2010-12-06 Thread Fujii Masao
On Mon, Dec 6, 2010 at 11:54 PM, Heikki Linnakangas wrote: > Hmm, that's actually a quite different timeout than what's required for > synchronous replication. In synchronous replication, you need to get an > acknowledgment within a timeout. This patch only puts a timeout on how long > we wait to

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Josh Berkus
Steve, > If you tell me which options to pgbench and which .conf file settings > you'd like to see I can probably arrange to run some tests on AIX. Compile and run test_fsync in PGSRC/src/tools/fsync. -- -- Josh Berkus Postg

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 08:38 PM, Tom Lane wrote: Josh Berkus writes: Making it support O_DIRECT would be possible but more complex; I don't see the point unless we think we're going to have open_sync_with_odirect as a seperate option. Whether it's complex or not isn't really the issue. The issue is

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Tom Lane
Josh Berkus writes: > Making it support O_DIRECT would be possible but more complex; I don't > see the point unless we think we're going to have open_sync_with_odirect > as a seperate option. Whether it's complex or not isn't really the issue. The issue is that what test_fsync is testing had bet

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Tom Lane
Greg Smith writes: > So my guess is that some small percentage of Windows users might notice > a change here, and some testing on FreeBSD would be useful too. That's > about it for platforms that I think anybody needs to worry about. To my mind, O_DIRECT is not really the key issue here, it's

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Steve Singer
On 10-12-06 06:56 PM, Greg Smith wrote: Tom Lane wrote: The various testing that's been reported so far is all for Linux and thus doesn't directly address the question of whether other kernels will have similar performance properties. Survey of some popular platforms: So my guess is that

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-06 Thread Josh Berkus
On 12/5/10 2:12 PM, Greg Smith wrote: > Josh Berkus wrote: >> I modified test_fsync in two ways to run this; first, to make it support >> O_DIRECT, and second to make it run in the *current* directory. > > Patch please? I agree with the latter change; what test_fsync does is > surprising. Attach

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
Josh Berkus writes: >> However, if you were doing something like parallel pg_dump you could >> just run the parent and child instances all against the slave, so the >> pg_dump scenario doesn't seem to offer much of a supporting use-case for >> worrying about this. When would you really need to be

[HACKERS] the number of file descriptors when using POSIX semaphore

2010-12-06 Thread flyusa2010 fly
Hi, folks, in src/template/darwin: # Select appropriate semaphore support. Darwin 6.0 (Mac OS X 10.2) and up # support System V semaphores; before that we have to use POSIX semaphores, # which are less good for our purposes because they eat a file descriptor # per backend per max_connection slot.

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Greg Smith
Tom Lane wrote: The various testing that's been reported so far is all for Linux and thus doesn't directly address the question of whether other kernels will have similar performance properties. Survey of some popular platforms: Linux: don't want O_DIRECT by default for reliability reasons, a

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Josh Berkus
> However, if you were doing something like parallel pg_dump you could > just run the parent and child instances all against the slave, so the > pg_dump scenario doesn't seem to offer much of a supporting use-case for > worrying about this. When would you really need to be able to do it? If you

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Kevin Grittner
Heikki Linnakangas wrote: > On 06.12.2010 22:53, Kevin Grittner wrote: >> What would be the correct way for a process to put itself to >> sleep, and for another process to later wake it up? > > See ProcWaitSignal/ProcSendSignal. Or the new 'latch' code. Is there a reason to prefer one over the

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Heikki Linnakangas
On 06.12.2010 22:53, Kevin Grittner wrote: What would be the correct way for a process to put itself to sleep, and for another process to later wake it up? See ProcWaitSignal/ProcSendSignal. Or the new 'latch' code. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Kevin Grittner
Florian Pflug wrote: > On Dec5, 2010, at 16:11 , Kevin Grittner wrote: >> The simple way to implement SERIALIZABLE READ ONLY DEFERRABLE >> under SSI would be to have each non-read-only serializable >> transaction acquire a heavyweight lock which can coexist with >> other locks at the same level (S

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
marcin mank writes: > On Sun, Dec 5, 2010 at 7:28 PM, Tom Lane wrote: >> IIRC, in old discussions of this problem we first considered allowing >> clients to pull down an explicit representation of their snapshot (which >> actually is an existing feature now, txid_current_snapshot()) and then >> u

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Heikki Linnakangas
On 06.12.2010 21:48, marcin mank wrote: On Sun, Dec 5, 2010 at 7:28 PM, Tom Lane wrote: IIRC, in old discussions of this problem we first considered allowing clients to pull down an explicit representation of their snapshot (which actually is an existing feature now, txid_current_snapshot()) an

Re: [HACKERS] We really ought to do something about O_DIRECT and data=journalled on ext4

2010-12-06 Thread Tom Lane
Greg Smith writes: > Regardless, I'm now leaning heavily toward the idea of avoiding > open_datasync by default given this bug, and backpatching that change to > at least 8.4. I'll do some more database-level performance tests here > just as a final sanity check on that. My gut feel is now th

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread marcin mank
On Sun, Dec 5, 2010 at 7:28 PM, Tom Lane wrote: > IIRC, in old discussions of this problem we first considered allowing > clients to pull down an explicit representation of their snapshot (which > actually is an existing feature now, txid_current_snapshot()) and then > upload that again to become

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Florian Pflug
On Dec5, 2010, at 16:11 , Kevin Grittner wrote: > The simple way to implement SERIALIZABLE READ ONLY DEFERRABLE under > SSI would be to have each non-read-only serializable transaction > acquire a heavyweight lock which can coexist with other locks at the > same level (SHARE looks good) on some com

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 11:36 AM, Tom Lane wrote: > There's a difference between whether an extension as such is considered > to belong to a schema and whether its contained objects do. We can't > really avoid the fact that functions, operators, etc must be assigned to > some particular schema. Rig

Re: [HACKERS] Per-column collation

2010-12-06 Thread Alexandre Riveira
Please It would be very important to us that the Brazilian LIKE collate worked with, and possible case-insensitive and accent-insensitive Tank's Alexandre Riveira Brazil Peter Eisentraut escreveu: On mån, 2010-12-06 at 10:01 -0800, David E. Wheeler wrote: I've been wondering if this pa

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Josh Berkus
> Please explain more precisely what is wrong with SET SESSION > AUTHORIZATION / SET ROLE. 1) Session GUCS do not change with a SET ROLE (this is a TODO I haven't had any time to work on) 2) Users can always issue their own SET ROLE and then "hack into" other users' data. --

Re: [HACKERS] Label switcher function

2010-12-06 Thread Robert Haas
2010/11/25 KaiGai Kohei : > The attached patch is a revised one. > > It provides two hooks; the one informs core PG whether the supplied > function needs to be hooked, or not. the other is an actual hook on > prepare, start, end and abort of function invocations. > >  typedef bool (*needs_function_

Re: [HACKERS] Per-column collation

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 11:29 AM, Peter Eisentraut wrote: > This has been touch upon several times during the discussions on past > patches. > > Essentially, the current patch only arranges that you can specify a sort > order for data. The system always breaks ties using a binary > comparison. This

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Tom Lane
"David E. Wheeler" writes: > The other question I have, though, is how important is it to have extensions > live in a particular schema since there seems to be no advantage to doing so. > With the current patch, I can put extension "foo" in schema "bar", but I > can't put any other extension na

Re: [HACKERS] Per-column collation

2010-12-06 Thread Peter Eisentraut
On mån, 2010-12-06 at 10:01 -0800, David E. Wheeler wrote: > I've been wondering if this patch will support case-insensitve > collations. If so, then citext should probably be revised to use one. This has been touch upon several times during the discussions on past patches. Essentially, the curre

Re: [HACKERS] Per-column collation

2010-12-06 Thread Peter Eisentraut
On mån, 2010-12-06 at 21:06 +0900, Itagaki Takahiro wrote: > On Sun, Dec 5, 2010 at 01:04, Peter Eisentraut wrote: > > Here is an updated patch to address the issues discussed during this > > commitfest. > > Here are comments and questions after I tested the latest patch: > > Issues >

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 11:12 AM, Tom Lane wrote: > Well, I don't put any stock in the idea that it's important for existing > module .sql files to be usable as-is as extension definition files. If > it happens to fall out that way, fine, but we shouldn't give up anything > else to get that. I agree,

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Tom Lane
"David E. Wheeler" writes: > On Dec 6, 2010, at 10:43 AM, Tom Lane wrote: >> (On the other hand, if we *could* avoid using any explicit >> substitutions, it would certainly ease testing of extension files no? >> They'd be sourceable into psql then.) > Yes. And extension authors would not have to

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> Tom Lane wrote: >>> No. See subtransactions. > >> Subtransactions are included in snapshots? > > Sure, see GetSnapshotData(). You could avoid it by setting > suboverflowed, but that comes at a nontrivial performance cost. Yeah, sorry for blurti

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 10:43 AM, Tom Lane wrote: > That's an interesting idea, but I'm not sure it's wise to design around > the assumption that we won't need substitutions ever. What I was > thinking was that we should try to limit knowledge of the substitution > behavior to the extension definition

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> No. See subtransactions. > Subtransactions are included in snapshots? Sure, see GetSnapshotData(). You could avoid it by setting suboverflowed, but that comes at a nontrivial performance cost. regards, tom lane -- Sent

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Tom Lane
"David E. Wheeler" writes: > On Dec 6, 2010, at 7:19 AM, Tom Lane wrote: >> On the whole I'd prefer not to have any substitution functionality >> hard-wired into pg_execute_file either, though I can see the argument >> that it's necessary for practical use. Basically I'm concerned that >> replace

Re: [HACKERS] allow COPY routines to read arbitrary numbers of fields

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 01:23 PM, Tom Lane wrote: Andrew Dunstan writes: On 12/06/2010 12:11 PM, Tom Lane wrote: Also, why did you change the setup code to not compute nfields in binary mode? That seems at best an unnecessary change, and at worst a breakage of the binary path --- did you test it? AF

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Kevin Grittner
Tom Lane wrote: > "Kevin Grittner" writes: >> Surely you can predict that any snapshot is no larger than a fairly >> small fixed portion plus sizeof(TransactionId) * MaxBackends? > > No. See subtransactions. Subtransactions are included in snapshots? -Kevin -- Sent via pgsql-hackers mai

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >>> I'm still not convinced that using shared memory is a bad way to >>> pass these around. Surely we're not talking about large numbers >>> of them. What am I missing here? >> >> They're not of a very predictable size. > Surely you can predict that

Re: [HACKERS] allow COPY routines to read arbitrary numbers of fields

2010-12-06 Thread Tom Lane
Andrew Dunstan writes: > On 12/06/2010 12:11 PM, Tom Lane wrote: >> Also, why did you change the setup code to >> not compute nfields in binary mode? That seems at best an unnecessary >> change, and at worst a breakage of the binary path --- did you test it? > AFAICT it's not used in binary mode

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Tom Lane
"Kevin Grittner" writes: > I reviewed the documentation and played around with this a bit and > can't find any areas where the current PostgreSQL implementation of > READ ONLY is incompatible with what is needed for the SSI > optimizations where it is used. There are a large number of tests > whi

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Kevin Grittner
Tom Lane wrote: >> I'm still not convinced that using shared memory is a bad way to >> pass these around. Surely we're not talking about large numbers >> of them. What am I missing here? > > They're not of a very predictable size. Surely you can predict that any snapshot is no larger than a

Re: [HACKERS] Per-column collation

2010-12-06 Thread Pavel Stehule
2010/12/6 David E. Wheeler : > On Dec 6, 2010, at 4:06 AM, Itagaki Takahiro wrote: > >> * contrib/citext raises an encoding error when COLLATE is specified >> even if it is the collation as same as the database default. >> We might need some special treatment for C locale. > > I've been wondering i

Re: [HACKERS] allow COPY routines to read arbitrary numbers of fields

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 12:11 PM, Tom Lane wrote: Andrew Dunstan writes: Attached is a patch that allows CopyReadAttibutesText() and CopyReadAttributesCSV() to read arbitrary numbers of attributes. Underflowing attributes are recorded as null, and space is made for overflowing attributes on a line. Wh

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 12:41 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Dec 5, 2010 at 6:01 PM, Tom Lane wrote: >>> Why is there a variadic replace() in this patch at all?  It seems just >>> about entirely unrelated to the stated purpose of the patch, as well >>> as being of dubious us

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 12:57 PM, Josh Berkus wrote: >>> At some point Hackers should look at pg vs MySQL multi tenantry but it >>> is way tangential today. >> >> My understanding is that our schemas work like MySQL databases; and >> our databases are an even higher level of isolation.  No? > > Tha

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 7:19 AM, Tom Lane wrote: > On the whole I'd prefer not to have any substitution functionality > hard-wired into pg_execute_file either, though I can see the argument > that it's necessary for practical use. Basically I'm concerned that > replace-equivalent behavior is not going

Re: [HACKERS] Per-column collation

2010-12-06 Thread David E. Wheeler
On Dec 6, 2010, at 4:06 AM, Itagaki Takahiro wrote: > * contrib/citext raises an encoding error when COLLATE is specified > even if it is the collation as same as the database default. > We might need some special treatment for C locale. I've been wondering if this patch will support case-insensi

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Josh Berkus
At some point Hackers should look at pg vs MySQL multi tenantry but it is way tangential today. My understanding is that our schemas work like MySQL databases; and our databases are an even higher level of isolation. No? That's correct. Drizzle is looking at implementing a feature like our

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 12:38 PM, Tom Lane wrote: > Robert Haas writes: >> One possible way to do make an improvement in this area would be to >> move the responsibility for accepting connections out of the >> postmaster.  Instead, you'd have a group of children that would all >> call accept() on

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 12:28 PM, Tom Lane wrote: Andrew Dunstan writes: Yeah. I'm still not convinced that using shared memory is a bad way to pass these around. Surely we're not talking about large numbers of them. What am I missing here? They're not of a very predictable size. Ah. Ok. cheers

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Josh Berkus
On 12/06/2010 09:38 AM, Tom Lane wrote: Another issue that would require some thought is what algorithm the postmaster uses for deciding to spawn new children. But that doesn't sound like a potential showstopper. We'd probably want a couple of different ones, optimized for different connectio

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Tom Lane
Robert Haas writes: > On Sun, Dec 5, 2010 at 6:01 PM, Tom Lane wrote: >> Why is there a variadic replace() in this patch at all? It seems just >> about entirely unrelated to the stated purpose of the patch, as well >> as being of dubious usefulness. When would it be superior to >>replac

Re: [HACKERS] profiling connection overhead

2010-12-06 Thread Tom Lane
Robert Haas writes: > One possible way to do make an improvement in this area would be to > move the responsibility for accepting connections out of the > postmaster. Instead, you'd have a group of children that would all > call accept() on the socket, and the OS would arbitrarily pick one to > r

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
Andrew Dunstan writes: > Why not just say give me the snapshot currently held by process ? There's not a unique snapshot held by a particular process. Also, we don't want to expend the overhead to fully publish every snapshot. I think it's really necessary that the "sending" process take som

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
Andrew Dunstan writes: > Yeah. I'm still not convinced that using shared memory is a bad way to > pass these around. Surely we're not talking about large numbers of them. > What am I missing here? They're not of a very predictable size. Robert's idea of publish() returning a temp file identifi

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-06 Thread Tom Lane
Robert Haas writes: > On Sun, Dec 5, 2010 at 12:41 PM, Andrew Dunstan wrote: >> Well, ISTM that amounts to not having "official topic branches" :-) I agree >> that this is supposed to be one of git's strengths (or more exactly a >> strength of distributed SCM's generally).  I don't really see any

Re: [HACKERS] serializable read only deferrable

2010-12-06 Thread Kevin Grittner
I wrote: > Tom Lane wrote: >> I assume this would have to be a "hard" definition of READ ONLY, >> not the rather squishy definition we use now? > I'm excluding temporary tables from SSI on the grounds that they > are only read and written by a single transaction and therefore > can't be a sour

Re: [HACKERS] allow COPY routines to read arbitrary numbers of fields

2010-12-06 Thread Tom Lane
Andrew Dunstan writes: > Attached is a patch that allows CopyReadAttibutesText() and > CopyReadAttributesCSV() to read arbitrary numbers of attributes. > Underflowing attributes are recorded as null, and space is made for > overflowing attributes on a line. Why are you still passing nfields as

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Dmitriy Igrishin
2010/12/6 Andrew Chernow > On 12/6/2010 11:40 AM, Dmitriy Igrishin wrote: > >> IMO, it would be better to implement some utility functions to >> make it easy to construct arrays dynamically for PQexecParams >> and PQexecPrepared. This seems to me more universal solution >> and it is useful for bo

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Andrew Chernow
On 12/6/2010 11:40 AM, Dmitriy Igrishin wrote: IMO, it would be better to implement some utility functions to make it easy to construct arrays dynamically for PQexecParams and PQexecPrepared. This seems to me more universal solution and it is useful for both -- high level libpq-libraries authors

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Dmitriy Igrishin
IMO, it would be better to implement some utility functions to make it easy to construct arrays dynamically for PQexecParams and PQexecPrepared. This seems to me more universal solution and it is useful for both -- high level libpq-libraries authors and for those who like to use libpq directly. 20

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 10:40 AM, Tom Lane wrote: Robert Haas writes: On Mon, Dec 6, 2010 at 9:45 AM, Heikki Linnakangas wrote: Well, then you need some sort of cross-backend communication, which is always a bit clumsy. A temp file seems quite sufficient, and not at all difficult. "Not at all diff

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 10:40 AM, Tom Lane wrote: > Robert Haas writes: >> On Mon, Dec 6, 2010 at 9:45 AM, Heikki Linnakangas >> wrote: >>> Well, then you need some sort of cross-backend communication, which is >>> always a bit clumsy. > >> A temp file seems quite sufficient, and not at all diffi

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Tom Lane
Robert Haas writes: > On Mon, Dec 6, 2010 at 9:45 AM, Heikki Linnakangas > wrote: >> Well, then you need some sort of cross-backend communication, which is >> always a bit clumsy. > A temp file seems quite sufficient, and not at all difficult. "Not at all difficult" is nonsense. To do that, yo

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 10:35 AM, Andrew Dunstan wrote: > On 12/06/2010 10:22 AM, Robert Haas wrote: >> >> On Mon, Dec 6, 2010 at 9:58 AM, Heikki Linnakangas >>  wrote: >>> >>> On 06.12.2010 15:53, Robert Haas wrote: I guess.  It still seems far too much like exposing the server's guts >

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Andrew Dunstan
On 12/06/2010 10:22 AM, Robert Haas wrote: On Mon, Dec 6, 2010 at 9:58 AM, Heikki Linnakangas wrote: On 06.12.2010 15:53, Robert Haas wrote: I guess. It still seems far too much like exposing the server's guts for my taste. It might not be as bad as the expression tree stuff, but there's

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Andrew Chernow
That would be a *HUGE* piece of software compared the relatively small thing I am suggesting... Sometimes complex and large solutions are required for the simplest of ideas. I believe this is one of those cases. You can't solve the "printf style PQexec" properly by merely implementing a

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Kenneth Marshall
On Mon, Dec 06, 2010 at 10:14:55AM -0500, Merlin Moncure wrote: > On Mon, Dec 6, 2010 at 9:55 AM, Marc Balmer wrote: > > Am 06.12.10 15:37, schrieb Merlin Moncure: > >> On Sun, Dec 5, 2010 at 5:10 AM, Magnus Hagander > >> wrote: > >>> On Sun, Dec 5, 2010 at 10:22, Marc Balmer wrote: > I am

Re: [HACKERS] FK's to refer to rows in inheritance child

2010-12-06 Thread Alvaro Herrera
Excerpts from Andrew Dunstan's message of dom dic 05 14:41:20 -0300 2010: > What I would like to see is people publishing the location of > development repos so that they can be pulled from or merged, especially > for any large patch. Yes, this is pretty useful. Dimitri published his repos for

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 9:58 AM, Heikki Linnakangas wrote: > On 06.12.2010 15:53, Robert Haas wrote: >> >> I guess.  It still seems far too much like exposing the server's guts >> for my taste.  It might not be as bad as the expression tree stuff, >> but there's nothing particularly good about it e

Re: [HACKERS] Timeout for asynchronous replication Re: Timeout and wait-forever in sync rep

2010-12-06 Thread Robert Haas
On Mon, Dec 6, 2010 at 9:54 AM, Heikki Linnakangas wrote: >>> This occurred to me that the timeout would be required even for >>> asynchronous streaming replication. So, how about implementing the >>> replication timeout feature before synchronous replication itself? >> >> Here is the patch. This

Re: [HACKERS] pg_execute_from_file review

2010-12-06 Thread Tom Lane
Dimitri Fontaine writes: > Tom Lane writes: >> Why is there a variadic replace() in this patch at all? It seems just >> about entirely unrelated to the stated purpose of the patch, as well >> as being of dubious usefulness. > It used not to being exposed at the SQL level, but just an internal l

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Merlin Moncure
On Mon, Dec 6, 2010 at 9:55 AM, Marc Balmer wrote: > Am 06.12.10 15:37, schrieb Merlin Moncure: >> On Sun, Dec 5, 2010 at 5:10 AM, Magnus Hagander wrote: >>> On Sun, Dec 5, 2010 at 10:22, Marc Balmer wrote: I am suggesting adding a function to libpq: PGresult *PQvexec(PGconn *conn

Re: [HACKERS] wal_sender_delay is still required?

2010-12-06 Thread Tom Lane
Fujii Masao writes: > Walsender doesn't need the periodic wakeups anymore, thanks to > the latch feature. So wal_sender_delay is basically useless now. > How about dropping wal_sender_delay or increasing the default > value? If we don't need it, we should remove it. regar

Re: [HACKERS] WIP patch for parallel pg_dump

2010-12-06 Thread Heikki Linnakangas
On 06.12.2010 15:53, Robert Haas wrote: I guess. It still seems far too much like exposing the server's guts for my taste. It might not be as bad as the expression tree stuff, but there's nothing particularly good about it either. Note that we already have txid_current_snapshot() function, wh

Re: [HACKERS] Suggesting a libpq addition

2010-12-06 Thread Marc Balmer
Am 06.12.10 15:37, schrieb Merlin Moncure: > On Sun, Dec 5, 2010 at 5:10 AM, Magnus Hagander wrote: >> On Sun, Dec 5, 2010 at 10:22, Marc Balmer wrote: >>> I am suggesting adding a function to libpq: >>> >>> PGresult *PQvexec(PGconn *conn, const char *fmt, ...); >>> >>> It behaves similar to PQex

  1   2   >