Re: [HACKERS] 9.4 regression

2013-08-08 Thread Andres Freund
On 2013-08-08 22:58:42 -0500, Jon Nelson wrote: > On Thu, Aug 8, 2013 at 9:27 PM, Andres Freund wrote: > > On 2013-08-08 16:12:06 -0500, Jon Nelson wrote: > ... > > >> At this point I'm convinced that the issue is a pathological case in > >> ext4. The performance impact disappears as soon as the

Re: [HACKERS] pg_dump and schema names

2013-08-08 Thread Tom Lane
Bruce Momjian writes: > pg_dump goes to great lengths not to hard-code the schema name into > commands like CREATE TABLE, instead setting the search_path before > creating the table; these commands: > CREATE SCHEMA xx; > CREATE TABLE xx.test(x int); > generates this output: >

Re: [HACKERS] pg_dump and schema names

2013-08-08 Thread James Sewell
I was under the impression that every command that references a relation makes use of the search path, regardless of what it is *doing*. Maybe this is different in older versions though? I actually ran into this recently and had to remove all the xx. schema components using vi before I could run t

[HACKERS] pg_dump and schema names

2013-08-08 Thread Bruce Momjian
pg_dump goes to great lengths not to hard-code the schema name into commands like CREATE TABLE, instead setting the search_path before creating the table; these commands: CREATE SCHEMA xx; CREATE TABLE xx.test(x int); generates this output: SET search_path = xx, pg_catal

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 9:27 PM, Andres Freund wrote: > On 2013-08-08 16:12:06 -0500, Jon Nelson wrote: ... >> At this point I'm convinced that the issue is a pathological case in >> ext4. The performance impact disappears as soon as the unwritten >> extent(s) are written to with real data. Thus,

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Tomonari Katsumata
Hi, I understood it's too late to change the feature. I hope it will be revised in 9.4! (2013/08/09 4:13), Josh Berkus wrote: > On 08/08/2013 11:01 AM, Andres Freund wrote: > >> I don't think anybody working on related areas of the code thinks it's >> rock solid. >> But anyway, I just don't see

Re: [HACKERS] Proposal for XML Schema Validation

2013-08-08 Thread Craig Ringer
On 08/09/2013 12:39 AM, Kodamasimham Pridhvi (MT2012066) wrote: > > Objective: To Add XML Schema validation and xmlvalidate functions (SQL:2008) > > Description: > We’ve gone through current support of xml in postgreSQL and found that t

[HACKERS] Proposal: leave a hint when switching logging away from stderr

2013-08-08 Thread Tom Lane
The attached patch is motivated by http://www.postgresql.org/message-id/cajyqwwryt9rmbzs-sh6ucr1otg4joxqkdf-fkoyp6pv12t0...@mail.gmail.com in which it appears that Oliver Elphick forgot to look in the configured log_directory directory for log output, and instead examined only the file that postmas

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Andres Freund
On 2013-08-08 16:12:06 -0500, Jon Nelson wrote: > On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing wrote: > > On 08/08/2013 05:28 PM, Jon Nelson wrote: > ... > > > Just an idea - can you check if using a fillfactor different form 100 > > changes anything > > > > pgbench -s 20 -p 54320 -d pgb -F 90 -

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 5:25 PM, Jon Nelson wrote: > On Thu, Aug 8, 2013 at 5:23 PM, Tom Lane wrote: >> Jon Nelson writes: >>> On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane wrote: Does your test program use all the same writing options that the real WAL writes do (like O_DIRECT)? >> >>> I b

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 5:23 PM, Tom Lane wrote: > Jon Nelson writes: >> On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane wrote: >>> Does your test program use all the same writing options that the real >>> WAL writes do (like O_DIRECT)? > >> I believe so. > >>> From xlog.c: > >> /* do not use get_sy

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Tom Lane
Jon Nelson writes: > On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane wrote: >> Does your test program use all the same writing options that the real >> WAL writes do (like O_DIRECT)? > I believe so. >> From xlog.c: > /* do not use get_sync_bit() here --- want to fsync only at end of fill */ >

Re: [HACKERS] [PATCH] Statistics collection for CLUSTER command

2013-08-08 Thread Vik Fearing
On 08/08/2013 07:57 PM, Stefan Kaltenbrunner wrote: > On 08/08/2013 01:52 PM, Vik Fearing wrote: >> I would add this to the next commitfest but I seem to be unable to log >> in with my community account (I can log in to the wiki). Help appreciated. > whould be a bit easier to diagnose if we knew

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 4:42 PM, Tom Lane wrote: > Jon Nelson writes: >> At this point I'm convinced that the issue is a pathological case in >> ext4. The performance impact disappears as soon as the unwritten >> extent(s) are written to with real data. Thus, even though allocating >> files with p

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Bruce Momjian
On Thu, Aug 8, 2013 at 04:33:05PM -0500, Jon Nelson wrote: > > How much slower would it be if we wrote it with zeros after > > posix_fallocate() --- that would still give use single extents. Has > > anyone tested to see if the write without test_fallocate() still gives > > us one extent? > > Act

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Tom Lane
Jon Nelson writes: > At this point I'm convinced that the issue is a pathological case in > ext4. The performance impact disappears as soon as the unwritten > extent(s) are written to with real data. Thus, even though allocating > files with posix_fallocate is - frequently - orders of magnitude >

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 4:24 PM, Bruce Momjian wrote: > On Thu, Aug 8, 2013 at 04:12:06PM -0500, Jon Nelson wrote: >> On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing wrote: >> > On 08/08/2013 05:28 PM, Jon Nelson wrote: >> ... >> >> > Just an idea - can you check if using a fillfactor different for

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Bruce Momjian
On Thu, Aug 8, 2013 at 04:12:06PM -0500, Jon Nelson wrote: > On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing wrote: > > On 08/08/2013 05:28 PM, Jon Nelson wrote: > ... > > > Just an idea - can you check if using a fillfactor different form 100 > > changes anything > > > > pgbench -s 20 -p 54320 -d

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
On Thu, Aug 8, 2013 at 2:50 PM, Hannu Krosing wrote: > On 08/08/2013 05:28 PM, Jon Nelson wrote: ... > Just an idea - can you check if using a fillfactor different form 100 > changes anything > > pgbench -s 20 -p 54320 -d pgb -F 90 -i > > >> pgbench -j 80 -c 80 -T 120 -p 54320 pgb >> pg_ctl -D tt

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Hannu Krosing
On 08/08/2013 05:28 PM, Jon Nelson wrote: > A follow-up. > I found this thread that seems to explain some things: > > http://comments.gmane.org/gmane.comp.file-systems.ext4/33024 > > Short version: if we are writing into the "middle" of the > newly-fallocated file on ext4 (with extents) the extent

Re: [HACKERS] question about HTTP API

2013-08-08 Thread Josh Berkus
On 08/08/2013 11:44 AM, Szymon Guz wrote: > Do we have any attempts of implementation the HTTP server described at > http://wiki.postgresql.org/wiki/HTTP_API? > > It seems like there are design ideas only. Are there any ideas about > implementation like using some existing http servers or writing

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Josh Berkus
On 08/08/2013 11:01 AM, Andres Freund wrote: > I don't think anybody working on related areas of the code thinks it's > rock solid. > But anyway, I just don't see the downside of allowing problem > analysis. You're free to do more testing, review, whatever before the > release. I'm 100% with you

Re: [HACKERS] StrategyGetBuffer optimization, take 2

2013-08-08 Thread Merlin Moncure
On Wed, Aug 7, 2013 at 11:52 PM, Amit Kapila wrote: >> -Original Message- >> From: pgsql-hackers-ow...@postgresql.org [mailto:pgsql-hackers- >> ow...@postgresql.org] On Behalf Of Merlin Moncure >> Sent: Thursday, August 08, 2013 12:09 AM >> To: Andres Freund >> Cc: PostgreSQL-development;

[HACKERS] question about HTTP API

2013-08-08 Thread Szymon Guz
Do we have any attempts of implementation the HTTP server described at http://wiki.postgresql.org/wiki/HTTP_API? It seems like there are design ideas only. Are there any ideas about implementation like using some existing http servers or writing everything from scratch? regards Szymon

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Andres Freund
On 2013-08-08 10:51:45 -0700, Josh Berkus wrote: > On 08/08/2013 10:34 AM, Andres Freund wrote: > > On 2013-08-08 10:15:14 -0700, Josh Berkus wrote: > >> Either we have confidence is fast promotion, or we don't. If we don't > >> have confidence, then either (a) more testing is needed, or (b) it >

Re: [HACKERS] [PATCH] Statistics collection for CLUSTER command

2013-08-08 Thread Stefan Kaltenbrunner
On 08/08/2013 01:52 PM, Vik Fearing wrote: > As part of routine maintenance monitoring, it is interesting for us to > have statistics on the CLUSTER command (timestamp of last run, and > number of runs since stat reset) like we have for (auto)ANALYZE and > (auto)VACUUM. Patch against today's HEAD

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Josh Berkus
On 08/08/2013 10:34 AM, Andres Freund wrote: > On 2013-08-08 10:15:14 -0700, Josh Berkus wrote: >> Either we have confidence is fast promotion, or we don't. If we don't >> have confidence, then either (a) more testing is needed, or (b) it >> shouldn't be the default. Again, here, we are coming up

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Andres Freund
On 2013-08-08 10:15:14 -0700, Josh Berkus wrote: > Bruce, all: > > > We seem to be all over the map with the fast promotion code --- some > > people don't trust it, some people want an option to enable the old > > method, and some people want the old method removed. > > Having read over this thre

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-08-08 Thread Andres Freund
On 2013-08-08 09:27:24 -0400, Robert Haas wrote: > On Tue, Aug 6, 2013 at 3:06 AM, Andres Freund wrote: > > The reason for that is that when we do the AtEO(Sub)?Xact_Inval(), we've > > already done a RecordTransactionAbort(true|false) and > > CurrentTransactionState->state = TRANS_ABORT. So the vi

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Josh Berkus
Bruce, all: > We seem to be all over the map with the fast promotion code --- some > people don't trust it, some people want an option to enable the old > method, and some people want the old method removed. Having read over this thread, the only reason given for retaining any ability to use "old

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Andres Freund
On 2013-08-08 12:50:31 -0400, Bruce Momjian wrote: > On Thu, Aug 8, 2013 at 01:27:35PM +0900, Michael Paquier wrote: > > > Why are we suddenly trying to make this even more complicated? It's too > > > late to redesign stuff without very good evidence that it's > > > needed. Renaming trigger files

Re: [HACKERS] Should we remove "not fast" promotion at all?

2013-08-08 Thread Bruce Momjian
On Thu, Aug 8, 2013 at 01:27:35PM +0900, Michael Paquier wrote: > > Why are we suddenly trying to make this even more complicated? It's too > > late to redesign stuff without very good evidence that it's > > needed. Renaming trigger files and changing their format certainly > > doesn't seem approp

Re: [HACKERS] Proposal for XML Schema Validation

2013-08-08 Thread Kodamasimham Pridhvi (MT2012066)
Objective: To Add XML Schema validation and xmlvalidate functions (SQL:2008) Description: We’ve gone through current support of xml in postgreSQL and found that there is a check for well-formedness of xml document while inserting and u

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Thom Brown
On 8 August 2013 04:05, Andres Freund wrote: > On 2013-08-07 20:23:55 +0100, Thom Brown wrote: >> >>> 269e78 was the commit immediately after 8800d8, so it appears that >> >>> introduced the regression. >> >>> >> >>> "Use posix_fallocate() for new WAL files, where available." >> >> >> >> This is c

Re: [HACKERS] 9.4 regression

2013-08-08 Thread Jon Nelson
A follow-up. I found this thread that seems to explain some things: http://comments.gmane.org/gmane.comp.file-systems.ext4/33024 Short version: if we are writing into the "middle" of the newly-fallocated file on ext4 (with extents) the extent tree can fragment badly, causing poor performance due

Re: [HACKERS] Patch for reserved connections for replication users

2013-08-08 Thread Robert Haas
On Mon, Aug 5, 2013 at 2:04 AM, Andres Freund wrote: >> Hmm. It seems like this match is making MaxConnections no longer mean >> the maximum number of connections, but rather the maximum number of >> non-replication connections. I don't think I support that >> definitional change, and I'm kinda

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-08 Thread Robert Haas
On Sun, Aug 4, 2013 at 4:26 PM, Dimitri Fontaine wrote: > Alvaro Herrera writes: >> I remind you that event triggers are not fired for global objects >> such as databases and roles. Do you intend to lift that restriction? > > That's not on my TODO list for 9.4. My understanding about implementin

Re: [HACKERS] mvcc catalo gsnapshots and TopTransactionContext

2013-08-08 Thread Robert Haas
On Tue, Aug 6, 2013 at 3:06 AM, Andres Freund wrote: > The reason for that is that when we do the AtEO(Sub)?Xact_Inval(), we've > already done a RecordTransactionAbort(true|false) and > CurrentTransactionState->state = TRANS_ABORT. So the visibility routines > have enough information to consider r

Re: [HACKERS] Proposal for XML Schema Validation

2013-08-08 Thread Andrew Dunstan
On 08/08/2013 12:42 AM, Kodamasimham Pridhvi (MT2012066) wrote: Hello pgsql-hackers , With reference to "Add XML Schema validation and xmlvalidate functions (SQL:2008)" in ToDo list, we have gone through pgsql-mailing list but we didn't find any significant work in this area, so we are

Re: [HACKERS] [PATCH] Statistics collection for CLUSTER command

2013-08-08 Thread Fabien COELHO
As part of routine maintenance monitoring, it is interesting for us to have statistics on the CLUSTER command (timestamp of last run, and number of runs since stat reset) like we have for (auto)ANALYZE and (auto)VACUUM. Patch against today's HEAD attached. I would add this to the next commitfe

Re: [HACKERS] [PATCH] pg_sleep(interval)

2013-08-08 Thread Fabien COELHO
Usage: SELECT pg_sleep(interval '2 minutes'); I would add this to the next commitfest but I seem to be unable to log in with my community account (I can log in to the wiki). Help appreciated. Done. -- Fabien. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make ch

[HACKERS] [PATCH] Statistics collection for CLUSTER command

2013-08-08 Thread Vik Fearing
As part of routine maintenance monitoring, it is interesting for us to have statistics on the CLUSTER command (timestamp of last run, and number of runs since stat reset) like we have for (auto)ANALYZE and (auto)VACUUM. Patch against today's HEAD attached. I would add this to the next commitfest

[HACKERS] [PATCH] pg_sleep(interval)

2013-08-08 Thread Vik Fearing
Someone on IRC a while ago was complaining that there was no way to specify an interval for pg_sleep, so I made one. Patch against today's HEAD attached. Usage: SELECT pg_sleep(interval '2 minutes'); I would add this to the next commitfest but I seem to be unable to log in with my community acco

Re: [HACKERS] Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-08-08 Thread Amit Kapila
On Monday, August 05, 2013 10:12 PM Josh Berkus wrote: > All, > > To limit the argument here, let's please not argue about things which > people already agree on. So: > > We seem to have consensus around: > > A) the inadvisability of storing GUCs in a system catalog. > > B) the utility of a co

Re: [HACKERS] MultiXactId concept underdocumented

2013-08-08 Thread Dimitri Fontaine
Peter Eisentraut writes: > The new MultiXactId concept appears in a number of user-facing error > messages, including in the scary context of transaction ID wraparound, > so it seems kind of important, but it doesn't appear to be documented in > any user-facing places. Should this be rectified?