Re: [HACKERS] Deparsing DDL command strings

2012-10-29 Thread Jim Nasby
On 10/29/12 4:30 PM, Dimitri Fontaine wrote: In some cases we may need to divert or reject DDL, but that's a >secondary concern. Reject is already in, just RAISE ERROR from the trigger code. Divert is another sell entirely, we currently miss that capability. I'm interested into it for some DDLs.

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 7:07 PM, Cédric Villemain wrote: >> But it also looks forgotten. Bringing it back to life would mean >> building the latest kernel with that patch included, replicating the >> benchmarks I ran here, sans pg patch, but with patched kernel, and >> reporting the (hopefully equ

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Cédric Villemain
> But it also looks forgotten. Bringing it back to life would mean > building the latest kernel with that patch included, replicating the > benchmarks I ran here, sans pg patch, but with patched kernel, and > reporting the (hopefully equally dramatic) performance improvements in > the kernel ML. Th

Re: [HACKERS] Deparsing DDL command strings

2012-10-29 Thread Dimitri Fontaine
Jim Nasby writes: > We need to deal with questions like "If we rename a table, what do we have > to do in londiste to accommodate that?" Except we're dealing with more than > just londiste. We have internal code that does things like track "seed > tables", which are tables that we need to dump dat

[HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-29 Thread Peter Geoghegan
On 29 October 2012 20:14, Christian Kruse wrote: > created a patch which implements MAP_HUGETLB for sysv shared memory segments > (PGSharedMemoryCreate). It is based on tests of Tom Lane and Andres Freund, I > added error handling, huge page size detection and a GUC variable. I have a few initial

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-29 Thread Andres Freund
On Monday, October 29, 2012 09:33:25 PM Tom Lane wrote: > Christian Kruse writes: > > I created a patch which implements MAP_HUGETLB for sysv shared memory > > segments (PGSharedMemoryCreate). It is based on tests of Tom Lane and > > Andres Freund, I added error handling, huge page size detection

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Jim Nasby
On 10/28/12 10:22 AM, David Lee wrote: Thanks. Is this something viable as a feature request? Possibly, but it's not terribly high on the list. In the meantime, we've built a user-space index daemon at $WORK that generally solves this issue. We intend to release it at some point, but if you n

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-29 Thread Tom Lane
Christian Kruse writes: > I created a patch which implements MAP_HUGETLB for sysv shared memory > segments > (PGSharedMemoryCreate). It is based on tests of Tom Lane and Andres Freund, I > added error handling, huge page size detection and a GUC variable. My recollection is we'd decided not to

Re: [HACKERS] Re: [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-29 Thread Tom Lane
Alvaro Herrera writes: > I think you may have converted some malloc() calls from Andres' patch > into palloc() -- because you have some palloc() calls which are later > checked for NULL results, which obviously doesn't make sense. At the > same time, if we're going to use malloc() instead of pall

Re: [HACKERS] WIP checksums patch

2012-10-29 Thread Jim Nasby
On 10/1/12 12:22 PM, Josh Berkus wrote: >Perhaps we don't allow this to be turned per page, but rather per >cluster, and per-cluster would require the entire cluster to be >rewritten. We dicussed this last year, and options which require a total rewrite of the database in order to turn on the op

[HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-29 Thread Christian Kruse
Hey, this is my first post to the -hackers lists, so be merciful ;-) I created a patch which implements MAP_HUGETLB for sysv shared memory segments (PGSharedMemoryCreate). It is based on tests of Tom Lane and Andres Freund, I added error handling, huge page size detection and a GUC variable. P

Re: [HACKERS] Deparsing DDL command strings

2012-10-29 Thread Jim Nasby
On 10/9/12 2:57 AM, Dimitri Fontaine wrote: Jim Nasby writes: I definitely want to be able to parse DDL commands to be able to either enforce things or to drive other parts of the system based on what's changing. Without the ability to capture (and parse) DDL commands I'm stuck creating wrapper

[HACKERS] Re: [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-29 Thread Alvaro Herrera
Heikki Linnakangas escribió: > Hmm. I was thinking that making this work in a non-backend context > would be too hard, so I didn't give that much thought, but I guess > there isn't many dependencies to backend functions after all. > palloc/pfree are straightforward to replace with malloc/free. Tha

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 4:17 PM, Cédric Villemain wrote: >> Ok, this is the best I could come up with, without some real test hardware. >> >> The only improvement I see in single-disk scenarios: >> * Huge speedup of back-sequential index-only scans >> * Marginal speedup on forward index-only s

Re: [HACKERS] Index creation in postgresql

2012-10-29 Thread Gavin Flower
On 29/10/12 06:22, nadeesh t v wrote: hi all, I am newbee in Postgresql.As a part of an assignment I need to create indexes for certain set of queries and evaluate its performance. Can u guys throw some light in this? Thanking in advance Reagards nanti Go to the web site and search for the

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Cédric Villemain
> Ok, this is the best I could come up with, without some real test hardware. > > The only improvement I see in single-disk scenarios: > * Huge speedup of back-sequential index-only scans > * Marginal speedup on forward index-only scans (5% or less) > * No discernible difference in heap-incl

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Pavel Stehule
2012/10/29 Tom Lane : > Hannu Krosing writes: >> Would just converting UNDEFINED to NULL be a very bad idea ? > > Yeah, it would, at least for the normal case where an undefined record > variable has no well-defined type either. Saying that it's NULL doesn't > provide an answer to what type it ha

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Pavel Stehule
2012/10/29 Hannu Krosing : > On 10/29/2012 05:36 PM, Merlin Moncure wrote: >> >> On Mon, Oct 29, 2012 at 11:26 AM, Pavel Stehule >> wrote: >>> >>> Hello >>> >>> >>> 2012/10/29 Hannu Krosing : Hi Is there a way to test for a variable being unassigned in pl/pgsql ? I'm

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Tom Lane
Hannu Krosing writes: > Would just converting UNDEFINED to NULL be a very bad idea ? Yeah, it would, at least for the normal case where an undefined record variable has no well-defined type either. Saying that it's NULL doesn't provide an answer to what type it has. In the case of OLD/NEW we do

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Hannu Krosing
On 10/29/2012 05:36 PM, Merlin Moncure wrote: On Mon, Oct 29, 2012 at 11:26 AM, Pavel Stehule wrote: Hello 2012/10/29 Hannu Krosing : Hi Is there a way to test for a variable being unassigned in pl/pgsql ? I'm writing an audit trigger where I'd like to save full before and after images int

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Merlin Moncure
On Mon, Oct 29, 2012 at 11:26 AM, Pavel Stehule wrote: > Hello > > > 2012/10/29 Hannu Krosing : >> Hi >> >> Is there a way to test for a variable being unassigned in pl/pgsql ? >> >> I'm writing an audit trigger where I'd like to save full before and >> after images into audit log and I really do

Re: [HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Pavel Stehule
Hello 2012/10/29 Hannu Krosing : > Hi > > Is there a way to test for a variable being unassigned in pl/pgsql ? > > I'm writing an audit trigger where I'd like to save full before and > after images into audit log and I really do not like to do > > IF TG_OP IN ('INSERT', 'UPDATE') ... > > I'd like

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Mon, Oct 29, 2012 at 12:53 PM, Claudio Freire wrote: >> Yes, I've seen that, though I thought it was only an improvement on >> PrefetchBuffer. That patch would interact quite nicely with mine. >> >> I'm now trying to prefetch heap tuples, and I got to a really nice >> place where I get an extra

Re: [HACKERS] [PATCH] Prefetch index pages for B-Tree index scans

2012-10-29 Thread Claudio Freire
On Tue, Oct 23, 2012 at 10:54 AM, Claudio Freire wrote: >> Indeed not. I also looked into prefetching for pure index scans for >> b-trees (and extension to use async io). >> http://archives.postgresql.org/message-id/BLU0-SMTP31709961D846CCF4F5EB4C2A3930%40phx.gbl > > Yes, I've seen that, though

[HACKERS] Is there a way to test for UNASSIGNED in pl/pgsql

2012-10-29 Thread Hannu Krosing
Hi Is there a way to test for a variable being unassigned in pl/pgsql ? I'm writing an audit trigger where I'd like to save full before and after images into audit log and I really do not like to do IF TG_OP IN ('INSERT', 'UPDATE') ... I'd like rather better if i could just write IF NEW IS AS

Re: [HACKERS] September 2012 commitfest

2012-10-29 Thread Alvaro Herrera
The September 2012 commitfest has been running now for six weeks, and we've been making some progress. That stalled a bit last week due to the European conference; hopefully there are now some renewed energies to let us finish it up soon. One thing I do *not* want to do is send many patches to th

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-10-29 Thread Amit Kapila
On Saturday, October 27, 2012 11:34 PM Noah Misch > On Sat, Oct 27, 2012 at 04:57:46PM +0530, Amit Kapila wrote: > > On Saturday, October 27, 2012 4:03 AM Noah Misch wrote: > > > Could you elaborate on your reason for continuing to treat TOAST as > a > > > special > > > case? As best I recall, the

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Alvaro Herrera
David Lee escribió: > Thanks for all the responses. > > I forgot to ask in my initial post: If not already available, is background > indexing a viable feature request? (This was why I sent to pgsql-hackers). > > My use case is a multi-tenant CMS where indexes can be created by a web > front-en

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-10-29 Thread Amit Kapila
On Sunday, October 28, 2012 12:28 AM Heikki Linnakangas wrote: > On 27.10.2012 14:27, Amit Kapila wrote: > > On Saturday, October 27, 2012 4:03 AM Noah Misch wrote: > >> In my previous review, I said: > >> > >>Given [not relying on the executor to know which columns changed], > >> why not > > >

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Amit Kapila
On Monday, October 29, 2012 7:11 PM Chris Corbyn > What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto file. Assuming only super users/sysadmins would have the abili

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Amit Kapila
On Monday, October 29, 2012 7:06 PM Merlin Moncure wrote: > On Mon, Oct 29, 2012 at 8:31 AM, Amit Kapila > wrote: > > SYNTAX: > > > > ALTER SYSTEM SET configuration_parameter = value COMMENT 'value'; > > > > > Comments/Suggestions about the value of this feature and > Implementation > > Idea? >

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Chris Corbyn
What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto file. Assuming only super users/sysadmins would have the ability to change things in the config file, wouldn't they

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Merlin Moncure
On Mon, Oct 29, 2012 at 8:31 AM, Amit Kapila wrote: > SYNTAX: > > ALTER SYSTEM SET configuration_parameter = value COMMENT 'value'; > > Comments/Suggestions about the value of this feature and Implementation > Idea? How is this better than simply updating them through the pg_settings view (in SQ

[HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-29 Thread Amit Kapila
SYNTAX: ALTER SYSTEM SET configuration_parameter = value COMMENT 'value'; DESIGN IDEA: (a) have a postgresql.conf.auto (b) add a default include for postgresql.conf.auto at the beginning of PostgreSQL.conf (c) SQL updates go to postgresql.conf.auto, which consists only of"setting = val

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread David Johnston
On Oct 27, 2012, at 19:20, David Lee wrote: > Hey folks, > > It seems like right now when you want to create an index concurrently, the > index creation will get canceled if you cancel the statement (i.e. you must > keep your statement open). > > Is there a way to "launch" an index creation i

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Jeff Janes
On Sat, Oct 27, 2012 at 4:20 PM, David Lee wrote: > Hey folks, > > It seems like right now when you want to create an index concurrently, the > index creation will get canceled if you cancel the statement (i.e. you must > keep your statement open). > > Is there a way to "launch" an index creatio

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Michael Paquier
On Sun, Oct 28, 2012 at 8:20 AM, David Lee wrote: > Hey folks, > > It seems like right now when you want to create an index concurrently, the > index creation will get canceled if you cancel the statement (i.e. you must > keep your statement open). > > Is there a way to "launch" an index creation

Re: [HACKERS] Index creation in postgresql

2012-10-29 Thread Kevin Grittner
nadeesh t v wrote: > I am newbee in Postgresql.As a part of an assignment I need to > create indexes for certain set of queries and evaluate its > performance. > Can u guys throw some light in this? This is the wrong list for this question. I recommend you read up on indexes and make some attemp

[HACKERS] Index creation in postgresql

2012-10-29 Thread nadeesh t v
hi all, I am newbee in Postgresql.As a part of an assignment I need to create indexes for certain set of queries and evaluate its performance. Can u guys throw some light in this? Thanking in advance Reagards nanti

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Jaime Casanova
On Sat, Oct 27, 2012 at 6:20 PM, David Lee wrote: > Hey folks, > > It seems like right now when you want to create an index concurrently, the > index creation will get canceled if you cancel the statement (i.e. you must > keep your statement open). > > Is there a way to "launch" an index creatio

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Kevin Grittner
David Lee wrote: > It seems like right now when you want to create an index > concurrently, the index creation will get canceled if you cancel > the statement (i.e. you must keep your statement open). > > Is there a way to "launch" an index creation in the background so > that the statement doesn

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread David Lee
Thanks for all the responses. I forgot to ask in my initial post: If not already available, is background indexing a viable feature request? (This was why I sent to pgsql-hackers). My use case is a multi-tenant CMS where indexes can be created by a web front-end. Since web requests should retur

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread Robert Haas
On Sat, Oct 27, 2012 at 7:20 PM, David Lee wrote: > It seems like right now when you want to create an index concurrently, the > index creation will get canceled if you cancel the statement (i.e. you must > keep your statement open). > > Is there a way to "launch" an index creation in the backgr

Re: [HACKERS] Creating indexes in the background

2012-10-29 Thread David Lee
Thanks. Is this something viable as a feature request? On Oct 28, 2012 7:48 AM, "David Johnston" wrote: > On Oct 27, 2012, at 19:20, David Lee wrote: > > > Hey folks, > > > > It seems like right now when you want to create an index concurrently, > the index creation will get canceled if you canc

Re: [HACKERS] foreign key locks

2012-10-29 Thread Kevin Grittner
Andres Freund wrote: > On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: >> Andres Freund wrote: >>> * Is it ok to make FOR UPDATE somewhat weaker than before? In 9.2 >>> and earlier you could be sure that if you FOR UPDATE'ed a row you >>> could delete it. Unless I miss something now th

Re: [HACKERS] foreign key locks

2012-10-29 Thread Simon Riggs
On 29 October 2012 12:27, Andres Freund wrote: >> This sounds like we need a GUC or table-level default to control >> whether FOR UPDATE means FOR UPDATE or FOR DELETE > > I don't like adding a new guc for something that should be solveable with some > creative naming. If a new user doesn't get a

Re: [HACKERS] foreign key locks

2012-10-29 Thread Andres Freund
On Sunday, October 28, 2012 11:47:16 PM Simon Riggs wrote: > On 27 October 2012 00:06, Andres Freund wrote: > > On Thursday, October 18, 2012 09:58:20 PM Alvaro Herrera wrote: > >> Here is version 22 of this patch. This version contains fixes to issues > >> reported by Andres, as well as a rebase

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-10-29 Thread Robert Haas
On Wed, Oct 24, 2012 at 4:20 PM, Jan Wieck wrote: > This patch does introduce three new postgresql.conf parameters, which I > would be happy to get rid of if we could derive them from something else. > Something based on the deadlock timeout may be possible. > > autovacuum_truncate_lock_check

Re: [HACKERS] Logical to physical page mapping

2012-10-29 Thread Robert Haas
On Sat, Oct 27, 2012 at 1:01 AM, Jan Wieck wrote: > The reason why we need full_page_writes is that we need to guard against > torn pages or partial writes. So what if smgr would manage a mapping between > logical page numbers and their physical location in the relation? This sounds a lot like ht