Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 02:52 AM, Peter Geoghegan wrote: Having been surprisingly successful at advancing our understanding of arguments for and against various approaches to "value locking", I decided to try the same thing out elsewhere. I have created a general-purpose UPSERT wiki page. The page is: htt

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 03:20 AM, Kevin Grittner wrote: My only concern from the benchmarks is that it seemed like there was a statistically significant increase in planning time: unpatched plan time average: 0.450 ms patched plan time average: 0.536 ms That *might* just be noise, but it seems likely t

Re: [HACKERS] bad estimation together with large work_mem generates terrible slow hash joins

2014-10-02 Thread Tomas Vondra
Dne 2 Říjen 2014, 2:20, Kevin Grittner napsal(a): > Tomas Vondra wrote: >> On 12.9.2014 23:22, Robert Haas wrote: > >>> My first thought is to revert to NTUP_PER_BUCKET=1, but it's >>> certainly arguable. Either method, though, figures to be better than >>> doing nothing, so let's do something. >>

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:07 AM, Heikki Linnakangas wrote: > Could you write down all of the discussed syntaxes, using a similar notation > we use in the manual, with examples on how to use them? And some examples on > what is possible with some syntaxes, and not with others? That would make it >

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Kyotaro HORIGUCHI
Hello, > > I propose the attached patch. It adds a new flag ImmediateDieOK, which is a > > weaker form of ImmediateInterruptOK that only allows handling a pending > > die-signal in the signal handler. > > > > Robert, others, do you see a problem with this? > > Per se I don't have a problem with

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Heikki Linnakangas
On 09/23/2014 09:24 PM, Andres Freund wrote: I've previously started two threads about replication identifiers. Check http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de and http://archives.postgresql.org/message-id/20131211153833.GB25227%40awork2.anarazel.de . Th

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Andres Freund
On 2014-10-02 17:47:39 +0900, Kyotaro HORIGUCHI wrote: > Hello, > > > > I propose the attached patch. It adds a new flag ImmediateDieOK, which is > > > a > > > weaker form of ImmediateInterruptOK that only allows handling a pending > > > die-signal in the signal handler. > > > > > > Robert, othe

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Andres Freund
On 2014-10-02 11:49:31 +0300, Heikki Linnakangas wrote: > On 09/23/2014 09:24 PM, Andres Freund wrote: > >I've previously started two threads about replication identifiers. Check > >http://archives.postgresql.org/message-id/20131114172632.GE7522%40alap2.anarazel.de > >and > >http://archives.postgre

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Andres Freund
On 2014-10-01 18:19:05 +0200, Ilya Kosmodemiansky wrote: > I have a patch which is actually not commitfest-ready now, but it > always better to start discussing proof of concept having some patch > instead of just an idea. That's a good way to start work on a topic like this. > From an Oracle DBA

Re: [HACKERS] Escaping from blocked send() reprised.

2014-10-02 Thread Kyotaro HORIGUCHI
Hi, > > > But this is imo prohibitive. Yes, we're doing it for a long while. But > > > no, that's not ok. It actually prompoted me into prototyping the latch > > > thing (in some other thread). I don't think existing practice justifies > > > expanding it further. > > > > I see, in that case, this

Re: [HACKERS] pgbench throttling latency limit

2014-10-02 Thread Heikki Linnakangas
On 09/15/2014 08:46 PM, Fabien COELHO wrote: I'm not sure I like the idea of printing a percentage. It might be unclear what the denominator was if somebody feels the urge to work back to the actual number of skipped transactions. I mean, I guess it's probably just the value you passed to -R,

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Ilya Kosmodemiansky
On Thu, Oct 2, 2014 at 11:50 AM, Andres Freund wrote: > Not just from a oracle DBA POV ;). Generally. sure >> Saying that, principally they mean an >> Oracle Wait Interface analogue. The Basic idea is to have counters or >> sensors all around database kernel to measure what a particular >> backe

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Ilya Kosmodemiansky
On Thu, Oct 2, 2014 at 5:25 AM, Craig Ringer wrote: > It's not at all clear to me that a DTrace-like (or perf-based, rather) > approach is unsafe, slow, or unsuitable for production use. > With appropriate wrapper tools I think we could have quite a useful > library of perf-based diagnostics and

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Heikki Linnakangas
On 10/01/2014 07:28 PM, Thomas Munro wrote: Hi SQL:2003 introduced the function NEXT VALUE FOR . Google tells me that at least DB2, SQL Server and a few niche databases understand it so far. As far as I can tell there is no standardised equivalent of currval and setval (but I only have access t

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-02 Thread Heikki Linnakangas
I looked at this briefly, and was surprised that there is no support for signing a message without encrypting it. Is that intentional? Instead of adding a function to encrypt and sign a message, I would have expected this to just add a new function for signing, and you could then pass it an alr

[HACKERS] proposal: doc: simplify examples of dynamic SQL

2014-10-02 Thread Pavel Stehule
Hi There are few less readable examples of dynamic SQL in plpgsql doc like: EXECUTE 'SELECT count(*) FROM ' || tabname::regclass || ' WHERE inserted_by = $1 AND inserted <= $2' INTO c USING checked_user, checked_date; or EXECUTE 'UPDATE tbl SET ' || quote_ident(colname)

Re: [HACKERS] pgcrypto: PGP signatures

2014-10-02 Thread Marko Tiikkaja
On 10/2/14 1:47 PM, Heikki Linnakangas wrote: I looked at this briefly, and was surprised that there is no support for signing a message without encrypting it. Is that intentional? Instead of adding a function to encrypt and sign a message, I would have expected this to just add a new function fo

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > > The patch https://commitfest.postgresql.org/action/patch_view?id=885 > > (discussion starts here I hope - > > http://www.postgresql.org/message-id/4fe8ca2c.3030...@uptime.jp) > > demonstrates performance problems; LWLOCK_STAT, LOCK_DEBUG and > > DT

Re: [HACKERS] Dynamic LWLock tracing via pg_stat_lwlock (proof of concept)

2014-10-02 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > > 1. I've decided to put pg_stat_lwlock into extension pg_stat_lwlock > > (simply for test purposes). Is it OK, or better to implement it > > somewhere inside pg_catalog or in another extension (for example > > pg_stat_statements)? > > I personally

Re: [HACKERS] Replication identifiers, take 3

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 4:49 AM, Heikki Linnakangas wrote: > An origin column in the table itself helps tremendously to debug issues with > the replication system. In many if not most scenarios, I think you'd want to > have that extra column, even if it's not strictly required. I like a lot of wha

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-02 Thread Robert Haas
On Wed, Oct 1, 2014 at 4:56 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Mon, Sep 29, 2014 at 12:05 PM, Stephen Frost wrote: >> > Perhaps I'm just being a bit over the top, but all this per-character >> > work feels a bit ridiculous.. When we're using MAXIMUM_ALI

Re: [HACKERS] pg_background (and more parallelism infrastructure patches)

2014-10-02 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Oct 1, 2014 at 4:56 PM, Stephen Frost wrote: > > Was just thinking that we might be able to work out what needs to be > > done without having to actually do it on a per-character basis. That > > said, I'm not sure it's really worth t

[HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread Michael Banck
Hi, we have seen repeatedly that users can be confused about why PostgreSQL is not shutting down even though they requested it. Usually, this is because `log_checkpoints' is not enabled and the final checkpoint is being written, delaying shutdown. As no message besides "shutting down" is written

Re: [HACKERS] Time measurement format - more human readable

2014-10-02 Thread Bogdan Pilch
> On 9/29/14, 1:08 AM, Andres Freund wrote: > >On 2014-09-28 20:32:30 -0400, Gregory Smith wrote: > >>There are already a wide range of human readable time interval output > >>formats available in the database; see the list at > >>http://www.postgresql.org/docs/current/static/datatype-datetime.htm

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Tom Lane
Thomas Munro writes: > SQL:2003 introduced the function NEXT VALUE FOR . Google > tells me that at least DB2, SQL Server and a few niche databases > understand it so far. As far as I can tell there is no standardised > equivalent of currval and setval (but I only have access to second > hand info

Re: [HACKERS] WITH CHECK and Column-Level Privileges

2014-10-02 Thread Stephen Frost
* Dean Rasheed (dean.a.rash...@gmail.com) wrote: > On 30 September 2014 20:17, Stephen Frost wrote: > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > >> One of the main things that detail is useful for is identifying the > >> failing row in a multi-row update. In most real-world cases, I woul

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera wrote: > Alvaro Herrera wrote: >> So in essence what we're going to do is that the balance mechanism >> considers only tables that don't have per-table configuration options; >> for those that do, we will use the values configured there without any >>

[HACKERS] "port/atomics/arch-*.h" are missing from installation

2014-10-02 Thread Kohei KaiGai
I got the following error when I try to build my extension towards the latest master branch. Is the "port/atomics/*.h" files forgotten on make install? [kaigai@magro pg_strom]$ make gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-at

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-09-26 10:28:21 -0400, Robert Haas wrote: > On Fri, Sep 26, 2014 at 8:55 AM, Oskari Saarenmaa wrote: > >> So you think a read barrier is the same thing as an acquire barrier > >> and a write barrier is the same as a release barrier? That would be > >> surprising. It's certainly not true i

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-09-25 10:42:29 -0400, Robert Haas wrote: > On Thu, Sep 25, 2014 at 10:24 AM, Andres Freund > wrote: > > On 2014-09-25 10:22:47 -0400, Robert Haas wrote: > >> On Thu, Sep 25, 2014 at 10:14 AM, Andres Freund > >> wrote: > >> > That leads me to wonder: Have you measured different, lower, n

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund wrote: >> OK. > > Given that the results look good, do you plan to push this? By "this", you mean the increase in the number of buffer mapping partitions to 128, and a corresponding increase in MAX_SIMUL_LWLOCKS? If so, and if you don't have any res

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 10:40:30 -0400, Robert Haas wrote: > On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund wrote: > >> OK. > > > > Given that the results look good, do you plan to push this? > > By "this", you mean the increase in the number of buffer mapping > partitions to 128, and a corresponding incre

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:34 AM, Andres Freund wrote: > It's actually more complex than that :( > > Simple things first: > > Oracle's definition seems pretty iron clad: > http://docs.oracle.com/cd/E18659_01/html/821-1383/gjzmf.html > __machine_acq_barrier is a clear superset of __machine_r_barrier

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 10:44 AM, Andres Freund wrote: > On 2014-10-02 10:40:30 -0400, Robert Haas wrote: >> On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund >> wrote: >> >> OK. >> > >> > Given that the results look good, do you plan to push this? >> >> By "this", you mean the increase in the numbe

Re: [HACKERS] "port/atomics/arch-*.h" are missing from installation

2014-10-02 Thread Andres Freund
Hi, On 2014-10-02 23:33:36 +0900, Kohei KaiGai wrote: > I got the following error when I try to build my extension > towards the latest master branch. > > Is the "port/atomics/*.h" files forgotten on make install? You're right. > The attached patch is probably right remedy. I've changed the or

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Thu, Oct 2, 2014 at 9:54 AM, Alvaro Herrera > wrote: > > Alvaro Herrera wrote: > >> So in essence what we're going to do is that the balance mechanism > >> considers only tables that don't have per-table configuration options; > >> for those that

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread David G Johnston
Michael Banck-2 wrote > Hi, > > we have seen repeatedly that users can be confused about why PostgreSQL > is not shutting down even though they requested it. Usually, this is > because `log_checkpoints' is not enabled and the final checkpoint is > being written, delaying shutdown. As no message b

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-10-02 10:55:06 -0400, Robert Haas wrote: > On Thu, Oct 2, 2014 at 10:34 AM, Andres Freund wrote: > > It's actually more complex than that :( > > > > Simple things first: > > > > Oracle's definition seems pretty iron clad: > > http://docs.oracle.com/cd/E18659_01/html/821-1383/gjzmf.html > >

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 10:56:05 -0400, Robert Haas wrote: > On Thu, Oct 2, 2014 at 10:44 AM, Andres Freund wrote: > > On 2014-10-02 10:40:30 -0400, Robert Haas wrote: > >> On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund > >> wrote: > >> >> OK. > >> > > >> > Given that the results look good, do you plan to

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 11:18 AM, Andres Freund wrote: >> So let's use those, then. > > Right, I've never contended that. OK, cool. >> A fully barrier on x86 should be an mfence, right? > > Right. I've not talked about changing full barrier semantics. What I was > referring to is that until the a

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Alvaro Herrera
Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > I agree with both of those arguments. I have run into very few > > customers who have used the autovacuum settings to customize behavior > > for particular tables, and anyone who hasn't should see no change > > (right?), so m

Re: [HACKERS] Log notice that checkpoint is to be written on shutdown

2014-10-02 Thread Michael Banck
Hi, Am Donnerstag, den 02.10.2014, 08:17 -0700 schrieb David G Johnston: > Michael Banck-2 wrote > > I've attached a trivial patch for this, should it be added to the next > > commitfest? > > Peeking at this provokes a couple of novice questions: > > While apparently it is impossible to have a

[HACKERS] DDL Damage Assessment

2014-10-02 Thread Dimitri Fontaine
Hi fellow hackers, I would like to work on a new feature allowing our users to assess the amount of trouble they will run into when running a DDL script on their production setups, *before* actually getting their services down. The main practical example I can offer here is the ALTER TABLE comman

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Fabrízio de Royes Mello
On Thu, Oct 2, 2014 at 1:30 PM, Dimitri Fontaine wrote: > > Hi fellow hackers, > > I would like to work on a new feature allowing our users to assess the > amount of trouble they will run into when running a DDL script on their > production setups, *before* actually getting their services down. >

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 1:46 PM, Fabrízio de Royes Mello wrote: > On Thu, Oct 2, 2014 at 1:30 PM, Dimitri Fontaine > wrote: >> >> Hi fellow hackers, >> >> I would like to work on a new feature allowing our users to assess the >> amount of trouble they will run into when running a DDL script on the

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Harold Giménez
I think the main issue is when a table rewrite is triggered on a DDL command on a large table, as this is what frequently leads to unavailability. The idea of introducing a NOREWRITE keyword to DDL commands then came up (credit: Peter Geoghegan). When the NOREWRITE keyword is used and the DDL state

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Heikki Linnakangas
On 10/02/2014 05:40 PM, Robert Haas wrote: On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund wrote: OK. Given that the results look good, do you plan to push this? By "this", you mean the increase in the number of buffer mapping partitions to 128, and a corresponding increase in MAX_SIMUL_LWLO

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Andres Freund
On 2014-10-02 20:04:58 +0300, Heikki Linnakangas wrote: > On 10/02/2014 05:40 PM, Robert Haas wrote: > >On Thu, Oct 2, 2014 at 10:36 AM, Andres Freund > >wrote: > >>>OK. > >> > >>Given that the results look good, do you plan to push this? > > > >By "this", you mean the increase in the number of b

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 11:30 AM, Dimitri Fontaine wrote: > Questions: > > 1. Do you agree that a systematic way to report what a DDL command > (or script, or transaction) is going to do on your production > database is a feature we should provide to our growin

Re: [HACKERS] Inefficient barriers on solaris with sun cc

2014-10-02 Thread Andres Freund
On 2014-10-02 11:35:32 -0400, Robert Haas wrote: > On Thu, Oct 2, 2014 at 11:18 AM, Andres Freund wrote: > >> > Which is why these acquire/release fences, in contrast to > >> > acquire/release operations, have more guarantees... You put your finger > >> > right onto the spot. > >> > >> But, uh, we

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joshua D. Drake
On 10/02/2014 09:30 AM, Dimitri Fontaine wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feature we should provide to our growing user base? I would say it is late to

[HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Robert Haas
make check-world dies ingloriously for me, like this: /bin/sh ../../../config/install-sh -c -d tmp_check/log make -C ../../.. DESTDIR='/Users/rhaas/pgsql/src/bin/initdb'/tmp_check/install install >'/Users/rhaas/pgsql/src/bin/initdb'/tmp_check/log/install.log 2>&1 cd . && TESTDIR='/Users/rhaas/pgsq

Re: [HACKERS] Scaling shared buffer eviction

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 1:07 PM, Andres Freund wrote: > Do a make check-world and it'll hopefully fail ;). Check > pg_buffercache_pages.c. Yep. Committed, with an update to the comments in lwlock.c to allude to the pg_buffercache issue. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com T

Re: [HACKERS] Proper query implementation for Postgresql driver

2014-10-02 Thread Robert Haas
On Tue, Sep 30, 2014 at 1:20 AM, Craig Ringer wrote: > Frankly, I suggest dropping "simple" entirely and using only the > parse/bind/describe/execute flow in the v3 protocol. The last time I checked, that was significantly slower. http://www.postgresql.org/message-id/ca+tgmoyjkfnmrtmhodwhnoj1jwc

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: > 1. Do you agree that a systematic way to report what a DDL command (or > script, or transaction) is going to do on your production database > is a feature we should provide to our growing user base? I definitely like the idea of such a

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Harold Giménez (har...@heroku.com) wrote: > I think the main issue is when a table rewrite is triggered on a DDL > command on a large table, as this is what frequently leads to > unavailability. The idea of introducing a NOREWRITE keyword to DDL > commands then came up (credit: Peter Geoghegan).

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Joshua D. Drake (j...@commandprompt.com) wrote: > > 2. What do you think such a feature should look like? > > I liked the other post that said: EXPLAIN or whatever. > Heck it could even be useful to have EXPLAIN ANALZYE > in case people want to run it on staging/test/dev environments to > jud

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Robert Haas
On Thu, Oct 2, 2014 at 7:27 AM, Heikki Linnakangas wrote: >> SQL:2003 introduced the function NEXT VALUE FOR . Google >> tells me that at least DB2, SQL Server and a few niche databases >> understand it so far. As far as I can tell there is no standardised >> equivalent of currval and setval (but

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Josh Berkus
> Questions: > > 1. Do you agree that a systematic way to report what a DDL command (or > script, or transaction) is going to do on your production database > is a feature we should provide to our growing user base? Yes. > 2. What do you think such a feature should look like? As with

Re: [HACKERS] UPSERT wiki page, and SQL MERGE syntax

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:54 AM, Peter Geoghegan wrote: > I've started off by adding varied examples of the use of the existing > proposed syntax. I'll expand on this soon. I spent some time today expanding on the details, and commenting on the issues around the custom syntax (exactly what it doe

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread José Luis Tallón
On 10/02/2014 06:30 PM, Dimitri Fontaine wrote: > Hi fellow hackers, > [snip] > Questions: > > 1. Do you agree that a systematic way to report what a DDL command (or > script, or transaction) is going to do on your production database > is a feature we should provide to our growing user ba

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Bruce Momjian
On Tue, Sep 30, 2014 at 02:57:43PM -0700, Josh Berkus wrote: > I don't know that that is the *expectation*. However, I personally > would find it *acceptable* if it meant that we could get efficient merge > semantics on other aspects of the syntax, since my primary use for MERGE > is bulk loading.

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Steven Lembark
> EXPLAIN ALTER TABLE ? Good thing: People recognize it. Bad thing: People might not be able to tell the difference between a DDL and DML result. What about "EXPLAIN DDL ..."? The extra keyword ("DDL") makes it a bit more explicit that the results are not comparable to the standar

Re: [HACKERS] PL/pgSQL 2

2014-10-02 Thread Steven Lembark
On Mon, 01 Sep 2014 12:00:48 +0200 Marko Tiikkaja wrote: > create a new language. There are enough problems with SQL in general, enough alternatives proposed over time that it might be worth coming up with something that Just Works. -- Steven Lembark

Re: [HACKERS] PL/pgSQL 2

2014-10-02 Thread Steven Lembark
> Python2 -> Python3 would've been a lot less painful if you could mark, > on a module-by-module basis, whether a module was python2 or python3 > code. It wasn't very practical for Python because python code can reach > deep into the guts of unrelated objects discovered at runtime - it can > add/

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost wrote: > * Joshua D. Drake (j...@commandprompt.com) wrote: >> > 2. What do you think such a feature should look like? >> >> I liked the other post that said: EXPLAIN or whatever. >> Heck it could even be useful to have EXPLAIN ANALZYE >> in case peo

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 12:40 PM, Stephen Frost wrote: > The downside of the 'explain' approach is that the script then has to be > modified to put 'explain' in front of everything and then you have to go > through each statement and consider it. Having a 'dry-run' transaction > type which then pr

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:37 PM, Peter Geoghegan wrote: > And yet, in theory session 2's impact > on production should not be minimal, if we consider something like > EXPLAIN output. "Should have been minimal", I mean. -- Peter Geoghegan -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 5:37 PM, Peter Geoghegan wrote: > Session 3 is an innocent bystander. It goes to query the same table in > an ordinary, routine way - a SELECT statement. Even though session 2's > lock is not granted yet, session 3 is not at liberty to skip the queue > and get its own Access

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:00 PM, Peter Geoghegan wrote: >> Granted, it's something that's not easily automatable, whereas a nowait is. >> >> However, rather than nowait, I'd prefer "cancellable" semantics, that >> would cancel voluntarily if any other transaction requests a >> conflicting lock, lik

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: > On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost wrote: > > The downside of the 'explain' approach is that the script then has to be > > modified to put 'explain' in front of everything and then you have to go > > through each statement and consider

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:52 PM, Claudio Freire wrote: > The explain would show the AccessExclusiveLock, so it would be enough > for a heads-up to kill all idle-in-transaction holding locks on the > target relation (if killable, or just wait). I think that there are very few problems with recogniz

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost wrote: > * Claudio Freire (klaussfre...@gmail.com) wrote: >> On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost wrote: >> > The downside of the 'explain' approach is that the script then has to be >> > modified to put 'explain' in front of everything and t

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 1:10 PM, Bruce Momjian wrote: > I think if we use the MERGE command for this feature we would need to > use a non-standard keyword to specify that we want OLTP/UPSERT > functionality. That would allow us to mostly use the MERGE standard > syntax without having surprises abo

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Peter Eisentraut
On 10/2/14 3:19 PM, Robert Haas wrote: > 1..2 > ok 1 - initdb with invalid option nonzero exit code > ok 2 - initdb with invalid option prints error message > # Looks like your test exited with 256 just after 2. > not ok 3 - initdb options handling > > # Failed test 'initdb optio

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Peter Geoghegan
On Thu, Oct 2, 2014 at 2:04 PM, Claudio Freire wrote: > I've done that manually (throw the DDL, and cancel if it takes more > than a couple of seconds) on modest but relatively busy servers with > quite some success. Fair enough, but that isn't the same as NOWAIT. It's something we'd have a hard

Re: [HACKERS] TAP test breakage on MacOS X

2014-10-02 Thread Andres Freund
On 2014-10-02 17:09:43 -0400, Peter Eisentraut wrote: > On 10/2/14 3:19 PM, Robert Haas wrote: > > 1..2 > > ok 1 - initdb with invalid option nonzero exit code > > ok 2 - initdb with invalid option prints error message > > # Looks like your test exited with 256 just after 2. > > not

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Thu, Oct 2, 2014 at 12:40 PM, Stephen Frost wrote: > > The downside of the 'explain' approach is that the script then has to be > > modified to put 'explain' in front of everything and then you have to go > > through each statement and consider it.

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Andres Freund
On 2014-10-02 17:03:59 -0400, Stephen Frost wrote: > * Claudio Freire (klaussfre...@gmail.com) wrote: > > On Thu, Oct 2, 2014 at 4:40 PM, Stephen Frost wrote: > > > The downside of the 'explain' approach is that the script then has to be > > > modified to put 'explain' in front of everything and t

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Andres Freund
On 2014-10-02 13:49:36 -0300, Claudio Freire wrote: > EXPLAIN ALTER TABLE ? I don't think that'll work - there's already EXPLAIN for some CREATE. At least CREATE TABLE ... AS, CREATE VIEW ... AS and SELECT INTO. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadr

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: > On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost wrote: > >> That sounds extremely complex. You'd have to implement the fake > >> columns, foreign keys, indexes, etc on most execution nodes, the > >> planner, and even system views. > > > > Eh? We ha

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Andres Freund (and...@2ndquadrant.com) wrote: > On 2014-10-02 17:03:59 -0400, Stephen Frost wrote: > > > That sounds extremely complex. You'd have to implement the fake > > > columns, foreign keys, indexes, etc on most execution nodes, the > > > planner, and even system views. > > > > Eh? We ha

Re: [HACKERS] Assertion failure in syncrep.c

2014-10-02 Thread Simon Riggs
On 18 September 2014 07:32, Pavan Deolasee wrote: > 564 /* > 565 * Set state to complete; see SyncRepWaitForLSN() for discussion > of > 566 * the various states. > 567 */ > 568 thisproc->syncRepState = SYNC_REP_WAIT_COMPLETE; > 569 > 570 /* > 571

Re: [HACKERS] INSERT ... ON CONFLICT {UPDATE | IGNORE}

2014-10-02 Thread Bruce Momjian
On Thu, Oct 2, 2014 at 02:08:30PM -0700, Peter Geoghegan wrote: > On Thu, Oct 2, 2014 at 1:10 PM, Bruce Momjian wrote: > > I think if we use the MERGE command for this feature we would need to > > use a non-standard keyword to specify that we want OLTP/UPSERT > > functionality. That would allow

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Claudio Freire
On Thu, Oct 2, 2014 at 6:19 PM, Stephen Frost wrote: > * Claudio Freire (klaussfre...@gmail.com) wrote: >> On Thu, Oct 2, 2014 at 6:03 PM, Stephen Frost wrote: >> >> That sounds extremely complex. You'd have to implement the fake >> >> columns, foreign keys, indexes, etc on most execution nodes,

[HACKERS] [PATCH] Simplify EXISTS subqueries containing LIMIT

2014-10-02 Thread Marti Raudsepp
Hi list, Attached patch allows semijoin/antijoin/hashed SubPlan optimization when an EXISTS subquery contains a LIMIT clause with a positive constant. It seems to be a fairly common meme to put LIMIT 1 into EXISTS() subqueries, and it even makes sense when you're not aware that the database alread

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Alvaro Herrera
Dimitri Fontaine wrote: > The main practical example I can offer here is the ALTER TABLE command. > Recent releases are including very nice optimisations to it, so much so > that it's becoming increasingly hard to answer some very basic > questions: > > - what kind of locks will be taken? (excl

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

2014-10-02 Thread Alvaro Herrera
Alvaro Herrera wrote: > Basically, if you are on 9.3.5 or earlier any per-table options for > autovacuum cost delay will misbehave (meaning: any such table will be > processed with settings flattened according to balancing of the standard > options, _not_ the configured ones). If you are on 9.3.6

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Alvaro Herrera
Peter Geoghegan wrote: > Why is NOWAIT only supported for SET TABLESPACE? I guess it's just a > particularly bad case. NOWAIT might be the wrong thing for DDL > generally. I didn't realize we supported NOWAIT for SET TABLESPACE. My hunch is that if we have that, it should really be supported for

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Dimitri Fontaine
Alvaro Herrera writes: >> - will the table have to be rewritten? the indexes? > > Please give my DDL deparsing patch a look. There is a portion there > about deparsing ALTER TABLE specifically; what it does is save a list of > subcommands, and for each of them we either report the OID of the ob

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Jan Wieck
On 10/02/2014 01:15 PM, Joe Conway wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 11:30 AM, Dimitri Fontaine wrote: Questions: 1. Do you agree that a systematic way to report what a DDL command (or script, or transaction) is going to do on your production database is a feat

Re: [HACKERS] Proposal for updating src/timezone

2014-10-02 Thread Tom Lane
John Cochran writes: > As it is, I've finished checking the differences between the postgres and > IANA code for zic.c after editing both to eliminate non-functional style > differences such as indentation, function prototypes, comparing strchr > results against NULL or 0, etc. It looks like the o

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Thomas Munro
On 2 October 2014 14:48, Tom Lane wrote: > Thomas Munro writes: >> SQL:2003 introduced the function NEXT VALUE FOR . Google >> tells me that at least DB2, SQL Server and a few niche databases >> understand it so far. As far as I can tell there is no standardised >> equivalent of currval and setv

Re: [HACKERS] Promise index tuples for UPSERT

2014-10-02 Thread Peter Geoghegan
On Wed, Oct 1, 2014 at 12:59 PM, Peter Geoghegan wrote: > On Wed, Oct 1, 2014 at 12:54 PM, Heikki Linnakangas > wrote: >> XactLockTableWait() waits until the end of transaction, that's not you want >> here. If the backend that inserted the promise tuple decides to not proceed >> with the insertio

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Thomas Munro
On 3 October 2014 00:01, Thomas Munro wrote: > On 2 October 2014 14:48, Tom Lane wrote: >> Thomas Munro writes: >>> SQL:2003 introduced the function NEXT VALUE FOR . Google >>> tells me that at least DB2, SQL Server and a few niche databases >>> understand it so far. As far as I can tell there

Re: [HACKERS] CREATE IF NOT EXISTS INDEX

2014-10-02 Thread Marti Raudsepp
On Wed, Oct 1, 2014 at 2:42 PM, Fabrízio de Royes Mello wrote: > So, what's the correct/best grammar? > CREATE [ IF NOT EXISTS ] [ UNIQUE ] INDEX index_name > or > CREATE [ UNIQUE ] INDEX [ IF NOT EXISTS ] index_name I've elected myself as the reviewer for this patch. Here are some preliminary co

Re: [HACKERS] NEXT VALUE FOR

2014-10-02 Thread Tom Lane
Thomas Munro writes: > On 2 October 2014 14:48, Tom Lane wrote: >> Have you checked the archives about this? My recollection is that one >> reason it's not in there (aside from having to reserve "NEXT") is that >> the standard-mandated semantics are not the same as nextval(). > Right, I found t

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 10/02/2014 06:43 PM, Jan Wieck wrote: > The real question is at what level of information, returned to the > user, does this feature become user friendly? > > It is one thing to provide information of the kind of > > TAKE ACCECSS EXCLUSIVE LOCK ON

[HACKERS] Fixed xloginsert_locks for 9.4

2014-10-02 Thread Greg Smith
I did a fair dive into double-checking the decision to just leave xloginsert_locks fixed at 8 for 9.4. My conclusion: good call, move along. Further improvements beyond what the 8-way split gives sure are possible. But my guess from chasing them a little is that additional places will pop u

Re: [HACKERS] DDL Damage Assessment

2014-10-02 Thread Stephen Frost
* Claudio Freire (klaussfre...@gmail.com) wrote: > On Thu, Oct 2, 2014 at 6:19 PM, Stephen Frost wrote: > >> And that needs locks, especially if you modify the underlying filesystem > >> layout. > > > > And we wouldn't be doing that, certainly. It's a dry-run. > > ... > > > (...) We might also

  1   2   >