Re: AS OF queries

2018-01-25 Thread Konstantin Knizhnik
On 26.01.2018 03:55, Bruce Momjian wrote: On Sat, Dec 23, 2017 at 11:53:19PM +0300, konstantin knizhnik wrote: On Dec 23, 2017, at 2:08 AM, Greg Stark wrote: On 20 December 2017 at 12:45, Konstantin Knizhnik wrote: It seems to me that it will be not so difficult to implement them in Postg

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Daniel Gustafsson
> On 26 Jan 2018, at 02:10, Michael Paquier wrote: > On Fri, Jan 26, 2018 at 12:27:16AM +0100, Daniel Gustafsson wrote: >> While only tangentially related to the issue this patch solves, converting >> be_tls_get_peerdn_name() to return const char * seems reasonable too to keep >> the API consiste

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Konstantin Knizhnik
Hi, I've spent the last weeks working on my LLVM compilation patchset. In the course of that I *heavily* revised it. While still a good bit away from committable, it's IMO definitely not a prototype anymore. Below are results on my system for Q1 TPC-H scale 10 (~13Gb database) Options

Re: STATISTICS retained in CREATE TABLE ... LIKE (INCLUDING ALL)?

2018-01-25 Thread David Rowley
On 21 January 2018 at 19:21, David Rowley wrote: > On 20 January 2018 at 18:50, Tom Lane wrote: >> Stephen Froehlich writes: >>> Are custom statistics in PG10 retained in LIKE (INCLUDING ALL) or do I need >>> to recreate the statistics by hand each time I create a new table? >> >> LIKE doesn't

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-25 Thread Amit Kapila
On Fri, Jan 26, 2018 at 12:00 PM, Peter Geoghegan wrote: > On Thu, Jan 25, 2018 at 10:00 PM, Amit Kapila wrote: >>> At this point, my preferred solution is for someone to go implement >>> Amit's WaitForParallelWorkersToAttach() idea [1] (Amit himself seems >>> like the logical person for the job)

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
On Mon, 2018-01-08 at 11:18 -0500, Robert Haas wrote: > I also don't agree with the idea that we should reject syntax that > doesn't appear in the SQL standard. We have a great deal of such > syntax already, and we add more of it in every release, and a good > deal of it is contributed by you and

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
On Sun, 2018-01-07 at 09:58 +, Simon Riggs wrote: > > The attached README explains the ALIGN operation step-by-step with > > a TEMPORAL LEFT OUTER JOIN example. That is, we start from a query > > input, show how we rewrite it during parser stage, and show how the > > final execution generates r

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
On Sat, 2018-01-06 at 20:29 +, Simon Riggs wrote: > * various PostJoinSetProjection() functions to be called as needed > So the idea is we enable Postgres to allow major new functionality, > as was done for PostGIS so successfully. If we use a post-join approach many intermediate result tuples

Re: [HACKERS] [PROPOSAL] Temporal query processing with range types

2018-01-25 Thread Peter Moser
On Thu, 2017-11-30 at 12:26 -0500, Robert Haas wrote: > I wonder if we could instead think about R NORMALIZE S ON R.x = S.y > WITH (R.time, S.time) as an ordinary join on R.x = S.y with some > extra processing afterwards. After finding all the join partners for > a row in R, extract all the lower

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-25 Thread Peter Geoghegan
On Thu, Jan 25, 2018 at 10:00 PM, Amit Kapila wrote: >> At this point, my preferred solution is for someone to go implement >> Amit's WaitForParallelWorkersToAttach() idea [1] (Amit himself seems >> like the logical person for the job). >> > > I can implement it and share a prototype patch with yo

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

2018-01-25 Thread Amit Kapila
On Wed, Jan 24, 2018 at 12:44 PM, amul sul wrote: > On Tue, Jan 23, 2018 at 7:01 PM, Amit Kapila wrote: >> On Fri, Jan 12, 2018 at 11:43 AM, amul sul wrote: > [] >> I have asked to change the message "tuple to be updated .." after >> heap_lock_tuple call not in nodeModifyTable.c, so please r

Re: General purpose hashing func in pgbench

2018-01-25 Thread Fabien COELHO
Hello Ildar, Applies, compiles, runs. I still have a few very minor comments, sorry for this (hopefully) last iteration from my part. I'm kind of iterative... The XML documentation source should avoid a paragraph on one very long line, but rather be indented like other sections. I'd propo

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-25 Thread Amit Kapila
On Fri, Jan 26, 2018 at 11:30 AM, Amit Kapila wrote: > On Thu, Jan 25, 2018 at 1:24 AM, Peter Geoghegan wrote: >> On Tue, Jan 23, 2018 at 9:43 PM, Amit Kapila wrote: >>> Right, but what if the worker dies due to something proc_exit(1) or >>> something like that before calling BarrierArriveAndWai

Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)

2018-01-25 Thread Amit Kapila
On Thu, Jan 25, 2018 at 1:24 AM, Peter Geoghegan wrote: > On Tue, Jan 23, 2018 at 9:43 PM, Amit Kapila wrote: >> Right, but what if the worker dies due to something proc_exit(1) or >> something like that before calling BarrierArriveAndWait. I think this >> is part of the problem we have solved i

Re: Redefining inet_net_ntop

2018-01-25 Thread Tom Lane
Craig Ringer writes: > Should we be using our own if the OS has it? I'm thinking of adding a test > to configure and omitting our own version if configure finds it. Objections? I can't imagine that there's any real upside here. The amount of code involved is barely over a kilobyte, and we'd be e

RE: Temporary tables prevent autovacuum, leading to XID wraparound

2018-01-25 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > I think we should consider having backends try to remove their temporary > schema on startup; then, if a temp table in a backend is old enough that > it's due for vacuum for wraparound, have autovacuum kill the connection. > The former is necessary

RE: Temporary tables prevent autovacuum, leading to XID wraparound

2018-01-25 Thread Tsunakawa, Takayuki
From: Masahiko Sawada [mailto:sawada.m...@gmail.com] > On Thu, Jan 25, 2018 at 3:14 PM, Tsunakawa, Takayuki > wrote: > > * Why does autovacuum launcher always choose only one database when that > database need vacuuming for XID wraparound? Shouldn't it also choose other > databases? > > Yeah, I'

Re: Redefining inet_net_ntop

2018-01-25 Thread Peter Eisentraut
On 1/25/18 22:24, Craig Ringer wrote: > port.h declares inet_net_ntop and we always compile our own > from port/inet_net_ntop.c . > > But it's part of -lresolv on Linux, and more importantly, it's declared > in . > > Should we be using our own if the OS has it? I'm thinking of adding a > test to

Re: WIP: Covering + unique indexes.

2018-01-25 Thread Thomas Munro
On Fri, Jan 26, 2018 at 3:01 AM, Anastasia Lubennikova wrote: > Thanks for the reminder. Rebased patches are attached. This is a really cool and also difficult feature. Thanks for working on it! Here are a couple of quick comments on the documentation, since I noticed it doesn't build: SGML->X

Re: [HACKERS] generated columns

2018-01-25 Thread Peter Eisentraut
On 1/19/18 00:18, Michael Paquier wrote: > Instead of leaving bits for a feature that may or may not be > implemented, have you considered just blocking STORED at parsing level > and remove those bits? There is little point in keeping the equivalent > of dead code in the tree. So I would suggest a

Redefining inet_net_ntop

2018-01-25 Thread Craig Ringer
Hi folks port.h declares inet_net_ntop and we always compile our own from port/inet_net_ntop.c . But it's part of -lresolv on Linux, and more importantly, it's declared in . Should we be using our own if the OS has it? I'm thinking of adding a test to configure and omitting our own version if co

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Peter Eisentraut
On 1/25/18 20:10, Michael Paquier wrote: > Peter, could you change ssl_version() and ssl_cipher() in sslinfo at the > same time please? I think that those should use the generic backend-side > APIs as well. sslinfo depends heavily on OpenSSL, OK, but if possible > getting this code more generic wil

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Peter Eisentraut
On 1/25/18 09:40, Tom Lane wrote: > Peter Eisentraut writes: >> On 1/23/18 13:39, Robert Haas wrote: >>> I don't understand. AAUI, it is currently the case that if you set >>> the options before the TOAST table exists, they are lost. > >> Well, that's also weird. > > Well, maybe the right answe

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-25 Thread Peter Eisentraut
On 1/24/18 07:53, Petr Jelinek wrote: > That depends on if we consider this to be part of sequence handling or > truncate statement replication handling. It's true that if we had > sequence replication, the restart would get propagated that way anyway. > OTOH, if we'll want to add option in the fut

Re: [HACKERS] Subscription code improvements

2018-01-25 Thread Peter Eisentraut
On 1/24/18 02:33, Masahiko Sawada wrote: > Thank you for notification. Since it seems to me that no one is > interested in this patch, it would be better to close out this patch. > This patch is a refactoring patch but subscription code seems to work > fine so far. If a problem appears around subsc

Re: [HACKERS] [PATCH]make pg_rewind to not copy useless WAL files

2018-01-25 Thread Michael Paquier
On Thu, Jan 25, 2018 at 07:38:37AM -0500, Stephen Frost wrote: > Michael, all, > > * Michael Paquier (michael.paqu...@gmail.com) wrote: >> On Wed, Jan 24, 2018 at 12:43:51PM -0500, Stephen Frost wrote: >>> * chenhj (chjis...@163.com) wrote: At 2018-01-23 09:56:48, "Stephen Frost" wrote:

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2018-01-25 Thread Stephen Frost
Craig, Michael, all, * Craig Ringer (cr...@2ndquadrant.com) wrote: > On 21 December 2017 at 11:31, Michael Paquier > wrote: > > > On Thu, Dec 21, 2017 at 11:46 AM, Alvaro Herrera > > wrote: > > > Michael Paquier wrote: > > >> Well, the idea is really to get rid of that as there are already > >

Re: \describe*

2018-01-25 Thread Vik Fearing
On 01/26/2018 02:11 AM, Corey Huinker wrote: > Some of the discussions about making psql more user friendly (more tab > completions help, exit, etc) got me thinking about other ways that psql > could be more friendly, and the one that comes to mind is our terse but > cryptic \d* commands. > > I th

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > If I understand correctly, those results are all just pg_test_fsync results. > That's not reflective of what will happen when the database is actually > running. When you use open_sync or open_datasync, you force WAL write and > WAL flush to happe

Re: Boolean partitions syntax

2018-01-25 Thread Amit Langote
Hi Stephen. On 2018/01/26 10:16, Stephen Frost wrote: > * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > Still compiles and passes regression tests, which is good. Thanks for looking at this. >>> I extended your test a bit to check whether partitions over booleans are >>> useful. >>> Not

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-25 Thread Robert Haas
On Thu, Jan 25, 2018 at 8:36 PM, Petr Jelinek wrote: > On 26/01/18 02:34, Robert Haas wrote: >> On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek >> wrote: >>> The patch will cascade truncation on downstream if cascade was specified >>> on the upstream, that can potentially be dangerous and we eithe

Re: Fix a typo in autoprewarm.c

2018-01-25 Thread Masahiko Sawada
On Fri, Jan 26, 2018 at 10:13 AM, Bruce Momjian wrote: > On Fri, Dec 22, 2017 at 02:30:56PM +0900, Masahiko Sawada wrote: >> Hi, >> >> Attached a patch for fixing a typo in autoprewarm.c. I'm sorry if I'm >> mistaken. >> >> s/withs/with/ > > Patch applied to head, thanks. Thank you! Regards, -

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Tsunakawa, Takayuki
From: Michael Paquier [mailto:michael.paqu...@gmail.com] > Or to put it short, the lack of granular syncs in ext3 kills performance > for some workloads. Tomas Vondra's presentation on such matters are a really > cool read by the way: > https://www.slideshare.net/fuzzycz/postgresql-on-ext4-xfs-btrf

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-25 Thread Petr Jelinek
On 26/01/18 02:34, Robert Haas wrote: > On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek > wrote: >> The patch will cascade truncation on downstream if cascade was specified >> on the upstream, that can potentially be dangerous and we either should >> not do it and only truncate the tables which wer

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Robert Haas
On Thu, Jan 25, 2018 at 8:32 PM, Tsunakawa, Takayuki wrote: > As I showed previously, regular file writes on PCIe flash, *not writes using > PMDK on persistent memory*, was 20% faster with open_datasync than with > fdatasync. If I understand correctly, those results are all just pg_test_fsync r

Re: [PATCH] Logical decoding of TRUNCATE

2018-01-25 Thread Robert Haas
On Wed, Jan 17, 2018 at 12:07 PM, Petr Jelinek wrote: > The patch will cascade truncation on downstream if cascade was specified > on the upstream, that can potentially be dangerous and we either should > not do it and only truncate the tables which were truncated upstream > (but without restricti

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com]> On Thu, Jan 25, 2018 at 7:08 PM, Tsunakawa, Takayuki > wrote: > > No, I'm not saying we should make the persistent memory mode the default. > I'm simply asking whether it's time to make open_datasync the default > setting. We can write a notice i

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Michael Paquier
On Thu, Jan 25, 2018 at 09:30:45AM -0500, Robert Haas wrote: > On Wed, Jan 24, 2018 at 10:31 PM, Tsunakawa, Takayuki > wrote: >>> This is just a guess, of course. You didn't mention what the underlying >>> storage for your test was? >> >> Uh, your guess was correct. My file system was ext3, wher

Re: Temporary tables prevent autovacuum, leading to XID wraparound

2018-01-25 Thread Robert Haas
On Thu, Jan 25, 2018 at 1:14 AM, Tsunakawa, Takayuki wrote: > * I think temporary tables should not require vacuuming for XID wraparound. > Furtherover, should updates/deletes to temporary tables be in-place instead > of creating garbage, so that any form of vacuum is unnecessary? Other > se

Re: Documentation of pgcrypto AES key sizes

2018-01-25 Thread Michael Paquier
On Fri, Jan 26, 2018 at 12:33:41PM +1300, Thomas Munro wrote: > I noticed that the documentation for encrypt()/decrypt() says "aes — > AES (Rijndael-128)", but in fact 192 and 256 bit keys are also > supported, whether you build --with-openssl or --without-openssl. > Should that say "AES (Rijndael-

Re: Boolean partitions syntax

2018-01-25 Thread Stephen Frost
Greetings Amit, * Amit Langote (langote_amit...@lab.ntt.co.jp) wrote: > On 2017/12/20 6:46, Mark Dilger wrote: > >> On Dec 12, 2017, at 10:32 PM, Amit Langote > >> wrote: > >> Added to CF: https://commitfest.postgresql.org/16/1410/ > > > > This compiles and passes the regression tests for me. >

Re: list partition constraint shape

2018-01-25 Thread Amit Langote
Fujita-san, Thanks for the review. On 2018/01/25 21:17, Etsuro Fujita wrote: > Thanks for the updated patch!  Some minor comments: > > +   /* > +    * Construct an ArrayExpr for the non-null partition > +    * values > +    */ > +  

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Robert Haas
On Thu, Jan 25, 2018 at 7:08 PM, Tsunakawa, Takayuki wrote: > No, I'm not saying we should make the persistent memory mode the default. > I'm simply asking whether it's time to make open_datasync the default > setting. We can write a notice in the release note for users who still use > ext3 e

Re: Removing WITH clause support in CREATE FUNCTION, for isCachable and isStrict

2018-01-25 Thread Michael Paquier
On Thu, Jan 25, 2018 at 06:30:04PM -0500, Tom Lane wrote: > I poked into the git log and confirmed Michael's statement that > CREATE FUNCTION ... WITH has been documented as deprecated since > 7.3 (commit 94bdc4855 to be exact). Thanks for the confirmation. > I think the original intention was th

Re: Fix a typo in autoprewarm.c

2018-01-25 Thread Bruce Momjian
On Fri, Dec 22, 2017 at 02:30:56PM +0900, Masahiko Sawada wrote: > Hi, > > Attached a patch for fixing a typo in autoprewarm.c. I'm sorry if I'm > mistaken. > > s/withs/with/ Patch applied to head, thanks. --- > > Regar

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Michael Paquier
On Fri, Jan 26, 2018 at 12:27:16AM +0100, Daniel Gustafsson wrote: >> On 25 Jan 2018, at 15:07, Peter Eisentraut >> wrote: >> >> On 1/19/18 13:43, Peter Eisentraut wrote: >>> Comparing the existing {be,fe}-secure-openssl.c with the proposed >>> {be,fe}-secure-gnutls.c, and with half an eye on th

\describe*

2018-01-25 Thread Corey Huinker
Some of the discussions about making psql more user friendly (more tab completions help, exit, etc) got me thinking about other ways that psql could be more friendly, and the one that comes to mind is our terse but cryptic \d* commands. I think it would be helpful and instructive to have correspon

Re: PATCH: psql tab completion for SELECT

2018-01-25 Thread Edmund Horner
On 26 January 2018 at 13:44, Vik Fearing wrote: > On 01/26/2018 01:28 AM, Edmund Horner wrote: >> The patch mentioned attempts to put savepoints around the tab >> completion query where appropriate. > > I am -1 on this idea. May I ask why? It doesn't stop psql working against older versions, as

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Michael Paquier
On Fri, Jan 19, 2018 at 01:55:30PM -0500, Robert Haas wrote: > On Wed, Jan 17, 2018 at 10:02 PM, Tom Lane wrote: >> Also, this isn't really a good argument against using uniform names >> for parameters that every implementation is certain to have, like >> ssl_key_file. > > Even then, it's not tha

Re: AS OF queries

2018-01-25 Thread Bruce Momjian
On Sat, Dec 23, 2017 at 11:53:19PM +0300, konstantin knizhnik wrote: > > On Dec 23, 2017, at 2:08 AM, Greg Stark wrote: > > > On 20 December 2017 at 12:45, Konstantin Knizhnik > > wrote: > > > >> It seems to me that it will be not so difficult to implement them in > >> Postgres - we already hav

Re: PATCH: psql tab completion for SELECT

2018-01-25 Thread Vik Fearing
On 01/26/2018 01:28 AM, Edmund Horner wrote: > On 19 January 2018 at 05:37, Vik Fearing wrote: >> On 01/18/2018 01:07 AM, Tom Lane wrote: >>> Edmund Horner writes: On 15 January 2018 at 15:45, Andres Freund wrote: > All worries like this are supposed to check the server version. >>> >>>

Re: PATCH: psql tab completion for SELECT

2018-01-25 Thread Edmund Horner
On 19 January 2018 at 05:37, Vik Fearing wrote: > On 01/18/2018 01:07 AM, Tom Lane wrote: >> Edmund Horner writes: >>> On 15 January 2018 at 15:45, Andres Freund wrote: All worries like this are supposed to check the server version. >> >>> In psql there are around 200 such tab completion qu

Re: ALTER TABLE ADD COLUMN fast default

2018-01-25 Thread Andrew Dunstan
On Thu, Jan 25, 2018 at 6:10 PM, Thomas Munro wrote: > On Wed, Jan 17, 2018 at 2:21 AM, Andrew Dunstan > wrote: >> Yeah, got caught by the bki/pg_attribute changes on Friday. here's an >> updated version. Thanks for looking. > > A boring semi-automated update: this no long compiles, because > 85

Re: PATCH: psql tab completion for SELECT

2018-01-25 Thread Edmund Horner
On 23 January 2018 at 21:47, Vik Fearing wrote: > Don't forget to include the list :-) > I'm quoting the entirety of the message---which I would normally trim---for > the archives. Thanks for spotting that. Sorry list! > If this were my patch, I'd have one query for 8.0, and then queries for al

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-01-25 Thread Tsunakawa, Takayuki
From: Robert Haas [mailto:robertmh...@gmail.com] > On Wed, Jan 24, 2018 at 10:31 PM, Tsunakawa, Takayuki > wrote: > > As you said, open_datasync was 20% faster than fdatasync on RHEL7.2, on > a LVM volume with ext4 (mounted with options noatime, nobarrier) on a PCIe > flash memory. > > So does th

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Stephen Frost
Alvaro, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > Stephen Frost wrote: > > Alvaro, Tom, > > > > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > > > > Crazy idea: maybe a large fraction of that test could be replaced with > > > comparisons of the "pg_restore -l" output file rather t

Should we introduce a "really strict" volatility about return values?

2018-01-25 Thread Andres Freund
Hi, Currently a good bit of time evaluation more complex expressions is spent checking whether a strict [transition] function should be called or not. There's a good number of cases where we could optimize that away based on the underlying data - if e.g. a slot column refers to a NOT NULL column (

Documentation of pgcrypto AES key sizes

2018-01-25 Thread Thomas Munro
Hi, I noticed that the documentation for encrypt()/decrypt() says "aes — AES (Rijndael-128)", but in fact 192 and 256 bit keys are also supported, whether you build --with-openssl or --without-openssl. Should that say "AES (Rijndael-128, -192 or -256)" instead? -- Thomas Munro http://www.enterpr

Re: Removing WITH clause support in CREATE FUNCTION, for isCachable and isStrict

2018-01-25 Thread Tom Lane
Daniel Gustafsson writes: >> On 15 Jan 2018, at 03:27, Michael Paquier wrote: >> As noticed by Daniel here: >> https://www.postgresql.org/message-id/d5f34c9d-3ab7-4419-af2e-12f67581d...@yesql.se > In that thread I proposed a patch to fix this, but I certainly don’t object to > just removing it t

Re: [HACKERS] GnuTLS support

2018-01-25 Thread Daniel Gustafsson
> On 25 Jan 2018, at 15:07, Peter Eisentraut > wrote: > > On 1/19/18 13:43, Peter Eisentraut wrote: >> Comparing the existing {be,fe}-secure-openssl.c with the proposed >> {be,fe}-secure-gnutls.c, and with half an eye on the previously proposed >> Apple Secure Transport implementation, I have id

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-01-25 Thread Daniel Gustafsson
> On 24 Jan 2018, at 16:45, Alvaro Herrera wrote: > > A quick suggestion from a passer-by -- would you put the new code in > src/backend/storage/ipc/backend_signal.c instead? Sounds like a better > place than utils/misc/; and "signal" is more general in nature than just > "cancel". A bunch of s

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-01-25 Thread Fabien COELHO
Hello Doug, This time with the revised patch file: pgbench11-ppoll-v8.patch Patch applies cleanly. Compiles cleanly and runs fine in both ppoll & select cases. I'm okay with having a preferred ppoll implementation because of its improved capability. A few minor additional comments/sugges

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Well, the current implementation compares a dozen of pg_dump output text > files, three hundred lines apiece, against a thousand of regexes (give > or take). Whenever there is a mismatch, what you get is "this regexp > failed to match " (or sometimes "matched when it > sh

Re: MCV lists for highly skewed distributions

2018-01-25 Thread Tom Lane
Dean Rasheed writes: > It occurs to me that maybe a better test to exclude a value from the > MCV list would be to demand that its relative standard error not be > too high. Such a test, in addition to the existing tests, might be > sufficient to solve the opposite problem of too many values in th

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Alvaro Herrera
Stephen Frost wrote: > Alvaro, Tom, > > * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > > Crazy idea: maybe a large fraction of that test could be replaced with > > comparisons of the "pg_restore -l" output file rather than pg_dump's > > text output (i.e. the TOC entry for each object, rather

Invalid result from hash_page_items function

2018-01-25 Thread Masahiko Sawada
Hi, While researching hash index, I found that hash_page_items could return an invalid result as follows. postgres(1:1056)=# create table hash_test (c int); CREATE TABLE postgres(1:1056)=# insert into hash_test select generate_series(1,50) % 5; INSERT 0 50 postgres(1:1056)=# create index hash_idx

Re: After dropping the rule - Not able to insert / server crash (one time ONLY)

2018-01-25 Thread Bruce Momjian
Can someone review this thread and determine if this patch is needed? Thanks. --- On Fri, Dec 22, 2017 at 04:58:47PM +0530, Dilip Kumar wrote: > On Mon, Dec 11, 2017 at 4:33 PM, Dilip Kumar wrote: > > On Mon, Dec 11,

Re: reducing isolation tests runtime

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It > > can be further reduced, but not by more than a second or two unless you > > get in the business of modifying other tests. (I only modified > > deadlock-soft-2 because it saves 5

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2018-01-25 Thread Thomas Munro
On Fri, Jan 26, 2018 at 9:38 AM, Claudio Freire wrote: > I had the tests running in a loop all day long, and I cannot reproduce > that variance. > > Can you share your steps to reproduce it, including configure flags? Here are two build logs where it failed: https://travis-ci.org/postgresql-cfbo

Re: reducing isolation tests runtime

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Here's a concrete proposal. Runtime is 45.7 seconds on my laptop. It > can be further reduced, but not by more than a second or two unless you > get in the business of modifying other tests. (I only modified > deadlock-soft-2 because it saves 5 seconds). Looks reasonab

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Stephen Frost
Alvaro, Tom, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > Tom Lane wrote: > > > The changes in t/002_pg_dump.pl largely failed to apply, which is > > partially due to the age of the patch but IMO speaks more to the > > unmaintainability of that TAP test. It still didn't run after I'd > >

Re: proposal: alternative psql commands quit and exit

2018-01-25 Thread Bruce Momjian
On Mon, Jan 15, 2018 at 11:10:44AM -0500, Robert Haas wrote: > On Mon, Jan 15, 2018 at 10:57 AM, Tom Lane wrote: > > Robert Haas writes: > >> I've discovered one thing about this design that is not so good, which > >> is that if you open a single, double, or dollar quote, then the > >> instructio

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > The changes in t/002_pg_dump.pl largely failed to apply, which is > partially due to the age of the patch but IMO speaks more to the > unmaintainability of that TAP test. It still didn't run after I'd > manually fixed the merge failures, so I gave up in disgust and > did not pus

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2018-01-25 Thread Claudio Freire
On Thu, Jan 25, 2018 at 10:56 AM, Claudio Freire wrote: > On Thu, Jan 25, 2018 at 4:11 AM, Thomas Munro > wrote: >> On Thu, Jan 18, 2018 at 9:17 AM, Claudio Freire >> wrote: >>> Huh. That was simpler than I thought. >>> >>> Attached rebased versions. >> >> Hi Claudio, >> >> FYI the regression t

Re: reducing isolation tests runtime

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > I think we could solve this by putting in the same parallel group only > > slow tests that mostly sleeps, ie. nothing that would monopolize CPU for > > long enough to cause a problem. Concretely: > > test: timeouts tuplelock-update deadlock-hard deadl

Re: [HACKERS] Patch: Add --no-comments to skip COMMENTs with pg_dump

2018-01-25 Thread Tom Lane
Robins Tharakan writes: > Attached is an updated patch (v4) with basic tests for pg_dump / pg_dumpall. I've reviewed and pushed this, after making some changes so that the switch works in pg_restore as well, and minor cosmetic adjustments. The changes in t/002_pg_dump.pl largely failed to apply,

Re: PATCH: Exclude unlogged tables from base backups

2018-01-25 Thread David Steele
On 1/25/18 12:31 AM, Masahiko Sawada wrote: > On Thu, Jan 25, 2018 at 3:25 AM, David Steele wrote: >>> >>> Here is the first review comments. >>> >>> + unloggedDelim = strrchr(path, '/'); >>> >>> I think it doesn't work fine on windows. How about using >>> last_dir_separator() instead? >> >>

Re: [HACKERS] Proposal: Local indexes for partitioned table

2018-01-25 Thread Alvaro Herrera
Alvaro Herrera wrote: > I think this is the right fix for this problem. I wonder about > exploring other callers of RelationGetIndexList to see who else could be > confused ... CLUSTER was also affected (and ALTER TABLE .. CLUSTER ON). Patched both and added simple tests. Couldn't detect any ot

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Andres Freund
Hi, On 2018-01-25 18:40:53 +0300, Konstantin Knizhnik wrote: > As far as I understand generation of native code is now always done for all > supported expressions and individually by each backend. Mostly, yes. It's done "always" done, because there's cost based checks whether to do so or not. >

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Andres Freund
Hi, On 2018-01-25 10:00:14 +0100, Pierre Ducroquet wrote: > I don't know when this would be released, August-October range. > but the minimal supported LLVM > version will have a strong influence on the availability of that feature. If > today this JIT compiling was released with only LLVM 5/

Re: Possible performance regression with pg_dump of a large number of relations

2018-01-25 Thread Stephen Frost
Luke, * Luke Cowell (lcow...@gmail.com) wrote: > > On Jan 24, 2018, at 2:56 PM, Stephen Frost wrote: > >>> ERROR: relation "pg_init_privs" does not exist > >>> LINE 139: LEFT JOIN pg_init_privs pip > > > > I certainly hope that works on 9.6, since that's when pg_init_privs was > > added.. > >

Re: plpgsql function startup-time improvements

2018-01-25 Thread Pavel Stehule
Hi 2018-01-25 0:16 GMT+01:00 Tom Lane : > Pavel Stehule writes: > > please, can you rebase all three patches necessary for patching? > > Done. These now need to be applied over > https://www.postgresql.org/message-id/833.1516834...@sss.pgh.pa.us Thank you I checked it 1. there are no proble

Re: CONSTANT/NOT NULL/initializer properties for plpgsql record variables

2018-01-25 Thread Daniel Gustafsson
> On 25 Jan 2018, at 16:34, Tom Lane wrote: > Daniel Gustafsson writes: >> One tiny thing: while not introduced in this patch, I wonder if it would be >> worth adding an errhint in the following hunk when applied to arrays, to >> clarify what CONSTANT in an array declaration mean. I have seen c

Re: Possible performance regression with pg_dump of a large number of relations

2018-01-25 Thread Luke Cowell
> On Jan 24, 2018, at 2:56 PM, Stephen Frost wrote: > > Hi there! > > >>> ERROR: relation "pg_init_privs" does not exist >>> LINE 139: LEFT JOIN pg_init_privs pip > > I certainly hope that works on 9.6, since that's when pg_init_privs was > added.. My mistake. That error is from my 9.5 ser

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Nikolay Shaplov
В письме от 25 января 2018 11:29:34 пользователь Tom Lane написал: > Alvaro Herrera writes: > > Tom Lane wrote: > >> Well, maybe the right answer is to address that. It's clear to me > >> why that would happen if we store these things as reloptions on the > >> toast table, but can't they be store

Re: [HACKERS] [POC] Faster processing at Gather node

2018-01-25 Thread Robert Haas
On Tue, Jan 9, 2018 at 7:09 PM, Andres Freund wrote: >> + * mq_sender and mq_bytes_written can only be changed by the sender. >> + * mq_receiver and mq_sender are protected by mq_mutex, although, >> importantly, >> + * they cannot change once set, and thus may be read without a lock once >> this

Re: Further cleanup of pg_dump/pg_restore item selection code

2018-01-25 Thread Tom Lane
"David G. Johnston" writes: > Should pg_restore fail if asked to --create without a database entry in the > TOC? Yeah, I wondered about that too. This patch makes it a non-issue for archives created with v11 or later pg_dump, but there's still a hazard if you're restoring from an archive made by

Re: [PROPOSAL] Shared Ispell dictionaries

2018-01-25 Thread Arthur Zakirov
Hello, Thank you for your review! Good catches. On Thu, Jan 25, 2018 at 03:26:46PM +0300, Ildus Kurbangaliev wrote: > In 0001 there are few lines where is only indentation has changed. Fixed. > 0002: > - TsearchShmemSize - calculating size using hash_estimate_size seems > redundant since you us

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Well, maybe the right answer is to address that. It's clear to me >> why that would happen if we store these things as reloptions on the >> toast table, but can't they be stored on the parent table? > Actually, Nikolay provided a possible solution: if

Re: Doc tweak for huge_pages?

2018-01-25 Thread Peter Eisentraut
On 1/22/18 01:10, Thomas Munro wrote: > Sorry, right, that was 100% wrong. It would probably be correct to > remove the "not", but let's just remove that bit. New version > attached. Committed that. I reordered some of the existing material because it seemed to have gotten a bit out of order wi

Re: unique indexes on partitioned tables

2018-01-25 Thread Jesper Pedersen
Hi Alvaro, On 01/22/2018 05:55 PM, Alvaro Herrera wrote: Alvaro Herrera wrote: Version 4 of this patch, rebased on today's master. Passes make check-world. Maybe add a test case to indexing.sql that highlights that hash indexes doesn't support UNIQUE; although not unique to partitioned i

CONSTANT/NOT NULL/initializer properties for plpgsql record variables

2018-01-25 Thread David G. Johnston
On Thursday, January 25, 2018, Tom Lane wrote: > > The documentation currently says > > The CONSTANT option prevents the variable from being assigned to > after initialization, so that its value will remain constant for > the duration of the block. > While we don't really

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Alvaro Herrera
Tom Lane wrote: > Peter Eisentraut writes: > > On 1/23/18 13:39, Robert Haas wrote: > >> I don't understand. AAUI, it is currently the case that if you set > >> the options before the TOAST table exists, they are lost. > > > Well, that's also weird. > > Well, maybe the right answer is to addres

Re: JIT compiling with LLVM v9.0

2018-01-25 Thread Konstantin Knizhnik
On 24.01.2018 10:20, Andres Freund wrote: Hi, I've spent the last weeks working on my LLVM compilation patchset. In the course of that I *heavily* revised it. While still a good bit away from committable, it's IMO definitely not a prototype anymore. There's too many small changes, so I'm only

Re: CONSTANT/NOT NULL/initializer properties for plpgsql record variables

2018-01-25 Thread Tom Lane
Daniel Gustafsson writes: > I’ve reviewed this patch (disclaimer: I did not review the patches listed > above > which it is based on) and the functionality introduced. The code is straight- > forward, there are ample tests and I can’t make it break however many weird > combinations thrown at it.

Re: [HACKERS] SERIALIZABLE with parallel query

2018-01-25 Thread Robert Haas
On Wed, Jan 24, 2018 at 7:39 PM, Thomas Munro wrote: > This started crashing some time yesterday with an assertion failure in > the isolation tests after commit 2badb5af landed. Reordering of code > in parallel.c confused patch's fuzz heuristics leading > SetSerializableXact() to be called too so

Re: PATCH: pgbench - option to build using ppoll() for larger connection counts

2018-01-25 Thread Fabien COELHO
ISTM that the v7 patch version you sent is identical to v6. -- Fabien.

Re: reducing isolation tests runtime

2018-01-25 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> Alvaro Herrera writes: >>> On the subject of test total time, we could paralelize isolation tests. >> BTW, one small issue there is that the reason the timeouts test is so >> slow is that we have to use multi-second timeouts to be sure slower >> buildf

Re: [HACKERS] Vacuum: allow usage of more than 1GB of work mem

2018-01-25 Thread Alvaro Herrera
Claudio Freire wrote: > On Thu, Jan 25, 2018 at 4:11 AM, Thomas Munro > wrote: > > *** 128,134 > > SELECT pg_relation_size('vactst', 'main'); > >pg_relation_size > > -- > > ! 0 > > (1 row) > > > > SELECT count(*) FROM vactst; > > --- 128,134 >

Re: [PATCH][PROPOSAL] Refuse setting toast.* reloptions when TOAST table does not exist

2018-01-25 Thread Tom Lane
Peter Eisentraut writes: > On 1/23/18 13:39, Robert Haas wrote: >> I don't understand. AAUI, it is currently the case that if you set >> the options before the TOAST table exists, they are lost. > Well, that's also weird. Well, maybe the right answer is to address that. It's clear to me why th

  1   2   >