Re: pgbench's expression parsing & negative numbers

2018-08-10 Thread Fabien COELHO
Hello, The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, passed Spec compliant: tested, passed Documentation:not tested Patch does not apply cleanly on the master branch, anyways

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Fabien COELHO
Hello Marina, I'd suggest to let lookupCreateVariable, putVariable* as they are, call pgbench_error with a level which does not stop the execution, and abort if necessary from the callers with a "aborted because of putVariable/eval/... error" message, as it was done before. There's one more

Re: xact_start meaning when dealing with procedures?

2018-08-10 Thread Peter Eisentraut
On 09/08/2018 20:25, Vik Fearing wrote: > On 09/08/18 20:13, Peter Eisentraut wrote: >> On 09/08/2018 19:57, hubert depesz lubaczewski wrote: >>> I just noticed that when I called a procedure that commits and rollbacks >>> - the xact_start in pg_stat_activity is not updated. Is it intentional? >> >

log_temp_files associated with "wrong" statement

2018-08-10 Thread Peter Eisentraut
log_temp_files makes a log entry when a temporary file is deleted. Temporary file deletion is usually organized by the resource owner mechanism. So usually it happens at the end of a query. But when the query is run through a cursor, it happens whenever the cursor is closed. So you might get a l

Re: Typo in doc or wrong EXCLUDE implementation

2018-08-10 Thread KES
huh, maybe you are right, I missread that. English is not my native language. Actually I come there from FK constraints. Would it be sufficient for FK require not UNIQUEs, but **allow** "EXCLUDE with operators that act like equality"? 09.08.2018, 22:31, "Tom Lane" : > Bruce Momjian writes: >>

Get funcid when create function

2018-08-10 Thread 王翔宇
I'm developing a extension for pg. Now I have create a event trigger on ddl_command_end, and this function will be called after I enter create function statement. In this function I can only get parseTree. In pg source code, I found a function named "pg_event_trigger_ddl_commands" seems provide cmd

Re: Postgres 11 release notes

2018-08-10 Thread Alexander Korotkov
On Fri, Aug 10, 2018 at 8:08 AM Masahiko Sawada wrote: > I found that the release note says "Add pgtrgm function > strict_word_similarity() to compute the similarity of whole words" but > I think "pgtrgm" should be "pg_trgm". Attached patch fixes it. > Right. Pushed, thanks! -- Alexander K

xact_start meaning when dealing with procedures?

2018-08-10 Thread hubert depesz lubaczewski
Hi I just noticed that when I called a procedure that commits and rollbacks - the xact_start in pg_stat_activity is not updated. Is it intentional? I'm on newest 12devel, built today. Best regards, depesz -- Hubert Lubaczewski (depesz) | DBA hlubaczew...@instructure.com Instructure.com

Re: csv format for psql

2018-08-10 Thread Daniel Verite
Pavel Stehule wrote: > > On the whole I'm inclined to resubmit the patch with > > fieldsep_csv and some minor changes based on the rest > > of the discussion. > > > > +1 PFA an updated version. Usage from the command line: $ psql --csv # or -P format=csv $ psql --csv -P fieldsep

Re: Constraint documentation

2018-08-10 Thread Peter Eisentraut
On 09/08/2018 23:32, Alvaro Herrera wrote: > I agree that we should point this out in *some* way, just not sure how. > Maybe something like "Postgres does not currently support CHECK > constraints containing queries, therefore we recommend to avoid them." > I would not mention pg_dump by name, just

Re: Doc patch for index access method function

2018-08-10 Thread Alexander Korotkov
Hi! On Fri, Aug 10, 2018 at 6:24 AM Tatsuro Yamada wrote: > > Attached patch for fixing documents of "61.2. Index Access Method Functions" > and > "61.6. Index Cost Estimation Functions". > > I added a variable "double *indexPages" introduced by commit 5262f7a4f to the > documents and > also ad

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 11:33, Fabien COELHO wrote: Hello Marina, I'd suggest to let lookupCreateVariable, putVariable* as they are, call pgbench_error with a level which does not stop the execution, and abort if necessary from the callers with a "aborted because of putVariable/eval/... error" message,

Re: Doc patch for index access method function

2018-08-10 Thread Alexander Korotkov
On Fri, Aug 10, 2018 at 1:37 PM Alexander Korotkov wrote: > On Fri, Aug 10, 2018 at 6:24 AM Tatsuro Yamada > wrote: > > > > Attached patch for fixing documents of "61.2. Index Access Method > > Functions" and > > "61.6. Index Cost Estimation Functions". > > > > I added a variable "double *indexP

Re: pgbench exit code

2018-08-10 Thread Fabien COELHO
The attached patch changes this so that it exits with status 1 and also prints a line below the summary advising that the results are incomplete. BTW, I have added this patch to the next CF: https://commitfest.postgresql.org/19/1751/ -- Fabien.

Re: Postgres, fsync, and OSs (specifically linux)

2018-08-10 Thread Thomas Munro
On Sun, Jul 29, 2018 at 6:14 PM, Thomas Munro wrote: > As a way of poking this thread, here are some more thoughts. I am keen to move this forward, not only because it is something we need to get fixed, but also because I have some other pending patches in this area and I want this sorted out fir

Re: Reopen logfile on SIGHUP

2018-08-10 Thread Alexander Kuzmenkov
On 08/09/2018 10:33 AM, Kyotaro HORIGUCHI wrote: - Since I'm not sure unlink is signal-handler safe on all supported platforms, I moved unlink() call out of checkLogrotateSignal() to SysLoggerMain, just before rotating log file. Which platforms specifically do you have in mind? unlink

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Arthur Zakirov
On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote: > > * ErrorLevel > > > > If ErrorLevel is used for things which are not errors, its name should > > not include "Error"? Maybe "LogLevel"? > > On the one hand, this sounds better for me too. On the other hand, will not > this be in

Re: csv format for psql

2018-08-10 Thread Fabien COELHO
Hello Daniel, PFA an updated version. Usage from the command line: $ psql --csv # or -P format=csv $ psql --csv -P fieldsep_csv=";" # for non-comma csv separator From inside psql: \pset format csv \pset fieldsep_csv '\t' Patch applies cleanly, compiles, global make check ok. Doc

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 15:53, Arthur Zakirov wrote: On Thu, Aug 09, 2018 at 06:17:22PM +0300, Marina Polyakova wrote: > * ErrorLevel > > If ErrorLevel is used for things which are not errors, its name should > not include "Error"? Maybe "LogLevel"? On the one hand, this sounds better for me too. On the

Re: Constraint documentation

2018-08-10 Thread Tom Lane
Peter Eisentraut writes: > I think it would be very easy to restore check constraints separately > after all tables in pg_dump. There is already support for that, but > it's only used when necessary, for things like not-valid constraints. > The argument in favor of keeping the constraint with the

Re: [HACKERS] WIP: long transactions on hot standby feedback replica / proof of concept

2018-08-10 Thread Alexander Korotkov
Hi! On Thu, Aug 9, 2018 at 11:26 PM Ivan Kartyshov wrote: > Alexander Korotkov писал 2018-06-20 20:54: > > Thinking about that more I found that adding vacuum mark as an extra > > argument to LockAcquireExtended is also wrong. It would be still hard > > to determine if we should log the lock in

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Arthur Zakirov
On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote: > > +1 from me to keep initial name "pgbench_error". "pgbench_log" for new > > function looks nice to me. I think it is better than just "log", > > because "log" may conflict with natural logarithmic function (see "man 3 > > log"). >

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-08-10 Thread Marina Polyakova
On 10-08-2018 17:19, Arthur Zakirov wrote: On Fri, Aug 10, 2018 at 04:46:04PM +0300, Marina Polyakova wrote: > +1 from me to keep initial name "pgbench_error". "pgbench_log" for new > function looks nice to me. I think it is better than just "log", > because "log" may conflict with natural logar

Creating extensions for non-superusers

2018-08-10 Thread Alexandra Ryzhevich
Hello! In an environment where we control the host system and all installed extensions, we need to allow postgresql non-superuser to install all of them, without opening gaps that will let this user gain superuser privileges. We have a sample solution to add a new default role pg_create_extension

Re: Creating extensions for non-superusers

2018-08-10 Thread Stephen Frost
Greetings, * Alexandra Ryzhevich (aryzhev...@google.com) wrote: > In an environment where we control the host system and all installed > extensions, we need to allow postgresql non-superuser to install all of > them, without opening gaps that will let this user gain superuser > privileges. We have

Re: Constraint documentation

2018-08-10 Thread David Fetter
On Fri, Aug 10, 2018 at 12:27:49PM +0200, Peter Eisentraut wrote: > On 09/08/2018 23:32, Alvaro Herrera wrote: > > I agree that we should point this out in *some* way, just not sure how. > > Maybe something like "Postgres does not currently support CHECK > > constraints containing queries, therefor

Re: Constraint documentation

2018-08-10 Thread David Fetter
On Fri, Aug 10, 2018 at 09:47:09AM -0400, Tom Lane wrote: > Peter Eisentraut writes: > > I think it would be very easy to restore check constraints separately > > after all tables in pg_dump. There is already support for that, but > > it's only used when necessary, for things like not-valid const

Re: Constraint documentation

2018-08-10 Thread Andres Freund
On August 10, 2018 7:17:09 PM GMT+05:30, Tom Lane wrote: >Peter Eisentraut writes: >> I think it would be very easy to restore check constraints separately >> after all tables in pg_dump. There is already support for that, but >> it's only used when necessary, for things like not-valid constr

Re: Improve behavior of concurrent TRUNCATE

2018-08-10 Thread Michael Paquier
On Thu, Aug 09, 2018 at 05:55:54PM +, Bossart, Nathan wrote: > On 8/9/18, 11:31 AM, "Michael Paquier" wrote: >> Thanks, I have updated the patch as you suggested. Any more >> improvements to it that you can foresee? > > Looks good to me. Okay, pushed to HEAD. Now remains the cases for VACU

Re: libpq should append auth failures, not overwrite

2018-08-10 Thread Tom Lane
Michael Paquier writes: > On Thu, Aug 09, 2018 at 11:44:27AM -0400, Tom Lane wrote: >> I noticed that, although most error reports during libpq's connection >> setup code append to conn->errorMessage, the ones in fe-auth.c and >> fe-auth-scram.c don't: they're all printfPQExpBuffer() not >> appen

Re: xact_start meaning when dealing with procedures?

2018-08-10 Thread Michael Paquier
On Thu, Aug 09, 2018 at 07:49:31PM +0200, hubert depesz lubaczewski wrote: > I just noticed that when I called a procedure that commits and rollbacks > - the xact_start in pg_stat_activity is not updated. Is it intentional? > > I'm on newest 12devel, built today. That sounds incorrect to me. I d

Re: Documentaion fix.

2018-08-10 Thread Michael Paquier
On Thu, Aug 09, 2018 at 03:58:09PM -0400, Alvaro Herrera wrote: > Yeah. I suggest never changing subject lines, because Gmail has the > nasty (mis-)feature of making such a response into a completely new > thread. I don't know if Google paid mail service behaves in the same > way. If one uses Gm

Re: NLS handling fixes.

2018-08-10 Thread Michael Paquier
On Fri, Aug 10, 2018 at 03:21:31PM +0900, Kyotaro HORIGUCHI wrote: > The cause is GetConfigOptionByNum is forgetting to retrieve > translations for texts that have been marked with gettext_noop. > > Regarding GUCs, group names, short desc and long desc have > translations so the attached first patc

Re: NLS handling fixes.

2018-08-10 Thread Alvaro Herrera
On 2018-Aug-10, Michael Paquier wrote: > On Fri, Aug 10, 2018 at 03:21:31PM +0900, Kyotaro HORIGUCHI wrote: > > The cause is GetConfigOptionByNum is forgetting to retrieve > > translations for texts that have been marked with gettext_noop. > > > > Regarding GUCs, group names, short desc and long d

Re: Improve behavior of concurrent TRUNCATE

2018-08-10 Thread Alvaro Herrera
On 2018-Aug-06, Michael Paquier wrote: > Attached is a patch I have been working on which refactors the code of > TRUNCATE in such a way that we check for privileges before trying to > acquire a lock, without any user-facing impact (I have reworked a couple > of comments compared to the last versi

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2018-08-10 Thread Tom Lane
I wrote: >> ... I'll >> take a look at whipping up something that checks /etc/localtime. > Here's a draft patch. It seems to do what I expect on a couple of > different macOS releases as well as recent Fedora. The cfbot points out that this has suffered bit-rot, so here's a rebased version --- n

Re: Allowing printf("%m") only where it actually works

2018-08-10 Thread Tom Lane
In the hopes of getting the cfbot un-stuck (it's currently trying to test a known-not-to-work patch), here are updated versions of the two live patches we have in this thread. 0001 is the patch I originally proposed to adjust printf archetypes. 0002 is Thomas's patch to blow up on errno reference

Re: NLS handling fixes.

2018-08-10 Thread Tom Lane
Michael Paquier writes: > I have been looking at all the things you are proposing here, and it > seems to me that you are right for these. I lack a bit of knowledge > about the translation of items, but can such things be back-patched? I would certainly *not* back-patch the GetConfigOptionByNum

Re: Improve behavior of concurrent TRUNCATE

2018-08-10 Thread Michael Paquier
On Fri, Aug 10, 2018 at 02:03:28PM -0400, Alvaro Herrera wrote: > On 2018-Aug-06, Michael Paquier wrote: >> Like cbe24a6, perhaps we would not want to back-patch it? Based on the >> past history (and the consensus being reached for the REINDEX case would >> be to patch only HEAD), I would be actua

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-10 Thread Tomas Vondra
On 08/09/2018 07:47 PM, Alvaro Herrera wrote: > On 2018-Aug-09, Tomas Vondra wrote: > >> I suppose there are reasons why it's done this way, and admittedly the test >> that happens to trigger this is a bit extreme (essentially running pgbench >> concurrently with 'vacuum full pg_class' in a loo

Re: Allowing printf("%m") only where it actually works

2018-08-10 Thread Tom Lane
I wrote: > I think 0002 is probably pushable, really. The unresolved issue about > 0001 is whether it will create a spate of warnings on Windows builds, > and if so what to do about it. We might learn something from the > cfbot about that, but I think the full buildfarm is going to be the > only

Re: Improve behavior of concurrent TRUNCATE

2018-08-10 Thread Alvaro Herrera
On 2018-Aug-10, Michael Paquier wrote: > On Fri, Aug 10, 2018 at 02:03:28PM -0400, Alvaro Herrera wrote: > > On 2018-Aug-06, Michael Paquier wrote: > >> Like cbe24a6, perhaps we would not want to back-patch it? Based on the > >> past history (and the consensus being reached for the REINDEX case w

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-10 Thread Andres Freund
On 2018-08-10 22:57:57 +0200, Tomas Vondra wrote: > > > On 08/09/2018 07:47 PM, Alvaro Herrera wrote: > > On 2018-Aug-09, Tomas Vondra wrote: > > > >> I suppose there are reasons why it's done this way, and admittedly the test > >> that happens to trigger this is a bit extreme (essentially runni

Re: libpq compression

2018-08-10 Thread Andrew Dunstan
On 06/25/2018 05:32 AM, Konstantin Knizhnik wrote: On 18.06.2018 23:34, Robbie Harwood wrote: ### Documentation!  You're going to need it.  There needs to be enough around for other people to implement the protocol (or if you prefer, enough for us to debug the protocol as it exists). In

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-10 Thread Tomas Vondra
On 08/10/2018 11:13 PM, Andres Freund wrote: > On 2018-08-10 22:57:57 +0200, Tomas Vondra wrote: >> >> >> On 08/09/2018 07:47 PM, Alvaro Herrera wrote: >>> On 2018-Aug-09, Tomas Vondra wrote: >>> I suppose there are reasons why it's done this way, and admittedly the test that happens to

[sqlsmith] ERROR: partition missing from subplans

2018-08-10 Thread Andreas Seltenreich
Hi, running sqlsmith on REL_11_STABLE at 1b9d1b08fe for a couple hours yielded the previously-unseen internal error "partition missing from subplans". It is readily reproducible on the regression database with the following query: select * from public.fk_partitioned_fk as sample_0 tablesample sy

Re: Commitfest 2018-07 WOA items

2018-08-10 Thread Andrew Dunstan
On 08/09/2018 06:00 PM, Andrew Dunstan wrote: https://commitfest.postgresql.org/18/1644/ Add --include-table-data-where option to pg_dump, to export only a subset of table data I'm not really clear what we're waiting on the author for. https://commitfest.postgresql.org/18/1635/ libpq com

Commitfest 2018-07 is closed

2018-08-10 Thread Andrew Dunstan
All the remaining items have been moved to the next commitfest, in some cases with changed status. Thanks to all the authors, reviewers and committers. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training

Re: logical decoding / rewrite map vs. maxAllocatedDescs

2018-08-10 Thread Tomas Vondra
On 08/10/2018 11:59 PM, Tomas Vondra wrote: > > ... > > I suspect there's some other ingredient, e.g. some manipulation with the > subscription. Or maybe it's not needed at all and I'm just imagining things. > Indeed, the manipulation with the subscription seems to be the key here. I pretty rel

Re: [sqlsmith] ERROR: partition missing from subplans

2018-08-10 Thread David Rowley
On 11 August 2018 at 10:12, Andreas Seltenreich wrote: > running sqlsmith on REL_11_STABLE at 1b9d1b08fe for a couple hours > yielded the previously-unseen internal error "partition missing from > subplans". It is readily reproducible on the regression database with > the following query: > > sel

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-10 Thread Tom Lane
Peter Geoghegan writes: > On Wed, Aug 8, 2018 at 7:40 PM, Tom Lane wrote: >> Oooh ... but pg_class wouldn't be big enough to get a parallel >> index rebuild during that test, would it? > Typically not, but I don't think that we can rule it out right away. Didn't take long to show that the relma

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-10 Thread Peter Geoghegan
On Fri, Aug 10, 2018 at 7:45 PM, Tom Lane wrote: > Didn't take long to show that the relmapper issue wasn't it: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=coypu&dt=2018-08-10%2021%3A21%3A40 > > So we're back to square one. Although Tomas' recent report might > give us something n

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-10 Thread Tom Lane
Peter Geoghegan writes: > I'm concerned that this item has the potential to delay the release, > since, as you said, we're back to the drawing board. Me too. I will absolutely not vote to release 11.0 before we've solved this ... regards, tom lane

Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes

2018-08-10 Thread Peter Geoghegan
On Fri, Aug 10, 2018 at 8:02 PM, Tom Lane wrote: > Me too. I will absolutely not vote to release 11.0 before we've > solved this ... I believe that that's the right call, assuming things don't change. This is spooky in a way that creates a lot of doubts in my mind. I don't think it's at all adv

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2018-08-10 Thread Mark Rofail
I am still having problems rebasing this patch. I can not figure it out on my own. On Sun, 27 May 2018 at 5:31 pm, Mark Rofail wrote: > issue 1: `pg_constraint.c:564` > I need to check that `conppeqop` is not null and copy it but I don't know > how to check its type since its a char* > > issue 2

Re: [sqlsmith] ERROR: partition missing from subplans

2018-08-10 Thread David Rowley
On 11 August 2018 at 14:09, David Rowley wrote: > So my best idea on a > fix is simply to remove the code that raises the error. Here's a patch to do that. I've also included a further simplified test to ensure this case performs run-time pruning correctly. -- David Rowley ht

Re: csv format for psql

2018-08-10 Thread Pavel Stehule
2018-08-10 12:25 GMT+02:00 Daniel Verite : > Pavel Stehule wrote: > > > > On the whole I'm inclined to resubmit the patch with > > > fieldsep_csv and some minor changes based on the rest > > > of the discussion. > > > > > > > +1 > > PFA an updated version. > Usage from the command line: >