Re: [Patch] Create a new session in postmaster by calling setsid()

2018-09-11 Thread Michael Paquier
On Mon, Aug 06, 2018 at 12:11:26PM +0800, Paul Guo wrote: > Yes, if considering the case of starting postmaster manually, we can not > create > a new session in postmaster, so pg_ctl seems to be a good place for setsid() > call. Attached a newer patch. Thanks. Hmm. This patch breaks a feature of

Re: review printing ecpg program version

2018-09-11 Thread Ioseph Kim
check please pg_config --version too. 2018년 09월 12일 15:52에 Ioseph Kim 이(가) 쓴 글: hi, I found a little problem about printing ecpg version. unlike PostgreSQL ver.11 other programs, ecpg version string has only version number. $ ./ecpg --version ecpg 11beta3 $ ./psql --version psql (Postgre

review printing ecpg program version

2018-09-11 Thread Ioseph Kim
hi, I found a little problem about printing ecpg version. unlike PostgreSQL ver.11 other programs, ecpg version string has only version number. $ ./ecpg --version ecpg 11beta3 $ ./psql --version psql (PostgreSQL) 11beta3 $ ./pg_dump --version pg_dump (PostgreSQL) 11beta3 I hope PostgreSQL

Re: [HACKERS] pgbench - allow to store select results into variables

2018-09-11 Thread Michael Paquier
Hi Stephen, On Tue, Aug 14, 2018 at 01:38:21PM +0200, Fabien COELHO wrote: > I re-attached the v19 for a check on the list. You are marked as the committer of this patch in the CF app since last April and this patch is marked as ready for committer. Are you planning to look at it soon? -- Michae

Re: Loaded footgun open_datasync on Windows

2018-09-11 Thread Michael Paquier
On Mon, Sep 10, 2018 at 04:46:40PM +0200, Laurenz Albe wrote: > I didn't get pg_upgrade to work without the log file hacks; I suspect > that there is more than just log file locking going on, but my Windows > skills are limited. > > How shall I proceed? I do like this patch, and we have an occasi

RE: Protect syscache from bloating with negative cache entries

2018-09-11 Thread Ideriha, Takeshi
Hi, >Subject: Re: Protect syscache from bloating with negative cache entries > >Hello. The previous v4 patchset was just broken. >Somehow the 0004 was merged into the 0003 and applying 0004 results in >failure. I >removed 0004 part from the 0003 and rebased and repost it. I have some questions

Re: [HACKERS] Secondary index access optimizations

2018-09-11 Thread David Rowley
On 12 September 2018 at 08:32, Konstantin Knizhnik wrote: > Also the patch proposed by you is much simple and does mostly the same. Yes, > it is not covering CHECK constraints, > but as far as partitioning becomes now standard in Postgres, I do not think > that much people will use old inheritance

Re: [HACKERS] Restricting maximum keep segments by repslots

2018-09-11 Thread Masahiko Sawada
On Mon, Sep 10, 2018 at 7:19 PM, Kyotaro HORIGUCHI wrote: > Hello. > > At Thu, 6 Sep 2018 19:55:39 +0900, Masahiko Sawada > wrote in >> On Thu, Sep 6, 2018 at 4:10 PM, Kyotaro HORIGUCHI >> wrote: >> > Thank you for the comment. >> > >> > At Wed, 5 Sep 2018 14:31:10 +0900, Masahiko Sawada >>

Re: Retrieve memory size allocated by libpq

2018-09-11 Thread Pavel Stehule
2018-09-12 0:59 GMT+02:00 Tom Lane : > Pavel Stehule writes: > > The implementation of PQresultMemsize is simple and correct. > > Hm, well, only for small values of "correct". A quick look at PQclear > to see just what it clears shows that PGEvent objects associated with the > PGresult were over

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-11 Thread Michael Paquier
On Tue, Sep 11, 2018 at 11:27:08PM -0400, Tom Lane wrote: > Michael Paquier writes: >> It is possible to get away with the error by using _stat64(), which >> returns as result a _stat64 structure. Still, it has one difference >> with the native result returned by stat() (which maps to _stat64i32)

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-11 Thread Tom Lane
Michael Paquier writes: > At the end, I have finally been able to put my hands on a Windows VM > which uses VS2015, and I am able to see the problem. In short, Windows > definition of stat() is an utter mess as this documentation page, which > is the latest one available, nicely summarizes: > htt

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-11 Thread Michael Paquier
On Tue, Sep 11, 2018 at 10:36:51AM +, Higuchi, Daisuke wrote: > So, attached patch help me and strange message disappeared, > but I ignore the impact of this for others now. @@ -386,7 +386,6 @@ pgwin32_safestat(const char *path, struct stat *buf) return -1; } - retu

Re: [HACKERS] Can ICU be used for a database's default sort order?

2018-09-11 Thread Thomas Munro
On Sat, Jun 24, 2017 at 10:55 AM Peter Geoghegan wrote: > On Fri, Jun 23, 2017 at 11:32 AM, Peter Eisentraut > wrote: > > It's something I hope to address soon. > > I hope you do. I think that we'd realize significant benefits by > having ICU become the defacto standard collation provider, that m

Re: Retrieve memory size allocated by libpq

2018-09-11 Thread Tom Lane
Pavel Stehule writes: > The implementation of PQresultMemsize is simple and correct. Hm, well, only for small values of "correct". A quick look at PQclear to see just what it clears shows that PGEvent objects associated with the PGresult were overlooked. While that's not hard to fix (and I did

Re: libpq stricter integer parsing

2018-09-11 Thread Michael Paquier
On Tue, Sep 11, 2018 at 07:03:41PM +0200, Fabien COELHO wrote: > Attached Michael's simplified version updated with your remarks. Okay, this version looks good to me so pushed. Thanks Fabien and Peter. -- Michael signature.asc Description: PGP signature

Re: Stored procedures and out parameters

2018-09-11 Thread Peter Eisentraut
On 30/08/2018 21:35, Robert Haas wrote: > The semantics you've chosen for procedures are more like Oracle that > the existing function semantics, which, as I can attest from my work > experience, can be very useful for users looking to migrate. Worth > noting, however, is Oracle also has those sem

Re: Collation versioning

2018-09-11 Thread Peter Eisentraut
On 07/09/2018 23:34, Thomas Munro wrote: > On Thu, Sep 6, 2018 at 5:36 PM Thomas Munro > wrote: >> On Thu, Sep 6, 2018 at 12:01 PM Peter Eisentraut >> wrote: >>> Also, note that this mechanism only applies to collation objects, not to >>> database-global locales. So most users wouldn't be helped

Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

2018-09-11 Thread Tom Lane
Paul Guo writes: > [ 0001-Fix-pg_upgrade-test-failure-caused-by-the-DDL-below.v2.patch ] Actually, there's an even easier way to fix this, which is to discard the special case for BITOID/VARBITOID altogether, and let the "default" case handle it. Fixing things by removing code is always great wh

Re: [HACKERS] Secondary index access optimizations

2018-09-11 Thread Konstantin Knizhnik
Hi David, On 11.09.2018 06:49, David Rowley wrote: On 9 July 2018 at 13:26, David Rowley wrote: I started looking over this patch and have a few comments: Hi Konstantin, Wondering, are you going to be submitting an updated patch for this commitfest? If not then I think we can mark this as

Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)

2018-09-11 Thread Andres Freund
Hi, On 2018-09-11 16:13:15 -0400, Andrew Dunstan wrote: > I've pushed support for the latest MSVC compilers back to all live branches. Thanks! Greetings, Andres Freund

Re: Windows vs C99 (was Re: C99 compliance for src/port/snprintf.c)

2018-09-11 Thread Andrew Dunstan
On 08/24/2018 03:42 PM, Andrew Dunstan wrote: On 08/24/2018 02:38 PM, Tom Lane wrote: Andres Freund writes: On 2018-08-24 14:09:09 -0400, Andrew Dunstan wrote: However, we only support VS2017 down to 9.6 and Vs2015 down to 9.5. Perhaps we should consider backpatching support for those d

Re: libpq stricter integer parsing

2018-09-11 Thread Fabien COELHO
+ ... I would leave this out. We don't need to document every single refinement of parsing rules. This might better belong in the release notes. Why not, I'd be fine with that. The fact that the libpq parser was quite fuzzy was not documented, the behavior was really a side effect of th

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Andres Freund
On 2018-09-11 12:50:06 -0400, Tom Lane wrote: > I am not sure which part of "I will not fix this" you didn't understand. Maybe the "this is an open list, and we can discuss things" bit?

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Andres Freund writes: > On 2018-09-11 12:26:44 -0400, Tom Lane wrote: >> Well, there remains the fact that we've seen no field reports that seem >> to trace to failure-to-acquire-AEL since 9.6 came out. So arguing that >> this *could* be a probable scenario fails to comport with the available >>

Re: Prevent concurrent DROP SCHEMA when certain objects are being initially created in the namespace

2018-09-11 Thread Robert Haas
On Wed, Sep 5, 2018 at 1:05 AM, Tom Lane wrote: >> Would expanding this a git further really be that noticeable? > > Frankly, I think it would be not so much "noticeable" as "disastrous". > > Making the overhead tolerable would require very large compromises > in coverage, perhaps like "we'll only

Re: Slotification of partition tuple conversion

2018-09-11 Thread Amit Khandekar
On 3 September 2018 at 12:14, Amit Langote wrote: > Hi Amit, > > Thanks for the updated patch and sorry I couldn't reply sooner. > > On 2018/08/21 16:18, Amit Khandekar wrote: >> On 21 August 2018 at 08:12, Amit Langote >> wrote: >>> Here are some comments on the patch: >> >> Thanks for the revi

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Andres Freund
On 2018-09-11 12:26:44 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-09-11 12:18:59 -0400, Tom Lane wrote: > >> Doesn't matter: startup would hit a lock conflict and cancel the pg_dump > >> to get out of it, long before approaching locktable full. > > > Only if all that's happening i

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Andres Freund writes: > On 2018-09-11 12:18:59 -0400, Tom Lane wrote: >> Doesn't matter: startup would hit a lock conflict and cancel the pg_dump >> to get out of it, long before approaching locktable full. > Only if all that's happening in the same database, which is far from a > given. Well, t

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Andres Freund
On 2018-09-11 12:18:59 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-09-11 12:03:44 -0400, Tom Lane wrote: > >> If the startup process has acquired enough AELs to approach locktable > >> full, any concurrent pg_dump has probably failed already, because it'd > >> be trying to share-loc

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Andres Freund writes: > On 2018-09-11 12:03:44 -0400, Tom Lane wrote: >> If the startup process has acquired enough AELs to approach locktable >> full, any concurrent pg_dump has probably failed already, because it'd >> be trying to share-lock every table and so would have a huge conflict >> cross

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Andres Freund
Hi, On 2018-09-11 12:03:44 -0400, Tom Lane wrote: > Andres Freund writes: > > Isn't one of the most common ways to run into "out of shared memory" > > "You might need to increase max_locks_per_transaction." to run pg_dump? > > And that's pretty commonly done against standbys? > > If the startup

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Andres Freund writes: > On 2018-09-11 16:23:44 +0100, Simon Riggs wrote: >> It's hard to see how any reasonable workload would affect the standby. And >> if it did, you'd change the parameter and restart, just like you already >> have to do if someone changes max_connections on master first. > Is

Re: patch to allow disable of WAL recycling

2018-09-11 Thread Peter Eisentraut
On 10/09/2018 16:10, Jerry Jelinek wrote: > Thank you again for running all of these tests on your various hardware > configurations. I was not aware of the convention that the commented > example in the config file is expected to match the default value, so I > was actually trying to show what to

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Robert Haas writes: > On Tue, Sep 11, 2018 at 10:25 AM, Tom Lane wrote: >> The point of the previous coding here was that perhaps there's some >> range of number-of-locks-needed where kicking hot-standby queries >> off of locks would allow recovery to proceed. However, it is (as >> far as I know

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Andres Freund
Hi, On 2018-09-11 16:23:44 +0100, Simon Riggs wrote: > It's hard to see how any reasonable workload would affect the standby. And > if it did, you'd change the parameter and restart, just like you already > have to do if someone changes max_connections on master first. Isn't one of the most commo

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Fabien COELHO
Hello Marina, Hmm, but we can say the same for serialization or deadlock errors that were not retried (the client test code itself could not run correctly or the SQL sent was somehow wrong, which is also the client's fault), can't we? I think not. If a client asks for something "legal", bu

Re: Flexible configuration for full-text search

2018-09-11 Thread Tom Lane
Aleksandr Parfenov writes: > As I wrote few weeks ago, there is a issue with stopwords processing in > proposed syntax for full-text configurations. I want to separate word > normalization and stopwords detection to two separate dictionaries. The > problem is how to configure stopword detection di

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Simon Riggs
On 11 September 2018 at 16:11, Robert Haas wrote: > On Tue, Sep 11, 2018 at 10:25 AM, Tom Lane wrote: > > The point of the previous coding here was that perhaps there's some > > range of number-of-locks-needed where kicking hot-standby queries > > off of locks would allow recovery to proceed. H

Re: Bug report: Dramatic increase in conflict with recovery after upgrading 10.2->10.5

2018-09-11 Thread Justin Pryzby
On Mon, Sep 10, 2018 at 11:43:47AM +0200, Chris Travers wrote: > At present I believe this to likely be a regression. But if nobody else > knows otherwise, I should know more in a couple days. Do you have query logs or can you send details of the query ? We're not using replication, but I can't

Re: [HACKERS] Bug in to_timestamp().

2018-09-11 Thread Arthur Zakirov
On Sun, Sep 09, 2018 at 09:52:46PM +0300, Alexander Korotkov wrote: > So, pushed! Thanks to every thread participant for review and feedback. Great! Should we close the commitfest entry? There is FX part of the patch though. But it seems that nobody is happy with it. It could be done with a separ

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Robert Haas writes: > On Tue, Sep 11, 2018 at 5:54 AM, Simon Riggs wrote: >> Please explain why you think that would be with no restart. > Because the startup process will die, and if that happens, IIRC, > there's no crash-and-restart loop. You're just done. Unless we think that the startup pr

Re: Hint to set owner for tablespace directory

2018-09-11 Thread Peter Eisentraut
On 07/09/2018 17:59, Maksim Milyutin wrote: >> You are still proposing to hint that the permissions on the tablespace >> directory might be correct but that the main database instance is >> running under the wrong user. > Not exactly this way. The case that motivated me to make this patch was > th

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Robert Haas
On Tue, Sep 11, 2018 at 10:25 AM, Tom Lane wrote: > The point of the previous coding here was that perhaps there's some > range of number-of-locks-needed where kicking hot-standby queries > off of locks would allow recovery to proceed. However, it is (as > far as I know) unproven that that actual

Re: pgbench - add pseudo-random permutation function

2018-09-11 Thread Fabien COELHO
Hello Hironobu-san, I reviewed `pgbench-prp-func-1.patch` and found an incomplete implementation. Indeed, thanks a lot for the debug, and the proposed fix! I'm going to work a little bit more on the patch based on your proposed changes, and submit a v3, hopefully soon. -- Fabien.

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Robert Haas
On Tue, Sep 11, 2018 at 5:54 AM, Simon Riggs wrote: > Please explain why you think that would be with no restart. Because the startup process will die, and if that happens, IIRC, there's no crash-and-restart loop. You're just done. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The E

Re: CREATE ROUTINE MAPPING

2018-09-11 Thread David Fetter
On Mon, Sep 10, 2018 at 09:28:31AM +0200, Hannu Krosing wrote: > Hi Corey > > Have you looked at pl/proxy ? DBI-Link pre-dated PL/proxy by some years, and was a good bit more flexible as to what types of functions it could send where. Neither has a capability fundamentally similar to this because

Re: libpq stricter integer parsing

2018-09-11 Thread Peter Eisentraut
On 11/09/2018 11:00, Michael Paquier wrote: > diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml > index 5e7931ba90..bc7836d103 100644 > --- a/doc/src/sgml/libpq.sgml > +++ b/doc/src/sgml/libpq.sgml > @@ -1591,6 +1591,15 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname > > >

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Tom Lane
Simon Riggs writes: > On 10 September 2018 at 19:16, Robert Haas wrote: >> On Fri, Sep 7, 2018 at 6:37 PM, Tom Lane wrote: >>> So my inclination is to remove the reportMemoryError = false parameter, >>> and just let an error happen in the unlikely situation that we hit OOM >>> for the lock table

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 11-09-2018 16:47, Marina Polyakova wrote: On 08-09-2018 16:03, Fabien COELHO wrote: Hello Marina, I'd insist in a comment that "cnt" does not include "skipped" transactions (anymore). If you mean CState.cnt I'm not sure if this is practically useful because the code uses only the sum of a

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 08-09-2018 16:03, Fabien COELHO wrote: Hello Marina, v11-0003-Pgbench-errors-and-serialization-deadlock-retrie.patch - the main patch for handling client errors and repetition of transactions with serialization/deadlock failures (see the detailed description in the file). About patch v11

Re: Index Skip Scan

2018-09-11 Thread Jesper Pedersen
Hi Dmitry, On 9/10/18 5:47 PM, Dmitry Dolgov wrote: On Mon, 18 Jun 2018 at 17:26, Jesper Pedersen wrote: I've looked through the patch more closely, and have a few questions: Thanks for your review ! * Is there any reason why only copy function for the IndexOnlyScan node includes an i

RE: stat() on Windows might cause error if target file is larger than 4GB

2018-09-11 Thread Higuchi, Daisuke
Michael-san, From: Michael Paquier [mailto:mich...@paquier.xyz] >Does something like the patch attached help? >This makes sure that st_size is set correctly for files with a size larger >than 4GB. Thank you for creating patch, but this does not solve current problem. Of cause setting wrong siz

Re: CREATE ROUTINE MAPPING

2018-09-11 Thread Masahiko Sawada
Thank you for the comment. On Mon, Sep 10, 2018 at 4:16 PM, Kyotaro HORIGUCHI wrote: > Hello. > > At Tue, 4 Sep 2018 09:34:21 +0900, Masahiko Sawada > wrote in >> On Tue, Sep 4, 2018 at 5:48 AM, David Fetter wrote: >> > On Fri, Aug 31, 2018 at 05:18:26PM +0900, Masahiko Sawada wrote: >> >> On

Re: Flexible configuration for full-text search

2018-09-11 Thread Aleksandr Parfenov
Hello hackers! As I wrote few weeks ago, there is a issue with stopwords processing in proposed syntax for full-text configurations. I want to separate word normalization and stopwords detection to two separate dictionaries. The problem is how to configure stopword detection dictionary. The cause

Re: StandbyAcquireAccessExclusiveLock doesn't necessarily

2018-09-11 Thread Simon Riggs
On 10 September 2018 at 19:16, Robert Haas wrote: > On Fri, Sep 7, 2018 at 6:37 PM, Tom Lane wrote: > > So my inclination is to remove the reportMemoryError = false parameter, > > and just let an error happen in the unlikely situation that we hit OOM > > for the lock table. > > Wouldn't that tak

Re: Query is over 2x slower with jit=on

2018-09-11 Thread Amit Khandekar
On 10 September 2018 at 21:39, Andres Freund wrote: > Hi, > > On 2018-09-10 15:42:55 +0530, Amit Khandekar wrote: >> Attached is a patch that accumulates the per-worker jit counters into >> the leader process. > > Thanks! > > >> I think we better show per-worker jit info also. The current patch >>

Re: libpq stricter integer parsing

2018-09-11 Thread Michael Paquier
On Sun, Sep 09, 2018 at 09:01:15AM +0200, Fabien COELHO wrote: > Hmmm. This is what the sentence following the above tries to explain > implicitely: > > Versions of libpq before > PostgreSQL 12 accepted trailing garbage or overflows. > > Maybe I can rephrase it in one sentence, eg: > > "From

Re: [HACKERS] WIP Patch: Pgbench Serialization and deadlock errors

2018-09-11 Thread Marina Polyakova
On 08-09-2018 10:17, Fabien COELHO wrote: Hello Marina, Hello, Fabien! About the two first preparatory patches. v11-0001-Pgbench-errors-use-the-RandomState-structure-for.patch - a patch for the RandomState structure (this is used to reset a client's random seed during the repeating of tran

Re: Avoid extra Sort nodes between WindowAggs when sorting can be reused

2018-09-11 Thread Masahiko Sawada
On Sat, Jul 28, 2018 at 4:12 AM, Alexander Kuzmenkov wrote: > Daniel, > > Thanks for the update. > > > On 07/25/2018 01:37 AM, Daniel Gustafsson wrote: >> >> >>> Hmm, this is missing the eqop fields of SortGroupClause. I haven't >>> tested yet but does the similar degradation happen if two >>> Sor

Re: Pluggable Storage - Andres's take

2018-09-11 Thread Beena Emerson
Hello, On Mon, 10 Sep 2018, 19:33 Amit Kapila, wrote: > On Mon, Sep 10, 2018 at 1:12 PM Haribabu Kommi > wrote: > > > > On Wed, Sep 5, 2018 at 2:04 PM Haribabu Kommi > wrote: > >> > > pg_stat_get_tuples_hot_updated and others: > > /* > > * Counter tuples_hot_updated stores number of hot update

Re: stat() on Windows might cause error if target file is larger than 4GB

2018-09-11 Thread Michael Paquier
Higuchi-san, On Mon, Sep 10, 2018 at 03:44:54PM +0900, Michael Paquier wrote: > Yes, the fix needs to happen there. It seems to me that what we are > looking for here is to complete the calculation of st_size with > nFileSizeHigh, so as we are able to compile a correct 64-bit integer > size, allo