Re: hostorder and failover_timeout for libpq

2018-10-01 Thread Michael Paquier
On Wed, Sep 19, 2018 at 02:26:53PM +0200, Ildar Musin wrote: > Another approach would be to leave `pg_conn->connhost` as it is now (i.e. > not to create global addresses array) and just apply random permutations to > it if `hostorder=random` is specified. And probably apply permutations to > addres

Re: Hash Joins vs. Bloom Filters / take 2

2018-10-01 Thread Michael Paquier
On Thu, Mar 01, 2018 at 07:04:41PM -0500, David Steele wrote: > After reviewing the thread I also agree that this should be pushed to > 2018-09, so I have done so. > > I'm very excited by this patch, though. In general I agree with Peter that > a higher rate of false positives is acceptable to sa

Re: [HACKERS] SERIALIZABLE with parallel query

2018-10-01 Thread Michael Paquier
On Wed, Sep 19, 2018 at 04:50:40PM -0500, Kevin Grittner wrote: > I will spend a few more days in testing and review, but figured I > should pass along "first impressions" now. Kevin, it seems that this patch is pending on your input. I have moved this patch to next CF for now. -- Michael signa

Re: Relations being opened without any lock whatever

2018-10-01 Thread Amit Langote
On 2018/10/01 4:20, Tom Lane wrote: > Running the regression tests with the patch I showed in > https://www.postgresql.org/message-id/16565.1538327...@sss.pgh.pa.us > exposes two places where HEAD is opening relations without having > any lock at all on them: Maybe you've noticed but the relation_

Re: partition tree inspection functions

2018-10-01 Thread Amit Langote
On 2018/10/01 15:27, Michael Paquier wrote: > On Mon, Oct 01, 2018 at 03:16:32PM +0900, Amit Langote wrote: >> I wasn't able to respond to some of issues that Jesper brought up with the >> approach taken by the latest patch whereby there is no separate >> pg_partition_level function. He said that

Re: Query is over 2x slower with jit=on

2018-10-01 Thread Lukas Fittl
On Tue, Sep 25, 2018 at 1:17 PM, Andres Freund wrote: > > I've pushed the change without that bit - it's just a few additional > lines if we want to change that. > It seems that since this commit, JIT statistics are now only being printed for parallel query plans. This is due to ExplainPrintJIT b

[PROPOSAL]a new data type 'bytea' for ECPG

2018-10-01 Thread Matsumura, Ryo
Hi, Hackers # This is my first post. I will try to implement a new data type 'bytea' for ECPG. I think that the implementation is not complicated. Does anyone need it ? * Why do I need bytea ? Currently, ECPG program can treat binary data for bytea column with 'char' type of C language, but it

Re: [PROPOSAL] Shared Ispell dictionaries

2018-10-01 Thread Arthur Zakirov
On Thu, Jun 14, 2018 at 11:40:17AM +0300, Arthur Zakirov wrote: > I attached new version of the patch. The patch still applies to HEAD. I moved it to the next commitfest. -- Arthur Zakirov Postgres Professional: http://www.postgrespro.com Russian Postgres Company

Re: executor relation handling

2018-10-01 Thread Amit Langote
On 2018/10/01 2:18, Tom Lane wrote: > I wrote: >> 1. You set up transformRuleStmt to insert AccessExclusiveLock into >> the "OLD" and "NEW" RTEs for a view. This is surely wrong; we do >> not want to take exclusive lock on a view just to run a query using >> the view. It should (usually, anyway)

New vacuum option to do only freezing

2018-10-01 Thread Masahiko Sawada
Hi, Attached patch adds a new option FREEZE_ONLY to VACUUM command. This option is same as FREEZE option except for it disables reclaiming dead tuples. That is, with this option vacuum does pruning HOT chain, freezing live tuples and maintaining both visibility map and freespace map but does not c

Re: de-deduplicate code in DML execution hooks in postgres_fdw

2018-10-01 Thread Michael Paquier
On Mon, Jul 23, 2018 at 02:17:38PM +0530, Ashutosh Bapat wrote: > I used perform instead of execute since the later is usually > associated with local operation. I added "foreign" in the name of the > function to indicate that it's executed on foreign server. I am happy > with "remote" as well. I d

Re: Hash Joins vs. Bloom Filters / take 2

2018-10-01 Thread Tomas Vondra
On 10/01/2018 09:15 AM, Michael Paquier wrote: On Thu, Mar 01, 2018 at 07:04:41PM -0500, David Steele wrote: After reviewing the thread I also agree that this should be pushed to 2018-09, so I have done so. I'm very excited by this patch, though. In general I agree with Peter that a higher rat

Re: pgbench - add pseudo-random permutation function

2018-10-01 Thread Fabien COELHO
PostgreSQL Hackers Subject: Re: pgbench - add pseudo-random permutation function On Wed, Sep 26, 2018 at 01:20:49PM +0200, Fabien COELHO wrote: So, am I right to deducing that you are satisfied with the current status of the patch, with the nbits implementation either based on popcount (v4)

Re: Function for listing archive_status directory

2018-10-01 Thread Christoph Moench-Tegeder
## Michael Paquier (mich...@paquier.xyz): > Okay, could you add this patch to the next commit fest? Here it is: > https://commitfest.postgresql.org/20/ And here's the patch: https://commitfest.postgresql.org/20/1813/ Regards, Christoph -- Spare Space

Re: Proposal for Signal Detection Refactoring

2018-10-01 Thread Chris Travers
On Wed, Sep 26, 2018 at 9:54 AM Chris Travers wrote: > > > On Tue, Sep 25, 2018 at 3:23 PM Tom Lane wrote: > >> Chris Travers writes: >> > However, what I think one could do is use a struct of volatile >> > sig_atomic_t members and macros for checking/setting. Simply writing a >> > value is s

Re: Hint to set owner for tablespace directory

2018-10-01 Thread Peter Eisentraut
On 24/09/2018 14:50, Peter Eisentraut wrote: > On 11/09/2018 17:10, Peter Eisentraut wrote: >> On 07/09/2018 17:59, Maksim Milyutin wrote: >>> those directories was that user). The error message "could not set >>> permissions on directory ..." disoriented that user. The need to change >>> the own

Re: SSL tests failing with "ee key too small" error on Debian SID

2018-10-01 Thread Kyotaro HORIGUCHI
At Tue, 25 Sep 2018 14:26:42 +0900, Michael Paquier wrote in <20180925052642.gj1...@paquier.xyz> > On Tue, Sep 25, 2018 at 12:48:57PM +0900, Kyotaro HORIGUCHI wrote: > > Do you mean that cert/key files are generated on-the-fly while > > running 'make check'? It sounds reasonable as long as just

Re: SQL/JSON: documentation

2018-10-01 Thread Liudmila Mantrova
On 09/28/2018 08:29 PM, Peter Eisentraut wrote: On 28/06/2018 01:36, Nikita Glukhov wrote: Attached patch with draft of SQL/JSON documentation written by Liudmila Mantrova, Oleg Bartunov and me. Also it can be found in our sqljson repository on sqljson_doc branch: https://github.com/postgresp

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-10-01 Thread Daniel Gustafsson
> On 1 Oct 2018, at 01:19, Michael Paquier wrote: > > On Sun, Sep 30, 2018 at 10:51:44PM +0900, Michael Paquier wrote: >> You could have chosen something less complicated, like "ホゲ", which is >> the equivalent of "foo" in English. Anyway, non-ASCII characters should >> not be included in the fin

Re: Hint to set owner for tablespace directory

2018-10-01 Thread Maksim Milyutin
01.10.2018 15:15, Peter Eisentraut wrote: On 24/09/2018 14:50, Peter Eisentraut wrote: On 11/09/2018 17:10, Peter Eisentraut wrote: On 07/09/2018 17:59, Maksim Milyutin wrote: those directories was that user). The error message "could not set permissions on directory ..." disoriented that use

Re: de-deduplicate code in DML execution hooks in postgres_fdw

2018-10-01 Thread Etsuro Fujita
(2018/10/01 19:42), Michael Paquier wrote: On Mon, Jul 23, 2018 at 02:17:38PM +0530, Ashutosh Bapat wrote: I used perform instead of execute since the later is usually associated with local operation. I added "foreign" in the name of the function to indicate that it's executed on foreign server.

Re: Early WIP/PoC for inlining CTEs

2018-10-01 Thread Andreas Karlsson
On 07/27/2018 10:10 AM, David Fetter wrote: On Fri, Jul 27, 2018 at 02:55:26PM +1200, Thomas Munro wrote: On Thu, Jul 26, 2018 at 7:14 AM, David Fetter wrote: Please find attached the next version, which passes 'make check'. ... but not 'make check-world' (contrib/postgres_fdw's EXPLAIN is d

Re: Sync ECPG scanner with core

2018-10-01 Thread John Naylor
On 9/30/18, Michael Paquier wrote: > It would be nice to add this patch to the next commit fest: > https://commitfest.postgresql.org/20/ Done, thanks. -John Naylor

Re: Oid returned from AddSubscriptionRelState/UpdateSubscriptionRelState

2018-10-01 Thread Petr Jelinek
Hi, On 29/09/18 03:35, Andres Freund wrote: > > How come those two functions return oids, even though, as far as I > understand, the underlying relation is BKI_WITHOUT_OIDS. I assume > that's just an oversight? > Yeah, it's copy-pasto that managed to get through, those functions should return

Re: Relations being opened without any lock whatever

2018-10-01 Thread Tom Lane
Amit Langote writes: > On 2018/10/01 4:20, Tom Lane wrote: >> Running the regression tests with the patch I showed in >> https://www.postgresql.org/message-id/16565.1538327...@sss.pgh.pa.us > Maybe you've noticed but the relation_open calls coming from bootstrap.c > all pass NoLock which trigger

Re: executor relation handling

2018-10-01 Thread Tom Lane
Amit Langote writes: > On 2018/09/30 5:04, Tom Lane wrote: >> 3. There remain some cases where the RTE says RowExclusiveLock but >> the executor calculation indicates we only need AccessShareLock. >> AFAICT, this happens only when we have a DO ALSO rule that results >> in an added query that merel

Re: executor relation handling

2018-10-01 Thread Tom Lane
Amit Langote writes: > On 2018/10/01 2:18, Tom Lane wrote: >> I think that the call sites should ultimately look like >> Assert(CheckRelationLockedByMe(...)); >> but for hunting down the places where the assertion currently fails, >> it's more convenient if it's just an elog(WARNING). > Should th

Re: automatic restore point

2018-10-01 Thread Peter Eisentraut
On 01/10/2018 05:34, Alvaro Herrera wrote: > I don't see it as clear cut as all that ... particularly considering > that a useful event trigger runs *after* the DDL command in question has > already written all its WAL, so such a restore point would be completely > useless. (Or are ddl_command_sta

Re: Assert failed in snprintf.c

2018-10-01 Thread Tom Lane
Jaime Casanova writes: > sqlsmith made it again, attached is the query (run against regression > database) that makes the assert fail and the backtrace. > this happens in head only (or at least 11 is fine). Ah. Looks like the has_column_privilege stuff is incautious about whether it's handed a v

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-10-01 Thread Bossart, Nathan
On 9/27/18, 2:52 AM, "Michael Paquier" wrote: > Thanks for the new patches. So I have begun looking at the full set, > beginning by 0001 which introduces a new common routine to get the > elevel associated to a skipped relation. I have been chewing on this > one, and I think that the patch could

Re: pg_ls_tmpdir(); AND, Function for listing archive_status directory

2018-10-01 Thread Justin Pryzby
On Wed, Sep 26, 2018 at 10:36:03PM +0200, Laurenz Albe wrote: > Bossart, Nathan wrote: > > Attached is a patch to add a pg_ls_tmpdir() function that lists the > > contents of a specified tablespace's pgsql_tmp directory. This is > > very similar to the existing pg_ls_logdir() and pg_ls_waldir() >

Re: pg_ls_tmpdir()

2018-10-01 Thread Andres Freund
Hi, On 2018-09-26 22:36:03 +0200, Laurenz Albe wrote: > 2. There already are functions "pg_ls_logdir" and "pg_ls_waldir", >and I can imagine new requests, e.g. pg_ls_datafiles() to list the >data files in a database directory. > >It may make sense to have a generic function like > >

Re: buildfarm and git pull

2018-10-01 Thread Justin Pryzby
On Thu, Sep 27, 2018 at 06:32:59PM +0300, Alexander Kuzmenkov wrote: > It just has to checkout the remote branch as-is. It doesn't clean files, but I would suggest: git checkout -B branch remote/branch Justin

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Mark Wong
On Sun, Sep 30, 2018 at 12:38:46AM +0100, Andrew Gierth wrote: > > "Andrew" == Andrew Dunstan writes: > > >> What is the size of a C "int" on this platform? > > Andrew> 4. > > Hmm. > > Because int being more than 32 bits is the simplest explanation for this > difference. > > How about t

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Tom Lane
Mark Wong writes: > a | a > | uuid_cmp > --+--+- > ---- | ---- > |

Re: Proposal for Signal Detection Refactoring

2018-10-01 Thread Chris Travers
Moving this to documentation due to a general consensus that abstracting this is not necessarily worth it. If we don't want to refactor and abstract this, it is worth documenting the design as to how things work now so that others who face bugs can consult docs instead of trying to determine ac

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Andres Freund
On 2018-10-01 11:58:51 -0400, Tom Lane wrote: > Mark Wong writes: > > a | a > > | uuid_cmp > > --+--+- > > ----1

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Tom Lane
Andres Freund writes: > On 2018-10-01 11:58:51 -0400, Tom Lane wrote: >> Oooh ... apparently, on that platform, memcmp() is willing to produce >> INT_MIN in some cases. That's not a safe value for a sort comparator >> to produce --- we explicitly say that somewhere, IIRC. > Hm, that'd be pretty

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Andres Freund
On 2018-10-01 12:13:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-10-01 11:58:51 -0400, Tom Lane wrote: > >> Oooh ... apparently, on that platform, memcmp() is willing to produce > >> INT_MIN in some cases. That's not a safe value for a sort comparator > >> to produce --- we expli

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Tom Lane
Andres Freund writes: > On 2018-10-01 12:13:57 -0400, Tom Lane wrote: >> Yeah. So our choices are >> >> (1) Retain the current restriction on what sort comparators can >> produce. Find all the places where memcmp's result is returned >> directly, and fix them. (I wonder if strcmp has same issu

Re: [HACKERS] kqueue

2018-10-01 Thread Matteo Beccati
Hi Thomas, On 01/10/2018 01:09, Thomas Munro wrote: I don't know why the existence of the kqueue should make recvfrom() slower on the pgbench side. That's probably something to look into off-line with some FreeBSD guru help. Degraded performance for clients on the same machine does seem to be

Re: [HACKERS] kqueue

2018-10-01 Thread Andres Freund
On 2018-10-01 19:25:45 +0200, Matteo Beccati wrote: > On 01/10/2018 01:09, Thomas Munro wrote: > > I don't know why the existence of the kqueue should make recvfrom() > > slower on the pgbench side. That's probably something to look into > > off-line with some FreeBSD guru help. Degraded performa

has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
I wrote: > Jaime Casanova writes: >> sqlsmith made it again, attached is the query (run against regression >> database) that makes the assert fail and the backtrace. >> this happens in head only (or at least 11 is fine). > Ah. Looks like the has_column_privilege stuff is incautious about whether

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > I wrote: > > Jaime Casanova writes: > >> sqlsmith made it again, attached is the query (run against regression > >> database) that makes the assert fail and the backtrace. > >> this happens in head only (or at least 11 is fine). > > > Ah. Look

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> But it's not quite clear to me what we want the behavior for bad column >> name to be. A case could be made for either of: >> >> * If either the table OID is bad, or the OID is OK but there's no such >> column, return null. >> >>

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Joe Conway
On 10/01/2018 02:41 PM, Stephen Frost wrote: > Tom Lane (t...@sss.pgh.pa.us) wrote: >> But it's not quite clear to me what we want the behavior for bad column >> name to be. A case could be made for either of: >> >> * If either the table OID is bad, or the OID is OK but there's no such >> column,

Re: executor relation handling

2018-10-01 Thread David Rowley
On 1 October 2018 at 19:39, Amit Langote wrote: > For this and the other cases (AcquireRewriteLocks, AcquireExecutorLocks, > etc.), I wonder whether we couldn't just *not* recalculate the lock mode > based on inspecting the query tree to cross-check with rellockmode? Why > not just use rellockmod

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> But it's not quite clear to me what we want the behavior for bad column > >> name to be. A case could be made for either of: > >> > >> * If either the table OID is bad, or t

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> The general plan in the has_foo_privilege functions is to throw errors for >> failing name-based lookups, but return null for failing numerically-based >> lookups (object OID or column number). I'm inclined to think we should >> st

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> The general plan in the has_foo_privilege functions is to throw errors for > >> failing name-based lookups, but return null for failing numerically-based > >> lookups (object

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> You can't have it both ways. Either you throw an error if the name's >> not there, or you don't. > I'm not following why we couldn't handle a dropped column differently. Different from what? A name-based lookup isn't going to fi

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > >> You can't have it both ways. Either you throw an error if the name's > >> not there, or you don't. > > > I'm not following why we couldn't handle a dropped column differentl

settings to control SSL/TLS protocol version

2018-10-01 Thread Peter Eisentraut
There have been some requests to be able to select the TLS versions PostgreSQL is using. We currently only hardcode that SSLv2 and SSLv3 are disabled, but there is also some interest now in disabling TLSv1.0 and TLSv1.1. Also, I've had some issues in some combinations with the new TLSv1.3, so the

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Tom Lane
Amit Kapila writes: > On Sun, Sep 30, 2018 at 10:44 AM Amit Kapila wrote: >> Attached is a patch along these lines, let me know if you have >> something else in mind? I have manually verified this with >> force_parallel_mode=regress configuration in my development >> environment. I don't have a

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
Stephen Frost writes: > My complaint is specifically trying to do something like: > =*# select * > from > pg_class > join pg_attribute on (pg_class.oid = pg_attribute.attrelid) > join pg_namespace on (pg_class.relnamespace = pg_n

Re: buildfarm and git pull

2018-10-01 Thread Andrew Dunstan
On 10/01/2018 11:45 AM, Justin Pryzby wrote: On Thu, Sep 27, 2018 at 06:32:59PM +0300, Alexander Kuzmenkov wrote: It just has to checkout the remote branch as-is. It doesn't clean files, but I would suggest: git checkout -B branch remote/branch You can see what was done in yesterday's thr

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Tom Lane
I wrote: > The attached revised patch contains a test case that demonstrably triggers > the problem on gaur's host. Oddly, I do not get a crash either on a PPC > Mac or a Raspberry Pi 3 running Raspbian. I'm not very sure why; I traced > through things with gdb and it's definitely calling EA_flat

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Andrew Dunstan
On 10/01/2018 11:58 AM, Tom Lane wrote: Mark Wong writes: a | a | uuid_cmp --+--+- ---- | -1

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Tom Lane
Andrew Dunstan writes: > On 10/01/2018 11:58 AM, Tom Lane wrote: >> Oooh ... apparently, on that platform, memcmp() is willing to produce >> INT_MIN in some cases. That's not a safe value for a sort comparator >> to produce --- we explicitly say that somewhere, IIRC. I think we >> implement DESC

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Stephen Frost
Greetings, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Stephen Frost writes: > > My complaint is specifically trying to do something like: [...] > That code is kinda broken anyway, because it won't survive relation drops > either. What you *should* be writing is Yes, I agree, but it still seems li

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Larry Rosenman
On Mon, Oct 01, 2018 at 05:11:02PM -0400, Tom Lane wrote: > Andrew Dunstan writes: > > On 10/01/2018 11:58 AM, Tom Lane wrote: > >> Oooh ... apparently, on that platform, memcmp() is willing to produce > >> INT_MIN in some cases. That's not a safe value for a sort comparator > >> to produce --- w

Re: executor relation handling

2018-10-01 Thread Tom Lane
David Rowley writes: > On 1 October 2018 at 19:39, Amit Langote > wrote: >> For this and the other cases (AcquireRewriteLocks, AcquireExecutorLocks, >> etc.), I wonder whether we couldn't just *not* recalculate the lock mode >> based on inspecting the query tree to cross-check with rellockmode?

Re: settings to control SSL/TLS protocol version

2018-10-01 Thread Daniel Gustafsson
> On 1 Oct 2018, at 22:21, Peter Eisentraut > wrote: > > There have been some requests to be able to select the TLS versions > PostgreSQL is using. We currently only hardcode that SSLv2 and SSLv3 > are disabled, but there is also some interest now in disabling TLSv1.0 > and TLSv1.1. Also, I've

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Andrew Dunstan
On 10/01/2018 12:50 PM, Tom Lane wrote: Andres Freund writes: On 2018-10-01 12:13:57 -0400, Tom Lane wrote: Yeah. So our choices are (1) Retain the current restriction on what sort comparators can produce. Find all the places where memcmp's result is returned directly, and fix them. (I

Re: automatic restore point

2018-10-01 Thread Alvaro Herrera
On 2018-Oct-01, Peter Eisentraut wrote: > The following appears to work: > > CREATE FUNCTION evt_automatic_restart_point() RETURNS event_trigger > LANGUAGE plpgsql > AS $$ > BEGIN > PERFORM pg_create_restore_point(tg_tag); > END > $$; > > CREATE EVENT TRIGGER automatic_restart_point ON ddl_com

Re: Odd 9.4, 9.3 buildfarm failure on s390x

2018-10-01 Thread Thomas Munro
On Tue, Oct 2, 2018 at 10:55 AM Andrew Dunstan wrote: > On 10/01/2018 12:50 PM, Tom Lane wrote: > > Andres Freund writes: > >> On 2018-10-01 12:13:57 -0400, Tom Lane wrote: > >>> Yeah. So our choices are > >>> > >>> (1) Retain the current restriction on what sort comparators can > >>> produce.

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Lou Picciano
Wow, Tom. This is great stuff. Thanks for it. Lou Picciano > On Sep 9, 2018, at 11:27 PM, Tom Lane wrote: > > I wondered why buildfarm member chipmunk has been failing hard > for the last little while. Fortunately, it's supplying us with > a handy backtrace: > > Program terminated with signal

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Thomas Munro
On Tue, Oct 2, 2018 at 9:49 AM Tom Lane wrote: > Apparently the only somewhat-modern architecture that is resolutely > unaligned-unfriendly is MIPS. It's been a few years now since I worked on that architecture, but Sparc is somewhat-modern and resolutely unaligned-unfriendly. It's just that you

snprintf.c hammering memset()

2018-10-01 Thread Thomas Munro
Hello hackers, Mateusz Guzik was benchmarking PostgreSQL on FreeBSD investigating the kqueue thread and complained off-list about a lot of calls to memset() of size 256KB from dopr() in our snprintf.c code. Yeah, that says: PrintfArgType argtypes[NL_ARGMAX + 2]; ... MemSet(argtypes, 0, s

Re: has_column_privilege behavior (was Re: Assert failed in snprintf.c)

2018-10-01 Thread Tom Lane
Stephen Frost writes: > * Tom Lane (t...@sss.pgh.pa.us) wrote: >> Having said that, I'm fine with having it return NULL if the given >> attname matches an attisdropped column. > Ok, that's really all I was asking about. Ah, we were just talking past each other then :-(. That behavior existed al

Re: snprintf.c hammering memset()

2018-10-01 Thread Tom Lane
Thomas Munro writes: > Mateusz Guzik was benchmarking PostgreSQL on FreeBSD investigating the > kqueue thread and complained off-list about a lot of calls to memset() > of size 256KB from dopr() in our snprintf.c code. > Yeah, that says: > PrintfArgType argtypes[NL_ARGMAX + 2]; > ... > Me

Re: snprintf.c hammering memset()

2018-10-01 Thread Andres Freund
Hi, On 2018-10-01 19:52:40 -0400, Tom Lane wrote: > Thomas Munro writes: > > Mateusz Guzik was benchmarking PostgreSQL on FreeBSD investigating the > > kqueue thread and complained off-list about a lot of calls to memset() > > of size 256KB from dopr() in our snprintf.c code. > > > Yeah, that sa

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Tom Lane
Thomas Munro writes: > On Tue, Oct 2, 2018 at 9:49 AM Tom Lane wrote: >> Apparently the only somewhat-modern architecture that is resolutely >> unaligned-unfriendly is MIPS. > It's been a few years now since I worked on that architecture, but > Sparc is somewhat-modern and resolutely unaligned-u

Re: Add SKIP LOCKED to VACUUM and ANALYZE

2018-10-01 Thread Michael Paquier
On Mon, Oct 01, 2018 at 03:37:01PM +, Bossart, Nathan wrote: > Good idea. This patch looks good to me. Thanks, I have pushed this one now. > Without the find_all_inheritors() stuff, I think we would just need to > modify the ANALYZE documentation patch to say something like > > Specifie

Re: pgsql: Improve autovacuum logging for aggressive and anti-wraparound ru

2018-10-01 Thread Michael Paquier
On Fri, Sep 28, 2018 at 01:53:14PM +0900, Masahiko Sawada wrote: > I agree. Can we fix this simply by the attached patch? Thanks for sending a patch. +/* autovacuum cannot be anti-wraparound and not aggressive vacuum */ +Assert(aggressive); +msgfmt = _("automatic aggressive vacuum to

Re: snprintf.c hammering memset()

2018-10-01 Thread Tom Lane
Andres Freund writes: > On 2018-10-01 19:52:40 -0400, Tom Lane wrote: >> Ouch indeed. Quite aside from cycles wasted, that's way more stack than >> we want this to consume. I'm good with forcing this to 16 or so ... >> any objections? > Especially after your performance patch, shouldn't we actu

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-10-01 Thread Michael Paquier
On Mon, Oct 01, 2018 at 02:37:42PM +0200, Daniel Gustafsson wrote: >> On 1 Oct 2018, at 01:19, Michael Paquier wrote: >> Looking at the refactoring patch 0001, wouldn't signalfuncs.c make a >> better name for the new file? There are already multiple examples of >> this type, like logicalfuncs.c,

Re: snprintf.c hammering memset()

2018-10-01 Thread Tom Lane
Thomas Munro writes: > PrintfArgType is an enum, and we define NL_ARGMAX as 16 if the OS > didn't already define it. On FreeBSD 11, NL_ARGMAX was defined as 99 > in . On FreeBSD 12, it is defined as 65536... ouch. On a > Debian box I see it is 4096. Some further research: * My Red Hat boxes a

Re: snprintf.c hammering memset()

2018-10-01 Thread Andres Freund
Hi, On 2018-10-01 20:19:16 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-10-01 19:52:40 -0400, Tom Lane wrote: > >> Ouch indeed. Quite aside from cycles wasted, that's way more stack than > >> we want this to consume. I'm good with forcing this to 16 or so ... > >> any objections?

Re: snprintf.c hammering memset()

2018-10-01 Thread Andres Freund
Hi, On 2018-10-01 17:46:55 -0700, Andres Freund wrote: > On 2018-10-01 20:19:16 -0400, Tom Lane wrote: > > argtypes is only a small part of the stack-space issue, there's also > > argvalues which is (at least) twice as big. I don't think second-guessing > > the compiler about the most efficient r

Re: snprintf.c hammering memset()

2018-10-01 Thread Tom Lane
Andres Freund writes: > On 2018-10-01 20:19:16 -0400, Tom Lane wrote: >> That patch takes the memset out of the main line, but it'd still be >> a performance problem for formats using argument reordering; and the >> stack-space concern would remain the same. > What I mean is that it shouldn't be

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

2018-10-01 Thread Michael Paquier
On Mon, Sep 03, 2018 at 06:59:10PM -0700, Noah Misch wrote: > If you're going to keep this highly-simplified estimate, please expand the > comment to say why it doesn't matter or what makes it hard to do better. The > non-planunionor.c path for the same query computes its own estimate of the > sam

Re: GiST VACUUM

2018-10-01 Thread Michael Paquier
On Mon, Aug 06, 2018 at 11:12:00PM +0500, Andrey Borodin wrote: > Done. Added function bms_make_empty(int size) Andrey, your latest patch does not apply. I am moving this to the next CF, waiting for your input. -- Michael signature.asc Description: PGP signature

Re: patch to allow disable of WAL recycling

2018-10-01 Thread Michael Paquier
On Thu, Sep 13, 2018 at 02:56:42PM -0600, Jerry Jelinek wrote: > I'll take a look at that. I had been trying to keep the patch as minimal as > possible, but I'm happy to work through this. (Please be careful with top-posting) Jerry, the last status was from three weeks ago with the patch waiting

Re: speeding up planning with partitions

2018-10-01 Thread Michael Paquier
On Sat, Sep 29, 2018 at 07:00:02PM +0530, Dilip Kumar wrote: > I think we can delay allocating memory for rel->part_rels? And we can > allocate in add_rel_partitions_to_query only > for those partitions which survive pruning. This last review set has not been answered, and as it is recent I am mo

Re: speeding up planning with partitions

2018-10-01 Thread Amit Langote
On 2018/10/02 10:20, Michael Paquier wrote: > On Sat, Sep 29, 2018 at 07:00:02PM +0530, Dilip Kumar wrote: >> I think we can delay allocating memory for rel->part_rels? And we can >> allocate in add_rel_partitions_to_query only >> for those partitions which survive pruning. > > This last review s

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Amit Kapila
On Tue, Oct 2, 2018 at 5:31 AM Tom Lane wrote: > > Thomas Munro writes: > > On Tue, Oct 2, 2018 at 9:49 AM Tom Lane wrote: > >> Apparently the only somewhat-modern architecture that is resolutely > >> unaligned-unfriendly is MIPS. > > > It's been a few years now since I worked on that architectu

Re: [FEATURE PATCH] pg_stat_statements with plans (v02)

2018-10-01 Thread Michael Paquier
On Fri, Apr 06, 2018 at 03:03:30PM +0200, Julian Markwort wrote: > As I see it, planning duration, first date, and last update date would > be columns added to the pg_stat_statements view, i.e. they are tracked > for each kind of a (jumbled) query -- just as the good and bad plans, > their associat

Re: [HACKERS] Re: Improve OR conditions on joined columns (common star schema problem)

2018-10-01 Thread Tom Lane
Michael Paquier writes: > On Mon, Sep 03, 2018 at 06:59:10PM -0700, Noah Misch wrote: >> If you're going to keep this highly-simplified estimate, please expand the >> comment to say why it doesn't matter or what makes it hard to do better. The >> non-planunionor.c path for the same query computes

Re: [PATCH v18] GSSAPI encryption support

2018-10-01 Thread Michael Paquier
On Mon, Aug 06, 2018 at 05:23:28PM -0400, Robbie Harwood wrote: > If you're in a position where you're using Kerberos (or most other > things from the GSSAPI) for authentication, the encryption comes at > little to no additional setup cost. And then you get all the security > benefits outlined in

Re: SerializeParamList vs machines with strict alignment

2018-10-01 Thread Tom Lane
Amit Kapila writes: > I think if we do Analyze on the table after populating rows, it should > use just one worker and that should be sufficient to hit the case > being discussed. I would like to change the test so that it uses just > one worker. I thought that adding an ANALYZE would make the t

Re: TupleTableSlot abstraction

2018-10-01 Thread Kyotaro HORIGUCHI
At Tue, 25 Sep 2018 16:45:09 -0700, Andres Freund wrote in <20180925234509.3hrrf6tmvy5tf...@alap3.anarazel.de> > Hi, > > On 2018-09-04 18:35:34 +0530, Amit Khandekar wrote: > > Subject: [PATCH 05/14] Use tts_flags instead of previous bool members > > > > Pack the boolean members in TupleTableSl

Re: Vacuum: allow usage of more than 1GB of work mem

2018-10-01 Thread Michael Paquier
On Mon, Jul 16, 2018 at 02:33:17PM -0400, Andrew Dunstan wrote: > Ah, ok. Thanks. ignore the email I just sent about that. So... This thread has basically died of inactivity, while there have been a couple of interesting things discussed, like the version from Heikki here: https://www.postgresql.

Re: pg_dumpall --exclude-database option

2018-10-01 Thread Michael Paquier
On Fri, Aug 03, 2018 at 11:08:57PM +0200, Fabien COELHO wrote: > Patch applies cleanly, compiles, and works for me. Last review has not been addressed, so please note that this has been marked as returned with feedback. -- Michael signature.asc Description: PGP signature

Re: [HACKERS] Removing [Merge]Append nodes which contain a single subpath

2018-10-01 Thread Michael Paquier
On Mon, Jul 02, 2018 at 09:37:31AM +1200, David Rowley wrote: > Now that the September 'fest is open for new patches I'm going to move > this patch over there. This patch has become slightly less important > than some other stuff, but I'd still like to come back to it. Please note that the latest

Re: cost_sort() improvements

2018-10-01 Thread Michael Paquier
On Sun, Jul 22, 2018 at 10:22:10PM +0200, Tomas Vondra wrote: > Could you perhaps summarize the reasoning or at least point me to the > relevant parts of the sources, so that I know which parts to focus on? Teodor, this patch is waiting for some input from you. I have moved it to next CF for now,

Re: Push down Aggregates below joins

2018-10-01 Thread Michael Paquier
On Fri, Aug 03, 2018 at 04:50:11PM +0200, Antonin Houska wrote: > Antonin Houska wrote: > > > I didn't have enough time to separate "your functionality" and can do it > > when > > I'm back from vacation. > > So I've separated the code that does not use the 2-stage replication (and > therefore t

Re: Subplan result caching

2018-10-01 Thread Michael Paquier
On Mon, Jul 09, 2018 at 09:08:07PM +1200, David Rowley wrote: > I'll mark as waiting on author in the meantime. > > It's great to see someone working on this. Not that actively unfortunately. The patch status has remained unchanged since, so I am marking it as returned with feedback. -- Michael

Re: POC: GROUP BY optimization

2018-10-01 Thread Michael Paquier
On Sat, Jun 30, 2018 at 09:19:13AM +1200, Gavin Flower wrote: > Additionally put an upper limit threshold on the number of combinations to > check, fairly large by default? > > If first threshold is exceeded, could consider checking out a few more > selected at random from paths not yet checked, t

Re: Subplan result caching

2018-10-01 Thread David Rowley
On 19 July 2018 at 06:27, Robert Haas wrote: > On Mon, Jul 9, 2018 at 5:08 AM, David Rowley >> "LazyMaterialize" seems like a good option for a name. It seems better >> than "LazyHash" since you may not want to restrict it to a hash table >> based cache in the future. A binary search tree may be

  1   2   >