Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-04-20 Thread Justin Pryzby
On Tue, Apr 20, 2021 at 06:16:28PM -0700, Andres Freund wrote: > On 2021-04-20 20:03:13 -0500, Justin Pryzby wrote: > > That's a query over a 2 day period (midnight to midnight+2), so it's not > > hard > > for me to believe it sometimes exceeds 100k cost un

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 11:38:52PM +0900, Fujii Masao wrote: > On 2021/04/19 23:55, Justin Pryzby wrote: > > What does "kind" mean ? I think it means a "normalized" query or a "query > > structure". > > > > "a fewer kinds" is w

Re: track_planning causing performance regression

2021-04-21 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 12:13:17AM +0900, Fujii Masao wrote: > On 2021/04/21 23:53, Justin Pryzby wrote: > > Or: > > > > Enabling this parameter may incur a noticeable performance penalty, > > especially similar queries are executed by many

Re: TRUNCATE on foreign table

2021-04-22 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 03:36:25PM +0900, Fujii Masao wrote: > diff --git a/doc/src/sgml/fdwhandler.sgml b/doc/src/sgml/fdwhandler.sgml > index 553524553b..69aa66e73e 100644 > --- a/doc/src/sgml/fdwhandler.sgml > +++ b/doc/src/sgml/fdwhandler.sgml > @@ -1076,27 +1076,25 @@ ExecForeignTruncate(List

Re: truncating timestamps on arbitrary intervals

2021-04-22 Thread Justin Pryzby
On Fri, Apr 09, 2021 at 10:02:47PM +0200, Peter Eisentraut wrote: > On 30.03.21 18:50, John Naylor wrote: > > On Sat, Mar 27, 2021 at 1:06 PM Justin Pryzby wrote: > > > > > > The current docs seem to be missing a "synopsis", like > > > > > &

Re: TRUNCATE on foreign table

2021-04-22 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 05:09:02PM +0530, Bharath Rupireddy wrote: > > should it say "specified" instead of requested ? > > Or should it say "requested the RESTART IDENTITY behavior" ? > > > > Also, I think it should say "..otherwise, the CONTINUE IDENTITY behavior was > > requested". > > The orig

Re: TRUNCATE on foreign table

2021-04-22 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 07:41:06AM -0700, Zhihong Yu wrote: > > > > + Note that ONLY option specified in > > > > > > add "the" to say: "the ONLY" > > > > +1. > > Since 'the only option' is legitimate English phrase, I think the following > would be clearer: > > Note that the option ONLY ... I t

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-22 Thread Justin Pryzby
On Wed, Apr 21, 2021 at 07:06:49PM -0400, Alvaro Herrera wrote: > On 2021-Apr-09, Robert Haas wrote: > > Does this need to worry about new partitions getting attached to a > > partitioned table, or old ones getting detached? (Maybe it does > > already, not sure.) > > I was pinged because this is l

Re: INCLUDING COMPRESSION (sort enum fields)

2021-04-22 Thread Justin Pryzby
(moved to -hackers) https://www.postgresql.org/message-id/flat/54d30e66-dbd6-5485-aaf6-a291ed55919d%40oss.nttdata.com On Thu, Apr 15, 2021 at 11:24:07PM +0900, Fujii Masao wrote: > On 2021/04/15 11:54, Michael Paquier wrote: > > May I ask at the same time why gram.y (TableLikeOption) and > > par

Re: PoC/WIP: Extended statistics on expressions

2021-04-22 Thread Justin Pryzby
I suggest to add some kind of reference to stats expressions here. --- a/doc/src/sgml/maintenance.sgml +++ b/doc/src/sgml/maintenance.sgml Updating Planner Statistics statistics of the planner [...] @@ -330,10 +330,12 @@ Also, by default there is limited inf

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-04-23 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 12:43:46PM -0500, Justin Pryzby wrote: > Maybe the behavior should be documented, though. Actually, I thought the > pre-existing (non)behavior of autoanalyze would've been documented, and we'd > now update that. All I can find is this: > > https:

Re: [PATCH] force_parallel_mode and GUC categories

2021-04-23 Thread Justin Pryzby
On Wed, Apr 14, 2021 at 03:57:21PM +0900, Michael Paquier wrote: > On Tue, Apr 13, 2021 at 07:31:39AM -0500, Justin Pryzby wrote: > > Good point. > > Thanks. I have used the wording that Tom has proposed upthread, added > one GUC_NOT_IN_SAMPLE that you forgo

[PATCH] Remove extraneous whitespace in tags: > foo< and >bar

2021-04-23 Thread Justin Pryzby
More fixes like the one Peter committed as 9bd563aa9. I eyeballed the HTML to make sure this looks right. >From a8b782cde7c5d6eef1e3876636feb652bc5f3711 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 22 Apr 2021 21:10:49 -0500 Subject: [PATCH] Remove extraneous whitespace in tags

Re: Testing autovacuum wraparound (including failsafe)

2021-04-23 Thread Justin Pryzby
On Fri, Apr 23, 2021 at 01:43:06PM -0700, Andres Freund wrote: > 2) FAILSAFE_MIN_PAGES is 4GB - which seems to make it infeasible to test the >failsafe mode, we can't really create 4GB relations on the BF. While >writing the tests I've lowered this to 4MB... > For 2), I don't really have a

Re: pg_upgrade can result in early wraparound on databases with high transaction load

2021-04-23 Thread Justin Pryzby
On Fri, Apr 23, 2021 at 04:42:56PM -0700, Andres Freund wrote: > Hi, > > On 2019-06-15 11:37:59 -0700, Noah Misch wrote: > > On Tue, May 21, 2019 at 03:23:00PM -0700, Peter Geoghegan wrote: > > > On Mon, May 20, 2019 at 3:10 AM Jason Harvey wrote: > > > > This week I upgraded one of my large(2.8T

Re: [PATCH] force_parallel_mode and GUC categories

2021-04-23 Thread Justin Pryzby
On Sat, Apr 24, 2021 at 10:50:21AM +0900, Michael Paquier wrote: > On Fri, Apr 23, 2021 at 01:23:26PM -0500, Justin Pryzby wrote: > > The patch seems to apply cleanly on v12 but cherry-pick needs help for other > > branches... > > FWIW, this did not seem bad enough to me to

Re: Addition of authenticated ID to pg_stat_activity

2021-04-25 Thread Justin Pryzby
On Mon, Apr 26, 2021 at 11:34:16AM +0900, Michael Paquier wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7596,6 +7596,24 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > > > > + xreflabel="track_activity_authn_size"> > + track_activity_authn_size > (in

Re: tab-complete for ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2021-04-26 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 04:40:35PM -0400, Alvaro Herrera wrote: > Would anyone oppose me pushing this for tab-completing the new keywords > of ALTER TABLE .. DETACH PARTITION? +1 to apply tab completion for v14 -- Justin

Re: SQL-standard function body

2021-04-26 Thread Justin Pryzby
On Thu, Apr 22, 2021 at 04:04:18PM -0400, Jeff Janes wrote: > This commit break line continuation prompts for unbalanced parentheses in > the psql binary. Skimming through this thread, I don't see that this is > intentional or has been noticed before. > > with psql -X > > Before: > > jjanes=# a

Re: Performance degradation of REFRESH MATERIALIZED VIEW

2021-04-27 Thread Justin Pryzby
On Tue, Apr 27, 2021 at 03:43:07PM +0200, Tomas Vondra wrote: > On 4/27/21 7:34 AM, Masahiko Sawada wrote: > > On Tue, Apr 27, 2021 at 8:07 AM Andres Freund wrote: > > > On 2021-04-26 23:59:17 +0200, Tomas Vondra wrote: > > > > On 4/26/21 9:27 PM, Andres Freund wrote: > > > > > On 2021-04-26 15:31

Re: Addition of authenticated ID to pg_stat_activity

2021-04-27 Thread Justin Pryzby
On Tue, Apr 27, 2021 at 09:59:18AM +0900, Michael Paquier wrote: > On Mon, Apr 26, 2021 at 03:21:46PM -0400, Stephen Frost wrote: > > * Andres Freund (and...@anarazel.de) wrote: > >> I'm getting a bit worried about the incremental increase in > >> pg_stat_activity width - it's probably by far the v

Re: Some doubious error messages and comments

2021-04-28 Thread Justin Pryzby
ect_agg must be called with a multirange"))); typcache = multirange_get_typcache(fcinfo, mltrngtypoid); commit 8247b4034ed4c68241be9fbdec249bc967ceafd4 Author: Justin Pryzby Date: Tue Apr 27 07:57:50 2021 -0500 Comment typos: extended stats a4d75c86b and 518442c7f diff --git a/src/backend/parser/p

strange case of "if ((a & b))"

2021-04-28 Thread Justin Pryzby
These look strange to me - the inner parens don't do anything. I wouldn't write it with 2x parens for the same reason I wouldn't write it with 8x parens. diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c index 9f159eb3db..3bbc13c443 100644 --- a/contrib/amcheck/verify_

Re: Small issues with CREATE TABLE COMPRESSION

2021-04-28 Thread Justin Pryzby
On Tue, Apr 27, 2021 at 03:22:25PM +0900, Michael Paquier wrote: > Hi all, > > I have been looking at and testing the patch set for CREATE TABLE > COMPRESSION, and spotted a couple of things in parallel of some work > done by Jacob (added in CC). > > The behavior around CREATE TABLE AS and matvie

Re: [PATCH] force_parallel_mode and GUC categories

2021-04-28 Thread Justin Pryzby
On Fri, Apr 09, 2021 at 10:50:53AM +0900, Michael Paquier wrote: > - {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION, > + {"track_commit_timestamp", PGC_POSTMASTER, REPLICATION_SENDING, > I can get behind this change for clarity where it gets actively used. I'm not sure what you

Re: Patch to allow pg_filedump to support reading of pg_filenode.map

2021-04-28 Thread Justin Pryzby
This is separate from the postgresql server repo. https://git.postgresql.org/gitweb/?p=pg_filedump.git +#define RELMAPPER_FILEMAGIC 0x592717 +char magic_buffer[8]; ... + if ( (int) magic_buffer & RELMAPPER_FILEMAGIC ) { This is doing bitwise arithmetic on a pointer, which seems badly wrong.

Re: Remove post-increment in function quote_identifier of pg_upgrade

2021-04-29 Thread Justin Pryzby
On Thu, Apr 29, 2021 at 06:35:28PM +0530, Vaibhav Dalvi wrote: > Hi, > > The function quote_identifier has extra post-increment operation as > highlighted below, > > char * > quote_identifier(const char *s) > { >char *result = pg_malloc(strlen(s) * 2 + 3); >char *r = result; > >*

Re: Patch to allow pg_filedump to support reading of pg_filenode.map

2021-04-29 Thread Justin Pryzby
I think you should be able to avoid crashing if passed a non-relmapper file. Make sure not to loop over more mappings than exist in the relmapper file of the given size. I guess you should warn if the number of mappings is too large for the file's size. And then "cap" the number of mappings to th

Re: Remove redundant variable from transformCreateStmt

2021-04-29 Thread Justin Pryzby
On Thu, Apr 29, 2021 at 02:39:42PM -0400, Alvaro Herrera wrote: > On 2021-Apr-29, Tom Lane wrote: > > Alvaro Herrera writes: > > > I'd do it like this. Note I removed an if/else block in addition to > > > your changes. > > > > > I couldn't convince myself that this is worth pushing though; eithe

Re: pg_upgrade test for binary compatibility of core data types

2021-04-30 Thread Justin Pryzby
f. I changed to use regtype/OIDs, included range/multirange and stopped including only pg_catalog/information_schema. But didn't yet handle composites. > I don't know what we could do about contrib types. Maybe we should > figure that covering core types is already a step forward, and be > happy with getting th

Re: New assertion failed in lazy_scan_heap() on mereswine

2021-04-30 Thread Justin Pryzby
On Sat, May 01, 2021 at 04:43:21PM +1200, Thomas Munro wrote: > TRAP: FailedAssertion("!all_visible_according_to_vm || > prunestate.all_visible", File: > "/home/pgsql/build-farm/buildroot/HEAD/pgsql.build/../pgsql/src/backend/access/heap/vacuumlazy.c", > Line: 1347, PID: 16926) > 2021-04-30 04:1

Re: Extending amcheck to check toast size and compression

2021-05-04 Thread Justin Pryzby
+ /* Oversized toasted attributes should never be stored */ + if (toast_pointer.va_rawsize > VARLENA_SIZE_LIMIT) + report_corruption(ctx, + psprintf("toast value %u rawsize %u exceeds limit %u", +

Re: PG in container w/ pid namespace is init, process exits cause restart

2021-05-04 Thread Justin Pryzby
On Tue, May 04, 2021 at 01:35:50PM -0400, Greg Stark wrote: > Fwiw, I have a suspicion that the right check for being init is > whether `pid == ppid`. pryzbyj@pryzbyj:~$ ps -wwf 1 UIDPID PPID C STIME TTY STAT TIME CMD root 1 0 0 2020 ?Ss10:28 /sbin/init

Re: AlterSubscription_refresh "wrconn" wrong variable?

2021-05-04 Thread Justin Pryzby
On Tue, May 04, 2021 at 10:35:02PM -0400, Tom Lane wrote: > Peter Smith writes: > > This patch replaces the global "wrconn" in AlterSubscription_refresh with a > > local variable of the same name, making it consistent with other functions > > in subscriptioncmds.c (e.g. DropSubscription). > > Th

Re: Re: COPY table_name (single_column) FROM 'unknown.txt' DELIMITER E'\n'

2021-05-05 Thread Justin Pryzby
On Wed, May 05, 2021 at 02:45:41PM -0400, Tom Lane wrote: > > I'm currently using the pg_read_file()-hack in a project, > > and even though it can read files up to 1GB, > > using e.g. regexp_split_to_table() to split on E'\n' > > seems to need 4x as much memory, so it only > > works with files less

cache lookup failed for statistics object 123

2021-05-05 Thread Justin Pryzby
y has f(x) = 11, and the MCV list for the expression shows that 50% of the table has f(x)=11, then the query might decide to *not* use an index scan. -- Justin >From 6dec09300e4ad6cc7977acbfee9db7087420a9b5 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 5 May 2021 04:29:00 -0500

Re: Printing backtrace of postgres processes

2021-05-05 Thread Justin Pryzby
Here's a cleaned-up copy of the doc text. Send a request to the backend with the specified process ID to log its backtrace. The backtrace will be logged at message level LOG. It will appear in the server log based on the log configuration set (See for more information), but will not be sent to t

Re: [v15] ALTER TABLE ... SET ACCESS METHOD

2021-05-05 Thread Justin Pryzby
On Wed, May 05, 2021 at 08:45:50PM -0700, Jeff Davis wrote: > The attached patch implements ALTER TABLE ... SET ACCESS METHOD. > > For simplicity, I used the normal alter table path, ATRewriteTable(), > which does not follow the stricter isolation semantics that VACUUM FULL > follows. If someone t

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Sun, Jan 10, 2021 at 05:38:50PM -0500, Tom Lane wrote: > I wrote: > > The problems that I see in this area are first that there's no > > real standardization in libpq as to whether to append error messages > > together or just flush preceding messages; and second that no effort > > is made in mu

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: > I'm curious though why it took this long for anyone to complain. > I'd supposed that people were running sqlsmith against HEAD on > a pretty regular basis. I think it's also becase sqlsmith would need to run against the v14 *client* libra

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:10:53PM -0700, Jeff Davis wrote: > On Mon, 2021-03-08 at 16:30 +0900, Michael Paquier wrote: > > This toast issue is a kind of interesting one, and it seems rather > > right to rely on toast_build_flattened_tuple() to decompress things > > if > > both table AMs support to

Re: alter table set TABLE ACCESS METHOD

2021-05-06 Thread Justin Pryzby
On Thu, May 06, 2021 at 02:11:31PM -0700, Jeff Davis wrote: > On Thu, 2021-05-06 at 15:23 -0500, Justin Pryzby wrote: > > I think ALTER TABLE SET ACCESS METHOD should move all data off the > > old AM, > > including its toast table. > > Can you explain what you mean,

Re: plan with result cache is very slow when work_mem is not enough

2021-05-07 Thread Justin Pryzby
On Sat, May 08, 2021 at 02:26:44PM +1200, David Rowley wrote: > On Sat, 8 May 2021 at 09:16, Tomas Vondra > wrote: > > On 5/7/21 11:04 PM, David Rowley wrote: > > > Another thought I have is that maybe it would be ok just to move > > > memory accounting debug code so it only runs once in > > > Ex

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Justin Pryzby
On Sat, May 08, 2021 at 10:13:09PM +0900, Michael Paquier wrote: > + You need LZ4, if you want to support > + the compression of data with this method; see > + . I suggest to change "the compression" to "compression". I would write the whole thing like: | The LZ4 library is needed t

Re: Small issues with CREATE TABLE COMPRESSION

2021-05-08 Thread Justin Pryzby
| You need LZ4, if you want to support the compression of data with this method; see CREATE TABLE. I suggest that should reference guc-default-toast-compression instead of CREATE TABLE, since CREATE TABLE is large and very non-specific. Also, in at least 3 places there's extraneous trailing whi

Re: Inherited UPDATE/DELETE vs async execution

2021-05-09 Thread Justin Pryzby
On Sat, May 08, 2021 at 01:20:51AM +0900, Etsuro Fujita wrote: > I noticed this while working on the > EXPLAIN-ANALYZE-for-async-capable-nodes issue: > > DELETE FROM async_pt; > server closed the connection unexpectedly Confirmed, +Tomas, and added at https://wiki.postgresql.org/wiki/PostgreSQL_1

Re: PG 14 release notes, first draft

2021-05-10 Thread Justin Pryzby
Thanks for putting it together. I think these two should be merged: | Remove containment operators @ and ~ from contrib modules cube, hstore, intarray, and seg (Justin Pryzby) | Remove deprecated containment operators for built-in geometry data types (Justin Pryzby) | Improve autovacuum&#

Re: PG 14 release notes, first draft

2021-05-10 Thread Justin Pryzby
Same as the last couple years, I checked for missing items in the release notes, running something like this. git log --cherry-pick --oneline origin/REL_13_STABLE...origin/master Should any of these be included? f82de5c46b Do COPY FROM encoding conversion/verification in larger chunks. 9e596b65f

Re: parallel vacuum - few questions on docs, comments and code

2021-05-11 Thread Justin Pryzby
On Tue, May 11, 2021 at 05:37:50PM +0530, Bharath Rupireddy wrote: > 3) Should the Assert(nindexes > 0); in begin_parallel_vacuum just be > Assert(nindexes > 1); as this function is entered only when indexes > are > 1? I think you're right, at least with the current implementation that paralleliza

Re: PG 14 release notes, first draft

2021-05-11 Thread Justin Pryzby
dexes of a partitioned table (Justin Pryzby, Michael Paquier) It should actually say "all child tables or indexes of a partitioned relation", since you can REINDEX INDEX partitioned_index. | Add bit_xor XOR aggregate function (Alexey Bashtanov) Remove XOR ? | Remove support for

Re: PG 14 release notes, first draft

2021-05-11 Thread Justin Pryzby
On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > | Allow more than the common name (CN) to be matched for client certificate > > authentication (Andrew Dunstan) > > Your description makes it sound like arbitrary attributes can be compared. > > But > > the option just allows co

Re: PG 14 release notes, first draft

2021-05-12 Thread Justin Pryzby
On Tue, May 11, 2021 at 10:45:04PM -0400, Bruce Momjian wrote: > On Tue, May 11, 2021 at 05:13:21PM -0500, Justin Pryzby wrote: > > On Tue, May 11, 2021 at 10:35:23AM -0400, Bruce Momjian wrote: > > > > | Allow more than the common name (CN) to be matched for client

Re: Teaching users how they can get the most out of HOT in Postgres 14

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 04:27:47PM +0900, Michael Paquier wrote: > On Tue, May 11, 2021 at 04:42:27PM +0900, Michael Paquier wrote: > > Whatever the solution chosen, the thing I can see we agree on here is > > that we need to do something, at least in the shape of an on/off > > switch to have an es

amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
Per sqlsmith. postgres=# select amvalidate(123); ERROR: cache lookup failed for operator class 123 postgres=# \errverbose ERROR: XX000: cache lookup failed for operator class 123 LOCATION: amvalidate, amapi.c:125 The usual expectation is that sql callable functions should return null rather t

Re: amvalidate(): cache lookup failed for operator class 123

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 02:22:16PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > Per sqlsmith. > > postgres=# select amvalidate(123); > > ERROR: cache lookup failed for operator class 123 > > postgres=# \errverbose > > ERROR: XX000: cache lookup failed fo

Re: pgsql: autovacuum: handle analyze for partitioned tables

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 05:33:33PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/maintenance.sgml > @@ -817,6 +817,11 @@ analyze threshold = analyze base threshold + analyze > scale factor * number of tu > > is compared to the total number of tuples inserted, updated, or deleted >

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Mon, May 10, 2021 at 09:40:45AM -0500, Justin Pryzby wrote: > Should any of these be included? New SQL-accessible functionality should be included: > 8c15a29745 Allow ALTER TYPE to update an existing type's typsubscript value. These should be merged: > 756ab29124 Allow pageinsp

Re: PG 14 release notes, first draft

2021-05-13 Thread Justin Pryzby
On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > These should be merged: > > 756ab29124 Allow pageinspect to inspect GiST indexes (Andrey Borodin, > > Heikki Linnakangas) > > 9e596b65f4 Add "LP_DEAD item?" column to GiST pageinspect functions

Re: compute_query_id and pg_stat_statements

2021-05-14 Thread Justin Pryzby
On Fri, May 14, 2021 at 07:50:13PM -0400, Alvaro Herrera wrote: > +++ b/doc/src/sgml/config.sgml > @@ -7643,7 +7643,12 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' > WITH csv; > identifier to be computed. Note that an external module can > alternatively be used if the i

Re: PG 14 release notes, first draft

2021-05-15 Thread Justin Pryzby
; > > > > On Thu, May 13, 2021 at 09:01:41PM -0500, Justin Pryzby wrote: > > > > >> New SQL-accessible functionality should be included: > > > > >>> 8c15a29745 Allow ALTER TYPE to updat

Re: Removed extra memory allocations from create_list_bounds

2021-05-16 Thread Justin Pryzby
partition_list_value_cmp, (void *) key); -- 2.17.0 >From 0aeec1e5c9ddf2dea0045ed3ddbf0f7ae09d06a6 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 15 May 2021 15:57:12 -0500 Subject: [PATCH 2/5] allocate the PartitionListValue as a single chunk --- src/backend/partitioning/partb

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-05-16 Thread Justin Pryzby
d some issues. On Mon, Apr 19, 2021 at 12:41:36PM -0500, Justin Pryzby wrote: > > > Also, I just hit this assertion by cancelling the query with ^C / sigint. > > > But > > > I don't have a reprodcer for it. > > > > > > < 2021-04-17 19:14:23.509 ADT

Re: Removed extra memory allocations from create_list_bounds

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 08:22:25PM +0530, Nitin Jadhav wrote: > I agree and thanks for creating those patches. I am not able to apply > the patch on the latest HEAD. Kindly check and upload the modified > patches. The CFBOT had no issues with the patches, so I suspect an issue on your side. http:/

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-17 Thread Justin Pryzby
On Mon, May 17, 2021 at 01:48:03PM -0700, Andres Freund wrote: > pg_attribute is one of the biggest table in a new cluster, and often the > biggest table in production clusters. Its size is also quite relevant in > memory, due to all the TupleDescs we allocate. > > I just noticed that the new attc

Re: PG 14 release notes, first draft

2021-05-18 Thread Justin Pryzby
On Tue, May 18, 2021 at 10:07:25AM -0400, Bruce Momjian wrote: > On Tue, May 18, 2021 at 06:28:49PM +0900, Masahiko Sawada wrote: > > On Mon, May 10, 2021 at 3:03 PM Bruce Momjian wrote: > > > > > > I have committed the first draft of the PG 14 release notes. You can > > > see the most current b

Re: Different compression methods for FPI

2021-05-18 Thread Justin Pryzby
On Mon, May 17, 2021 at 04:44:11PM +0900, Michael Paquier wrote: > On Sun, Mar 21, 2021 at 02:30:04PM -0500, Justin Pryzby wrote: > > For this patch, this is going to require a bit more in terms of library > linking as the block decompression is done in xlogreader.c, so that'

Re: PG 14 release notes, first draft

2021-05-19 Thread Justin Pryzby
server variable client_connection_check_interval allows supporting > operating systems, e.g., Linux, to automatically cancel queries by > disconnected clients. say "some operating systems" ? > This can be disabled by turning client option "sslsni" off. "turning of

Re: Removed extra memory allocations from create_list_bounds

2021-05-19 Thread Justin Pryzby
On Tue, May 18, 2021 at 01:49:12PM -0400, Robert Haas wrote: > I see that you have made a theoretical argument for why this should be > good for performance, but it would be better to have some test results > that show that it works out in practice. Sometimes things seem like > they ought to be mor

Re: PG 14 release notes, first draft

2021-05-20 Thread Justin Pryzby
On Thu, May 20, 2021 at 03:44:46PM -0400, Bruce Momjian wrote: > > | Reduce the default value of vacuum_cost_page_miss (Peter Geoghegan) > > | This new default better reflects current hardware capabilities. > > Also say: the previous default was 10. > > Uh, we didn't report the new value, so wh

Re: array_cat anycompatible change is breaking xversion upgrade tests

2021-05-20 Thread Justin Pryzby
On Wed, Nov 04, 2020 at 07:43:51PM -0500, Tom Lane wrote: > crake is showing xversion upgrade failures since 9e38c2bb50: > > pg_restore: error: could not execute query: ERROR: function > array_cat(anyarray, anyarray) does not exist > Command was: CREATE AGGREGATE "public"."array_cat_accum"("anya

Re: Removed extra memory allocations from create_list_bounds

2021-05-23 Thread Justin Pryzby
On Sun, May 23, 2021 at 10:40:16PM +0530, Nitin Jadhav wrote: > I have used the same testing procedure as explained in the previous mail. > Please find the timing information of the last 10 creation of partitioned > tables as given below. > Without patch With 0001 and 0002 With all patch ... > 18.

Re: Different compression methods for FPI

2021-05-24 Thread Justin Pryzby
is going to be published, make sure it is psersistent */ + EnsureTopTransactionIdLogged(); + + PG_RETURN_FULLTRANSACTIONID(xid); } /* @@ -380,6 +387,9 @@ pg_current_xact_id_if_assigned(PG_FUNCTION_ARGS) if (!FullTransactionIdIsValid(topfxid)) PG_RETURN_NULL(); + /* the XID is going to be publi

Re: array_cat anycompatible change is breaking xversion upgrade tests

2021-05-25 Thread Justin Pryzby
On Thu, May 20, 2021 at 07:35:10PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Wed, Nov 04, 2020 at 07:43:51PM -0500, Tom Lane wrote: > >> As was discussed in the thread leading up to that commit, modifying the > >> signature of array_cat and friends could b

Re: fdatasync performance problem with large number of DB files

2021-05-25 Thread Justin Pryzby
On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote: > > > + { > > > + {"recovery_init_sync_method", PGC_POSTMASTER, > > > ERROR_HANDLING_OPTIONS, > > > + gettext_noop("Sets the method for synchronizing the > > > data directory before crash recovery.")

Re: automatic analyze: readahead - add "IO read time" log message

2021-05-25 Thread Justin Pryzby
This patch adds hits/misses/dirtied, but explain says hit/read/dirtied/written. Should it say "read" instead of "misses" ? src/backend/access/heap/vacuumlazy.c: _("buffer usage: %lld hits, %lld misses, %lld dirtied\n"), src/backend/commands/exp

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-25 Thread Justin Pryzby
On Sun, May 23, 2021 at 12:25:10PM -0400, Tom Lane wrote: > However, the more I looked at that code the less I liked it. > I think the way that compression selection is handled for indexes, > ie consult default_toast_compression on-the-fly, is *far* saner > than what is currently implemented for ta

Re: Incorrect GUC descriptions in docs and postgresql.conf.sample

2021-05-25 Thread Justin Pryzby
Your patch adds documentation about GUCs that can only be set at server start/config/commandline. But it's not true for any of these, which are all HUP/SUSET. Please double check your logic :) src/backend/utils/misc/guc.c: {"autovacuum_work_mem", PGC_SIGHUP, RESOURCES_MEM, src/backend/

Re: Incorrect GUC descriptions in docs and postgresql.conf.sample

2021-05-25 Thread Justin Pryzby
On Tue, May 25, 2021 at 08:43:14PM -0500, Justin Pryzby wrote: > Your patch adds documentation about GUCs that can only be set at server > start/config/commandline. Oh: I realized that I read too quickly and misinterpretted what "only be set in the config" means (I know I&#x

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-26 Thread Justin Pryzby
On Wed, May 26, 2021 at 11:13:46AM -0400, Tom Lane wrote: > Michael Paquier writes: > > Ah, the parallel with reltablespace and default_tablespace at database > > level is a very good point. It is true that currently the code would > > assign attcompression to a non-zero value once the relation i

Re: Speed up pg_checksums in cases where checksum already set

2021-05-26 Thread Justin Pryzby
In one of the checksum patches, there was an understanding that the pages should be written even if the checksum is correct, to handle replicas. >From the v19 patch: https://www.postgresql.org/message-id/F7AFCFCD-8F77-4546-8D42-C7F675A4B680%40yesql.se +* Mark the buffer as dirty an

Re: Move pg_attribute.attcompression to earlier in struct for reduced size?

2021-05-27 Thread Justin Pryzby
On Tue, May 25, 2021 at 08:33:47PM -0500, Justin Pryzby wrote: > On Sun, May 23, 2021 at 12:25:10PM -0400, Tom Lane wrote: > > However, the more I looked at that code the less I liked it. > > I think the way that compression selection is handled for indexes,

Re: terminate called after throwing an instance of 'std::bad_alloc'

2021-05-28 Thread Justin Pryzby
There's a memory leak affecting JIT expressions, even when inlining, optimization, and tuple deforming are disabled. The server that got OOM after installing PG13.3 (because your patch wasn't applied) still gets OOM even with inline_above_cost=-1, optimize_above_cost=-1, and deforming=off. Actual

Re: fdatasync performance problem with large number of DB files

2021-05-29 Thread Justin Pryzby
On Tue, May 25, 2021 at 07:13:59PM -0500, Justin Pryzby wrote: > On Sat, Mar 20, 2021 at 12:16:27PM +1300, Thomas Munro wrote: > > > > + { > > > > + {"recovery_init_sync_method", PGC_POSTMASTER, > > > > ERROR_HANDLING_OPTIONS, >

Re: O_DIRECT on macOS

2021-05-30 Thread Justin Pryzby
On Sun, May 30, 2021 at 04:39:48PM +1200, Thomas Munro wrote: > +BasicOpenFilePermDirect(const char *fileName, int fileFlags, mode_t fileMode, > + bool direct) > ... > +#if !defined(O_DIRECT) && defined(F_NOCACHE) > + /* macOS requires an extra s

Re: list of extended statistics on psql (\dX)

2021-05-30 Thread Justin Pryzby
On Wed, Jan 20, 2021 at 11:00:50PM +0100, Tomas Vondra wrote: > Thanks, I've pushed this. I had to tweak the regression tests a bit, for two > reasons: \dX isn't checking schema visibility rules, so accidentally shows stats objects outside of the search path. I noticed after installing the PG14b1

Re: Multiple hosts in connection string failed to failover in non-hot standby mode

2021-05-30 Thread Justin Pryzby
On Thu, May 06, 2021 at 01:22:27PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > 52a10224 broke sqlsmith, of all things. > > > It was using errmsg as a test of success, instead of checking if the > > connection > > result wasn't null: > >

Re: Remove source code display from \df+?

2023-01-19 Thread Justin Pryzby
On Wed, Jan 18, 2023 at 10:27:46AM -0500, Isaac Morland wrote: > On Wed, 18 Jan 2023 at 00:00, Pavel Stehule wrote: > > > út 17. 1. 2023 v 20:29 odesílatel Isaac Morland > > napsal: > > > >> I welcome comments and feedback. Now to try to find something manageable > >> to review. > > > > looks w

Re: doc: mentioned CREATE+ATTACH PARTITION as an alternative to CREATE TABLE..PARTITION OF

2023-01-19 Thread Justin Pryzby
On Thu, Jan 19, 2023 at 04:47:59PM -0500, Robert Treat wrote: > I think all of that feedback is useful, I guess the immediate question > becomes if Justin wants to try to proceed with his patch implementing > the change, or if adjusting the documentation for the current > implementation is the righ

Re: bug: copy progress reporting of backends which run multiple COPYs

2023-01-20 Thread Justin Pryzby
On Sat, Jan 21, 2023 at 01:51:28AM +0100, Matthias van de Meent wrote: > On Thu, 19 Jan 2023 at 06:47, Justin Pryzby wrote: > > > > pg_stat_progress_copy was added in v14 (8a4f618e7, 9d2d45700). > > > > But if a command JOINs file_fdw tables, the progress report gets

Re: Reduce timing overhead of EXPLAIN ANALYZE using rdtsc?

2023-01-20 Thread Justin Pryzby
On Fri, Jan 20, 2023 at 04:40:32PM -0800, Andres Freund wrote: > From 5a458d4584961dedd3f80a07d8faea66e57c5d94 Mon Sep 17 00:00:00 2001 > From: Andres Freund > Date: Mon, 16 Jan 2023 11:19:11 -0800 > Subject: [PATCH v8 4/5] wip: report nanoseconds in pg_test_timing > > - The i7-860 system m

Re: Remove source code display from \df+?

2023-01-21 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 12:18:34AM -0500, Isaac Morland wrote: > On Thu, 19 Jan 2023 at 13:02, Isaac Morland wrote: > > On Thu, 19 Jan 2023 at 11:30, Justin Pryzby wrote: > >> On Wed, Jan 18, 2023 at 10:27:46AM -0500, Isaac Morland wrote: > >> >

bug: ANALYZE progress report with inheritance tables

2023-01-22 Thread Justin Pryzby
pg_stat_progress_analyze was added in v13 (a166d408e). For tables with inheritance children, do_analyze_rel() and acquire_sample_rows() are called twice. The first time through, pgstat_progress_start_command() has memset() the progress array to zero. But the 2nd time, ANALYZE_BLOCKS_DONE is alre

Re: pg_stats and range statistics

2023-01-22 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 07:19:41PM +0100, Tomas Vondra wrote: > On 1/21/23 19:53, Egor Rogov wrote: > > Hi Tomas, > > On 21.01.2023 00:50, Tomas Vondra wrote: > >> This simply adds two functions, accepting/producing anyarray - one for > >> lower bounds, one for upper bounds. I don't think it can be

Re: Remove source code display from \df+?

2023-01-22 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 03:04:14PM -0500, Tom Lane wrote: > Isaac Morland writes: > > On Sun, 22 Jan 2023 at 14:26, Alvaro Herrera > > wrote: > >> This one would fail the sanity check that all roles created by > >> regression tests need to have names that start with "regress_". > > > Thanks for

Re: Remove source code display from \df+?

2023-01-22 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 04:28:21PM -0500, Isaac Morland wrote: > On Sun, 22 Jan 2023 at 15:04, Tom Lane wrote: > > > Isaac Morland writes: > > > On Sun, 22 Jan 2023 at 14:26, Alvaro Herrera > > > wrote: > > >> This one would fail the sanity check that all roles created by > > >> regression test

Re: Remove source code display from \df+?

2023-01-22 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 08:23:25PM -0500, Isaac Morland wrote: > > Were you able to test with your own github account ? > > I haven’t had a chance to try this. I must confess to being a bit confused > by the distinction between running the CI tests and doing "make check"; > ideally I would like to

Re: Remove source code display from \df+?

2023-01-22 Thread Justin Pryzby
On Sun, Jan 22, 2023 at 09:50:29PM -0500, Isaac Morland wrote: > However, one of the jobs (Windows - Server 2019, MinGW64 - Meson) is paused > and appears never to have run: > > https://cirrus-ci.com/task/6687014536347648 Yeah, mingw is currently set to run only when manually "triggered" by the r

Re: Add LZ4 compression in pg_dump

2023-01-23 Thread Justin Pryzby
On Mon, Jan 23, 2023 at 05:31:55PM +, gkokola...@pm.me wrote: > Please find attached v23 which reintroduces the split. > > 0001 is reworked to have a reduced footprint than before. Also in an attempt > to facilitate the readability, 0002 splits the API's and the uncompressed > implementation i

Re: Cygwin cleanup

2023-01-23 Thread Justin Pryzby
On Thu, Jan 12, 2023 at 10:17:55PM -0600, Justin Pryzby wrote: > On Thu, Jan 12, 2023 at 06:43:54PM -0800, Andres Freund wrote: > > > It looks like logical decoding may be the "most wrong" place that > > > wal_sync_method is being used, so maybe my change is reason

Re: Improve logging when using Huge Pages

2023-01-23 Thread Justin Pryzby
dea and nobody verbalized hatred of it, so I implemented it. In v15, we have shared_memory_size_in_huge_pages, so this adds effective_huge_pages. Feel free to suggest a better name. -- Justin >From 2bb0c48dfefff78325b1b9d31a3e54e982d44e4e Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 23

<    11   12   13   14   15   16   17   18   19   20   >