Re: Uniforms the errors msgs at tuplestore paths

2022-02-20 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 11:12:42AM -0300, Ranier Vilela wrote: > Like how the commit > https://github.com/postgres/postgres/commit/07daca53bfcad59618a9c6fad304e380cc9d2bc1 > The are some paths that were missed: I think these are all unified by the existing tuplestore patch. https://commitfest.post

Re: Adding CI to our tree (ccache)

2022-02-20 Thread Justin Pryzby
Have you tried to use the yet-to-be-released ccache with MSVC ? Also, do you know about msbuild /outputResultsCache ? When I tried that, it gave a bunch of error. https://cirrus-ci.com/task/5697497241747456 |[16:35:13.605] 1>c:\cirrus\pgsql.sln.metaproj : error : MSB4252: Project "c:\cirru

Re: Adding CI to our tree (ccache)

2022-02-20 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 12:47:31PM -0800, Andres Freund wrote: > > Did you ever try to use clcache (or others) ? > > > > When I tried, it refused to cache because of our debug settings > > (DebugInformationFormat) - which seem to be enabled even in release mode. > > > I wonder if that'll be an is

wal_compression=zstd

2022-02-22 Thread Justin Pryzby
lso adds support for zlib. I'm of the impression nobody cares about this, otherwise it would've been included 5-10 years ago. Only 0001 should be reviewed for pg15 - the others are optional/future work. >From 9253013c789ffb121272bfeeaa9dcdebbef79ced Mon Sep 17 00:00:00 2001 From:

Re: shared_preload_libraries = 'pg_stat_statements' failing with installcheck (compute_query_id = auto)

2022-02-22 Thread Justin Pryzby
On Tue, Feb 22, 2022 at 11:04:16AM +0900, Michael Paquier wrote: > On Fri, Feb 18, 2022 at 05:38:56PM +0800, Julien Rouhaud wrote: > > On Fri, Feb 18, 2022 at 05:22:36PM +0900, Michael Paquier wrote: > >> So, I have been looking at this problem, and I don't see a problem in > >> doing something lik

Re: CLUSTER on partitioned index

2022-02-23 Thread Justin Pryzby
On Tue, Jul 20, 2021 at 08:27:02PM -0400, Alvaro Herrera wrote: > I have to wonder if there really *is* a use case for CLUSTER in the > first place on regular tables, let alone on partitioned tables, which > are likely to be large and thus take a lot of time. What justifies > spending so much time

Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)

2022-02-23 Thread Justin Pryzby
+ Whenever the checkpoint operation is running, the + pg_stat_progress_checkpoint view will contain a + single row indicating the progress of the checkpoint. The tables below Maybe it should show a single row , unless the checkpointer isn't running at all (like in single user mode). +

Re: document that brin's autosummarize parameter is off by default

2022-02-24 Thread Justin Pryzby
Ten months ago, Jaime Casanova wrote: > Hi everyone, > > Just noted that the default value of autosummarize reloption for brin > indexes is not documented, or at least not well documented. > > I added the default value in create_index.sgml where other options > mention their own defaults, also ma

Re: set TESTDIR from perl rather than Makefile

2022-02-24 Thread Justin Pryzby
On Mon, Feb 21, 2022 at 07:00:54AM -0500, Andrew Dunstan wrote: > On 2/19/22 18:53, Justin Pryzby wrote: > > On Sat, Feb 19, 2022 at 05:41:49PM -0600, Justin Pryzby wrote: > >> I rebased and fixed the check-guc script to work, made it work with vpath > >> builds, and cle

Re: Add parameter jit_warn_above_fraction

2022-02-25 Thread Justin Pryzby
On Fri, Feb 25, 2022 at 04:16:01PM +0100, Magnus Hagander wrote: > + { > + {"jit_warn_above_fraction", PGC_SUSET, LOGGING_WHEN, > + gettext_noop("Sets the fraction of query time spent on > JIT before writing" > + "a w

Re: set TESTDIR from perl rather than Makefile

2022-02-25 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 04:39:08PM -0800, Andres Freund wrote: > On 2022-02-19 17:53:09 -0600, Justin Pryzby wrote: > > I also meant to also attach it. > > Is the patch actually independent of the other patches in your stack? Yes - I rearranged it that way for this thread. How

Re: Adding CI to our tree

2022-02-25 Thread Justin Pryzby
This is the other half of my CI patches, which are unrelated to the TAP ones on the other thread. >From 88c01c09ee26db2817629265fc12b2dbcd8c9a91 Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 17 Jan 2022 00:53:04 -0600 Subject: [PATCH 1/7] cirrus: include hints how to install OS packa

Re: Missed condition-variable wakeups on FreeBSD

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 02:07:05PM -0500, Tom Lane wrote: > I don't know much about how gdb interacts with kernel calls on > FreeBSD, but I speculate that the poll(2) call returns with EINTR > after gdb releases the process, and then things resume fine, > suggesting that we lost an interrupt somewh

Re: explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-02-26 Thread Justin Pryzby
:00 2001 From: Justin Pryzby Date: Sat, 22 Feb 2020 21:17:10 -0600 Subject: [PATCH 1/6] Add GUC: explain_regress This changes the defaults for explain to: costs off, timing off, summary off. It'd be reasonable to use this for new regression tests which are not intended to be backpatched.

Re: [Proposal] Global temporary tables

2022-02-26 Thread Justin Pryzby
"longrunning transactions". Have you looked at Gilles Darold's GTT extension ? >From b89f3cc5c78e7b4c3e10ab39ef527b524d0d112d Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Sat, 1 Jan 2022 17:02:30 -0600 Subject: [PATCH 02/11] f!0002-gtt-v64-doc.patch --- doc/src/sgml/ref/create_table.sgml |

Re: Commitfest manager for 2022-03

2022-02-26 Thread Justin Pryzby
Can I suggest to update the CF APP to allow: | Target version: 16 I also suggest to update patches to indicate which are (not) being considered for v15. A few specific ones from myself: |https://commitfest.postgresql.org/37/2573/ |pg_dump - read data for some options from external fileRe

Re: Adding CI to our tree

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 05:09:08PM -0800, Andres Freund wrote: > > XXX: if this is run in the same task, the configure flags should probably be > > consistent ? > > What do you mean? I mean that commit to run CompilerWarnings unconditionally built docs with different flags than the other stuff in

Re: Adding CI to our tree

2022-02-26 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 06:50:00PM -0800, Andres Freund wrote: > Hi, > > On 2022-02-26 20:43:52 -0600, Justin Pryzby wrote: > > This doesn't do the right thing - I just tried. > > https://cirrus-ci.org/guide/writing-tasks/#environment-variables > > | changesInclude

Re: Adding CI to our tree

2022-02-28 Thread Justin Pryzby
On Sat, Feb 26, 2022 at 08:08:38PM -0800, Andres Freund wrote: > On 2022-02-26 21:10:57 -0600, Justin Pryzby wrote: > > If someone renames or removes an xref target, shouldn't CI fail on its next > > run for a patch which tries to reference it ? > > Why wouldn't

Re: Pre-allocating WAL files

2022-03-01 Thread Justin Pryzby
On Thu, Dec 30, 2021 at 02:51:10PM +0300, Maxim Orlov wrote: > I did check the patch too and found it to be ok. Check and check-world are > passed. FYI: this is currently failing in cfbot on linux. https://cirrus-ci.com/task/4934371210690560 https://api.cirrus-ci.com/v1/artifact/task/493437121069

Re: Commitfest 2022-03 Patch Triage Part 1a.i

2022-03-01 Thread Justin Pryzby
Can I suggest to copy the patch authors on bulk emails like these ? (Obviously, an extended discussion about a particular patch should happen on its original thread, but that's not what this is about). -- Justin

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-02 Thread Justin Pryzby
On Wed, Mar 02, 2022 at 06:43:11PM +0400, Pavel Borisov wrote: > Hi hackers! > > Hi! Here is the rebased version. The patch doesn't apply - I suppose the patch is relative a forked postgres which already has other patches. http://cfbot.cputube.org/pavel-borisov.html Note also that I mentioned a

Re: using extended statistics to improve join estimates

2022-03-02 Thread Justin Pryzby
On Wed, Jan 19, 2022 at 06:18:09PM +0800, Julien Rouhaud wrote: > On Tue, Jan 04, 2022 at 03:55:50PM -0800, Andres Freund wrote: > > On 2022-01-01 18:21:06 +0100, Tomas Vondra wrote: > > > Here's an updated patch, rebased and fixing a couple typos reported by >

Re: using extended statistics to improve join estimates

2022-03-02 Thread Justin Pryzby
On Wed, Mar 02, 2022 at 11:38:21AM -0600, Justin Pryzby wrote: > Rebased over 269b532ae and muted compiler warnings. And attached. >From 587a5e9fe87c26cdcd9602fc349f092da95cc580 Mon Sep 17 00:00:00 2001 From: Tomas Vondra Date: Mon, 13 Dec 2021 14:05:17 +0100 Subject: [PATCH] Estimate

Re: Adding CI to our tree

2022-03-02 Thread Justin Pryzby
On Mon, Feb 28, 2022 at 02:58:02PM -0600, Justin Pryzby wrote: > I still think that if "Build Docs" is a separate cirrus task, it should > rebuild > docs on every CI run, even if they haven't changed, for any patch that touches > docs/. It'll be confusing if cfbo

Re: libpq compression (part 2)

2022-03-02 Thread Justin Pryzby
If there's no objection, I'd like to move this to the next CF for consideration in PG16. On Mon, Jan 17, 2022 at 10:39:19PM -0600, Justin Pryzby wrote: > On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > > => Since March, errmsg doesn't ne

Re: Column Filtering in Logical Replication

2022-03-02 Thread Justin Pryzby
I applied this patch in my branch with CI hacks to show code coverage on cirrus. https://api.cirrus-ci.com/v1/artifact/task/6186186539532288/coverage/coverage/00-index.html Eyeballing it looks good. But GetActionsInPublication() isn't being hit at all? I think the queries in pg_dump should be wr

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-03-02 Thread Justin Pryzby
Rebased to appease cfbot. I ran these paches under a branch which shows code coverage in cirrus. It looks good to my eyes. https://api.cirrus-ci.com/v1/artifact/task/5212346552418304/coverage/coverage/00-index.html Are these patches being considered for v15 ? >From 30707da3e5eb68d1efbc5594696da4

Re: Adding CI to our tree (ccache)

2022-03-03 Thread Justin Pryzby
On Sun, Feb 20, 2022 at 12:47:31PM -0800, Andres Freund wrote: > On 2022-02-20 13:36:55 -0600, Justin Pryzby wrote: > > Have you tried to use the yet-to-be-released ccache with MSVC ? > > Yes, it doesn't work, because it requires cl.exe to be used in a specific way > (o

Re: [Proposal] vacuumdb --schema only

2022-03-04 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 10:11:28AM +0100, Gilles Darold wrote: > The attached patch implements that. Option -n | --schema can be used > multiple time and can not be used together with options -a or -t. Yes, thanks. I suggest there should also be an --exclude-schema. > I do not propose to extend

Re: wal_compression=zstd

2022-03-04 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 04:19:32PM +0900, Michael Paquier wrote: > On Tue, Feb 22, 2022 at 05:19:48PM -0600, Justin Pryzby wrote: > > > As writen, this patch uses zstd level=1 (whereas the ZSTD's default compress > > level is 6). > > Why? ZSTD using this default

Re: Add LZ4 compression in pg_dump

2022-03-04 Thread Justin Pryzby
The patch is failing on cfbot/freebsd. http://cfbot.cputube.org/georgios-kokolatos.html Also, I wondered if you'd looked at the "high compression" interfaces in lz4hc.h ? Should pg_dump use that ? On Fri, Feb 25, 2022 at 08:03:40AM -0600, Justin Pryzby wrote: > Thanks for work

Re: Make unlogged table resets detectable

2022-03-04 Thread Justin Pryzby
Is this patch targetting pg15 ? There's no discussion since June. Latest at 2021-06-08 21:29:25 by Jeff Davis 2022-02-02 16:37:58 Julien Rouhaud (rjuju) Closed in commitfest 2022-01 with status: Moved to next CF 2021-12-03 06:18:05 Michael Paquier (michael-kun) Closed in commitfest

Re: [Proposal] vacuumdb --schema only

2022-03-06 Thread Justin Pryzby
appendPQExpBufferStr(&catalog_query, ")\n"); > } Maybe it's clearer to write this with =ANY() / != ALL() ? See 002. -- Justin >From 9f1b7f2fb0849a810dc1fa5c1c03d5ff4e2e7d55 Mon Sep 17 00:00:00 2001 From: Gilles Darold Date: Sun, 6 Mar 2022 09:39:37

Re: Adding CI to our tree (ccache)

2022-03-06 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 05:30:03PM -0800, Andres Freund wrote: > I tried to use it, but saw that no caching was happening, and debugged > it. Which yielded that it can't be used due to the way output files are > specified (and due to multiple files, but that can be prevented with an > msbuild param

Re: Add parameter jit_warn_above_fraction

2022-03-07 Thread Justin Pryzby
On Mon, Mar 07, 2022 at 01:10:32PM +0100, Magnus Hagander wrote: > On Fri, Feb 25, 2022 at 5:23 PM Justin Pryzby wrote: > > > > I think it should be a NOTICE (or less?) > > Hmm. I'm not so sure. > > Other similar parameters often use LOG, but the downside of th

Re: Adding CI to our tree (ccache)

2022-03-07 Thread Justin Pryzby
On Mon, Mar 07, 2022 at 11:10:54AM -0800, Andres Freund wrote: > On 2022-03-06 10:16:54 -0600, Justin Pryzby wrote: > > On Fri, Mar 04, 2022 at 05:30:03PM -0800, Andres Freund wrote: > > > I tried to use it, but saw that no caching was happening, and debugged > > > it.

Re: Add pg_freespacemap extension sql test

2022-03-08 Thread Justin Pryzby
On Tue, Mar 08, 2022 at 11:39:08PM +0900, Dong Wook Lee wrote: > Hi, > I just added some tests for the pg_freespacemap extension because the test > coverage was 0 percent. > But I don't know if I did it correctly. The patch only touches doc/*.sgml. I suppose you forgot to use "git add". -- Justi

Re: wal_compression=zstd

2022-03-09 Thread Justin Pryzby
On Fri, Mar 04, 2022 at 05:44:06AM -0600, Justin Pryzby wrote: > On Fri, Mar 04, 2022 at 04:19:32PM +0900, Michael Paquier wrote: > > On Tue, Feb 22, 2022 at 05:19:48PM -0600, Justin Pryzby wrote: > > > > > As writen, this patch uses zstd level=1 (whereas the ZSTD&#

Re: Printing backtrace of postgres processes

2022-03-09 Thread Justin Pryzby
_log_backtrace(integer)', 'EXECUTE'); -- no + +GRANT EXECUTE ON FUNCTION pg_log_backtrace(integer) + TO regress_log_backtrace; + +SELECT has_function_privilege('regress_log_backtrace', + 'pg_log_backtrace(integer)', 'EXECUTE'); -- yes + +SET RO

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-09 Thread Justin Pryzby
patch set seems isn't ready due to lstat() on windows. >From 47dde043c27840ef43c037f968b268270dc3206d Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 16 Mar 2020 14:12:55 -0500 Subject: [PATCH v34 01/15] Document historic behavior of links to directories.. Backpatch to 9.5: pg_stat_

Re: Adding CI to our tree

2022-03-09 Thread Justin Pryzby
I'm curious what you think of this patch. It makes check-world on freebsd over 30% faster - saving 5min. Apparently gcc -Og was added in gcc 4.8 (c. 2013). On Wed, Mar 02, 2022 at 02:50:58PM -0600, Justin Pryzby wrote: > From d180953d273c221a30c5e9ad8d74b1b4dfc60bd1 Mon Sep 17 00:00

Re: Adding CI to our tree

2022-03-09 Thread Justin Pryzby
On Wed, Mar 09, 2022 at 10:12:54AM -0800, Andres Freund wrote: > On 2022-03-09 11:47:23 -0600, Justin Pryzby wrote: > > I'm curious what you think of this patch. > > > > It makes check-world on freebsd over 30% faster - saving 5min. > > That's nice!

Re: [Proposal] vacuumdb --schema only

2022-03-09 Thread Justin Pryzby
On Mon, Mar 07, 2022 at 08:38:04AM +0100, Gilles Darold wrote: > > Maybe it's clearer to write this with =ANY() / != ALL() ? > > See 002. > > I have applied your changes and produced a new version v3 of the patch, > thanks for the improvements. The patch have been added to commitfest > interface,

Re: Adding CI to our tree

2022-03-10 Thread Justin Pryzby
On Thu, Mar 10, 2022 at 12:50:15PM -0800, Andres Freund wrote: > > - setup_cores_script: | > > + setup_os_script: | > > mkdir -m 770 /tmp/cores > > chown root:postgres /tmp/cores > > sysctl kern.corefile='/tmp/cores/%N.%P.core' > > +#pkg install -y ... > > Would you mind if I

Re: Adding CI to our tree

2022-03-10 Thread Justin Pryzby
See attached, or at https://github.com/justinpryzby/postgres/runs/5503079878 >From c631c3d9bdb8325aaaecc5dcdfac46eca7bd907a Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Mon, 17 Jan 2022 00:53:04 -0600 Subject: [PATCH 1/7] cirrus: include hints how to install OS packages.. This is use

Re: refactoring basebackup.c

2022-03-10 Thread Justin Pryzby
I'm getting errors from pg_basebackup when using both -D- and --compress=server-* The issue seems to go away if I use --no-manifest. $ ./src/bin/pg_basebackup/pg_basebackup -h /tmp -Ft -D- --wal-method none --compress=server-gzip >/dev/null ; echo $? pg_basebackup: error: tar member has empty na

Re: wal_compression=zstd

2022-03-10 Thread Justin Pryzby
On Fri, Mar 11, 2022 at 12:23:59PM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 07:14:11AM -0600, Justin Pryzby wrote: > > Anyway there's no compelling reason to not use the default. If we were to > > use > > a non-default default, we'd have to choose be

Re: refactoring basebackup.c

2022-03-11 Thread Justin Pryzby
would help to make them all less magic. commit 1fb1e21ba7a500bb2b85ec3e65f59130fcdb4a7e Author: Justin Pryzby Date: Thu Mar 10 21:22:16 2022 -0600 pg_basebackup: make magic numbers less magic The magic 8 for .gz should actually be a 7. .tar.gz 1234567 .tar.l

Re: wal_compression=zstd

2022-03-11 Thread Justin Pryzby
While rebasing, I realized this should have bumped XLOG_PAGE_MAGIC. Also, there's a dangling "and". +The supported methods are pglz, +lz4 (if PostgreSQL +was compiled with --with-lz4) and +zstd (if PostgreSQL +was compiled with --with-zstd) and +The

Re: Add parameter jit_warn_above_fraction

2022-03-12 Thread Justin Pryzby
On Mon, Mar 07, 2022 at 04:02:16PM +0100, Magnus Hagander wrote: > On Mon, Mar 7, 2022 at 2:09 PM Justin Pryzby wrote: > > On Mon, Mar 07, 2022 at 01:10:32PM +0100, Magnus Hagander wrote: > > > On Fri, Feb 25, 2022 at 5:23 PM Justin Pryzby > > > wrote: > > &

Re: support for MERGE

2022-03-12 Thread Justin Pryzby
On Sat, Jan 29, 2022 at 12:03:35AM -0600, Justin Pryzby wrote: > Note that MergeWhenClause and MergeAction have the node definition in a > different order than the header, which is a bit confusing. The .h files still order these fields differently than the other .h files, and then the node

Re: preserving db/ts/relfilenode OIDs across pg_upgrade (was Re: storing an explicit nonce)

2022-03-12 Thread Justin Pryzby
On Tue, Jan 25, 2022 at 10:19:53AM +0530, Shruthi Gowda wrote: > On Tue, Jan 25, 2022 at 1:14 AM Robert Haas wrote: > > On Sat, Jan 22, 2022 at 2:20 AM Shruthi Gowda wrote: > > > Agree. In the latest patch, the template0 and postgres OIDs are fixed > > > to unused manually assigned OIDs 4 and 5 r

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-12 Thread Justin Pryzby
On Sun, Mar 13, 2022 at 09:45:35AM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 10:50:45AM -0600, Justin Pryzby wrote: > > Rebased over 9e9858389 (Michael may want to look at the tuplestore part?). > > Are you referring to the contents of 0003 here that changes the

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 09:41:35PM +0530, Dipesh Pandit wrote: > I tried to implement support for parallel ZSTD compression. The > library provides an option (ZSTD_c_nbWorkers) to specify the > number of compression workers. The number of parallel > workers can be set as part of compression paramet

Re: refactoring basebackup.c (zstd workers)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 01:02:20PM -0400, Robert Haas wrote: > On Mon, Mar 14, 2022 at 12:35 PM Justin Pryzby wrote: > > I suggest to use a syntax that's more general than that, maybe something > > like > > > > :[level=]N,parallel=N,flag,flag,... > > > &g

Re: [PATCH] Fix various spelling errors

2022-03-14 Thread Justin Pryzby
ality. Some fixes are > also in variable names but they should be safe to change, as the change is > consistent in all occurrences of the variable. LGTM - I found a few of these myself. Attached now, in case it's useful to handle them together. >From 7a64f847db6129b347aa5ce2560111639f77a97c

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-03-14 Thread Justin Pryzby
On Mon, Mar 14, 2022 at 01:53:54PM +0900, Michael Paquier wrote: > On Wed, Mar 09, 2022 at 10:50:45AM -0600, Justin Pryzby wrote: > > I also changed pg_ls_dir_recurse() to handle concurrent removal of a dir, > > which > > I noticed caused an infrequent failure on CI. Howe

Re: Assert in pageinspect with NULL pages

2022-03-15 Thread Justin Pryzby
On Tue, Mar 15, 2022 at 06:32:44PM +0900, Michael Paquier wrote: > + if (!IS_BRIN(indexRel)) > + ereport(ERROR, > + (errcode(ERRCODE_WRONG_OBJECT_TYPE), > + errmsg("\"%s\" is not a %s index", > +

Re: pg14 psql broke \d datname.nspname.relname

2022-03-15 Thread Justin Pryzby
On Wed, Oct 13, 2021 at 11:54:26AM -0500, Justin Pryzby wrote: > It seems unfortunate if names from log messages qualified with datname were > now > rejected. Like this one: > > | automatic analyze of table "ts.child.cdrs_2021_10_12"... Mark mentioned this &quo

Re: document the need to analyze partitioned tables

2022-03-15 Thread Justin Pryzby
be eaten. If someone reads the existing documentation, they might conclude that their partitioned tables don't need to be analyzed, and they would've been better served by not reading the docs. -- Justin >From c237d91ec258ebbf24ebd3a38e139777582817f6 Mon Sep 17 00:00:00 2001 From: Just

Re: refactoring basebackup.c (zstd negative compression)

2022-03-16 Thread Justin Pryzby
e\": 227, \"Last-Modified\": \"2022-03-16 02:29:11 GMT\", \"Checksum-Algorithm\": \"CRC32C\", \"Checksum\": \"46f69d99\" },\n{ \"Pa"... ... (gdb) p mystreamer->base.bbs_buffer.maxlen $1 = 524288 (gdb) p (int) LZ4F_compr

Re: Teach pg_receivewal to use lz4 compression

2022-03-17 Thread Justin Pryzby
On Sat, Feb 12, 2022 at 12:52:40PM +0900, Michael Paquier wrote: > On Fri, Feb 11, 2022 at 10:07:49AM -0500, Robert Haas wrote: > > Over in > > http://postgr.es/m/CA+TgmoYUDEJga2qV_XbAZ=pgebaosgfmzz6ac4_srwom_+u...@mail.gmail.com > > I was noticing that CreateWalTarMethod doesn't support LZ4 > > c

Re: refactoring basebackup.c (zstd workers)

2022-03-17 Thread Justin Pryzby
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 9178c779ba..00c593f1af 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -2731,14 +2731,24 @@ The commands accepted in replication mode are: + + For gzip the compression level shoul

Re: PROPOSAL: Support global and local disabling of indexes

2022-03-18 Thread Justin Pryzby
On Thu, Mar 17, 2022 at 11:16:24PM -0700, Paul Martinez wrote: > I propose adding an ALTER INDEX command that can enable or disable an > index on a global level: See also this thread: https://commitfest.postgresql.org/34/2937/ https://www.postgresql.org/message-id/flat/CANbhV-H35fJsKnLoZJuhkYqg2MO

Re: Remove INT64_FORMAT in translatable strings

2022-03-18 Thread Justin Pryzby
On Fri, Mar 18, 2022 at 01:12:40PM -0400, Tom Lane wrote: > Japin Li writes: > > we can rely on %lld/%llu and we decided to use them in translatable strings. > > Seems like good cleanup, so pushed. I think though that project style > is to use "long long" or "unsigned long long", without the unn

Re: Add 64-bit XIDs into PostgreSQL 15

2022-03-19 Thread Justin Pryzby
On Fri, Mar 11, 2022 at 08:26:26PM +0300, Aleksander Alekseev wrote: > Hi hackers, > > > Here is a new version of the patchset. SLRU refactoring was moved to a > > separate patch. Both v14-0003 (XID_FMT macro) and v14-0004 (SLRU > > refactoring) can be delivered in PG15. > > Here is a new version

Re: Commitfest manager for 2022-03

2022-03-19 Thread Justin Pryzby
On Sun, Feb 27, 2022 at 04:51:16PM +0800, Julien Rouhaud wrote: > On Sat, Feb 26, 2022 at 06:37:21PM -0600, Justin Pryzby wrote: > > Can I suggest to update the CF APP to allow: > > | Target version: 16 > > > > I also suggest to update patches to indicate which are

Re: refactoring basebackup.c (zstd workers)

2022-03-20 Thread Justin Pryzby
On Sun, Mar 20, 2022 at 03:05:28PM -0400, Robert Haas wrote: > On Thu, Mar 17, 2022 at 3:41 PM Justin Pryzby wrote: > > -errmsg("unrecognized > > compression algorithm: \"%s\"", > > +

Re: refactoring basebackup.c (zstd workers)

2022-03-21 Thread Justin Pryzby
On Sun, Mar 20, 2022 at 09:38:44PM -0400, Robert Haas wrote: > > This patch also needs to update the other user-facing docs. > > Which ones exactly? I mean pg_basebackup -Z -Z level -Z [{client|server}-]method[:level] --compress=level --compress=[{client|server}-]method[:level]

Re: refactoring basebackup.c (zstd workers)

2022-03-21 Thread Justin Pryzby
On Mon, Mar 21, 2022 at 12:57:36PM -0400, Robert Haas wrote: > > typo: contain a an > I searched for the "contain a an" typo that you mentioned but was not able to > find it. Can you give me a more specific pointer? Here: + * during parsing, and will otherwise contain a an appropriate error messa

Re: New Object Access Type hooks

2022-03-22 Thread Justin Pryzby
If I'm not wrong, this is still causing issues at least on cfbot/windows, even since f0206d99. https://cirrus-ci.com/task/5266352712712192 https://cirrus-ci.com/task/5061218867085312 https://cirrus-ci.com/task/5663822005403648 https://cirrus-ci.com/task/5744257246953472 https://cirrus-ci.com/task

Re: Probable CF bot degradation

2022-03-22 Thread Justin Pryzby
On Wed, Mar 23, 2022 at 12:44:09PM +1300, Thomas Munro wrote: > On Mon, Mar 21, 2022 at 12:46 PM Thomas Munro wrote: > > On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro > > wrote: > > > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > > > wrote: > > > > Additionally, are there plans to va

Re: Adding CI to our tree

2022-03-22 Thread Justin Pryzby
On Fri, Mar 18, 2022 at 03:45:03PM -0700, Andres Freund wrote: > Pushed 0001, 0002. Only change I made was to add Thanks - is there any reason not to do the MSVC compiler warnings one, too ? I see that it'll warn about issues with at least 3 patches (including one of yours). -- Justin

Re: SQL/JSON: functions

2022-03-23 Thread Justin Pryzby
At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, per COPY_PARSE_PLAN_TREES. +ERROR: unrecognized node type: 157

Re: SQL/JSON: functions

2022-03-23 Thread Justin Pryzby
On Wed, Mar 23, 2022 at 03:49:17PM -0400, Andrew Dunstan wrote: > > On 3/23/22 08:24, Justin Pryzby wrote: > > At least 0002-SQL-JSON-constructors-v64.patch has an issue with nodes, > > per COPY_PARSE_PLAN_TREES. > > > > +ERROR: unrecognized node type: 157 > >

Re: Adding CI to our tree

2022-03-23 Thread Justin Pryzby
On Thu, Mar 24, 2022 at 09:52:39AM +1300, Thomas Munro wrote: > On Thu, Mar 10, 2022 at 9:37 AM Justin Pryzby wrote: > > -Og > > Adding this to CXXFLAGS caused a torrent of warnings from g++ about > LLVM headers, which I also see on my local system for LLVM 11 and LLVM >

Re: multithreaded zstd backup compression for client and server

2022-03-23 Thread Justin Pryzby
std_compress.c:int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; } lib/zstd.h:#define ZSTD_TARGETLENGTH_MAXZSTD_BLOCKSIZE_MAX lib/zstd.h:#define ZSTD_BLOCKSIZE_MAX (1<>From 80f45cfbe13d6fc0f16e49b7ea76f1e50afb632c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Thu, 10 Mar 2022

Re: multithreaded zstd backup compression for client and server

2022-03-23 Thread Justin Pryzby
On Wed, Mar 23, 2022 at 04:34:04PM -0400, Robert Haas wrote: > be, spawning threads inside the PostgreSQL backend. Short of cats and > dogs living together, it's hard to think of anything more terrifying, > because the PostgreSQL backend is very much not thread-safe. However, > a lot of the things

Re: error_severity of brin work item

2020-11-30 Thread Justin Pryzby
On Mon, Nov 30, 2020 at 08:47:32PM -0300, Alvaro Herrera wrote: > The more I look at this, the less I like it. This would set a precedent > that any action that can be initiated from an autovac work-item has a > requirement of silently being discarded when it referenced a > non-existant relation.

Re: ALTER TABLE .. DETACH PARTITION CONCURRENTLY

2020-11-30 Thread Justin Pryzby
On Tue, Nov 03, 2020 at 08:56:06PM -0300, Alvaro Herrera wrote: > Here's an updated version of this patch. > > Apart from rebasing to current master, I made the following changes: > > * On the first transaction (the one that marks the partition as > detached), the partition is locked with ShareLo

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-11-30 Thread Justin Pryzby
On Tue, Dec 01, 2020 at 11:46:55AM +0900, Michael Paquier wrote: > On Mon, Nov 30, 2020 at 05:12:42PM +0300, Alexey Kondratov wrote: > > Thanks. I have rebased the remaining patches on top of 873ea9ee to use > > 'utility_option_list' instead of 'common_option_list'. > > Thanks, that helps a lot.

Re: error_severity of brin work item

2020-12-01 Thread Justin Pryzby
On Tue, Dec 01, 2020 at 11:07:30AM -0300, Alvaro Herrera wrote: > > Should it be done in an AtCommit hook or something like that ? > > I didn't like this idea much on first read, on extensibility grounds, > but perhaps it's not so bad because we can generalize it whenever > there's pressure to add

Re: error_severity of brin work item

2020-12-01 Thread Justin Pryzby
On Tue, Dec 01, 2020 at 03:57:24PM -0300, Alvaro Herrera wrote: > On 2020-Dec-01, Justin Pryzby wrote: > > > This was an idea I made up - I don't know any of the details of this, but if > > you give a hint I could look at it more. There'd (still) be a race window, &

Re: should INSERT SELECT use a BulkInsertState?

2020-12-02 Thread Justin Pryzby
One loose end in this patch is how to check for volatile default expressions. copyfrom.c is a utility statement, so it can look at the parser's column list: COPY table(c1,c2)... However, for INSERT, in nodeModifyTable.c, it looks like parsing, rewriting, and planning are done, at which point I do

Re: [HACKERS] [PATCH] Generic type subscripting

2020-12-02 Thread Justin Pryzby
On Wed, Dec 02, 2020 at 08:18:08PM +0100, Dmitry Dolgov wrote: > > On Wed, Dec 02, 2020 at 12:58:51PM -0500, Tom Lane wrote: > > So ... one of the things that's been worrying me about this patch > > from day one is whether it would create a noticeable performance > > penalty for existing use-cases.

Re: pg_dump, ATTACH, and independently restorable child partitions

2020-12-02 Thread Justin Pryzby
On Wed, Nov 25, 2020 at 06:35:19PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > 1. Maybe pg_restore ExecuteSqlCommandBuf() should (always?) call > > ExecuteSimpleCommands() instead of ExecuteSqlCommand(). It doesn't seem to > > break anything (although tha

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-02 Thread Justin Pryzby
ern Oid IndexGetRelation(Oid indexId, bool missing_ok); extern void reindex_index(Oid indexId, bool skip_constraint_checks, char relpersistence, int options, Oid tablespaceOid); +extern bool set_rel_tablespace(Oid reloid, Oid tablespaceOid); + /* Flag bits for reindex_relation(): */ #define REIN

copy.sgml and partitioned tables

2020-12-03 Thread Justin Pryzby
https://www.postgresql.org/docs/current/sql-copy.html |. COPY FROM can be used with plain, foreign, or partitioned tables or with views that have INSTEAD OF INSERT triggers. |. COPY only deals with the specific table named; IT DOES NOT COPY DATA TO OR FROM CHILD TABLES. ... That language in comm

Re: please update ps display for recovery checkpoint

2020-12-03 Thread Justin Pryzby
On Thu, Dec 03, 2020 at 09:18:07PM +, Bossart, Nathan wrote: > I considered also checking that update_process_title was enabled, but > I figured that these ps display updates should happen sparsely enough > that it wouldn't make much of an impact. Since bf68b79e5, update_ps_display is responsi

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-03 Thread Justin Pryzby
lowlevel routines. Note that reindex_relation is called during cluster/vacuum, and in the later patches, I moved the test from from cluster() and ExecVacuum() to rebuild_relation(). -- Justin >From df43fe542081178ea74ffb2d1d77342e6c657c2f Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 2 Dec 2020

Re: pg_dump, ATTACH, and independently restorable child partitions

2020-12-04 Thread Justin Pryzby
On Fri, Dec 04, 2020 at 12:13:05PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > [ v4-0001-pg_dump-output-separate-object-for-ALTER-TABLE.AT.patch ] > > The cfbot is being picky about this: > > 3218pg_dump.c: In function ‘dumpTableAttach’: > 3219pg_dump.c:

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-04 Thread Justin Pryzby
. Yes, thanks. This also fixes some minor formatting and rebase issues, including broken doc/. -- Justin >From 5d874da6f93341cb5aed4d3cc54137cbc341d57c Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 2 Dec 2020 20:54:47 -0600 Subject: [PATCH v33 1/5] ExecReindex and ReindexParams TODO: typedef --- src/back

Re: should INSERT SELECT use a BulkInsertState?

2020-12-05 Thread Justin Pryzby
On Thu, Dec 03, 2020 at 10:59:34AM +0530, Bharath Rupireddy wrote: > On Wed, Dec 2, 2020 at 10:24 PM Justin Pryzby wrote: > > > > One loose end in this patch is how to check for volatile default > > expressions. > > I think we should be doing all the necessary checks

pg_upgrade test for binary compatibility of core data types

2020-12-06 Thread Justin Pryzby
I'm finally returning to this 14 month old thread: (was: Re: BUG #16045: vacuum_db crash and illegal memory alloc after pg_upgrade from PG11 to PG12) On Tue, Oct 15, 2019 at 09:07:25AM +0200, Tomas Vondra wrote: > On Mon, Oct 14, 2019 at 11:41:18PM -0500, Justin Pryzby wrote: > >

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-12-09 Thread Justin Pryzby
On Fri, Dec 04, 2020 at 12:23:23PM -0500, Tom Lane wrote: > Justin Pryzby writes: > [ v24-0001-Document-historic-behavior-of-links-to-directori.patch ] > > The cfbot is unhappy with one of the test cases you added: > Maybe it could be salvaged by reversing the sense of the WHERE

create table like: ACCESS METHOD

2020-12-09 Thread Justin Pryzby
ng. https://www.postgresql.org/message-id/20190818193533.gl11...@telsasoft.com On Sun, Aug 18, 2019 at 02:35:33PM -0500, Justin Pryzby wrote: > . What do you think about pg_restore --no-tableam; similar to >--no-tablespaces, it would allow restoring a table to a different AM: &g

Re: please update ps display for recovery checkpoint

2020-12-10 Thread Justin Pryzby
Isn't the sense of "reset" inverted ? I think maybe you mean to do set_ps_display(""); in the "if reset". On Fri, Dec 11, 2020 at 12:54:22PM +0900, Michael Paquier wrote: > +update_checkpoint_display(int flags, bool restartpoint, bool reset) > +{ > + if (reset) > + { > + char

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-12 Thread Justin Pryzby
ol isTopLevel); +extern ReindexOptions ReindexParseOptions(ParseState *pstate, ReindexStmt *stmt); +extern void ReindexIndex(RangeVar *indexRelation, ReindexOptions options, bool isTopLevel); +extern Oid ReindexTable(RangeVar *relation, ReindexOptions options, bool isTopLevel); extern void Reinde

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-12-12 Thread Justin Pryzby
On Sat, Dec 12, 2020 at 01:45:26PM -0600, Justin Pryzby wrote: > On Sat, Dec 12, 2020 at 09:20:35AM +0100, Peter Eisentraut wrote: > > On 2020-12-11 21:27, Alvaro Herrera wrote: > > > By the way-- What did you think of the idea of explictly marking the > > > types us

  1   2   3   4   5   6   7   8   9   10   >