Re: Add A Glossary

2020-03-27 Thread Justin Pryzby
On Fri, Mar 20, 2020 at 11:32:25PM +0100, Jürgen Purtz wrote: > > > +Archiver > > Can you change that to archiver process ? > > I prefer the short term without the addition of 'process' - concerning > 'Archiver' as well as the other cases. But I'm not an native English > speaker. I didn't lik

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-27 Thread Justin Pryzby
On Thu, Mar 26, 2020 at 11:01:06PM -0500, Justin Pryzby wrote: > > Another issue is this: > > > +VACUUM ( FULL [, ...] ) [ TABLESPACE > > class="parameter">new_tablespace ] [ > > class="parameter">table_and_columns [, ...] ] > > As you

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-27 Thread Justin Pryzby
On Thu, Mar 26, 2020 at 11:01:06PM -0500, Justin Pryzby wrote: > > Another issue is this: > > > +VACUUM ( FULL [, ...] ) [ TABLESPACE > > class="parameter">new_tablespace ] [ > > class="parameter">table_and_columns [, ...] ] > > As you

Re: error context for vacuum to include block number

2020-03-27 Thread Justin Pryzby
On Sat, Mar 28, 2020 at 06:28:38AM +0530, Amit Kapila wrote: > > Hm, but I caused a crash *without* adding CHECK_FOR_INTERRUPTS, just > > kill+sleep. The kill() could come from running pg_cancel_backend(). And > > the > > sleep() just encourages a context switch, which can happen at any time. >

Re: error context for vacuum to include block number

2020-03-27 Thread Justin Pryzby
On Sat, Mar 28, 2020 at 06:59:10AM +0530, Amit Kapila wrote: > On Sat, Mar 28, 2020 at 6:46 AM Justin Pryzby wrote: > > > > On Sat, Mar 28, 2020 at 06:28:38AM +0530, Amit Kapila wrote: > > > > Hm, but I caused a crash *without* adding CHECK_FOR_INTERRUPTS, just &g

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-28 Thread Justin Pryzby
On Sat, Mar 28, 2020 at 01:13:54PM -0400, Tom Lane wrote: > The buildfarm just showed up another instability in the test cases > we added: Yea, as you said, this is an issue with the *testcase*. The function behavior didn't change, we just weren't previously exercising it. > select (w).size = :

debian bugrept involving fast default crash in pg11.7

2020-03-28 Thread Justin Pryzby
I happened across this bugreport, which seems to have just enough information to be interesting. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953204 |Version: 11.7-0+deb10u1 |2020-03-05 16:55:55.511 UTC [515] LOG: background worker "parallel worker" (PID 884) was terminated by signal 11: Se

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-29 Thread Justin Pryzby
On Sun, Mar 29, 2020 at 12:37:05PM -0400, Tom Lane wrote: > I wrote: > > Justin Pryzby writes: > >> Maybe we should lstat() the file to determine if it's a dangling link; if > >> lstat() fails, then skip it. Currently, we use stat(), which shows > >>

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-29 Thread Justin Pryzby
On Sun, Mar 29, 2020 at 01:22:04PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > On Sun, Mar 29, 2020 at 12:37:05PM -0400, Tom Lane wrote: > >> After looking at the callers of pg_ls_dir_files, and noticing that > >> it's already defined to ignore anything t

Re: error context for vacuum to include block number

2020-03-30 Thread Justin Pryzby
it just to save an argument to two functions? -- Justin >From 85672d7f071c91f3ec9190be7feb293f0e49cf8a Mon Sep 17 00:00:00 2001 From: Justin Pryzby Date: Wed, 26 Feb 2020 19:22:55 -0600 Subject: [PATCH v39 1/2] Avoid some calls to RelationGetRelationName --- src/backend/access/heap/vacuumlazy.c | 20 ++

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-03-30 Thread Justin Pryzby
On Mon, Mar 30, 2020 at 09:02:22PM +0300, Alexey Kondratov wrote: > Hmm, I went through the well known to me SQL commands in Postgres and a bit > more. Parenthesized options list is mostly used in two common cases: There's also ANALYZE(VERBOSE), REINDEX(VERBOSE). There was debate a year ago [0] as

Re: BUG #16109: Postgres planning time is high across version (Expose buffer usage during planning in EXPLAIN)

2020-03-30 Thread Justin Pryzby
On Wed, Jan 29, 2020 at 12:15:59PM +0100, Julien Rouhaud wrote: > Rebase due to conflict with 3ec20c7091e97. This is failing to apply probably since 4a539a25ebfc48329fd656a95f3c1eb2cda38af3. Could you rebase? (Also, not sure if this can be set as RFC?) -- Justin

Re: [PATCH] Opclass parameters

2020-03-30 Thread Justin Pryzby
to docs for this commit. --word-diff to follow. commit d3f077b813efa90b25a162bf8d227f3e4218c248 Author: Justin Pryzby Date: Mon Mar 30 20:55:06 2020 -0500 Doc review: Implement operator class parameters commit 911e70207703799605f5a0e8aad9f06cff067c63 Author: Alexander Korotkov di

Re: error context for vacuum to include block number

2020-03-30 Thread Justin Pryzby
On Tue, Mar 31, 2020 at 07:50:45AM +0530, Amit Kapila wrote: > On Mon, Mar 30, 2020 at 9:56 PM Justin Pryzby wrote: > > > > On Mon, Mar 30, 2020 at 02:31:53PM +0530, Amit Kapila wrote: > > > The v37-0003-Avoid-some-calls-to-RelationGetRelationName.patch looks > > &

Re: pg11+: pg_ls_*dir LIMIT 1: temporary files .. not closed at end-of-transaction

2020-03-31 Thread Justin Pryzby
On Tue, Mar 31, 2020 at 07:36:03AM +0200, Fabien COELHO wrote: > > > As I wrote about an earlier version of the patch, ISTM that instead of > > > reinventing, extending, adapting various ls variants (with/without > > > metadata, which show only files, which shows target of links, which shows > > >

Re: Add A Glossary

2020-03-31 Thread Justin Pryzby
On Tue, Mar 31, 2020 at 04:13:00PM +0200, Jürgen Purtz wrote: > Please find some minor suggestions in the attachment. They are based on > Corey's last patch 0001-glossary-v4.patch. > @@ -220,7 +220,7 @@ >Records to the file system and creates a special >checkpoint record. This proc

Re: Add A Glossary

2020-03-31 Thread Justin Pryzby
On Mon, Mar 30, 2020 at 01:10:19PM -0400, Corey Huinker wrote: > + > +Aggregating > + > + > + The act of combining a collection of data (input) values into > + a single output value, which may not be of the same type as the > + input values. I think we maybe already

Re: Add A Glossary

2020-03-31 Thread Justin Pryzby
On Sun, Oct 13, 2019 at 04:52:05PM -0400, Corey Huinker wrote: > 1. It's obviously incomplete. There are more terms, a lot more, to add. How did you come up with the initial list of terms ? Here's some ideas; I'm *not* suggesting to include all of everything, but hopefully start with a coherent,

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2020-03-31 Thread Justin Pryzby
On Tue, Mar 17, 2020 at 02:04:01PM -0500, Justin Pryzby wrote: > > The example in the documentation could be better indented. Also, ISTM that > > there are two implicit laterals (format & pg_ls_dir_recurse) that I would > > make explicit. I'd use the pcs alias expl

Re: control max length of parameter values logged

2020-03-31 Thread Justin Pryzby
Hi, On Wed, Apr 01, 2020 at 01:52:48AM +0100, Alexey Bashtanov wrote: > +++ b/doc/src/sgml/config.sgml > + xreflabel="log_parameter_max_length"> > + log_parameter_max_length > (integer) > + > + log_parameter_max_length configuration > parameter > + > + > +

Re: control max length of parameter values logged

2020-04-01 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 10:10:55AM +0100, Alexey Bashtanov wrote: > Hi, > > > +If greater than zero, bind parameter values reported in non-error > > > +statement-logging messages are trimmed to no more than this many > > > bytes. > > Can I suggest to say: > > > > "Limit bind param

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-01 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 03:03:34PM +0900, Michael Paquier wrote: > On Tue, Mar 31, 2020 at 01:56:07PM +0300, Alexey Kondratov wrote: > > I am fine with allowing REINDEX (CONCURRENTLY), but then we will have to > > support both syntaxes as we already do for VACUUM. Anyway, if we agree to > > add par

Re: [PATCH] Opclass parameters

2020-04-01 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 02:53:41PM +0300, Alexander Korotkov wrote: > On Tue, Mar 31, 2020 at 12:15 PM Alexander Korotkov > wrote: > > What is XXX supposed to be? > > > > The rest of patch looks good to me. > > I've pushed the patch excepts XXX. Thank you. > You're welcome to clarify XXX and/or

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-01 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 06:57:18AM -0500, Justin Pryzby wrote: > Alexey suggested that those changes should be done as a separate patch, with > the tablespace options built on top. Which makes sense. I had quite some fun > rebasing these with patches in that order. > > Howeve

Re: Add A Glossary

2020-04-01 Thread Justin Pryzby
On Tue, Mar 31, 2020 at 03:26:02PM -0400, Corey Huinker wrote: > Just so I can prioritize my work, which of these things, along with your > suggestions in previous emails, would you say is a barrier to considering > this ready for a committer? To answer your off-list inquiry, I'm not likely to mar

Re: control max length of parameter values logged

2020-04-01 Thread Justin Pryzby
Thanks for updating the patch. On Thu, Apr 02, 2020 at 01:29:04AM +0100, Alexey Bashtanov wrote: > +If greater than zero, bind parameter values reported in non-error > +statement-logging messages are trimmed to no more than this many > bytes. > +If this value is specified

Re: User Interface for WAL usage data

2020-04-01 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 10:13:18AM +0530, Amit Kapila wrote: > In thread [1], we are discussing to expose WAL usage data for each > statement in a way quite similar to how we expose BufferUsage data. > The way it exposes seems reasonable to me and no one else raises any > objection. It could be th

Re: Allow continuations in "pg_hba.conf" files

2020-04-01 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 07:25:36AM +0200, Fabien COELHO wrote: > > Hello, > > > FWIW, I don't think so. Generally a trailing backspace is an escape > > character for the following newline. And '\ ' is a escaped space, > > which is usualy menas a space itself. > > > > In this case escape charact

Re: User Interface for WAL usage data

2020-04-01 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 11:32:16AM +0530, Amit Kapila wrote: > On Thu, Apr 2, 2020 at 11:28 AM Kyotaro Horiguchi > wrote: > > > > At Thu, 2 Apr 2020 00:41:20 -0500, Justin Pryzby > > wrote in > > > Regarding v10-0004-Add-option-to-report-WAL-usage-in-EXPLAIN-

Re: ALTER tbl rewrite loses CLUSTER ON index

2020-04-01 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 03:14:21PM +0900, Michael Paquier wrote: > Now, regarding patch 0002, note that you have a problem for this part: > -tuple = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexOid)); > -if (!HeapTupleIsValid(tuple))/* probably can't happen */ > -

Re: Add A Glossary

2020-04-02 Thread Justin Pryzby
On Thu, Apr 02, 2020 at 07:09:32PM -0300, Alvaro Herrera wrote: > "partition" instead). If you (or anybody) have suggestions for the > definition of "client" and "session", I'm all ears. We already have Session: A Connection to the Database. I propose: Client: A host (or a process o

Re: User Interface for WAL usage data

2020-04-02 Thread Justin Pryzby
> > > > > Regarding > > > > > v10-0004-Add-option-to-report-WAL-usage-in-EXPLAIN-and-au.patch: > > > > > I think there should be additional spaces before "full" and before > > > > > "bytes": > > > > > > > > > > >WAL: records=2359 full page records=42 bytes=447788 > > > > > > > > > > Compare w

Re: User Interface for WAL usage data

2020-04-02 Thread Justin Pryzby
On Fri, Apr 03, 2020 at 10:52:02AM +0530, Amit Kapila wrote: > On Fri, Apr 3, 2020 at 10:41 AM Justin Pryzby wrote: > > > > > > > > > >WAL: records=2359 full page records=42 bytes=447788 > > > > 1) records; 2) pages ("full page images");

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-03 Thread Justin Pryzby
On Wed, Apr 01, 2020 at 08:08:36AM -0500, Justin Pryzby wrote: > Or maybe you'd want me to squish my changes into yours and resend after any > review comments ? I didn't hear any feedback, so I've now squished all "parenthesized" and "fix" commits. 0004

Re: Add A Glossary

2020-04-03 Thread Justin Pryzby
On Fri, Apr 03, 2020 at 05:51:43PM -0300, Alvaro Herrera wrote: > - The internal representation of one value of a SQL > + The internal representation of one value of an SQL I'm not sure about this one. The new glossary says "a SQL" seven times, and doesn't say "an sql" at all. "An SQL" d

Re: backup manifests

2020-04-03 Thread Justin Pryzby
;s no comma before the first one or after the last one. * To make that work, adding a file to the manifest starts by terminating * the most recently added line, with a comma if appropriate, but does not -- Justin >From 0bc8211dd325e8fec55eab4a3089ed1768563502 Mon Sep 17 00

Re: VACUUM memory management

2020-04-04 Thread Justin Pryzby
On Fri, Apr 03, 2020 at 09:04:34PM +0500, Ibrar Ahmed wrote: > Here is the latest patch rebased with master > (19db23bcbda99e93321cb0636677ec9c6e121a2a) Fri Apr 3 12:20:42 2020. Patch > fix all the issues, after the parallel vacuum patch. The patch works in > case of a non-parallel option and allo

Re: VACUUM memory management

2020-04-05 Thread Justin Pryzby
On Wed, Dec 11, 2019 at 09:29:17PM +0500, Ibrar Ahmed wrote: > > Did you modify Claudio's patch or write a totally new one? > > I wrote completely new patch. I tried multiple techniques like using a list > instead of fixed size array which I thought was most suitable here, but > leave that because

Re: WAL usage calculation patch

2020-04-06 Thread Justin Pryzby
On Mon, Apr 06, 2020 at 05:01:30PM +0200, Peter Eisentraut wrote: > I noticed in some of the screenshots that were tweeted that for example in > > WAL: records=1 bytes=56 > > there are two spaces between pieces of data. This doesn't match the rest of > the EXPLAIN output. Can that be adju

Re: User Interface for WAL usage data

2020-04-06 Thread Justin Pryzby
ts/pg_stat_statements.c:int64 wal_num_fpw;/* # of WAL full page image records generated */ doc/src/sgml/ref/explain.sgml: number of records, number of full page image records and amount of WAL -- Justin >From 6819c8c61c7fe2294b132a72c3897dc5cb801c4f Mon Sep 17 00:00

Re: Allow CLUSTER, VACUUM FULL and REINDEX to change tablespace on the fly

2020-04-06 Thread Justin Pryzby
neral, but true here and true for system relations ts=# SELECT COUNT(1) FROM pg_index i JOIN pg_class a ON i.indrelid=a.oid JOIN pg_class b ON i.indexrelid=b.oid WHERE a.reltablespace != b.reltablespace; count | 0 -- Justin >From 9ba04c383248e370ecaf33c3ada4ec164116f28b Mon Sep 17 00:00:00 200

Re: Avoiding smgrimmedsync() during nbtree index builds

2022-01-17 Thread Justin Pryzby
On Sun, Jan 16, 2022 at 02:25:59PM -0600, Justin Pryzby wrote: > On Thu, Jan 13, 2022 at 09:52:55AM -0600, Justin Pryzby wrote: > > This is failing on windows CI when I use initdb --data-checksums, as > > attached. > > > > https://cirrus-ci.com/task/5612464120266752 &g

Re: \d with triggers: more than one row returned by a subquery used as an expression

2022-01-17 Thread Justin Pryzby
On Mon, Jan 17, 2022 at 05:02:00PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Dec 17, 2021 at 09:43:56AM -0600, Justin Pryzby wrote: > >> I want to mention that the 2nd problem I mentioned here is still broken. > >> https://www.postgresql.org/mes

Re: libpq compression (part 2)

2022-01-17 Thread Justin Pryzby
On Tue, Jan 18, 2022 at 02:06:32AM +0500, Daniil Zakhlystov wrote: > > => Since March, errmsg doesn't need extra parenthesis around it (e3a87b4). > I’ve resolved the stuck tests and added zlib support for CI Windows builds to > patch 0003-*. Thanks > for the suggestion, all tests seem to be OK n

Re: Adding CI to our tree

2022-01-18 Thread Justin Pryzby
On Mon, Jan 17, 2022 at 12:16:19PM -0800, Andres Freund wrote: > I think it might still be worth adding stopgap way of running all tap tests on > windows though. Having a vcregress.pl function to find all directories with t/ > and run the tests there, shouldn't be a lot of code... I started doing

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Justin Pryzby
On Wed, Jan 19, 2022 at 05:13:18PM +0900, Michael Paquier wrote: > On Tue, Jan 11, 2022 at 10:08:13PM -0600, Justin Pryzby wrote: > > I asked about that before. Right now, it'll exit(1) when mkdir fails. > > > > I had written a patch to allow "." by skipping m

Re: pg_upgrade should truncate/remove its logs before running

2022-01-19 Thread Justin Pryzby
On Thu, Jan 20, 2022 at 12:01:29PM +0900, Michael Paquier wrote: > On Wed, Jan 19, 2022 at 06:05:40PM -0600, Justin Pryzby wrote: > > > I'm not sure these restrictions are needed ? > > This could lead to issues with rmtree() if we are not careful enough, > no? We&

Re: Poor performance PostgreSQL13/PostGIS 3.x

2022-01-20 Thread Justin Pryzby
On Thu, Jan 20, 2022 at 10:31:15PM +, Lugosi, Jim wrote: > We are struggling to figure out what is going on. We are migrating from > PostgreSQL 9.6 to PostgreSQL 13 w/ PostGIS. Our 9.6 version was compiled > from source and the new version (13) was installed using Yum. BTW, the new > vers

Re: document the need to analyze partitioned tables

2022-01-21 Thread Justin Pryzby
Thanks for looking at this On Fri, Jan 21, 2022 at 06:21:57PM +0100, Tomas Vondra wrote: > Hi, > > On 10/8/21 14:58, Justin Pryzby wrote: > > Cleaned up and attached as a .patch. > > > > The patch implementing autoanalyze on partitioned tables should > > rever

pg_upgrade/test.sh and v9.5

2022-01-22 Thread Justin Pryzby
ASE \"regression\" SET bytea_output TO 'hex';ALTER DATABASE \"regression\" SET timezone_abbreviations TO 'Default';" "postgres" pg_regress was changed to do that recently: commit f45dc59a38cab1d2af6baaedb79559fe2e9b3781 A

typos

2022-01-23 Thread Justin Pryzby
Useful in both initdb and the backend. @@ -8,6 +8,8 @@ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California * + * IDENTIFICATION + * src/port/pgcheckdir.c *---

makefiles writing to $@ should first write to $@.new

2022-01-23 Thread Justin Pryzby
There are many Makefile rules like foo: bar ./tool $< > $@ If the rule is interrupted (due to ^C or ENOSPC), foo can be 0 bytes or partially written, but won't be rebuilt until someone runs distclean or debugs it and removes the individual file, as I did for errcodes.h. It'd be better if

Re: [PATCH] Implement INSERT SET syntax

2022-01-23 Thread Justin Pryzby
Hi, On Fri, Jan 21, 2022 at 05:24:51PM +0800, wenjing zeng wrote: > Since this feature adds INSERT OVERRIDING SET syntax, it is recommended to > add some related testcases. Thanks for proposing some more tests. Note that your patch caused Gareth's patches to break under the cfbot. http://cfbot.

Re: typos

2022-01-23 Thread Justin Pryzby
On Mon, Jan 24, 2022 at 04:01:47PM +0900, Michael Paquier wrote: > On Sun, Jan 23, 2022 at 09:00:01PM -0600, Justin Pryzby wrote: > > Feel free to ignore this for now and revisit in April... > > I don't mind fixing that now. That means less to do later. Thanks. > > @

Re: pg_upgrade should truncate/remove its logs before running

2022-01-24 Thread Justin Pryzby
On Mon, Jan 24, 2022 at 12:39:30PM -0500, Bruce Momjian wrote: > On Mon, Jan 24, 2022 at 10:59:40AM +0900, Michael Paquier wrote: > > On Thu, Jan 20, 2022 at 07:51:37PM +0900, Michael Paquier wrote: > > > Neat idea. That would work fine for my case. So I am fine to stick > > > with this suggestio

explain_regress, explain(MACHINE), and default to explain(BUFFERS) (was: BUFFERS enabled by default in EXPLAIN (ANALYZE))

2022-01-24 Thread Justin Pryzby
I'm renaming this thread for better visibility, since buffers is a small, optional part of the patches I sent. I made a CF entry here. https://commitfest.postgresql.org/36/3409/ On Wed, Dec 01, 2021 at 06:58:20PM -0600, Justin Pryzby wrote: > On Mon, Nov 15, 2021 at 01:09:54PM -0600

Re: GUC flags

2022-01-24 Thread Justin Pryzby
On Wed, Jan 05, 2022 at 11:36:41PM -0600, Justin Pryzby wrote: > On Thu, Jan 06, 2022 at 02:19:08PM +0900, Michael Paquier wrote: > > > + initStringInfo(&ret); > > > + appendStringInfoChar(&ret, '{'); > > > + > > > + if (flags & GUC_NO_

Re: pg_upgrade should truncate/remove its logs before running

2022-01-25 Thread Justin Pryzby
On Mon, Jan 24, 2022 at 10:59:40AM +0900, Michael Paquier wrote: > On Thu, Jan 20, 2022 at 07:51:37PM +0900, Michael Paquier wrote: > > Neat idea. That would work fine for my case. So I am fine to stick > > with this suggestion. > > I have been looking at this idea, and the result is quite nice

Re: GUC flags

2022-01-25 Thread Justin Pryzby
On Tue, Jan 25, 2022 at 11:47:14AM +0100, Peter Eisentraut wrote: > On 25.01.22 02:07, Justin Pryzby wrote: > > +CREATE TABLE pg_settings_flags AS SELECT name, category, > > + 'NO_SHOW_ALL' =ANY(flags) AS no_show_all, > > + 'NO_RESET_ALL' =ANY(flags

Re: pg_ls_tmpdir to show directories and shared filesets (and pg_ls_*)

2022-01-25 Thread Justin Pryzby
mbers', 'pg_multixact/offsets', 'pg_wal', 'pg_wal/archive_status') +-- (type='d' or path~'^(global/.*|PG_VERSION|postmaster\.opts|postmaster\.pid|pg_logical/replorigin_checkpoint)$') and filename!~'[0-9]' +order by path collate &q

Re: GUC flags

2022-01-25 Thread Justin Pryzby
On Wed, Jan 26, 2022 at 09:54:43AM +0900, Michael Paquier wrote: > On Tue, Jan 25, 2022 at 12:07:51PM -0600, Justin Pryzby wrote: > > On Tue, Jan 25, 2022 at 11:47:14AM +0100, Peter Eisentraut wrote: > >> Does this stuff have any value for users? I'm worried we are exposin

Re: Output clause for Upsert aka INSERT...ON CONFLICT

2022-01-26 Thread Justin Pryzby
On Thu, Jan 27, 2022 at 10:24:14AM +0530, Anand Sowmithiran wrote: > The INSERT...ON CONFLICT is used for doing upserts in one of our app. > Our app works with both MS SQL and Postgresql, based on customer needs. > > Unlike the MS SQL MERGE command's OUTPUT clause that gives the $action >

Re: Write visibility map during CLUSTER/VACUUM FULL

2022-01-27 Thread Justin Pryzby
On Sun, Dec 26, 2021 at 08:59:31PM -0600, Justin Pryzby wrote: > Rebased on 8e1fae193864527c931a704bd7908e4fbc983f5c. > > Would someone step up to "own" this patch ? > > If not, its CF entry may need to be closed (there's no status for "needs > author"

Re: warn if GUC set to an invalid shared library

2022-01-27 Thread Justin Pryzby
On Sun, Jan 09, 2022 at 11:58:18AM -0800, Maciek Sakrejda wrote: > On Sat, Jan 8, 2022 at 2:07 PM Justin Pryzby wrote: > > Unfortunately, the output for dlopen() is not portable, which (I think) > > means > > most of what I wrote can't be made to work.. Since it doesn

Re: do only critical work during single-user vacuum?

2022-01-27 Thread Justin Pryzby
On Fri, Jan 21, 2022 at 05:41:58PM -0500, John Naylor wrote: > On Wed, Jan 19, 2022 at 5:26 PM Michael Paquier wrote: > > > > Could you avoid introducing a new grammar pattern in VACUUM? Any new > > option had better be within the parenthesized part as it is extensible > > at will with its set of

Re: GUC flags

2022-01-27 Thread Justin Pryzby
On Wed, Jan 26, 2022 at 03:29:29PM +0900, Michael Paquier wrote: > On Tue, Jan 25, 2022 at 09:44:26PM -0600, Justin Pryzby wrote: > > It seems like an arbitrary and short-sighted policy to expose a handful of > > flags in the view for the purpose of retiring ./check_guc, but not ex

Re: support for MERGE

2022-01-28 Thread Justin Pryzby
On Fri, Jan 28, 2022 at 05:27:37PM -0300, Alvaro Herrera wrote: > The one thing I'm a bit bothered about is the fact > that we expose a lot of executor functions previously static. I am now > wondering if it would be better to move the MERGE executor support > functions into nodeModifyTable.c, whi

Re: warn if GUC set to an invalid shared library

2022-01-28 Thread Justin Pryzby
Thanks for loooking On Fri, Jan 28, 2022 at 11:36:20PM +, Cary Huang wrote: > This is fine as this is what these patches are aiming to provide. However, > when I try to restart the server, it fails to start because abc.so and xyz.so > do not exist. Setting the parameters "local_preload_libra

Re: GUC flags

2022-01-29 Thread Justin Pryzby
On Sat, Jan 29, 2022 at 03:38:53PM +0900, Michael Paquier wrote: > +-- Three exceptions as of transaction_* > +SELECT name FROM pg_settings_flags > + WHERE NOT no_show_all AND no_reset_all > + ORDER BY 1; > + name > + > + transaction_deferrable > + trans

Re: Why is INSERT-driven autovacuuming based on pg_class.reltuples?

2022-01-30 Thread Justin Pryzby
On Thu, Jan 27, 2022 at 01:59:38PM -0800, Peter Geoghegan wrote: > On Thu, Jan 27, 2022 at 12:20 PM Peter Geoghegan wrote: > > Both VACUUM and ANALYZE update pg_class.reltuples. But this code seems > > to assume that it's only something that VACUUM can ever do. Why > > wouldn't we expect a plain A

Re: pg_walinspect - a new extension to get raw WAL data and WAL stats

2022-01-31 Thread Justin Pryzby
thought this was simpler. https://www.postgresql.org/message-id/20210622031358.gf29...@telsasoft.com On Mon, Jun 21, 2021 at 10:13:58PM -0500, Justin Pryzby wrote: > +/* compression methods supported */ > +#define BKPIMAGE_COMPRESS_PGLZ 0x04 > +#define BKPIMAGE_COMPRESS_ZLIB 0x08 > +#d

Re: GUC flags

2022-01-31 Thread Justin Pryzby
On Mon, Jan 31, 2022 at 02:17:41PM +0900, Michael Paquier wrote: > With all those doc fixes, applied after an extra round of review. So > this makes us rather covered with the checks on the flags. Thanks > Now, what do we do with the rest of check_guc that involve a direct > lookup at what's on

Re: Condition pushdown: why (=) is pushed down into join, but BETWEEN or >= is not?

2022-02-01 Thread Justin Pryzby
> Subject: [PATCH v1 1/6] Rebaee David's patch against the latest code. If you use git-am, then the author/commit information is preserved. It's probably good to include a link to the patch in any case. > Subject: [PATCH v1 4/6] remove duplicated qual executing.

Re: Adding CI to our tree

2022-02-02 Thread Justin Pryzby
On Tue, Jan 18, 2022 at 03:08:47PM -0600, Justin Pryzby wrote: > On Mon, Jan 17, 2022 at 12:16:19PM -0800, Andres Freund wrote: > > I think it might still be worth adding stopgap way of running all tap tests > > on > > windows though. Having a vcregress.pl function to find

Re: do only critical work during single-user vacuum?

2022-02-03 Thread Justin Pryzby
On Tue, Feb 01, 2022 at 04:50:31PM -0500, John Naylor wrote: > On Thu, Jan 27, 2022 at 8:28 PM Justin Pryzby wrote: > > > I'm sure you meant "&" here (fixed in attached patch to appease the cfbot): > > + if (options | VACOPT_MINIMAL) > >

Re: do only critical work during single-user vacuum?

2022-02-03 Thread Justin Pryzby
On Thu, Feb 03, 2022 at 07:26:01PM -0800, Andres Freund wrote: > Which reminds me: Perhaps we ought to hint about reducing / removing > autovacuum cost limits in this situation? And perhaps make autovacuum absorb > config changes while running? It's annoying that an autovac halfway into a > huge ta

Re: Adding CI to our tree

2022-02-03 Thread Justin Pryzby
On Thu, Feb 03, 2022 at 11:57:18AM -0800, Andres Freund wrote: > On 2022-02-02 21:58:28 -0600, Justin Pryzby wrote: > > FYI: I've rebased these against your cirrus/windows changes. > > Did you put then on a dedicated branch, or only intermixed with other changes? Yes it

Re: Release notes for February minor releases

2022-02-04 Thread Justin Pryzby
Build extended statistics for partitioned tables (Justin Pryzby) + + + A previous bug fix disabled building of extended statistics for + old-style inheritance trees, but it also prevented building them for + partitioned tables, which was an unnecessary restriction. +

Re: Release notes for February minor releases

2022-02-04 Thread Justin Pryzby
On Fri, Feb 04, 2022 at 04:29:19PM -0500, Tom Lane wrote: > > + A previous bug fix disabled building of extended statistics for > > + old-style inheritance trees, but any existing statistics data was > > + not removed, and that data would become more and more out-of-date > > + o

Re: Release notes for February minor releases

2022-02-06 Thread Justin Pryzby
On Sun, Feb 06, 2022 at 02:22:25PM -0500, Jonathan S. Katz wrote: > I'm working on the release announcement and have been following this thread. > > Are there steps we can provide to help a user detect that this occurred, > even though it's a low-probability? It's the same question as raised here

Re: GUC flags

2022-02-06 Thread Justin Pryzby
Thanks for working on it. Your test is checking that stuff in sample.conf is actually a GUC and not marked NOT_IN_SAMPLE. But those are both unlikely mistakes to make. The important/interesting test is the opposite: that all GUCs are present in the sample file. It's a lot easier for someone to

Re: 2022-02-10 release announcement draft

2022-02-06 Thread Justin Pryzby
On Sun, Feb 06, 2022 at 08:01:02PM -0500, Jonathan S. Katz wrote: > Hi, > > Attached is a draft for the release announcement for the 2022-02-10 > cumulative update release. > > Please review for technical accuracy or if you believe any items should be > added/removed. Please provide feedback no l

Re: GUC flags

2022-02-07 Thread Justin Pryzby
On Tue, Feb 08, 2022 at 10:44:07AM +0900, Michael Paquier wrote: > What do you think about the updated version attached? I have applied > the addition of config_data() separately. Looks fine > + # Check if this line matches a GUC parameter. > + if ($line =~ m/^#?([_[:alpha:]]+) (= .*|[^

Re: is the base backup protocol used by out-of-core tools?

2022-02-08 Thread Justin Pryzby
On Tue, Feb 08, 2022 at 11:26:41AM -0500, Robert Haas wrote: > Patches for the nuking are attached. If nobody writes back, I'm going > to assume that means nobody cares, and commit these some time before > feature freeze. If one or more people do write back, then my plan is > to see what they have

Re: wrong fds used for refilenodes after pg_upgrade relfilenode changes Reply-To:

2022-02-09 Thread Justin Pryzby
On Wed, Feb 09, 2022 at 02:00:04PM -0800, Andres Freund wrote: > On linux we can do so by a) checking if readlink(/proc/self/fd/$fd) points to > a filename ending in " (deleted)", b) doing fstat(fd) and checking if st_nlink > == 0. You could also stat() the file in proc/self/fd/N and compare st_in

Re: warn if GUC set to an invalid shared library

2022-02-09 Thread Justin Pryzby
-- diff --git a/src/test/regress/sql/rules.sql b/src/test/regress/sql/rules.sql index 8bdab6dec30..6308e5d27fd 100644 --- a/src/test/regress/sql/rules.sql +++ b/src/test/regress/sql/rules.sql @@ -1198,6 +1198,7 @@ drop table hat_data; -- test for pg

Re: can we add subscription TAP test option "vcregress subscriptioncheck" for MSVC builds?

2022-02-10 Thread Justin Pryzby
On Thu, Feb 10, 2022 at 10:21:08PM +0530, Bharath Rupireddy wrote: > Thanks for the comments. Above looks good to me, changed that way, PSA v2. I spy a typo: subcription -- Justin

Re: refactoring basebackup.c

2022-02-11 Thread Justin Pryzby
On Fri, Feb 11, 2022 at 08:35:25PM +0530, Jeevan Ladhe wrote: > Thanks Robert for the bravity :-) FYI: there's a couple typos in the last 2 patches. I added them to my typos branch; feel free to wait until April if you'd prefer to see them fixed in bulk. diff --git a/doc/src/sgml/ref/pg_baseback

Re: pgsql: Add TAP test to automate the equivalent of check_guc

2022-02-11 Thread Justin Pryzby
On Fri, Feb 11, 2022 at 09:59:55AM -0500, Tom Lane wrote: > Christoph Berg writes: > > this test is failing at Debian package compile time: > > Could not open /usr/share/postgresql/15/postgresql.conf.sample: No such > > file or directory at t/003_check_guc.pl line 47. > > > So it's trying to rea

Re: pgsql: Add TAP test to automate the equivalent of check_guc

2022-02-11 Thread Justin Pryzby
On Fri, Feb 11, 2022 at 10:41:27AM -0500, Tom Lane wrote: > Justin Pryzby writes: > > On Fri, Feb 11, 2022 at 09:59:55AM -0500, Tom Lane wrote: > >> This seems like a pretty bad idea even if it weren't failing outright. > >> We should be examining the version

Re: support for MERGE

2022-02-11 Thread Justin Pryzby
On Fri, Feb 11, 2022 at 03:21:43PM -0300, Alvaro Herrera wrote: > On 2022-Jan-28, Justin Pryzby wrote: > > Have you looked at code coverage ? I have an experimental patch to add > > that to > > cirrus, and ran it with this patch; visible here: > > https://cirrus-c

Re: support for MERGE

2022-02-11 Thread Justin Pryzby
On Fri, Feb 11, 2022 at 05:25:49PM -0300, Alvaro Herrera wrote: > > I'm not sure git diff --cherry-pick is widely known/used, but I think > > using that relative to master may be good enough. > > I had never heard of git diff --cherry-pick, and the manpages I found > don't document it, so frankl

Re: refactoring basebackup.c

2022-02-12 Thread Justin Pryzby
The LZ4 patches caused new compiler warnings. It's the same issue that was fixed at 71ce8 for gzip. I think they would've been visible in the CI environment, too. https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=wrasse&dt=2022-02-12%2005%3A08%3A48&stg=make "/export/home/nm/farm/st

buildfarm warnings

2022-02-12 Thread Justin Pryzby
Is there any check for warnings from new code, other than those buildfarm members with -Werror ? It'd be better to avoid warnings, allowing members to use -Werror, rather than to allow/ignore warnings, which preclude that possibility. A circular problem. I checked for warnings on master during "

Re: Adding CI to our tree

2022-02-12 Thread Justin Pryzby
On Tue, Jan 18, 2022 at 05:16:26PM -0800, Andres Freund wrote: > On 2022-01-18 15:08:47 -0600, Justin Pryzby wrote: > > On Mon, Jan 17, 2022 at 12:16:19PM -0800, Andres Freund wrote: > > > I think it might still be worth adding stopgap way of running all tap > > > te

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2022-02-12 Thread Justin Pryzby
On Sat, Feb 12, 2022 at 06:00:44PM -0800, Andres Freund wrote: > Hi, > > On 2022-02-11 16:19:12 -0500, Robert Haas wrote: > > I somewhat hope we never end up with THREE strategies for creating a new > > database, but now that I think about it, we might. Somebody might want to > > use a fancy FS pr

Re: Adding CI to our tree

2022-02-12 Thread Justin Pryzby
On Sat, Feb 12, 2022 at 04:24:20PM -0800, Andres Freund wrote: > > > e5286ede1b4 cirrus: avoid unnecessary double star ** > > > > Can't get excited about this, but whatever. > > > > What I am excited about is that some of your other changes showed that we > > don't need separate *_artifacts for s

Re: Adding CI to our tree

2022-02-12 Thread Justin Pryzby
On Sat, Feb 12, 2022 at 05:20:08PM -0800, Andres Freund wrote: > On 2022-02-03 23:04:04 -0600, Justin Pryzby wrote: > > > I'm a bit worried about the increased storage and runtime overhead due to > > > the > > > docs changes. We probably can make it a good bit

Re: Adding CI to our tree

2022-02-13 Thread Justin Pryzby
On Sat, Feb 12, 2022 at 04:24:20PM -0800, Andres Freund wrote: > > What I am excited about is that some of your other changes showed that we > > don't need separate *_artifacts for separate directories anymore. That used > > to > > be the case, but an array of paths is now supported. Putting log,

Re: Adding CI to our tree

2022-02-13 Thread Justin Pryzby
On Sun, Feb 13, 2022 at 01:23:16PM -0800, Andres Freund wrote: > If you're seeing this on windows on one of your test branches, that's much > more likely to be caused by the alltaptests stuff, than by the change in > artifact instruction. Oh - I suppose you're right. That's an unfortunate consequ

<    9   10   11   12   13   14   15   16   17   18   >