Re: Patch: Range Merge Join

2021-11-10 Thread Thomas
sider an implementation as a CustomScan, as we feel the join is rather general, can be implemented using a small extension of the existing Merge Join, and would require a substantial duplication of the Merge Join code. Kind regards, Thomas, Anton, Johann, Michael, Peter [1] https://doi.org/10.1007/s

Re: Patch: Range Merge Join

2021-11-17 Thread Thomas
Thank you for the feedback and sorry for the oversight. I fixed the bug and attached a new version of the patch. Kind Regards, Thomas Am Mi., 17. Nov. 2021 um 15:03 Uhr schrieb Daniel Gustafsson < dan...@yesql.se>: > This patch fails to compile due to an incorrect function n

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-10-25 Thread thomas
se. For users, what is more surprising: A foot-gun that sounds safe, or one option that overrides another? -- typedef struct me_s { char name[] = { "Thomas Habets" }; char email[] = { "tho...@habets.se" }; char kernel[]= { "Linux" }; char *pgpKey[

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2022-11-01 Thread thomas
On Mon, 31 Oct 2022 23:36:16 +, Jacob Champion said: >> I wanted to get feedback on the approach before wordsmithing too >> much. > > I've added this to tomorrow's CF [1]. Thomas, if you get (or already > have) a PG community username, I can add you as an au

Re: Patch: Range Merge Join

2021-06-10 Thread Thomas
Thank you for the feedback. I removed the redundant comments from the patch and added this thread to the July CF [1]. Best Regards, Thomas Mannhart [1] https://commitfest.postgresql.org/33/3160/ Am Do., 10. Juni 2021 um 05:10 Uhr schrieb David Rowley < dgrowle...@gmail.com>: > On Th

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2021-09-06 Thread thomas
t CA file. But could be that I missed a config option? -- typedef struct me_s { char name[] = { "Thomas Habets" }; char email[] = { "tho...@habets.se" }; char kernel[]= { "Linux" }; char *pgpKey[] = { "http://www.habets.pp.se/pubkey.txt"

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2021-09-07 Thread thomas
with LE that every user of SSL starts using "real" certs. Postgres default settings imply that most people who even enable SSL will not verify the CA nor the name, which is a shame. -- typedef struct me_s { char name[] = { "Thomas Habets" }; char email[] = { "

Re: [PATCH] Add `verify-system` sslmode to use system CA pool for server cert

2021-09-28 Thread thomas
rable. -- typedef struct me_s { char name[] = { "Thomas Habets" }; char email[] = { "tho...@habets.se" }; char kernel[]= { "Linux" }; char *pgpKey[] = { "http://www.habets.pp.se/pubkey.txt"; }; char pgp[] = {

Re: killing perl2host

2022-02-20 Thread Thomas Munro
On Sun, Feb 20, 2022 at 10:51 AM Andrew Dunstan wrote: > On 2/19/22 16:34, Andres Freund wrote: > >> The first > >> removes perl2host completely and adjusts all the places that use it. > >> The second removes almost all the remaining msys special processing in > >> TAP tests. Very nice improveme

Re: Missed condition-variable wakeups on FreeBSD

2022-02-27 Thread Thomas Munro
On Sun, Feb 27, 2022 at 8:07 AM Tom Lane wrote: > I don't know much about how gdb interacts with kernel calls on > FreeBSD, but I speculate that the poll(2) call returns with EINTR > after gdb releases the process, and then things resume fine, Yeah, at least FreeBSD and macOS interrupt system cal

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-02-27 Thread Thomas Munro
L); + WaitLatch(NULL, WL_EXIT_ON_PM_DEATH | WL_TIMEOUT, 10, WAIT_EVENT_SYNC_REQUEST); But... really, this should be waiting on a condition variable that the checkpointer broadcasts on when the queue goes from full to not full, no? Perhaps for master only? From 23abdf95dea74b914dc56a46739

Re: Missed condition-variable wakeups on FreeBSD

2022-02-28 Thread Thomas Munro
On Sun, Feb 27, 2022 at 11:18 AM Melanie Plageman wrote: > How could it be that worker 2 is waiting on the build barrier in > PHJ_BUILD_HASHING_INNER and worker 1 and the leader are waiting on it > with it supposedly in PHJ_BUILD_HASHING_OUTER? That'd be consistent with a wakeup going missing, so

Re: Missed condition-variable wakeups on FreeBSD

2022-02-28 Thread Thomas Munro
On Sun, Feb 27, 2022 at 9:44 AM Andres Freund wrote: > > (gdb) p debug_query_string > > $1 = 0x21873090 "select count(*) from simple r join simple s using (id);" > > (gdb) bt > > #0 _poll () at _poll.S:4 > > #1 0x21701361 in __thr_poll (fds=0x219dc170, nfds=2, timeout=-1) at > > /usr/src/lib/li

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-03-01 Thread Thomas Munro
On Mon, Feb 28, 2022 at 2:36 PM Andres Freund wrote: > On February 27, 2022 4:19:21 PM PST, Thomas Munro > wrote: > >It seems a little strange to introduce a new wait event that will very > >often appear into a stable branch, but ... it is actually telling the > >truth,

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-04 Thread Thomas Munro
y break something on Windows? From 6f001ec46c5e5f6ffa8e103f2b0d711e6904b763 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Fri, 4 Mar 2022 23:07:46 +1300 Subject: [PATCH] Fix warning in basebackup of in-place tablespaces. While building the tablespace map for a base backup, don't call readlink() on direct

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-04 Thread Thomas Munro
On Fri, Mar 4, 2022 at 10:37 PM Bharath Rupireddy wrote: > It looks like regression tests are failing on Windows Server 2019 [1]. > Ignore if it's reported elsewhere. It's broken since 46ab07ff "Clean up assorted failures under clang's -fsanitize=undefined checks.": https://github.com/postgres/p

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-04 Thread Thomas Munro
On Sat, Mar 5, 2022 at 7:10 AM Andres Freund wrote: > Or perhaps pg_ctl ought to pass CREATE_NEW_PROCESS_GROUP to CreateProcess()? > The lack of a process group would explain why we're getting signalled on > ctrl-c... I thought that sounded promising, given that the recent Cirrus agent commit you

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-04 Thread Thomas Munro
On Sat, Mar 5, 2022 at 10:56 AM Andres Freund wrote: > I suspect one also needs the console detach thing. I tried adding DETACHED_PROCESS (which should be like calling FreeConsole() in child process) and then I tried CREATE_NEW_CONSOLE instead, on top of CREATE_NEW_PROCESS_GROUP. Neither helped,

Re: Regression tests failures on Windows Server 2019 - on master at commit # d816f366b

2022-03-04 Thread Thomas Munro
On Sat, Mar 5, 2022 at 4:19 PM Andres Freund wrote: > https://github.com/cirruslabs/cirrus-ci-agent/issues/218#issuecomment-1059657781 Oof. Nice detective work. > As indicated in the message above it, there's a workaround. Not sure if worth > committing, if they were to fix it in a few days? c

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-07 Thread Thomas Munro
On Tue, Mar 8, 2022 at 12:58 AM Michael Paquier wrote: > On Fri, Mar 04, 2022 at 11:26:43PM +1300, Thomas Munro wrote: > > + /* Skip in-place tablespaces (testing use only) */ > > + if (get_dirent_type(fullpath, de, false, ERROR) == PGFILETYPE_DIR) > > +

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-07 Thread Thomas Munro
On Tue, Mar 8, 2022 at 10:39 AM Thomas Munro wrote: > Test on a Win10 VM. Erm, "Tested" (as in, I tested), I meant to write...

Re: Adding CI to our tree

2022-03-09 Thread Thomas Munro
On Thu, Mar 10, 2022 at 9:37 AM Justin Pryzby wrote: > On Wed, Mar 09, 2022 at 10:12:54AM -0800, Andres Freund wrote: > > On 2022-03-09 11:47:23 -0600, Justin Pryzby wrote: > > > I'm curious what you think of this patch. > > > > > > It makes check-world on freebsd over 30% faster - saving 5min. >

Re: Adding CI to our tree

2022-03-09 Thread Thomas Munro
On Thu, Mar 10, 2022 at 4:33 PM Andres Freund wrote: > On 2022-03-10 15:43:16 +1300, Thomas Munro wrote: > > I'm confused. > > The "terrible IO wait" thing was before we reduced the number of CPUs and > concurrent jobs. It makes sense to me that with just two CPUs

Re: WIP: WAL prefetch (another approach)

2022-03-10 Thread Thomas Munro
On Fri, Mar 11, 2022 at 6:31 PM Thomas Munro wrote: > Thanks for your review of 0001! It gave me a few things to think > about and some good improvements. And just in case it's useful, here's what changed between v21 and v22.. diff --git a/src/backend/access/transam/xlogreader.

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 8, 2022 at 4:01 PM Kyotaro Horiguchi wrote: > At Tue, 8 Mar 2022 10:28:46 +0900, Michael Paquier > wrote in > > On Tue, Mar 08, 2022 at 10:06:50AM +0900, Kyotaro Horiguchi wrote: > > > At Tue, 8 Mar 2022 10:39:06 +1300, Thomas Munro > > > wrote

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 15, 2022 at 2:33 PM Thomas Munro wrote: > As for the complaint about pg_tablespace_location() failing, would it > be better to return an empty string? That's what was passed in as > LOCATION. Something like the attached. (Hrrmm, the contract for pgwin32_is_junction

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-14 Thread Thomas Munro
On Tue, Mar 15, 2022 at 2:50 PM Michael Paquier wrote: > On Tue, Mar 15, 2022 at 02:33:17PM +1300, Thomas Munro wrote: > > As for the complaint about pg_tablespace_location() failing, would it > > be better to return an empty string? That's what was passed in as > > LOC

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-15 Thread Thomas Munro
On Tue, Mar 15, 2022 at 10:30 PM Michael Paquier wrote: > So, which one of a relative path or an absolute path do you think > would be better for the user? My preference tends toward the relative > path, as we know that all those tablespaces stay in pg_tblspc/ so one > can make the difference wit

Re: Checkpointer sync queue fills up / loops around pg_usleep() are bad

2022-03-15 Thread Thomas Munro
On Wed, Mar 2, 2022 at 10:58 AM Andres Freund wrote: > On 2022-03-02 06:46:23 +1300, Thomas Munro wrote: > > From a9344bb2fb2a363bec4be526f87560cb212ca10b Mon Sep 17 00:00:00 2001 > > From: Thomas Munro > > Date: Mon, 28 Feb 2022 11:27:05 +1300 > > Subject: [PATCH v2

Re: USE_BARRIER_SMGRRELEASE on Linux?

2022-03-15 Thread Thomas Munro
On Sat, Feb 19, 2022 at 7:01 AM Nathan Bossart wrote: > Here is a new patch. This is essentially the same as v1, but I've spruced > up the comments and the commit message. Hi Nathan, Pushed and back-patched (it's slightly different before 12). Thanks!

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Thomas Munro
On Thu, Mar 17, 2022 at 3:53 PM Michael Paquier wrote: > > On Wed, Mar 16, 2022 at 05:15:58PM +0900, Kyotaro Horiguchi wrote: > > I'm not sure that the "of the symbolic link in pg_tblspc/" is > > needed. And allow_in_place_tablespaces alone doesn't create in-place > > tablespace. So this might nee

Re: pg_tablespace_location() failure with allow_in_place_tablespaces

2022-03-16 Thread Thomas Munro
On Thu, Mar 17, 2022 at 7:18 PM Michael Paquier wrote: > On Thu, Mar 17, 2022 at 04:34:30PM +1300, Thomas Munro wrote: > > I think what Horiguchi-san was pointing out above is that you need to > > enable the GUC *and* say LOCATION '', which the new paragraph doesn&#x

Re: WIP: WAL prefetch (another approach)

2022-03-17 Thread Thomas Munro
On Mon, Mar 14, 2022 at 8:17 PM Julien Rouhaud wrote: > Great! I'm happy with 0001 and I think it's good to go! I'll push 0001 today to let the build farm chew on it for a few days before moving to 0002.

Re: Declare PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY for aarch64

2022-03-17 Thread Thomas Munro
On Thu, Mar 17, 2022 at 1:32 AM Yura Sokolov wrote: > So I believe it is safe to define PG_HAVE_8BYTE_SINGLE_COPY_ATOMICITY > for aarch64 Agreed, and pushed. There was another thread that stalled, so I added a reference and a reviewer from that to your commit message. This should probably also

Re: Probable CF bot degradation

2022-03-18 Thread Thomas Munro
On Sat, Mar 19, 2022 at 5:07 AM Julien Rouhaud wrote: > On Fri, Mar 18, 2022 at 07:43:47PM +0400, Pavel Borisov wrote: > > Hi, hackers! > > I've noticed that CF bot hasn't been running active branches from yesterday: > > https://github.com/postgresql-cfbot/postgresql/branches/active > > > > Also,

Re: [PATCH] Add native windows on arm64 support

2022-03-18 Thread Thomas Munro
On Wed, Feb 23, 2022 at 11:09 PM Niyas Sait wrote: > I have created a patch that adds support for building Postgres for the > windows/arm64 platform using the MSVC toolchain. Following changes have been > included > > 1. Extend MSVC scripts to handle ARM64 platform. > 2. Add arm64 definition of

Re: Probable CF bot degradation

2022-03-18 Thread Thomas Munro
On Sat, Mar 19, 2022 at 9:41 AM Pavel Borisov wrote: >> >> remote: Internal Server Error >> To github.com:postgresql-cfbot/postgresql.git >> ! [remote rejected] commitfest/37/3489 -> commitfest/37/3489 >> (Internal Server Error) >> error: failed to push some refs to >> 'github.com:postgres

Re: A test for replay of regression tests

2022-03-19 Thread Thomas Munro
Another failure under 027_stream_regress.pl: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2022-03-16%2005%3A58%3A05 vacuum ... FAILED 3463 ms I'll try to come up with the perl needed to see the regression.diffs next time...

Re: WIP: WAL prefetch (another approach)

2022-03-19 Thread Thomas Munro
On Fri, Mar 18, 2022 at 9:59 AM Thomas Munro wrote: > I'll push 0001 today to let the build farm chew on it for a few days > before moving to 0002. Clearly 018_wal_optimize.pl is flapping and causing recoveryCheck to fail occasionally, but that predates the above commit. I didn&#

Re: WIP: WAL prefetch (another approach)

2022-03-20 Thread Thomas Munro
On Sun, Mar 20, 2022 at 5:36 PM Thomas Munro wrote: > Clearly 018_wal_optimize.pl is flapping Correction, 019_replslot_limit.pl, discussed at https://www.postgresql.org/message-id/flat/83b46e5f-2a52-86aa-fa6c-8174908174b8%40iki.fi .

Re: A test for replay of regression tests

2022-03-20 Thread Thomas Munro
On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro wrote: > Another failure under 027_stream_regress.pl: > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2022-03-16%2005%3A58%3A05 > > vacuum ... FAILED 3463 ms > > I'l

Re: A test for replay of regression tests

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 2:34 AM Andrew Dunstan wrote: > On 3/20/22 05:36, Thomas Munro wrote: > > On Sun, Mar 20, 2022 at 5:20 PM Thomas Munro wrote: > >> I'll try to come up with the perl needed to see the regression.diffs > >> next time... > > Here'

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent wrote: > Would you know how long the expected bitrot re-check period for CF > entries that haven't been updated is, or could the bitrot-checking > queue be displayed somewhere to indicate the position of a patch in > this queue? I see that you

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > wrote: > > Would you know how long the expected bitrot re-check period for CF > > entries that haven't been updated is, or could the bitrot-checking > > qu

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 1:41 PM Peter Geoghegan wrote: > BTW, I think that the usability of the CFBot website would be improved > if there was a better visual indicator of what each "green tick inside > a circle" link actually indicates -- what are we testing for each > green tick/red X shown? > >

Re: Probable CF bot degradation

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 3:11 PM Andres Freund wrote: > On 2022-03-21 14:44:55 +1300, Thomas Munro wrote: > > Those tooltips come from the "name" elements of the .cirrus.yml file > > where tasks are defined, with Cirrus's task status appended. If we > > had a s

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-20 Thread Thomas Munro
ation option to be specified"); + goto bad_argument; + } + if ((optind + 2) < argc) { pg_log_error("too many command-line arguments (first is \"%s\")", @@ -1148,6 +1261,19 @@ main(int argc, char **argv) config.filter_by_xid != record->xl_xid) conti

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-20 Thread Thomas Munro
On Mon, Mar 21, 2022 at 4:36 PM Thomas Munro wrote: > On Sat, Feb 26, 2022 at 7:58 AM David Christensen > wrote: > > Attached is V2 with additional feedback from this email, as well as the > > specification of the > > ForkNumber and FPW as specifiable options. > &

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-21 Thread Thomas Munro
waldump. This feature allows you to only output records that are touch a specific RelFileNode and optionally BlockNumber and ForkNum in this relation. We also add the independent ability to filter Full Page Write records. Author: David Christensen Reviewed-by: Peter Geoghegan Reviewed-by:

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-21 Thread Thomas Munro
I have a new socket abstraction patch that should address the known Windows socket/event bugs, but it's a little bigger than I thought it would be, not quite ready, and now too late to expect people to review for 15, so I think it should go into the next cycle. I've bounced https://commitfest.post

Broken make dependency somewhere near win32ver.o?

2022-03-21 Thread Thomas Munro
Hi, Here's a strange one-off failure seen on CI[1], in the CompilerWarnings task where we check that mingw cross-compile works: [10:48:29.045] time make -s -j${BUILD_JOBS} world-bin [10:48:38.705] x86_64-w64-mingw32-gcc: error: win32ver.o: No such file or directory [10:48:38.705] make[3]: *** [Ma

Re: Why is src/test/modules/committs/t/002_standby.pl flaky?

2022-03-21 Thread Thomas Munro
On Tue, Mar 22, 2022 at 4:13 PM Tom Lane wrote: > Thomas Munro writes: > > I have a new socket abstraction patch that should address the known > > Windows socket/event bugs, but it's a little bigger than I thought it > > would be, not quite ready, and now too late

Re: A test for replay of regression tests

2022-03-21 Thread Thomas Munro
Well, at least the first one. Here's a patch like that. From ba05f03c202bf66c7692787ec24ece13b193a897 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Tue, 22 Mar 2022 16:54:42 +1300 Subject: [PATCH] Try to stabilize vacuum test. As commits b700f96c and 3414099c did for the reloptions te

Re: New Object Access Type hooks

2022-03-21 Thread Thomas Munro
On Fri, Mar 18, 2022 at 4:22 PM Mark Dilger wrote: > (FYI, I got a test failure from src/test/recovery/t/013_crash_restart.pl when > testing v1-0001. I'm not sure yet what that is about.) Doesn't look like 0001 has anything to do with that... Are you on a Mac? Did it look like this recent fa

Re: Broken make dependency somewhere near win32ver.o?

2022-03-21 Thread Thomas Munro
On Tue, Mar 22, 2022 at 4:14 PM Andres Freund wrote: > The problem looks to be that pg_dumpall doesn't have a dependency on OBJs, > which in turn is what contains the dependency on WIN32RES, which in turn > contains win32ver.o. So the build succeeds if pg_dump/restores's dependencies > are built f

Re: MDAM techniques and Index Skip Scan patch

2022-03-22 Thread Thomas Munro
On Tue, Mar 22, 2022 at 2:34 PM Andres Freund wrote: > IMO it's pretty clear that having "duelling" patches below one CF entry is a > bad idea. I think they should be split, with inactive approaches marked as > returned with feeback or whatnot. I have the impression that this thread is getting so

Re: Probable CF bot degradation

2022-03-22 Thread Thomas Munro
On Mon, Mar 21, 2022 at 12:46 PM Thomas Munro wrote: > On Mon, Mar 21, 2022 at 12:23 PM Thomas Munro wrote: > > On Mon, Mar 21, 2022 at 1:58 AM Matthias van de Meent > > wrote: > > > Additionally, are there plans to validate commits of the main branch > > > b

Re: cpluspluscheck vs ICU

2022-03-22 Thread Thomas Munro
On Wed, Mar 23, 2022 at 3:23 PM Andres Freund wrote: > On 2022-03-22 17:20:24 -0700, Andres Freund wrote: > > I was about to propose adding headerscheck / cpluspluscheck to the CI file > > so > > that cfbot can catch future issues. > > The attached patch does so, with ICU disabled to avoid the pr

Re: [PATCH] Add native windows on arm64 support

2022-03-22 Thread Thomas Munro
On Tue, Mar 22, 2022 at 11:30 PM Julien Rouhaud wrote: > On Tue, Mar 22, 2022 at 09:37:46AM +, Niyas Sait wrote: > > Yes, we could look into providing a build machine. Do you have any > > reference to what the CI system looks like now for PostgresSQL and how to > > add new workers etc.? > > It

Re: Adding CI to our tree

2022-03-23 Thread Thomas Munro
On Thu, Mar 10, 2022 at 9:37 AM Justin Pryzby wrote: > -Og Adding this to CXXFLAGS caused a torrent of warnings from g++ about LLVM headers, which I also see on my local system for LLVM 11 and LLVM 14: [19:47:11.047] /usr/lib/llvm-11/include/llvm/ADT/Twine.h: In member function ‘llvm::CallInst*

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2022-03-23 Thread Thomas Munro
On Thu, Mar 24, 2022 at 9:59 AM Peter Geoghegan wrote: > On Wed, Mar 23, 2022 at 1:53 PM Robert Haas wrote: > > And therefore I favor defining it to mean that we don't > > skip any work at all. > > But even today DISABLE_PAGE_SKIPPING won't do pruning when we cannot > acquire a cleanup lock on a

Re: [PATCH] add relation and block-level filtering to pg_waldump

2022-03-23 Thread Thomas Munro
On Thu, Mar 24, 2022 at 9:53 AM Peter Eisentraut wrote: > On 21.03.22 05:55, Thomas Munro wrote: > > [04:30:50.630] pg_waldump.c:963:26: error: format ‘%u’ expects > > argument of type ‘unsigned int *’, but argument 3 has type ‘ForkNumber > > *’ [-Werror=format=] > &

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-03-23 Thread Thomas Munro
On Wed, Feb 16, 2022 at 12:11 PM Nathan Bossart wrote: > On Tue, Feb 15, 2022 at 10:37:58AM -0800, Nathan Bossart wrote: > > On Tue, Feb 15, 2022 at 10:10:34AM -0800, Andres Freund wrote: > >> I generally think it'd be a good exercise to go through an use > >> get_dirent_type() in nearly all ReadD

Checking pgwin32_is_junction() errors

2022-03-23 Thread Thomas Munro
to work around that problem, but I don't like it, and would need to come up with something better... Sketch code attached. From 8e0e51359c0191cf673c3ddc87a3262b13f530a4 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Wed, 16 Mar 2022 23:05:39 +1300 Subject: [PATCH] Raise errors in pgwin32_i

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Thomas Munro
On Mon, Nov 30, 2020 at 6:59 PM Thomas Munro wrote: > On Wed, Nov 25, 2020 at 8:00 AM Pavel Borisov wrote: > > The new status of this patch is: Ready for Committer > ... That's the version I plan to commit tomorrow, unless > there are further comments or objections.

Re: BUG #16663: DROP INDEX did not free up disk space: idle connection hold file marked as deleted

2020-11-30 Thread Thomas Munro
On Tue, Dec 1, 2020 at 3:55 PM Michael Paquier wrote: > On Mon, Nov 30, 2020 at 06:59:40PM +1300, Thomas Munro wrote: > > So I fixed that, by adding a return value to do_truncate() and > > checking it. That's the version I plan to commit tomorrow, unless > > the

PG vs LLVM 12 on seawasp, next round

2020-11-30 Thread Thomas Munro
Hi Since seawasp's bleeding edge LLVM installation moved to "trunk 20201114 c8f4e06b 12.0.0" ~16 days ago, it has been red. Further updates didn't help it and it's now on "trunk 20201127 6ee22ca6 12.0.0". I wonder if there is something in Fabien's scripting that needs to be tweaked, perhaps a sy

Re: Recent eelpout failures on 9.x branches

2020-12-01 Thread Thomas Munro
On Wed, Dec 2, 2020 at 11:36 AM Tom Lane wrote: > Perhaps this is just a question of the machine being too slow to complete > the test, in which case we ought to raise wal_sender_timeout. But it's > weird that it would've started to fail just now, because I don't really > see any changes in those

Re: Recent eelpout failures on 9.x branches

2020-12-01 Thread Thomas Munro
On Wed, Dec 2, 2020 at 12:07 PM Tom Lane wrote: > I'm also wondering a bit why the issue isn't affecting the newer > branches. It's certainly not because we made the test shorter ... I looked at htop while it was building the 9.x branches and saw pg_basebackup sitting in D state waiting for glac

Re: Commitfest 2020-11 is closed

2020-12-02 Thread Thomas Munro
On Thu, Dec 3, 2020 at 10:00 AM Tom Lane wrote: > This is actually a bit problematic, because now the cfbot is ignoring > those patches (or if it's not, I don't know where it's displaying the > results). Please go ahead and move the remaining open patches, or > else re-open the CF if that's possi

Re: Commitfest 2020-11 is closed

2020-12-02 Thread Thomas Munro
On Thu, Dec 3, 2020 at 10:51 AM David Steele wrote: > On 12/2/20 4:15 PM, Thomas Munro wrote: > > On Thu, Dec 3, 2020 at 10:00 AM Tom Lane wrote: > >> This is actually a bit problematic, because now the cfbot is ignoring > >> those patches (or if it's not, I don&

Github Actions (CI)

2020-12-02 Thread Thomas Munro
rhn8bH836OB%2B3SboiaeEcgOtrJS58Bki4%3D5yeVqToxgw%40mail.gmail.com From 424bcae7f1fcb1ada0e7046bfc5e0c5254c6f439 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Thu, 3 Dec 2020 10:58:16 +1300 Subject: [PATCH] Github CI WIP --- .github/workflows/ci-linux.yml | 43 +

Re: Commitfest 2020-11 is closed

2020-12-02 Thread Thomas Munro
On Thu, Dec 3, 2020 at 12:02 PM Josef Šimánek wrote: > st 2. 12. 2020 v 23:36 odesílatel Andrew Dunstan napsal: > > On 12/2/20 5:13 PM, Thomas Munro wrote: > > > I'm experimenting with Github's built in CI. All other ideas welcome. > > > > I'd lo

Re: SELECT INTO deprecation

2020-12-03 Thread Thomas Kellerer
r versions get to see that warning. Then target 15 or 16 as the release for removal, but not 14 Thomas

Re: Github Actions (CI)

2020-12-03 Thread Thomas Munro
On Fri, Dec 4, 2020 at 2:55 AM Josef Šimánek wrote: > Any chance to also share links to failing/passing testing builds? https://github.com/macdice/postgres/runs/1490727809 https://github.com/macdice/postgres/runs/1490727838 However, looking these in a clean/cookieless browser, I see that it won'

Re: Index Skip Scan (new UniqueKeys)

2020-12-05 Thread Thomas Munro
On Wed, Dec 2, 2020 at 9:59 AM Heikki Linnakangas wrote: > On 01/12/2020 22:21, Dmitry Dolgov wrote: > >> On Mon, Nov 30, 2020 at 04:42:20PM +0200, Heikki Linnakangas wrote: > >> - I'm surprised you need a new index AM function (amskip) for this. Can't > >> you just restart the scan with index_res

Re: Blocking I/O, async I/O and io_uring

2020-12-07 Thread Thomas Munro
On Tue, Dec 8, 2020 at 3:56 PM Craig Ringer wrote: > I thought I'd start the discussion on this and see where we can go with it. > What incremental steps can be done to move us toward parallelisable I/O > without having to redesign everything? > > I'm thinking that redo is probably a good first

Re: Commitfest 2020-11 is closed

2020-12-08 Thread Thomas Munro
On Fri, Dec 4, 2020 at 1:29 AM Peter Eisentraut wrote: > On 2020-12-02 23:13, Thomas Munro wrote: > > I'm experimenting with Github's built in CI. All other ideas welcome. > > You can run Linux builds on AppVeyor, too. Yeah. This would be the easiest change to mak

Re: Autovacuum worker doesn't immediately exit on postmaster death

2020-12-10 Thread Thomas Munro
On Fri, Dec 11, 2020 at 8:34 AM Robert Haas wrote: > On Thu, Oct 29, 2020 at 5:36 PM Alvaro Herrera > wrote: > > Maybe instead of thinking specifically in terms of vacuum, we could > > count buffer accesses (read from kernel) and check the latch once every > > 1000th such, or something like that

Re: Cache relation sizes?

2020-12-16 Thread Thomas Munro
Hi Andy, On Thu, Dec 17, 2020 at 7:29 PM Andy Fan wrote: > I spent one day studying the patch and I want to talk about one question for > now. > What is the purpose of calling smgrimmedsync to evict a DIRTY sr (what will > happen > if we remove it and the SR_SYNCING and SR_JUST_DIRTIED flags)?

pg_preadv() and pg_pwritev()

2020-12-19 Thread Thomas Munro
63cb0 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Thu, 26 Nov 2020 15:48:31 +1300 Subject: [PATCH 07/11] Add pg_preadv() and pg_pwritev(). Provide synchronous scatter/gather I/O routines. These map to preadv(), pwritev() with various fallbacks for systems that don't have them. -

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Thomas Munro
On Sun, Dec 20, 2020 at 12:34 PM Tom Lane wrote: > Thomas Munro writes: > > I want to be able to do synchronous vectored file I/O, so I made > > wrapper macros for preadv() and pwritev() with fallbacks for systems > > that don't have them. Following the prece

Re: pg_preadv() and pg_pwritev()

2020-12-20 Thread Thomas Munro
On Sun, Dec 20, 2020 at 8:07 PM Tom Lane wrote: > One minor thought is that in > > + struct iovec iov[Min(IOV_MAX, 1024)]; /* cap stack space */ > > it seems like pretty much every use of IOV_MAX would want some > similar cap. Should we centralize that idea with, say, > > #define

Re: pg_preadv() and pg_pwritev()

2020-12-21 Thread Thomas Munro
On Mon, Dec 21, 2020 at 8:25 PM Jakub Wartak wrote: > > > I'm drawing a blank on trivial candidate uses for preadv(), without > > > infrastructure from later patches. > > > > Can't immediately think of something either. > > This might be not that trivial , but maybe acquire_sample_rows() from > a

Re: pg_preadv() and pg_pwritev()

2020-12-22 Thread Thomas Munro
On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > On 2020-12-20 16:26:42 +1300, Thomas Munro wrote: > > > 1. port.h cannot assume that has already been included; > > > nor do I want to fix that by including there. Do we > > > really need to define a fal

Re: Cache relation sizes?

2020-12-23 Thread Thomas Munro
On Thu, Dec 17, 2020 at 10:22 PM Andy Fan wrote: > Let me try to understand your point. Suppose process 1 extends a file to > 2 blocks from 1 block, and fsync is not called, then a). the lseek *may* still > return 1 based on the comments in the ReadBuffer_common ("because > of buggy Linux kernels

Re: Re: Cache relation sizes?

2020-12-23 Thread Thomas Munro
On Wed, Dec 23, 2020 at 1:31 AM 陈佳昕(步真) wrote: > I studied your patch these days and found there might be a problem. > When execute 'drop database', the smgr shared pool will not be removed > because of no call 'smgr_drop_sr'. Function 'dropdb' in dbcommands.c remove > the buffer from bufferpool

Re: doc review for v14

2020-12-28 Thread Thomas Munro
that is managed automatically. > > The current formulation is not that confusing, but I agree that this > is an improvement. Thomas, you are behind this one. What do you > think? LGTM.

Re: Parallel Full Hash Join

2020-12-28 Thread Thomas Munro
n true if this worker is elected to do the unmatched inner scan + */ +bool +ExecParallelPrepHashTableForUnmatched(HashJoinState *hjstate) Comment name doesn't match function name. From 9199bfcfa84acbcfeb9a8d3c21962096c7ff645c Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Thu, 5 Nov 20

Re: Let's start using setenv()

2020-12-28 Thread Thomas Munro
On Tue, Dec 29, 2020 at 4:21 PM Tom Lane wrote: > Back in 2001 we decided to prefer putenv() over setenv() because the > latter wasn't in POSIX (SUSv2) at the time. That decision has been > overtaken by events: more recent editions of POSIX not only provide > setenv() but recommend it over putenv

Re: Let's start using setenv()

2020-12-29 Thread Thomas Munro
On Wed, Dec 30, 2020 at 5:45 AM Tom Lane wrote: > Since the cfbot seems happy with v1, here's a v2 that runs around > and converts all putenv() uses to setenv(). In some places there's > no notational savings, but it seems to me that standardizing > on just one way to do it is beneficial. +

Re: Cache relation sizes?

2020-12-29 Thread Thomas Munro
On Mon, Dec 28, 2020 at 5:24 PM Andy Fan wrote: > lseek(..., SEEK_END) = 9670656 > write(...) = 8192 > lseek(..., SEEK_END) = 9678848 > fsync(...) = -1 > lseek(..., SEEK_END) = 9670656 > > I got 2 information from above. a) before the fsync, the lseek(fd, 0, > SEEK_END) > returns a correct valu

Re: Re: Re: Cache relation sizes?

2020-12-29 Thread Thomas Munro
be better to increment sr->generation. That was already done in the "eviction" code path, where other processes might still have references to the SR object, and I don't think it's possible for anyone to access a dropped relation, but I suppose it's more consistent

Re: Re: Re: Cache relation sizes?

2020-12-29 Thread Thomas Munro
On Wed, Dec 30, 2020 at 5:52 PM Thomas Munro wrote: > and requires it on retry s/requires/reacquires/

Re: pgbench: option delaying queries till connections establishment?

2020-12-31 Thread Thomas Munro
On Sun, Nov 15, 2020 at 4:53 AM Fabien COELHO wrote: > In the attached version, I just comment out the call and add an > explanation about why it is commented out. If pg overall version > requirements are changed on windows, then it could be reinstated. It looks like macOS doesn't have pthread ba

Re: new heapcheck contrib module

2020-12-31 Thread Thomas Munro
On Tue, Oct 27, 2020 at 5:12 AM Mark Dilger wrote: > The v20 patches 0002, 0003, and 0005 still apply cleanly, but 0004 required a > rebase. (0001 was already committed last week.) > > Here is a rebased set of 4 patches, numbered 0002..0005 to be consistent with > the previous naming. There ar

Re: WIP: BRIN multi-range indexes

2021-01-01 Thread Thomas Munro
On Sun, Dec 20, 2020 at 1:16 PM Tomas Vondra wrote: > Attached is an updated version of the patch series, rebased on current > master, and results for benchmark comparing the various bloom variants. Perhaps src/include/utils/inet.h needs to include , because FreeBSD says: brin_minmax_multi.c:169

Re: psql \df choose functions by their arguments

2021-01-01 Thread Thomas Munro
On Thu, Dec 31, 2020 at 7:01 AM Greg Sabino Mullane wrote: > Attached is the latest patch against HEAD - basically fixes a few typos. Hi Greg, It looks like there is a collation dependency here that causes the test to fail on some systems: === ./src/test/regress/regression.diffs === diff -U3 /t

Re: Context diffs

2021-01-04 Thread Thomas Munro
On Tue, Jan 5, 2021 at 8:07 AM Bruce Momjian wrote: > * "git apply" and "git am" can't process context diffs (they throw an >error once a context-like section of the diff is hit; simple >adding/removing lines in a block works) > > * the commit-fest doesn't recognized context diff attachm

Re: Handing off SLRU fsyncs to the checkpointer

2021-01-04 Thread Thomas Munro
On Mon, Jan 4, 2021 at 3:35 AM Tomas Vondra wrote: > Seems this commit left behind a couple unnecessary prototypes in a bunch > of header files. In particular, it removed these functions > > - ShutdownCLOG(); > - ShutdownCommitTs(); > - ShutdownSUBTRANS(); > - ShutdownMultiXact(); Thanks. Fixed.

  1   2   3   4   5   6   7   8   9   10   >