pg_stat_wal: tracking the compression effect

2022-08-25 Thread Ken Kato
Hi hackers, We can specify compression method (for example, lz4, zstd), but it is hard to know the effect of compression depending on the method. There is already a way to know the compression effect using pg_waldump. However, having these statistics in the view makes it more accessible. I a

Re: pg_basebackup: add test about zstd compress option

2022-08-25 Thread Dong Wook Lee
On Tue, Aug 23, 2022 at 11:37 AM Michael Paquier wrote: > It seems to me that checking that the contents generated are valid is > equally necessary. We do that with zlib with gzip --test, and you > could use ${ZSTD} in the context of this test. Thank you for the good points. I supplemented the

Re: add checkpoint stats of snapshot and mapping files of pg_logical dir

2022-08-25 Thread Kyotaro Horiguchi
At Wed, 24 Aug 2022 13:34:54 +0900, Michael Paquier wrote in > On Wed, Aug 24, 2022 at 10:48:01AM +0900, Kyotaro Horiguchi wrote: > > By the way, I think we use INSTR_TIME_* macros to do masure internal > > durations (mainly for the monotonic clock characteristics, and to > > reduce performance

Re: Avoid unecessary MemSet call (src/backend/utils/cache/relcache.c)

2022-08-25 Thread Alvaro Herrera
On 2022-Aug-24, Peter Eisentraut wrote: > I don't follow how this is a backpatching hazard. It changes code. Any bugfix in the surrounding code would have to fix a conflict. That is nonzero effort. Is it a huge risk? No, it is very small risk and a very small cost to fix such a conflict; but

Re: Letter case of "admin option"

2022-08-25 Thread Alvaro Herrera
On 2022-Aug-25, Kyotaro Horiguchi wrote: > At Tue, 23 Aug 2022 09:58:47 -0400, Robert Haas wrote > in > I would translate "ADMIN OPTION" to "ADMIN OPTION" in Japanese but > "admin option" is translated to "管理者オプション" which is a bit hard > for the readers to come up with the connection to "ADMIN

Re: pg_receivewal and SIGTERM

2022-08-25 Thread Daniel Gustafsson
> On 23 Aug 2022, at 02:15, Michael Paquier wrote: > > On Mon, Aug 22, 2022 at 04:05:16PM +0200, Christoph Berg wrote: >> Do you mean it can, or can not be backpatched? (I'd argue for >> backpatching since the behaviour is slightly broken at the moment.) > > I mean that it is fine to backpatch t

Re: Column Filtering in Logical Replication

2022-08-25 Thread vignesh C
On Tue, Aug 23, 2022 at 7:52 AM Peter Smith wrote: > > On Mon, Aug 22, 2022 at 9:25 PM vignesh C wrote: > > > ... > > > Few comments: > > 1) I felt no expressions are allowed in case of column filters. Only > > column names can be specified. The second part of the sentence > > confuses what is al

Re: use SSE2 for is_valid_ascii

2022-08-25 Thread John Naylor
v3 applies on top of the v9 json_lex_string patch in [1] and adds a bit more to that, resulting in a simpler patch that is more amenable to additional SIMD-capable platforms. [1] https://www.postgresql.org/message-id/CAFBsxsFV4v802idV0-Bo%3DV7wLMHRbOZ4er0hgposhyGCikmVGA%40mail.gmail.com -- John

Re: Making Vars outer-join aware

2022-08-25 Thread Richard Guo
On Thu, Aug 25, 2022 at 5:18 AM Tom Lane wrote: > Richard Guo writes: > > Do we need to also > > generate two SpecialJoinInfos for the B/C join in the first order, with > > and without the A/B join in its min_lefthand? > > No, the SpecialJoinInfos would stay as they are now. It's already the >

Insertion Sort Improvements

2022-08-25 Thread Benjamin Coutu
Hello, Inspired by the recent discussions[1][2] around sort improvements, I took a look around the code and noticed the use of a somewhat naive version of insertion sort within the broader quicksort code. The current implementation (see sort_template.h) is practically the textbook version of i

Re: pg_receivewal and SIGTERM

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 11:19:05AM +0200, Daniel Gustafsson wrote: > I think this can be argued both for and against backpatching. Catching > SIGTERM > makes a lot of sense, especially given systemd's behavior. On the other hand, > This adds functionality to something arguably working as intende

Re: Avoid unecessary MemSet call (src/backend/utils/cache/relcache.c)

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 10:38:41AM +0200, Alvaro Herrera wrote: > It changes code. Any bugfix in the surrounding code would have to fix a > conflict. That is nonzero effort. Is it a huge risk? No, it is very > small risk and a very small cost to fix such a conflict; but my claim is > that this

Re: Perform streaming logical transactions by background workers and parallel apply

2022-08-25 Thread Amit Kapila
On Wed, Aug 24, 2022 at 7:17 PM houzj.f...@fujitsu.com wrote: > > On Friday, August 19, 2022 4:49 PM Amit Kapila > > > > > 8. It is not clear to me how APPLY_BGWORKER_EXIT status is used. Is it > > required > > for the cases where bgworker exists due to some error and then apply worker > > uses

Re: making relfilenodes 56 bits

2022-08-25 Thread Dilip Kumar
On Tue, Aug 23, 2022 at 8:00 PM Robert Haas wrote: > > On Tue, Aug 23, 2022 at 2:06 AM Dilip Kumar wrote: > > OTOH, if we keep the two separate ranges for the user and system table > > then we don't need all this complex logic of conflict checking. > > True. That's the downside. The question is w

Re: Avoid erroring out when unable to remove or parse logical rewrite files to save checkpoint work

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 10:48:08AM +0530, Bharath Rupireddy wrote: > On Tue, Aug 23, 2022 at 11:37 PM Nathan Bossart > wrote: >> IIUC an error in get_dirent_type() could cause slots to be skipped here, >> which is a behavior change. > > That behaviour hasn't changed, no? Currently, if lstat() fa

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Peter Eisentraut
On 25.08.22 02:14, Andrew Dunstan wrote: In short: if the Cygwin Perl headers redefine PGDLLEXPORT (unlikely) or somehow #define "__attribute__()" or "visibility()" into no-ops (perhaps more likely) then we could explain this failure, and that would also explain why it doesn't fail elsewhere. I

Re: [RFC] building postgres with meson - v11

2022-08-25 Thread Peter Eisentraut
On 24.08.22 17:30, Andres Freund wrote: 258f6dc0a7 Don't hardcode tmp_check/ as test directory for tap tests 8ecc33cf04 Split TESTDIR into TESTLOGDIR and TESTDATADIR I think these patches are split up a bit incorrectly. If you apply the first patch by itself, then the output appears in tab_comp

Re: archive modules

2022-08-25 Thread talk to ben
Here is a patch with the proposed wording. From 7fce0073f8a53b3e9ba84fa10fbc7b8efef36e97 Mon Sep 17 00:00:00 2001 From: benoit Date: Mon, 22 Aug 2022 12:00:46 +0200 Subject: [PATCH] basic_archive parameter visibility doc patch Module parameters are only visible from the pg_settings view once the

postgres_fdw hint messages

2022-08-25 Thread Peter Eisentraut
The postgres_fdw tests contain this (as amended by patch 0001): ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw'); ERROR: invalid option "password" HINT: Valid options in this context are: service, passfile, channel_binding, connect_timeout, dbname, host, hostaddr, port, options, ap

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Tom Lane
Peter Eisentraut writes: >>> In short: if the Cygwin Perl headers redefine PGDLLEXPORT (unlikely) >>> or somehow #define "__attribute__()" or "visibility()" into no-ops >>> (perhaps more likely) then we could explain this failure, and that >>> would also explain why it doesn't fail elsewhere. > T

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Robert Haas
On Wed, Aug 24, 2022 at 10:31 PM Tom Lane wrote: > git blame blames that whole mechanism on me: 60cfe25e68d. It looks > like the reason I did it like that is that I was replacing use of > system(3) with execl(), and system(3) is defined thus by POSIX: > > execl(, "sh", "-c", command, (cha

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Tom Lane
Robert Haas writes: > But what we've got is: > [rhaas pgsql]$ git grep execl\( > src/bin/pg_ctl/pg_ctl.c: (void) execl("/bin/sh", "/bin/sh", "-c", cmd, > (char *) NULL); > src/test/regress/pg_regress.c: execl(shellprog, shellprog, "-c", > cmdline2, (char *) NULL); Right. I wouldn't really

Re: replacing role-level NOINHERIT with a grant-level option

2022-08-25 Thread Robert Haas
On Wed, Aug 24, 2022 at 10:23 AM tushar wrote: > On 8/24/22 12:28 AM, Robert Haas wrote: > > This patch needed to be rebased pretty extensively after commit > > ce6b672e4455820a0348214be0da1a024c3f619f. Here is a new version. > Thanks, Robert, I have retested this patch with my previous scenarios

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 10:13 AM Tom Lane wrote: > My point is that that *isn't* what POSIX specifies. They say in so > many words that the path actually used by system(3) is unspecified. > They do NOT say that it's the value of $SHELL, and given that you're > allowed to set $SHELL to a non-POSIX

Re: Letter case of "admin option"

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 4:58 AM Alvaro Herrera wrote: > I ran a quick poll in a Spanish community. Everyone who responded (not > many admittedly) agreed with this idea -- they find the message clearer > if the keyword is mentioned explicitly in the translation. Makes sense. I didn't really doubt

libpq error message refactoring

2022-08-25 Thread Peter Eisentraut
libpq now contains a mix of error message strings that end with newlines and don't end with newlines, due to some newer code paths with new ways of passing errors around. This has now gotten me confused a few too many times both during development and translation. So I looked into whether we

Re: pg_basebackup: add test about zstd compress option

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 3:52 AM Dong Wook Lee wrote: > Could you check if I missed anything? There is already a far better test for this in src/bin/pg_verifybackup/t/009_extract.pl -- Robert Haas EDB: http://www.enterprisedb.com

[PATCH] Fix alter subscription concurrency errors

2022-08-25 Thread Jelte Fennema
Without this patch concurrent ALTER/DROP SUBSCRIPTION statements for the same subscription could result in one of these statements returning the following error: ERROR: XX000: tuple concurrently updated This patch fixes that by re-fetching the tuple after acquiring the lock on the subscription.

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Tom Lane
Robert Haas writes: > I mean, I can see you're on the warpath here and I don't care enough > to fight about it very much, but as a matter of theory, I believe that > hard-coded pathnames suck. Giving the user a way to survive if /bin/sh > doesn't exist on their system or isn't the path they want t

Re: shadow variables - pg15 edition

2022-08-25 Thread David Rowley
On Thu, 25 Aug 2022 at 14:08, Justin Pryzby wrote: > Here, I've included the rest of your list. OK, I've gone through v3-remove-var-declarations.txt, v4-reuse.txt v4-reuse-more.txt and committed most of what you had and removed a few that I thought should be renames instead. I also added some ad

Re: shadow variables - pg15 edition

2022-08-25 Thread David Rowley
On Thu, 25 Aug 2022 at 13:46, David Rowley wrote: > I've attached a patch which I think improves the code in > gistRelocateBuildBuffersOnSplit() so that there's no longer a shadowed > variable. I also benchmarked this method in a tight loop and can > measure no performance change from getting the

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 10:48 AM Tom Lane wrote: > If we were executing a program that the user needs to have some control > over, sure, but what we have here is an implementation detail that I > doubt anyone cares about. The fact that we're using a shell at all is > only because nobody has cared

Re: pg_receivewal and SIGTERM

2022-08-25 Thread Christoph Berg
Re: Michael Paquier > FWIW, I've worked on an archiver integration a few years ago and got > annoyed that we use SIGINT while SIGTERM was the default (systemd was > not directly used there but the signal problem was the same, so we had > to go through some loops to make the stop signal configurable

has_privs_of_role vs. is_member_of_role, redux

2022-08-25 Thread Robert Haas
Hi, We've had some previous discussions about when to use has_privs_of_role and when to use is_member_of_role, and has_privs_of_role has mostly won the fight. That means that, if role "robert" is set to NOINHERIT and you "GRANT stuff TO robert", for the most part "robert" will not actually be able

re: postgres_fdw hint messages

2022-08-25 Thread Ranier Vilela
>The postgres_fdw tests contain this (as amended by patch 0001): >ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw'); >ERROR: invalid option "password" >HINT: Valid options in this context are: service, passfile, >channel_binding, connect_timeout, dbname, host, hostaddr, port, options, >a

Re: Schema variables - new implementation for Postgres 15

2022-08-25 Thread Pavel Stehule
st 24. 8. 2022 v 10:04 odesílatel Erik Rijkers napsal: > Op 24-08-2022 om 08:37 schreef Pavel Stehule: > >> > > > > I fixed these. > > > > > [v20220824-1-*.patch] > > Hi Pavel, > > I noticed just now that variable assignment (i.e., LET) unexpectedly > (for me anyway) cast the type of the input v

V14 and later build the backend with -lpthread

2022-08-25 Thread Tom Lane
I realized $SUBJECT while wondering why my new buildfarm animal chickadee (NetBSD on gaur's old hardware) fails the plpython tests on v13 and earlier. After a bit of investigation I realized it *should* be failing, because neither NetBSD nor Python have done anything about the problem documented i

Re: postgres_fdw hint messages

2022-08-25 Thread Ranier Vilela
Em qui., 25 de ago. de 2022 às 14:31, Ranier Vilela escreveu: > >The postgres_fdw tests contain this (as amended by patch 0001): > > >ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw'); > >ERROR: invalid option "password" > >HINT: Valid options in this context are: service, passfile, > >

Re: pg_receivewal and SIGTERM

2022-08-25 Thread Magnus Hagander
On Thu, Aug 25, 2022 at 5:13 PM Christoph Berg wrote: > > Re: Michael Paquier > > FWIW, I've worked on an archiver integration a few years ago and got > > annoyed that we use SIGINT while SIGTERM was the default (systemd was > > not directly used there but the signal problem was the same, so we ha

Re: has_privs_of_role vs. is_member_of_role, redux

2022-08-25 Thread Joe Conway
On 8/25/22 12:12, Robert Haas wrote: So far, so good. But it's clearly not the case that "GRANT stuff TO robert" has conferred no privileges at all on robert. At the very least, it's enabled him to "SET ROLE stuff", but what else? I decided to go through the code and make a list of the things tha

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2022-08-25 Thread Andres Freund
Hi, On 2022-08-22 13:15:18 -0400, Melanie Plageman wrote: > v28 attached. > > I've added the new structs I added to typedefs.list. > > I've split the commit which adds all of the logic to track > IO operation statistics into two commits -- one which includes all of > the code to count IOOps for

Re: postgres_fdw hint messages

2022-08-25 Thread Ibrar Ahmed
On Thu, Aug 25, 2022 at 6:42 PM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > The postgres_fdw tests contain this (as amended by patch 0001): > > ALTER SERVER loopback_nopw OPTIONS (ADD password 'dummypw'); > ERROR: invalid option "password" > HINT: Valid options in this context

Re: Cleaning up historical portability baggage

2022-08-25 Thread Andres Freund
Hi, On 2022-08-18 18:13:38 +1200, Thomas Munro wrote: > Here's a slightly better AF_INET6 one. I'm planning to push it > tomorrow if there are no objections. You didn't yet, I think. Any chance you could? The HAVE_IPV6 stuff is wrong/ugly in the meson build, right now, and I'd rather not spend t

Re: pg_regress: lookup shellprog in $PATH

2022-08-25 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 25, 2022 at 10:48 AM Tom Lane wrote: >> If we were executing a program that the user needs to have some control >> over, sure, but what we have here is an implementation detail that I >> doubt anyone cares about. The fact that we're using a shell at all is >> on

Re: archive modules

2022-08-25 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 03:29:41PM +0200, talk to ben wrote: > Here is a patch with the proposed wording. Here is the same patch with a couple more links. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 92a6d8669d9e5b527a7ac9af7eb359a86526775b Mon Sep 17 00:00:00 2001 From: b

Re: has_privs_of_role vs. is_member_of_role, redux

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 3:03 PM Joe Conway wrote: > Nice analysis, and surprising (to me) Thanks. > > I argue that #3 is a clear bug. robert can't select from stuff's > > tables or change privileges on stuff's objects, so why can he change > > stuff's default privileges? is_member_of_role() has

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Robert Haas
On Thu, Aug 25, 2022 at 1:41 PM Tom Lane wrote: > I realized $SUBJECT while wondering why my new buildfarm animal chickadee > (NetBSD on gaur's old hardware) fails the plpython tests on v13 and > earlier. After a bit of investigation I realized it *should* be failing, > because neither NetBSD nor

Re: has_privs_of_role vs. is_member_of_role, redux

2022-08-25 Thread Tom Lane
Robert Haas writes: > I really hate back-patching this kind of change but it's possible that > it's the right thing to do. There's no real security exposure because > the member could always SET ROLE and then do the exact same thing, so > back-patching feels to me like it has a significantly highe

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Thomas Munro
On Fri, Aug 26, 2022 at 8:40 AM Robert Haas wrote: > On Thu, Aug 25, 2022 at 1:41 PM Tom Lane wrote: > > I realized $SUBJECT while wondering why my new buildfarm animal chickadee > > (NetBSD on gaur's old hardware) fails the plpython tests on v13 and > > earlier. After a bit of investigation I r

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Tom Lane
Robert Haas writes: > On Thu, Aug 25, 2022 at 1:41 PM Tom Lane wrote: >> ... On the third hand, poking at backends with ldd shows that at >> least on Linux, we've been linking the backend with -lpthread for >> quite some time, back to 9.4 or so. The new-in-v14 behavior is that >> it's getting in

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Tom Lane
Thomas Munro writes: > I suspect we will end up linked against the threading library anyway > in real-world builds via --with-XXX (I see that --with-icu has that > effect on my FreeBSD system, but I know that details about threading > are quite different in NetBSD). I may lack imagination but I'm

PostgreSQL 15 Beta 4

2022-08-25 Thread Jonathan S. Katz
Hi, We will be releasing a PostgreSQL 15 Beta 4 on September 8, 2022. Please have open items[1] completed and committed no later than September 5, 2022 0:00 AoE[2]. Thanks, Jonathan [1] https://wiki.postgresql.org/wiki/PostgreSQL_15_Open_Items [2] https://en.wikipedia.org/wiki/Anywhere_on_E

New strategies for freezing, advancing relfrozenxid early

2022-08-25 Thread Peter Geoghegan
Attached patch series is a completely overhauled version of earlier work on freezing. Related work from the Postgres 15 cycle became commits 0b018fab, f3c15cbe, and 44fa8488. Recap = The main high level goal of this work is to avoid painful, disruptive antiwraparound autovacuums (and other ag

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Tom Lane
I wrote: > (I'm also still struggling to explain why mamba, with the *exact* > same NetBSD code on a different hardware platform, isn't showing > the same failures as chickadee. More news if I figure that out.) Hah: I left --with-libxml out of chickadee's configuration, because libxml2 seemed to

Re: V14 and later build the backend with -lpthread

2022-08-25 Thread Andres Freund
Hi, On 2022-08-25 17:04:37 -0400, Tom Lane wrote: > (I'm also still struggling to explain why mamba, with the *exact* > same NetBSD code on a different hardware platform, isn't showing > the same failures as chickadee. More news if I figure that out.) I'd guess it's because of the different depe

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Andrew Dunstan
On 2022-08-25 Th 09:43, Tom Lane wrote: > Peter Eisentraut writes: In short: if the Cygwin Perl headers redefine PGDLLEXPORT (unlikely) or somehow #define "__attribute__()" or "visibility()" into no-ops (perhaps more likely) then we could explain this failure, and that would

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Andres Freund
Hi, On 2022-08-25 17:39:35 -0400, Andrew Dunstan wrote: > On 2022-08-25 Th 09:43, Tom Lane wrote: > > Peter Eisentraut writes: > In short: if the Cygwin Perl headers redefine PGDLLEXPORT (unlikely) > or somehow #define "__attribute__()" or "visibility()" into no-ops > (perhaps more

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Andrew Dunstan
On 2022-08-25 Th 17:47, Andres Freund wrote: > Hi, > > On 2022-08-25 17:39:35 -0400, Andrew Dunstan wrote: >> On 2022-08-25 Th 09:43, Tom Lane wrote: >>> Peter Eisentraut writes: >> In short: if the Cygwin Perl headers redefine PGDLLEXPORT (unlikely) >> or somehow #define "__attribute__(

Re: Strip -mmacosx-version-min options from plperl build

2022-08-25 Thread Andres Freund
Hi, On 2022-08-25 18:04:34 -0400, Andrew Dunstan wrote: > On 2022-08-25 Th 17:47, Andres Freund wrote: > >> $ egrep '_PG_init|Pg_magic_func'  plperl.i > >> extern __attribute__((visibility("default"))) void _PG_init(void); > >> extern __attribute__((visibility("default"))) const Pg_magic_struct >

Re: Cleaning up historical portability baggage

2022-08-25 Thread Thomas Munro
On Fri, Aug 26, 2022 at 7:47 AM Andres Freund wrote: > On 2022-08-18 18:13:38 +1200, Thomas Munro wrote: > > Here's a slightly better AF_INET6 one. I'm planning to push it > > tomorrow if there are no objections. > > You didn't yet, I think. Any chance you could? The HAVE_IPV6 stuff is > wrong/ug

Re: New strategies for freezing, advancing relfrozenxid early

2022-08-25 Thread Peter Geoghegan
On Thu, Aug 25, 2022 at 3:35 PM Jeremy Schneider wrote: > We should be careful here. IIUC, the current autovac behavior helps > bound the "spread" or range of active multixact IDs in the system, which > directly determines the number of distinct pages that contain those > multixacts. If the propos

Re: New strategies for freezing, advancing relfrozenxid early

2022-08-25 Thread Peter Geoghegan
On Thu, Aug 25, 2022 at 4:23 PM Peter Geoghegan wrote: > As a general rule VACUUM will tend to do more eager freezing with the > patch set compared to HEAD, though it should never do less eager > freezing. Not even in corner cases -- never. Come to think of it, I don't think that that's quite tru

Re: pg_receivewal and SIGTERM

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 08:45:05PM +0200, Magnus Hagander wrote: > I'm leaning towards considering it a feature-change and thus not > something to backpatch (I'd be OK sneaking it into 15 though, as that > one is not released yet and it feels like a perfectly *safe* change). > Not enough to insist

Re: SYSTEM_USER reserved word implementation

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 08:21:05PM +0200, Drouvot, Bertrand wrote: > system_user() now returns a text and I moved it to miscinit.c in the new > version attached (I think it makes more sense now). +/* kluge to avoid including libpq/libpq-be.h here */ +struct ClientConnectionInfo; +extern void Initi

Fix japanese translation of log messages

2022-08-25 Thread Shinya Kato
Hi hackers, I've found typos in ja.po, and fixed them. The patch is attached. -- Regards, -- Shinya Kato Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATIONdiff --git a/src/backend/po/ja.po b/src/backend/po/ja.po index 0925465d22..189000792c 100644 --

Re: [PATCH] Add native windows on arm64 support

2022-08-25 Thread Andres Freund
Hi, On 2022-05-10 10:01:55 +0900, Michael Paquier wrote: > On Mon, May 09, 2022 at 12:44:22PM +0100, Niyas Sait wrote: > > Microsoft updated documentation [1] and clarified that ASLR cannot be > > disabled for Arm64 targets. > > > > Because ASLR can't be disabled on ARM, ARM64, or ARM64EC archite

Re: Column Filtering in Logical Replication

2022-08-25 Thread Peter Smith
On Thu, Aug 25, 2022 at 7:38 PM vignesh C wrote: > ... > > PSA the v3* patch set. > > Thanks for the updated patch. > Few comments: > 1) We can shuffle the columns in publisher table and subscriber to > show that the order of the column does not matter > + > +Create a publication p1. A colu

Re: [PATCH] Add native windows on arm64 support

2022-08-25 Thread Michael Paquier
On Thu, Aug 25, 2022 at 06:29:07PM -0700, Andres Freund wrote: > I accidentally did a lot of testing with DYNAMICBASE - I accidentally > mistranslated MSBuildProject.pm's false to adding > /DYNAMICBASE in the meson port. Survived several hundred CI cycles and dozens > of local runs without observin

Re: [PATCH] Fix alter subscription concurrency errors

2022-08-25 Thread Amit Kapila
On Thu, Aug 25, 2022 at 8:17 PM Jelte Fennema wrote: > > Without this patch concurrent ALTER/DROP SUBSCRIPTION statements for > the same subscription could result in one of these statements returning the > following error: > > ERROR: XX000: tuple concurrently updated > > This patch fixes that by

Re: pg_stat_wal: tracking the compression effect

2022-08-25 Thread Kyotaro Horiguchi
At Thu, 25 Aug 2022 16:04:50 +0900, Ken Kato wrote in > Accumulating the values, which indicates how much space is saved by > each compression (size before compression - size after compression), > and keep track of how many times compression has happened. So that one > can know how much space is

Re: pg_stat_wal: tracking the compression effect

2022-08-25 Thread Kyotaro Horiguchi
At Fri, 26 Aug 2022 11:55:27 +0900 (JST), Kyotaro Horiguchi wrote in > At Thu, 25 Aug 2022 16:04:50 +0900, Ken Kato wrote > in > > Accumulating the values, which indicates how much space is saved by > > each compression (size before compression - size after compression), > > and keep track of

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-25 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 01:35:45PM +0700, John Naylor wrote: > - For the following comment, pgindent will put spaced operands on a > separate line which is not great for readability. and our other > reference to the Stanford bithacks page keeps the in-page link, and I > see no reason to exclude it

Re: [PoC] Reducing planning time when tables have many partitions

2022-08-25 Thread David Rowley
On Fri, 26 Aug 2022 at 12:40, Yuya Watari wrote: > This performance degradation is due to the heavy processing of the > get_ec***_indexes***() functions. These functions are the core part of > the optimization we are working on in this thread, but they are > relatively heavy when the number of par

Re: use SSE2 for is_valid_ascii

2022-08-25 Thread Nathan Bossart
On Thu, Aug 25, 2022 at 04:41:53PM +0700, John Naylor wrote: > v3 applies on top of the v9 json_lex_string patch in [1] and adds a > bit more to that, resulting in a simpler patch that is more amenable > to additional SIMD-capable platforms. LGTM -- Nathan Bossart Amazon Web Services: https://aw

Re: [PATCH] Add native windows on arm64 support

2022-08-25 Thread Andres Freund
Hi, On 2022-08-26 11:09:41 +0900, Michael Paquier wrote: > On Thu, Aug 25, 2022 at 06:29:07PM -0700, Andres Freund wrote: > > I accidentally did a lot of testing with DYNAMICBASE - I accidentally > > mistranslated MSBuildProject.pm's false to adding > > /DYNAMICBASE in the meson port. Survived sev

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-25 Thread John Naylor
On Thu, Aug 25, 2022 at 11:57 AM Nathan Bossart wrote: > > On Thu, Aug 25, 2022 at 10:38:34AM +0700, John Naylor wrote: > > On Thu, Aug 25, 2022 at 1:01 AM Nathan Bossart > > wrote: > >> On Wed, Aug 24, 2022 at 11:07:03AM +0700, John Naylor wrote: > >> > - Can a user on ARM64 ever get a runtime

Re: [PATCH] Optimize json_lex_string by batching character copying

2022-08-25 Thread John Naylor
On Fri, Aug 26, 2022 at 10:14 AM Nathan Bossart wrote: > > > +test_lfind8_internal(uint8 key) [...] > > + elog(ERROR, "pg_lfind8() found nonexistent element <= > > '0x%x'", key + 1); > > +} > > nitpick: Shouldn't the elog() calls use "==" instead of "<=" for this one? Good catch, wil

Re: Perform streaming logical transactions by background workers and parallel apply

2022-08-25 Thread Dilip Kumar
On Thu, Aug 11, 2022 at 12:13 PM Amit Kapila wrote: > > Since we will later consider applying non-streamed transactions in > > parallel, I > > think "apply streaming worker" might not be very suitable. I think > > PostgreSQL > > also has the worker "parallel worker", so for "apply parallel worke

Re: Handle infinite recursion in logical replication setup

2022-08-25 Thread Dilip Kumar
On Mon, Aug 22, 2022 at 9:19 AM houzj.f...@fujitsu.com wrote: > > > Jonathan, Sawada-San, Hou-San, and others, what do you think is the best way > > to move forward here? > > I think it's fine to throw a WARNING in this case given that there is a > chance of inconsistent data. IMHO, since the use

Small cleanups to tuplesort.c and a bonus small performance improvement

2022-08-25 Thread David Rowley
Hi, Since doing some work for PG15 for speeding up sorts, I've been a little irritated by the fact that dumptuples() calls WRITETUP, (which is now always calling writetuple()) and calls pfree() on the tuple only for dumptuples() to do MemoryContextReset(state->base.tuplecontext) directly afterward

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-25 Thread Nathan Bossart
On Fri, Aug 26, 2022 at 10:45:10AM +0700, John Naylor wrote: > On Thu, Aug 25, 2022 at 11:57 AM Nathan Bossart > wrote: >> The ARM literature appears to indicate that Neon support is pretty standard >> on aarch64, and AFAICT it's pretty common to just assume it's available. > > This doesn't exact

wal_sync_method=fsync_writethrough

2022-08-25 Thread Thomas Munro
Hi, We allow $SUBJECT on Windows. I'm not sure exactly how we finished up with that, maybe a historical mistake, but I find it misleading today. Modern Windows flushes drive write caches for fsync (= _commit()) and fdatasync (= FLUSH_FLAGS_FILE_DATA_SYNC_ONLY). In fact it is possible to tell Win

Re: Fix japanese translation of log messages

2022-08-25 Thread Kyotaro Horiguchi
At Fri, 26 Aug 2022 10:23:01 +0900, Shinya Kato wrote in > I've found typos in ja.po, and fixed them. > The patch is attached. (This is not for -hackers but I'm fine with it being posted here;p) Thanks for the report! Pushed to 10 to 15 of translation repository with some minor changes. They

Re: pg_stat_wal: tracking the compression effect

2022-08-25 Thread Andrey Borodin
> On 25 Aug 2022, at 12:04, Ken Kato wrote: > > What do you think? I think users will need to choose between Lz4 and Zstd. So they need to know tradeoff - compression ratio vs cpu time spend per page(or any other segment). I know that Zstd must be kind of "better", but doubt it have enough

Re: Reducing the chunk header sizes on all memory context types

2022-08-25 Thread David Rowley
On Tue, 23 Aug 2022 at 21:03, David Rowley wrote: > Finally, the v5 patch with the fixes mentioned above. The CFbot just alerted me to the cplusplus check was failing with the v5 patch, so here's v6. > I'm pretty keen to get this patch committed early next week. This is > quite core infrastruct

Re: Extensible storage manager API - smgr hooks

2022-08-25 Thread Andrey Borodin
> On 16 Jun 2022, at 13:41, Kirill Reshke wrote: > > Hello Yura and Anastasia. FWIW this technology is now a part of Greenplum [0]. We are building GP extension that automatically offloads cold data to S3 - a very simplified version of Neon for analytical workloads. When a segment of a tabl

Re: Fix japanese translation of log messages

2022-08-25 Thread torikoshia
On 2022-08-26 10:23, Shinya Kato wrote: Hi hackers, I've found typos in ja.po, and fixed them. The patch is attached. Thanks for the patch! LGTM. I had found a similar typo before in ja.po, so I added that as well. @@ -12739,7 +12739,7 @@ msgstr "ロールオプション\"%s\"が認識できません" #: gram.y:1588 gra

Re: Perform streaming logical transactions by background workers and parallel apply

2022-08-25 Thread Amit Kapila
On Fri, Aug 26, 2022 at 9:30 AM Dilip Kumar wrote: > > On Thu, Aug 11, 2022 at 12:13 PM Amit Kapila wrote: > > > Since we will later consider applying non-streamed transactions in > > > parallel, I > > > think "apply streaming worker" might not be very suitable. I think > > > PostgreSQL > > > a

Re: use ARM intrinsics in pg_lfind32() where available

2022-08-25 Thread Nathan Bossart
Here is a new patch set that applies on top of v9-0001 in the json_lex_string patch set [0] and v3 of the is_valid_ascii patch [1]. [0] https://postgr.es/m/CAFBsxsFV4v802idV0-Bo%3DV7wLMHRbOZ4er0hgposhyGCikmVGA%40mail.gmail.com [1] https://postgr.es/m/CAFBsxsFFAZ6acUfyUALiem4DpCW%3DApXbF02zrc0G0o

HOT chain validation in verify_heapam()

2022-08-25 Thread Himanshu Upadhyaya
Hi, On Mon, Apr 4, 2022 at 11:46 PM Andres Freund wrote: > > I think there's a few more things that'd be good to check. For example > amcheck > doesn't verify that HOT chains are reasonable, which can often be spotted > looking at an individual page. Which is a bit unfortunate, given how many >

Re: Fix japanese translation of log messages

2022-08-25 Thread Kyotaro Horiguchi
At Fri, 26 Aug 2022 14:28:26 +0900, torikoshia wrote in > " > > #: gram.y:1588 gram.y:1604 > > #, c-format > > msgid "CREATE SCHEMA IF NOT EXISTS cannot include schema elements" > > -msgstr "CREATE SCHEMA IF NOT EXISTSんはスキーマ要素を含めることはでき > > -ません" > > +msgstr "CREATE SCHEMA IF NOT EXISTSはスキーマ要

Re: Fix japanese translation of log messages

2022-08-25 Thread torikoshia
On 2022-08-26 15:20, Kyotaro Horiguchi wrote: At Fri, 26 Aug 2022 14:28:26 +0900, torikoshia wrote in > #: gram.y:1588 gram.y:1604 > #, c-format > msgid "CREATE SCHEMA IF NOT EXISTS cannot include schema elements" > -msgstr "CREATE SCHEMA IF NOT EXISTSんはスキーマ要素を含めることはでき > -ません" > +msgstr "CRE