Re: Non-text mode for pg_dumpall

2025-01-10 Thread Mahendra Singh Thalor
On Sat, 11 Jan 2025 at 11:19, jian he wrote: > Thanks Jian for the review and testing. > in src/bin/pg_dump/pg_dumpall.c main > i think you need do > > archDumpFormat = parseDumpFormat(formatName); > /* > * Open the output file if required, otherwise use stdout. If required, >

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread David Rowley
On Sat, 11 Jan 2025 at 12:32, Tom Lane wrote: > > David Rowley writes: > > hmm, I didn't think of that scenario. I think since > > verify_compact_attribute() does nothing when USE_ASSERT_CHECKING isn't > > defined that we might as well define a ((void) 0) macro to avoid the > > undefined symbol

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Tom Lane
David Rowley writes: > On Sat, 11 Jan 2025 at 12:32, Tom Lane wrote: >> No, this completely fails to address the problem. The concern is >> that the extension has been compiled under USE_ASSERT_CHECKING, >> so it will try to call the function. If the function's not there >> in core, kaboom. >

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread David Rowley
On Sat, 11 Jan 2025 at 12:56, Andrew Kane wrote: > I've updated the patch to make verify_compact_attribute a no-op. > > The extension sets USE_ASSERT_CHECKING, which is why the macro approach > doesn't work (it won't take that path). > > Also, it looks like it fails when creating the extension /

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Andrew Kane
Thank you both!

Re: Proposal: add new API to stringinfo

2025-01-10 Thread Tatsuo Ishii
> No objections here. V4 patch pushed. Thanks. -- Tatsuo Ishii SRA OSS K.K. English: http://www.sraoss.co.jp/index_en/ Japanese:http://www.sraoss.co.jp

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread David Rowley
On Sat, 11 Jan 2025 at 11:27, Andrew Kane wrote: > Prior to 6f3820f, extensions could be compiled with -DUSE_ASSERT_CHECKING > whether or not the Postgres installation was configured with --enable-cassert > (to enable at least some assertion checking). However, after 6f3820f, linking > fails wi

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-10 Thread Melanie Plageman
On Fri, Jan 10, 2025 at 6:00 PM Daniel Gustafsson wrote: > > > On 10 Jan 2025, at 23:09, Melanie Plageman > > wrote: > > > > On Thu, Jan 9, 2025 at 5:05 PM Daniel Gustafsson wrote: > >> > >> I think this is a really good restructuring which will make life easier > >> for our > >> users. Some

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Tom Lane
David Rowley writes: > hmm, I didn't think of that scenario. I think since > verify_compact_attribute() does nothing when USE_ASSERT_CHECKING isn't > defined that we might as well define a ((void) 0) macro to avoid the > undefined symbol error. That'll avoid the useless call in your debug > build

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-10 Thread Melanie Plageman
On Fri, Jan 10, 2025 at 11:18 AM Tomas Vondra wrote: > > On 1/10/25 15:54, Melanie Plageman wrote: > > On Thu, Jan 9, 2025 at 6:59 PM Tomas Vondra wrote: > > I think this is because we get the batch based on > > > > *batchno = pg_rotate_right32(hashvalue, hashtable->log2_nbuckets) & > > (nbatch -

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Tom Lane
Andrew Kane writes: > I've updated the patch to make verify_compact_attribute a no-op. > The extension sets USE_ASSERT_CHECKING, which is why the macro approach > doesn't work (it won't take that path). LGTM > Also, it looks like it fails when creating the extension / loading the > shared librar

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Andrew Kane
I've updated the patch to make verify_compact_attribute a no-op. The extension sets USE_ASSERT_CHECKING, which is why the macro approach doesn't work (it won't take that path). Also, it looks like it fails when creating the extension / loading the shared library (on Ubuntu), not when linking (as

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread David Rowley
On Sat, 11 Jan 2025 at 12:48, Tom Lane wrote: > > David Rowley writes: > > On Sat, 11 Jan 2025 at 12:32, Tom Lane wrote: > >> No, this completely fails to address the problem. The concern is > >> that the extension has been compiled under USE_ASSERT_CHECKING, > >> so it will try to call the fun

Re: Memory leak in plpython3u (with testcase and patch)

2025-01-10 Thread Tom Lane
Mat Arye writes: > I found a memory leak in plpython3u. It happens when converting the > argument for an spi plan execution (via plpy.execute() or similar). I've > attached a sql file to create two plpython3u functions to reproduce the > issue (memory_leak_test.sql). I see the leak all right, and

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-10 Thread Melanie Plageman
On Thu, Jan 9, 2025 at 5:05 PM Daniel Gustafsson wrote: > > I think this is a really good restructuring which will make life easier for > our > users. Some of the comments below are on wording which wasn't introduced in > this patch, but which I hadn't thought about before, so feel free to ignor

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-10 Thread Abhishek Chanda
Thanks, Daniel. Here is an updated patch with all previous changes, added a simple connection test and another check to make sure file mode is correct, and the env var fix. Please let me know if anything needs to be changed. I tested this locally using meson running all TAP tests, and also manually

Re: Non-text mode for pg_dumpall

2025-01-10 Thread jian he
in src/bin/pg_dump/pg_dumpall.c main i think you need do archDumpFormat = parseDumpFormat(formatName); /* * Open the output file if required, otherwise use stdout. If required, * then create new files with global.dat and map.dat names. */ if (archDumpFormat != archNull)

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-10 Thread Masahiko Sawada
On Thu, Jan 9, 2025 at 3:29 AM Ashutosh Bapat wrote: > > On Tue, Dec 31, 2024 at 10:15 AM Masahiko Sawada > wrote: > > > > Hi all, > > > > Logical decoding (and logical replication) are available only when > > wal_level = logical. As the documentation says[1], Using the 'logical' > > level incre

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-10 Thread Ryo Kanbayashi
On Thu, Jan 9, 2025 at 9:27 PM Ryo Kanbayashi wrote: > > > On 2025/01/09 20:34, Ryo Kanbayashi wrote: > > > Dear Tom, Fujii-san, Kuroda-san, > > > > > > I saw comments of yours and recognized that better fix is below. > > > > > > - Fix of first attached patch which does not change warning message

Re: Adding a '--two-phase' option to 'pg_createsubscriber' utility.

2025-01-10 Thread Ajin Cherian
On Fri, Dec 27, 2024 at 5:36 PM Shubham Khanna wrote: > The patch no longer applies on HEAD. Please do rebase. regards, Ajin Cherian Fujitsu Australia

Re: IWYU annotations

2025-01-10 Thread Peter Eisentraut
On 02.01.25 17:15, Tom Lane wrote: It's a fair point that some documentation could be provided. I suppose we don't want to verbosely explain each pragma individually. Should there be some central explanation, maybe in src/tools/pginclude/README? That might do, but perhaps instead in the "Post

Re: Conflict detection for update_deleted in logical replication

2025-01-10 Thread Nisha Moond
On Wed, Jan 8, 2025 at 3:02 PM Masahiko Sawada wrote: > > On Thu, Dec 19, 2024 at 11:11 PM Nisha Moond wrote: > > [3] Test with pgbench run on both publisher and subscriber. > > > > Test setup: > > - Tests performed on pgHead + v16 patches > > - Created a pub-sub replication system. > > - Paramet

Re: Incorrect CHUNKHDRSZ in nodeAgg.c

2025-01-10 Thread David Rowley
On Thu, 9 Jan 2025 at 09:50, Jeff Davis wrote: > Attached POC patch, which reduces memory usage by ~15% for a simple > distinct query on an integer key. Performance is the same or perhaps a > hair faster. > > It's not many lines of code, but the surrounding code might benefit > from some refactori

Re: AIO v2.0

2025-01-10 Thread Ants Aasma
On Thu, 9 Jan 2025 at 22:53, Andres Freund wrote: > Workstation w/ 2x Xeon Gold 6442Y: > >march memresult > native 100246.13766ms @ 33.282 GB/s > native 10456.08080ms @ 17.962 GB

RE: Windows meson build

2025-01-10 Thread Kohei Harikae (Fujitsu)
Hi, Thank you for your advice. Based on your advice, I have added a description of the vcpkg and -Dextra_include_dirs options. How do you think about this? Regards, Kohei Harikae installation-meson.patch Description: installation-meson.patch

Re: Some ExecSeqScan optimizations

2025-01-10 Thread Vladlen Popolitov
Amit Langote писал(а) 2025-01-10 16:22: On Fri, Jan 10, 2025 at 1:06 PM David Rowley wrote: On Fri, 10 Jan 2025 at 02:46, Amit Langote wrote: > > On Mon, Jan 6, 2025 at 10:18 PM David Rowley wrote: > > I've attached my workings of what I was messing around with. It seems > > to perform about

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-10 Thread Antonin Houska
Pavel Stehule wrote: > Hi > > čt 9. 1. 2025 v 14:35 odesílatel Alvaro Herrera > napsal: > > On 2024-Dec-11, Antonin Houska wrote: > > > Oh, it was too messy. I think I was thinking of too many things at once > (such > > as locking the old heap, the new heap and the new heap's TOAST). Als

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

2025-01-10 Thread Masahiko Sawada
Hi, On Tue, Jan 7, 2025 at 11:30 PM Peter Smith wrote: > > Hi Sawada-San. > > FWIW, I also thought it was a good idea suggested by Bertrand [1] to > "hide" everything behind the slot create/delete, and thereby eliminate > the need for user intervention using those new > pg_activate/deactivate_log

Re: per backend WAL statistics

2025-01-10 Thread Bertrand Drouvot
Hi, Michael Paquier wrote: > An extra thing I have finished by doing is removing > PgStat_BackendPendingIO, then applied the change. It was useful when > returned as a result of pgstat_prep_backend_pending(), but not so much > with the new PgStat_BackendPending that includes all the pending stats

RE: Conflict detection for update_deleted in logical replication

2025-01-10 Thread Zhijie Hou (Fujitsu)
On Friday, January 10, 2025 8:43 AM Masahiko Sawada wrote: Hi, > > On Wed, Jan 8, 2025 at 7:26 PM Zhijie Hou (Fujitsu) > wrote: > > > > On Thursday, January 9, 2025 9:48 AM Masahiko Sawada > wrote: > > > > Hi, > > > > > > > > On Wed, Jan 8, 2025 at 3:00 AM Zhijie Hou (Fujitsu) > > > > wrote

Re: Potential null pointer dereference in postgres.c

2025-01-10 Thread Maxim Orlov
I have to admit I was wrong with previous v2 patch. Sorry. Apparently, the chances of committing this very low, but here is the correct one. -- Best regards, Maxim Orlov. v3-0001-Use-pstrdup-for-remote_host-and-remote_port-save-.patch Description: Binary data

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-10 Thread chiranmoy.bhattacha...@fujitsu.com
Hello Nathan, We tried auto-vectorization and observed no performance improvement. The instructions in src/include/port/simd.h are based on older SIMD architectures like NEON, whereas the patch uses the newer SVE, so some of the instructions used in the patch may not have direct equivalents in N

Re: why there is not VACUUM FULL CONCURRENTLY?

2025-01-10 Thread Marcos Pegoraro
Em sex., 10 de jan. de 2025 às 06:31, Antonin Houska escreveu: > Thus I understand Alvaro's objections against VACUUM (FULL, CONCURRENTLY). > > Therefore I can imagine adding a new command that acts like VACUUM (FULL, > CONCURRENTLY), but does not try to be CLUSTER (CONCURRENTL). > If VACUUM F

Re: [PATCH] SVE popcount support

2025-01-10 Thread chiranmoy.bhattacha...@fujitsu.com
Hi all, Here is the updated patch using pg_attribute_target("arch=armv8-a+sve") to compile the arch-specific function instead of using compiler flags. --- Chiranmoy v3-0001-SVE-support-for-popcount-and-popcount-masked.patch Description: v3-0001-SVE-support-for-popcount-and-popcount-masked.p

Re: Moving the vacuum GUCs' docs out of the Client Connection Defaults section

2025-01-10 Thread Daniel Gustafsson
> On 10 Jan 2025, at 23:09, Melanie Plageman wrote: > > On Thu, Jan 9, 2025 at 5:05 PM Daniel Gustafsson wrote: >> >> I think this is a really good restructuring which will make life easier for >> our >> users. Some of the comments below are on wording which wasn't introduced in >> this patch

Re: Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread David Rowley
On Sat, 11 Jan 2025 at 13:05, David Rowley wrote: > > On Sat, 11 Jan 2025 at 12:56, Andrew Kane wrote: > > I've updated the patch to make verify_compact_attribute a no-op. > > > > The extension sets USE_ASSERT_CHECKING, which is why the macro approach > > doesn't work (it won't take that path).

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
Thanks for the review! > === 1 > > + endtime = GetCurrentTimestamp(); > pgstat_report_vacuum(RelationGetRelid(rel), > rel->rd_rel->relisshared, > Max(vacrel->new_live_tuples, > 0), >

Restore support for USE_ASSERT_CHECKING in extensions only

2025-01-10 Thread Andrew Kane
Prior to 6f3820f, extensions could be compiled with -DUSE_ASSERT_CHECKING whether or not the Postgres installation was configured with --enable-cassert (to enable at least some assertion checking). However, after 6f3820f, linking fails with `undefined symbol: verify_compact_attribute`. I'm not sure

Re: Memory leak in plpython3u (with testcase and patch)

2025-01-10 Thread Tom Lane
I wrote: > However, I don't really see why we need to use that scratch context. > PLy_spi_execute_plan runs a subtransaction, so we could perfectly well > use the subtransaction's CurTransactionContext. Nah, I'm wrong about that: I forgot CurTransactionContext goes away on subtransaction abort, bu

Re: psql: Add leakproof field to \dAo+ meta-command results

2025-01-10 Thread Dean Rasheed
On Wed, 4 Dec 2024 at 11:21, Yugo NAGATA wrote: > > > Looking through the complete list of psql meta-commands, "leakproof" > > could plausibly be added to the output of each of the following: > > > > \dAo+ > > \dC+ > > \df+ > > \do+ > > I've attached a updated patch (v3-0001) that include changes

Re: Some ExecSeqScan optimizations

2025-01-10 Thread Amit Langote
On Fri, Jan 10, 2025 at 7:36 PM David Rowley wrote: > On Fri, 10 Jan 2025 at 22:53, Vladlen Popolitov > wrote: > > In case of query > > select count(*) from test_table where a_1 = 100; > > I would expect increase of query time due to additional if...else . It > > is not clear > > what code

Re: per backend I/O statistics

2025-01-10 Thread Bertrand Drouvot
Hi, On Thu, Dec 19, 2024 at 06:12:04AM +, Bertrand Drouvot wrote: > I think I'll start a dedicated thread to discuss the > stats_fetch_consistency/'snapshot' > point (will be easier to follow than resuming the discussion in this thread). I gave more thoughts on it and did some research in th

tests for pg_stat_progress_copy.tuples_skipped

2025-01-10 Thread jian he
hi. seems no regress tests for pg_stat_progress_copy.tuples_skipped. CopyFrom pgstat_progress_update_param(PROGRESS_COPY_TUPLES_SKIPPED, cstate->num_errors); there is coverage for it. but in regress test, we didn't really print out this value (cstate->num_errors) Th

Re: Vacuum statistics

2025-01-10 Thread Alena Rybakina
Hi! I thought about this problem again and I think I have a solution. On 02.12.2024 23:12, Alena Rybakina wrote: * I've read the previous discussion on how important to keep all these fields regarding vacuum statistics including points by Andrei and Jim. It still worrying me that statistics volu

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-10 Thread Melanie Plageman
On Thu, Jan 9, 2025 at 6:59 PM Tomas Vondra wrote: > > > > On 1/9/25 21:42, Melanie Plageman wrote: > > > > I was excited about your raw file experiment. As Robert and you point > > out -- we may need a file per batch, but for most of the hash join's > > execution we don't need to keep buffers for

Re: Adding extension default version to \dx

2025-01-10 Thread Michael Banck
Hi, On Fri, Jan 10, 2025 at 01:04:14PM +0100, Magnus Hagander wrote: > In an effort to make at least a couple of more people realize they have to > run ALTER EXTENSION UPDATE after they've upgraded an extension, as well as > make it a bit easier to realize when you have to do it, I propose we add

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Tom Lane
Fujii Masao writes: > Before this commit, pgbench used pqsignal() from port/pqsignal.c > to set the signal handler for SIGALRM. This version of pqsignal() > sets SA_RESTART for frontend code, so fgets() in runShellCommand() > wouldn't return NULL even if SIGALRM arrived during fgets(), > preventin

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Tom Lane
Nazir Bilal Yavuz writes: > I ran it ~500 times on HEAD but the issue did not occur on my machine. What platform are you testing on? regards, tom lane

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Fujii Masao
On 2025/01/10 21:41, Fujii Masao wrote: On 2025/01/10 16:09, Andy Fan wrote: Andy Fan writes: Hi: I run into the {subject} issue with the below setup. cat foo.sql \setshell txn_mode echo ${TXN_MODE} \setshell speed echo ${SPEED} \setshell sleep_ms echo ${SLEEP_MS} \setshell subtxn_mod

Re: Adding extension default version to \dx

2025-01-10 Thread Julien Rouhaud
Hi, On Fri, Jan 10, 2025 at 01:04:14PM +0100, Magnus Hagander wrote: > In an effort to make at least a couple of more people realize they have to > run ALTER EXTENSION UPDATE after they've upgraded an extension, as well as > make it a bit easier to realize when you have to do it, I propose we add

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Fujii Masao
On 2025/01/10 16:09, Andy Fan wrote: Andy Fan writes: Hi: I run into the {subject} issue with the below setup. cat foo.sql \setshell txn_mode echo ${TXN_MODE} \setshell speed echo ${SPEED} \setshell sleep_ms echo ${SLEEP_MS} \setshell subtxn_mode echo ${SUBTXN_MODE} select 1; $ TXN_MOD

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Yura Sokolov
09.01.2025 19:03, Zhou, Zhiguo пишет: On 1/7/2025 10:49 AM, Юрий Соколов wrote: On 6 Jan 2025, at 09:46, Zhou, Zhiguo wrote: Hi Yura and Wenhui, Thanks for kindly reviewing this work! On 1/3/2025 9:01 PM, wenhui qiu wrote: Hi Thank you for your path,NUM_XLOGINSERT_LOCKS increase to 128,I

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 Jan 2025 at 10:10, Andy Fan wrote: > > Andy Fan writes: > > > Hi: > > > > I run into the {subject} issue with the below setup. > > > > cat foo.sql > > > > \setshell txn_mode echo ${TXN_MODE} > > \setshell speed echo ${SPEED} > > \setshell sleep_ms echo ${SLEEP_MS} > > \setshell su

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Alena Rybakina
Hi! On 04.01.2025 06:41, wenhui qiu wrote: Hi Sami  Thank you for your path,it seems some path monitor vacuum status,Can we synthesize their good ideas together。 I've been working on a patch that collects vacuum statistics since May [0]. It includes heap and index relation vacuum statistics,

Re: Some ExecSeqScan optimizations

2025-01-10 Thread Vladlen Popolitov
Amit Langote писал(а) 2025-01-10 18:22: On Fri, Jan 10, 2025 at 7:36 PM David Rowley wrote: On Fri, 10 Jan 2025 at 22:53, Vladlen Popolitov wrote: > In case of query > select count(*) from test_table where a_1 = 100; > I would expect increase of query time due to additional if...else . I

Re: [PoC] Federated Authn/z with OAUTHBEARER

2025-01-10 Thread Peter Eisentraut
On 09.01.25 20:18, Jacob Champion wrote: It'd be sad to copy-paste the API bug into a new place, though. If we're going to disconnect this API from SOCKET, can we use uintptr_t instead on Windows? If someone eventually adds an alternative to PQsocket(), as Tom suggested in [2], it'd be nice not t

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Bertrand Drouvot
Hi, On Thu, Jan 02, 2025 at 12:24:06PM -0600, Sami Imseih wrote: > Having the total (auto)vacuum elapsed time > along side the existing (auto)vaccum_count > allows a user to track the average time an > operating overtime and to find vacuum tuning > opportunities. > > The same can also be said for

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Nazir Bilal Yavuz
Hi, On Fri, 10 Jan 2025 at 17:58, Tom Lane wrote: > > Nazir Bilal Yavuz writes: > > I ran it ~500 times on HEAD but the issue did not occur on my machine. > > What platform are you testing on? My local machine is: Linux 6.12.8-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.12.8-1 (2025-01-02) x86_64 GNU

Re: pgbench error: (setshell) of script 0; execution of meta-command failed

2025-01-10 Thread Tom Lane
I wrote: > Since legacy-pqsignal is really not supposed to be used by clients > anymore, maybe we could just adjust it to set SA_RESTART for SIGALRM? > The other alternatives I can think of amount to re-introducing > link order dependencies, which would be horrid. Actually, after re-reading the th

Adding extension default version to \dx

2025-01-10 Thread Magnus Hagander
In an effort to make at least a couple of more people realize they have to run ALTER EXTENSION UPDATE after they've upgraded an extension, as well as make it a bit easier to realize when you have to do it, I propose we add the default version of an extension to \dx in psql. We currently show the i

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-10 Thread Nathan Bossart
On Fri, Jan 10, 2025 at 11:10:03AM +, chiranmoy.bhattacha...@fujitsu.com wrote: > We tried auto-vectorization and observed no performance improvement. Do you mean that the auto-vectorization worked and you observed no performance improvement, or the auto-vectorization had no effect on the cod

Re: Adding support for SSLKEYLOGFILE in the frontend

2025-01-10 Thread Daniel Gustafsson
> On 10 Jan 2025, at 04:59, Abhishek Chanda wrote: Thanks for the new version. + {"sslkeylogfile", "SSLKEYLOGFILE", The env var should be PGSSLKEYLOGFILE with the PG prefix. > 3. Added docs and tests There is no test in the attached patch, did you write one but forgot to git add it befor

Re: Vacuum statistics

2025-01-10 Thread Alena Rybakina
Sorry, I made a typo due to lack of sleep, I've marked below where exactly just in case. On 10.01.2025 15:04, Alena Rybakina wrote: Hi, I have updated the patch. Fix minor mistakes in the document, added the wraparound_failsafe_count statistics - it accounts the number of times when the vacu

Re: Adjusting hash join memory limit to handle batch explosion

2025-01-10 Thread Tomas Vondra
On 1/10/25 15:54, Melanie Plageman wrote: > On Thu, Jan 9, 2025 at 6:59 PM Tomas Vondra wrote: >> >> >> >> On 1/9/25 21:42, Melanie Plageman wrote: >>> >>> I was excited about your raw file experiment. As Robert and you point >>> out -- we may need a file per batch, but for most of the hash joi

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
> > this information gives a complete picture of vacuum efficiency, because > > analyzing only total_time statistics does not give a complete picture of > > what vacuum did: was it cleaning almost huge index, cleaning tables or > > just sleeping. > > The purpose of total_time is to be able to calcu

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2025-01-10 Thread Michail Nikolaev
Hello, Matthias! Updated patches attached. Changes: * cleanup test logic a little bit * resolve issue with rescan in GIST (item->blkno == InvalidBlockNumber) * move test to the main isolation suite * add test for SpGist * update comment I mentioned before * allow GIST to set LP_DEAD in cases it i

Re: POC: track vacuum/analyze cumulative time per relation

2025-01-10 Thread Sami Imseih
> this information gives a complete picture of vacuum efficiency, because > analyzing only total_time statistics does not give a complete picture of > what vacuum did: was it cleaning almost huge index, cleaning tables or > just sleeping. The purpose of total_time is to be able to calculate the av

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-10 Thread Nathan Bossart
On Wed, Jan 08, 2025 at 04:38:02PM -0600, Nathan Bossart wrote: > On Wed, Jan 08, 2025 at 05:25:24PM -0500, Andres Freund wrote: >> I wonder if you're hitting an extreme case of binary-layout related effects? >> I've never seen them at this magnitude though. I'd suggest using either lld >> or mold

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Matthias van de Meent
On Fri, 10 Jan 2025 at 13:42, Yura Sokolov wrote: > > BTW, your version could make alike trick for guaranteed atomicity: > - change XLogRecord's `XLogRecPtr xl_prev` to `uint32 xl_prev_offset` > and store offset to prev record's start. -1, I don't think that is possible without degrading what our

Re: pg_dump, pg_dumpall, pg_restore: Add --no-policies option

2025-01-10 Thread Greg Sabino Mullane
Looks good to me. Would ideally like to see some tests: should be easy enough to add to t/002_pg_dump.pl, but probably not worth it just for a simple flag like this? We don't test a lot of other flags, but on the other hand, that's what a test suite is supposed to do. Cheers, Greg

Proposal: "query_work_mem" GUC, to distribute working memory to the query's individual operators

2025-01-10 Thread James Hunter
I want customers to be able to run large OLAP queries on PostgreSQL, using as much memory as possible, to avoid spilling — without running out of memory. There are other ways to run out of memory, but the fastest and easiest way, on an OLAP query, is to use a lot of work_mem. (This is true for any

Re: IANA timezone abbreviations versus timezone_abbreviations

2025-01-10 Thread Tom Lane
I wrote: > Hmph ... the cfbot doesn't like that one bit. It appears that the > "LMT" abbrev is known in my machine's system-provided tzdata tree, > but not when building our own timezone files. I wonder if this is > exposing an existing bug (perhaps our copy of zic is too far out of > date?). Wi

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-10 Thread Andres Freund
Hi, On 2025-01-09 16:50:45 +0300, Nazir Bilal Yavuz wrote: > On Wed, 8 Jan 2025 at 22:26, Andres Freund wrote: > === 0005 > > > I think this is much better than before. I don't love PM_WAIT_XLOG_ARCHIVAL, > > but I can't think of anything better. > > I liked this, I think it is good enough. >

Memory leak in plpython3u (with testcase and patch)

2025-01-10 Thread Mat Arye
Hi all, I found a memory leak in plpython3u. It happens when converting the argument for an spi plan execution (via plpy.execute() or similar). I've attached a sql file to create two plpython3u functions to reproduce the issue (memory_leak_test.sql). When running ``` set client_min_messages = 'de

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-10 Thread Andres Freund
Hi, Thanks for the reviews! On 2025-01-09 12:01:05 +, Bertrand Drouvot wrote: > On Wed, Jan 08, 2025 at 02:26:15PM -0500, Andres Freund wrote: > === 2 > > + PM_WAIT_XLOG_ARCHIVAL, /* waiting for archiver and walsenders to > > > I don't love PM_WAIT_XLOG_ARCHIVAL, but I can't think

Re: [RFC] Lock-free XLog Reservation from WAL

2025-01-10 Thread Yura Sokolov
10.01.2025 19:53, Matthias van de Meent пишет: On Fri, 10 Jan 2025 at 13:42, Yura Sokolov wrote: BTW, your version could make alike trick for guaranteed atomicity: - change XLogRecord's `XLogRecPtr xl_prev` to `uint32 xl_prev_offset` and store offset to prev record's start. -1, I don't think

Re: pg_dump, pg_dumpall, pg_restore: Add --no-policies option

2025-01-10 Thread Nikolay Samokhvalov
Thank you Greg. 002_pg_dump.pl already deals with CREATE POLICY and ALTER TABLE .. ENABLE ROW LEVEL SECURITY, so I just added "--no-policies" there, to have basic coverage. Nik On Fri, Jan 10, 2025 at 9:44 AM Greg Sabino Mullane wrote: > Looks good to me. Would ideally like to see some tests:

Re: Reorder shutdown sequence, to flush pgstats later

2025-01-10 Thread Andres Freund
Hi, I've pushed 0001-0005. I think 0006 isn't far from ready. However, I think 0007 needs a bit more work. One thing that worries me a bit is that using SIGINT for triggering the shutdown checkpoint could somehow be unintentionally triggered? Previously checkpointer would effectively have ignor

Re: use a non-locking initial test in TAS_SPIN on AArch64

2025-01-10 Thread Nathan Bossart
On Fri, Jan 10, 2025 at 10:51:40AM -0600, Nathan Bossart wrote: > Actually, I think I may have just had back luck and/or not warmed things up > enough. I just re-ran the test a few dozen times, carefully ensuring the > data was in the cache and periodically alternating between the binary with > th

Re: Possible integer overflow in bringetbitmap()

2025-01-10 Thread James Hunter
On Fri, Dec 20, 2024 at 3:22 PM Michael Paquier wrote: > > On Tue, Dec 10, 2024 at 12:33:08PM +0900, Michael Paquier wrote: > > Sure, you could do (a) and (b) together. It also seems to me that it > > is just simpler to make totalpages a int64 to map automatically with > > the result expected by

Re: Proposal: add new API to stringinfo

2025-01-10 Thread Nathan Bossart
On Fri, Jan 10, 2025 at 11:31:34AM +0900, Tatsuo Ishii wrote: > If there's no objection, I am going to commit the patch. No objections here. -- nathan

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2025-01-10 Thread Bertrand Drouvot
Hi, On Fri, Jan 10, 2025 at 10:15:52AM +0300, Nazir Bilal Yavuz wrote: > Hi, > > On Fri, 10 Jan 2025 at 04:47, Michael Paquier wrote: > > > > On Thu, Jan 09, 2025 at 03:30:37PM +, Bertrand Drouvot wrote: > > > We first use an Assert in is_ioop_tracked_in_bytes() and then we return > > > a va

Re: Some ExecSeqScan optimizations

2025-01-10 Thread Amit Langote
On Fri, Jan 10, 2025 at 1:06 PM David Rowley wrote: > On Fri, 10 Jan 2025 at 02:46, Amit Langote wrote: > > > > On Mon, Jan 6, 2025 at 10:18 PM David Rowley wrote: > > > I've attached my workings of what I was messing around with. It seems > > > to perform about the same as your version. I think

Re: ecpg command does not warn COPY ... FROM STDIN;

2025-01-10 Thread Ryo Kanbayashi
On Fri, Jan 10, 2025 at 5:45 PM Ryo Kanbayashi wrote: > > On Thu, Jan 9, 2025 at 9:27 PM Ryo Kanbayashi > wrote: > > > > > On 2025/01/09 20:34, Ryo Kanbayashi wrote: > > > > Dear Tom, Fujii-san, Kuroda-san, > > > > > > > > I saw comments of yours and recognized that better fix is below. > > > >

Re: Some ExecSeqScan optimizations

2025-01-10 Thread David Rowley
On Fri, 10 Jan 2025 at 22:53, Vladlen Popolitov wrote: > In case of query > select count(*) from test_table where a_1 = 100; > I would expect increase of query time due to additional if...else . It > is not clear > what code was eliminated to decrease query time. Are you talking about the c

Re: Some ExecSeqScan optimizations

2025-01-10 Thread David Rowley
On Fri, 10 Jan 2025 at 22:22, Amit Langote wrote: > > On Fri, Jan 10, 2025 at 1:06 PM David Rowley wrote: > > I can look if performance tests show > > that it might be worthwhile considering. > > Sure, that would be great. What I wanted to know was if 0002 shows any additional gains over just 00

Re: Add Pipelining support in psql

2025-01-10 Thread Anthonin Bonnefoy
> I feel there's a large overlap between \flush and \flushrequest. On > the other hand, if people want to test the behaviour of pushing data > with and without a flush request message, then \flush can be useful. I've been looking into some issues related to a backend process stuck in ClientWrite s

Re: SQL function which allows to distinguish a server being in point in time recovery mode and an ordinary replica

2025-01-10 Thread m . litsarev
Hi! Michael, sorry for such a long time to deliver next version of the patch from my side. In this version I have fixed all your propositions, hopefully correct. There is one point that I would like to emphasize, namely Shouldn't STANDBY_MODE be only used in the local flag, as well as an ARCH

Re: [PATCH] Hex-coding optimizations using SVE on ARM.

2025-01-10 Thread Nathan Bossart
On Fri, Jan 10, 2025 at 09:38:14AM -0600, Nathan Bossart wrote: > On Fri, Jan 10, 2025 at 11:10:03AM +, chiranmoy.bhattacha...@fujitsu.com > wrote: >> We tried auto-vectorization and observed no performance improvement. > > Do you mean that the auto-vectorization worked and you observed no >