Re: Sort functions with specialized comparators

On Sat, Dec 21, 2024 at 12:16 AM Andrey M. Borodin wrote: > > > > > On 16 Dec 2024, at 14:02, John Naylor wrote: > > > > Sorry, I forgot this part earlier. Yes, let's have the private function. > > PFA v6. v6-0001: +static int +unique_cmp(const void *a, const void *b) +{ + int32 aval = *((const

Re: Proposal: add new API to stringinfo

>> With opinions from Michael and David , what about following additional >> APIs? >> >> #define STRINGINFO_DEFAULT_SIZE 1024 /* default initial allocation size >> #*/ >> #define STRINGINFO_SMALL_SIZE 64 /* small initial allocation size */ >> >> #define makeStringInfo makeStringInfoExtended(STRINGI

Re: FileFallocate misbehaving on XFS

Hi Andres On Wed, 1 Jan 2025 at 02:31, Andres Freund wrote: > Note that there's > a) a few hours between messages, whereas previous they were more frequent > b) f_bfree increased substantially. > > I assume that somewhere around 2AM some script prunes old partitions? Correct. Data is imported co

Re: Logical Replication of sequences

On Fri, 3 Jan 2025 at 09:07, Peter Smith wrote: > > Hi Vignesh, > > Some minor review comments for the patch v20241230-0003. > > == > src/backend/replication/logical/syncutils.c > > 1. > + * syncutils.c > + * PostgreSQL logical replication: common synchronization code > + * > + * Copyright (

Re: POC: track vacuum/analyze cumulative time per relation

Hi Sami Thank you for your path，it seems some path monitor vacuum status，Can we synthesize their good ideas together。 On Fri, 3 Jan 2025 at 02:24, Sami Imseih wrote: > Hi, > > After a recent question regarding tracking vacuum start_time in > pg_stat_all_tables [1], it dawned on me that this vie

Re: Incorrect CHUNKHDRSZ in nodeAgg.c

On Thu, Jan 2, 2025 at 7:24 AM David Rowley wrote: > > Bump wouldn't work due to the SH_FREE() in SH_GROW() when resizing the > table. If sizeof(TupleHashEntryData) were a power-of-two, then there'd > be no wastage as the hash table always has a power-of-two bucket count > and two powers-of-two mu

Re: Revisiting {CREATE INDEX, REINDEX} CONCURRENTLY improvements

On Wed, 1 Jan 2025 at 17:17, Michail Nikolaev wrote: > > Hello, everyone! > > I’ve added several updates to the patch set: > > * Automatic auxiliary index removal where applicable. > * Documentation updates to reflect recent changes. > * Optimization for STIR indexes: skipping datum setup, as they

Re: Fwd: Re: A new look at old NFS readdir() problems?

Thomas Munro writes: > I doubt that hides all potential problems though, if I have understood > the vague outline of this bug correctly: perhaps if you ran large > enough rm -r, and you unlinked a file concurrently with that loop, you > could break it, that is, cause it to skip innocent files othe

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

On Tue, 3 Dec 2024 at 17:21, Peter Geoghegan wrote: > > On Mon, Dec 2, 2024 at 8:18 PM Peter Geoghegan wrote: > > Attached is a refined version of a test case I posted earlier on [2], > > decisively proving that GiST index-only scans are in fact subtly > > broken. Right now it fails, showing a wr

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

On Fri, Jan 3, 2025 at 6:31 AM Euler Taveira wrote: > > On Fri, Jan 3, 2025, at 10:14 AM, Bertrand Drouvot wrote: > > If we don't want to force wal_level = logical to enable logical decoding (your > second idea) then I think that it would be better to "hide" everything behind > logical replication

Re: Modern SHA2- based password hashes for pgcrypto

On Fri, 03 Jan 2025 at 17:55, Bernd Helmle wrote: > Am Freitag, dem 03.01.2025 um 23:57 +0800 schrieb Japin Li: >> >> Greate!  I have some questions after using it. >> >> When I use the following query, it crashed! >> >> [local]:4012204 postgres=# select crypt('hello', >> '$5$$6$rounds=1$/Z

Re: Fwd: Re: A new look at old NFS readdir() problems?

On Sat, Jan 4, 2025 at 7:12 AM Tom Lane wrote: > One thing I noticed while testing yesterday is that "rm -rf foo" > worked even in cases where "rmtree foo" didn't. I did not look > into FreeBSD's rm code, but I'll bet it has the sort of retry > logic that was recommended to us upthread. It punts

Re: Proposal: add new API to stringinfo

On Fri, Dec 27, 2024 at 9:46 PM Tatsuo Ishii wrote: > > > With opinions from Michael and David , what about following additional APIs? ... > #define makeStringInfo > makeStringInfoExtended(STRINGINFO_DEFAULT_SIZE) Minor nit: the definition of the macro should contain parentheses, like s

Re: Proposal: add new API to stringinfo

On 2024-12-28 Sa 12:45 AM, Tatsuo Ishii wrote: Michael said: New APIs are materials for HEAD, so recompilation needs to happen anyway. Using a macro makes things slightly simpler and it would not break unnecessarily the compilation of existing extensions. Ok. David said: I didn't review th

Re: Add the ability to limit the amount of memory that can be allocated to backends.

On Thu, Jan 2, 2025 at 7:21 PM Tomas Vondra wrote: > > I'm not opposed to having a some sort of "workload management" (similar > to what's available in some databases), but my guess is that's (at > least) an order of magnitude more complex than introducing the memory > limit discussed here. I can

Re: Proposal to Enable/Disable Index using ALTER INDEX (with patch)

+ This is the + default state for newly created indexes. This is not needed in the ALTER INDEX docs, IMO.ss + + Disable the specified index. A disabled index is not used for queries, but it + is still updated when the underlying table data changes and will still be +

Re: pg_attribute_noreturn(), MSVC, C11

Peter Eisentraut writes: > I suggest we define pg_noreturn as > > 1. If C11 is supported, then _Noreturn, else > 2. If GCC-compatible, then __attribute__((noreturn)), else Would it be worth also checking __has_attribute(noreturn)? Or do all compilers that have __attribute__((noreturn)) claim to

Re: Add the ability to limit the amount of memory that can be allocated to backends.

I think this discussion is getting away from a manageable scope of work... On Thu, Jan 2, 2025 at 1:09 PM Jim Nasby wrote: > That’s basically my argument for having workload management. If a system > becomes loaded enough for the global limit to start kicking in it’s likely > that query respon

Re: Vacuum statistics

> I guess one question is how realistic it is to try and put everything about > (auto)vacuum in a single view. > Given the complexity, the answer to that might just be “no”. In that case > leaving existing fields in pg_stat_all_tables > is a lot more reasonable. Agree. I also think the total_tim

Re: Adding OLD/NEW support to RETURNING

On Wed, Jan 1, 2025 at 3:20 AM Dean Rasheed wrote: > On Thu, 28 Nov 2024 at 11:45, Dean Rasheed wrote: > Attached is an updated patch with some additional tidying up, plus the > following changes: > Hey Dean, This is really nice work. I was curious what you think the status of this patch is at

Re: Vacuum statistics

On Jan 2, 2025, at 4:33 PM, Sami Imseih wrote: > >> While backwards compatibility is important, there’s definitely precedent for >> changing >> what shows up in the catalog. IMHO it’s better to bite the bullet and move >> those fields >> instead of having vacuum stats spread across two differen

improve DEBUG1 logging of parallel workers for CREATE INDEX?

While reviewing patch [1], I realized that the DEBUG1 message for CREATE INDEX could do better in providing information about parallel workers launched. Currently, the message just shows how many workers are planned, but a user may want to ensure that they have the appropriate number of workers lau

Re: Fwd: Re: A new look at old NFS readdir() problems?

Peter Eisentraut writes: > On 03.01.25 02:58, Tom Lane wrote: >> I poked at this a little further. I made the attached stand-alone >> test case (you don't need any more than "cc -o rmtree rmtree.c" >> to build it, then point the script at some NFS-mounted directory). >> This fails with my NAS at

Re: Fwd: Re: A new look at old NFS readdir() problems?

On 03.01.25 02:58, Tom Lane wrote: I wrote: Thomas Munro writes: For what little it's worth, I'm not quite convinced yet that FreeBSD's client isn't more broken than it needs to be. I'm suspicious of that too. I poked at this a little further. I made the attached stand-alone test case (y

Re: Question about behavior of deletes with REPLICA IDENTITY NOTHING

On Wed, Dec 18, 2024 at 5:56 AM Amit Kapila wrote: > > On Thu, Feb 8, 2024 at 9:57 AM Laurenz Albe wrote: > > > > On Thu, 2024-02-08 at 13:40 +1100, Peter Smith wrote: > > > - how to set the replica identity. If a table without a replica > > > identity is > > > + how to set the replica iden

Re: Modern SHA2- based password hashes for pgcrypto

Am Freitag, dem 03.01.2025 um 23:57 +0800 schrieb Japin Li: > > Greate!  I have some questions after using it. > > When I use the following query, it crashed! > > [local]:4012204 postgres=# select crypt('hello', > '$5$$6$rounds=1$/Zg436s2vmTwsoSz'); > server closed the connection unexpectedl

Re: Fwd: Re: A new look at old NFS readdir() problems?

On 1/3/25 09:47, Greg Sabino Mullane wrote: On Fri, Jan 3, 2025 at 8:33 AM Robert Haas > wrote: We tried to make our code as robust as it could be in the face of kernel code that behaved in a manner that was fairly ridiculous relative to our needs. This

Re: Logging parallel worker draught

> * Centralization of logs: The maintenance operation can be planned in > cron jobs. In that context, it could certainly be logged separately via > the script. However, when I am doing an audit on a client database, I > think it's useful to have all the relevant information in PostgreSQL logs. Mai

Re: relfilenode statistics

Hi, On Tue, Dec 03, 2024 at 10:31:15AM +, Bertrand Drouvot wrote: > Hi, > > On Fri, Nov 29, 2024 at 08:52:13PM +0500, Kirill Reshke wrote: > > On Fri, 29 Nov 2024 at 20:20, Bertrand Drouvot > > wrote: > > > On Fri, Nov 29, 2024 at 11:23:12AM +0500, Kirill Reshke wrote: > > > > If we don’t ha

Re: Autovacuum giving up on tables after crash because of lack of stats

On Fri, Jan 03, 2025 at 09:47:58AM -0500, Andres Freund wrote: > On 2024-12-30 19:12:54 +0900, Michael Paquier wrote: > > On Mon, Dec 30, 2024 at 09:44:45AM +, Bertrand Drouvot wrote: > > > I think that replicating stats that are used by autovacuum would be an > > > additional > > > benefit, s

Re: Exporting float_to_shortest_decimal_buf(n) with Postgres 17 on Windows

I tested this with a patched version of the EDB PG 17 installer that includes this patch and it fixed the issue I was having described in: https://www.postgresql.org/message-id/01db1df8%2449765bb0%24dc631310%24%40pcorp.us It would be great if this was backported to PG17. The new status of t

Re: Modern SHA2- based password hashes for pgcrypto

On Tue, 31 Dec 2024 at 17:06, Bernd Helmle wrote: > Hi Hackers, > > Some of you might already arrived 2025, so first a Happy New Year to > everyone already there ;) > > Please find attached a patch to pgcrypto to add modern SHA-2 based > password hashes sha256crypt (256 bit) and sha512crypt (512 b

Re: [PATCH] Fix jsonb comparison for raw scalar pseudo arrays

On 2024-12-09 Mo 11:16 AM, jian he wrote: On Mon, Dec 9, 2024 at 9:27 PM Yan Chengpeng wrote: Sorry, I uploaded the wrong file. I uploaded a new patch with the modified document. Please take a review. Thanks! sorry. maybe i didn't mention it explicitly. i mean something like: diff --git

Re: Remove unused rel parameter in lookup_var_attr_stats

On Fri, Jan 3, 2025 at 11:09 AM Ilia Evdokimov < ilya.evdoki...@tantorlabs.com> wrote: > Hi hackers, > > I've attached a small patch that remove unused parameter 'rel' from > the static function lookup_var_attr_stats() in extended_stats.c > > While exploring src/backend/statistics, I noticed that

Re: Autovacuum giving up on tables after crash because of lack of stats

On 2024-12-30 19:12:54 +0900, Michael Paquier wrote: > On Mon, Dec 30, 2024 at 09:44:45AM +, Bertrand Drouvot wrote: > > I think that replicating stats that are used by autovacuum would be an > > additional > > benefit, so +1 for the idea number 2). This is an "issue" that has been > > raised

Re: Fwd: Re: A new look at old NFS readdir() problems?

On Fri, Jan 3, 2025 at 8:33 AM Robert Haas wrote: > We tried to make our code as robust as it could be in the face of kernel > code that behaved in a manner that was fairly ridiculous relative to our > needs. This case doesn't seem that different to me. > +1. Seems a shame that freebsd chooses s

Re: Autovacuum giving up on tables after crash because of lack of stats

Hi, On 2024-12-25 10:10:44 +0900, Michael Paquier wrote: > While digging again into the support for pgstats flushes across > checkpoints, mainly to avoid autovacuum giving up on tables after a > crash if stats entries cannot be found, I've been reminded about this > point raised by Heikki: > https

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

On Fri, Jan 3, 2025, at 10:14 AM, Bertrand Drouvot wrote: > If we don't want to force wal_level = logical to enable logical decoding (your > second idea) then I think that it would be better to "hide" everything behind > logical replication slot creation / deletion. That would mean that having at >

Re: Vacuum statistics

> > While backwards compatibility is important, there’s definitely precedent > for changing what shows up in the catalog. IMHO it’s better to bite the > bullet and move those fields instead of having vacuum stats spread across > two different views. > -1. That's a huge change, and pg_stat_all_tabl

Remove unused rel parameter in lookup_var_attr_stats

Hi hackers, I've attached a small patch that remove unused parameter 'rel' from the static function lookup_var_attr_stats() in extended_stats.c While exploring src/backend/statistics, I noticed that there are several other parameters which appear to be unused. However, since the discussion a

Re: Define STATS_MIN_ROWS for minimum rows of stats in ANALYZE

On 10.12.2024 16:32, Ilia Evdokimov wrote: On 09.12.2024 16:10, Ilia Evdokimov wrote: Hi hackers, The repeated use of the number 300 in the ANALYZE-related code creates redundancy and relies on scattered, sometimes unclear, comments to explain its purpose. This can make the code harder to

Re: magical eref alias names

On Thu, Jan 2, 2025 at 5:11 PM Tom Lane wrote: > Okay, but then we still have the problem of how to ensure that in > a query that has inline'd some views. I think solving the sort of > I-want-to-reference-this problem you describe would require > that we unique-ify the aliases in the rewriter, ju

Re: Fwd: Re: A new look at old NFS readdir() problems?

On Thu, Jan 2, 2025 at 4:52 PM Tom Lane wrote: > Are you prepared to buy into "we will make every bit of code that uses > readdir() proof against arbitrary lies from readdir()"? I'm not: > I cannot see how to do that in any but the simplest cases -- rmtree() > being about the simplest. Even if w

Re: POC: enable logical decoding when wal_level = 'replica' without a server restart

Hi, On Mon, Dec 30, 2024 at 10:44:38PM -0600, Masahiko Sawada wrote: > Hi all, > > Logical decoding (and logical replication) are available only when > wal_level = logical. As the documentation says[1], Using the 'logical' > level increases the WAL volume which could negatively affect the > perfo

Re: [RFC] Lock-free XLog Reservation from WAL

Hi Thank you for your path，NUM_XLOGINSERT_LOCKS increase to 128，I think it will be challenged，do we make it guc ？ On Fri, 3 Jan 2025 at 20:36, Yura Sokolov wrote: > 02.01.2025 10:05, Zhou, Zhiguo wrote: > > Hi all, > > > > I am reaching out to solicit your insights and comments on a recent

Re: [RFC] Lock-free XLog Reservation from WAL

02.01.2025 10:05, Zhou, Zhiguo wrote: Hi all, I am reaching out to solicit your insights and comments on a recent proposal regarding the "Lock-free XLog Reservation from WAL." We have identified some challenges with the current WAL insertions, which require space reservations in the WAL buffer

Re: Conflict detection for update_deleted in logical replication

On Thu, Jan 2, 2025 at 2:57 PM vignesh C wrote: > > Conflict detection of truncated updates is detected as update_missing > and deleted update is detected as update_deleted. I was not sure if > truncated updates should also be detected as update_deleted, as the > document says truncate operation i

Re: per backend I/O statistics

Hi, On Tue, Dec 24, 2024 at 02:35:16PM +0900, Michael Paquier wrote: > On Fri, Dec 20, 2024 at 09:57:19AM +, Bertrand Drouvot wrote: > > BTW, now that the per backend I/O statistics is done, I'll start working on > > per > > backend wal statistics. > > I think that this is a good idea. Than

Re: Conflict detection for update_deleted in logical replication

On Fri, Jan 3, 2025 at 2:34 PM vignesh C wrote: > > Few comments: > 1) In case there are no logical replication workers, the launcher > process just logs a warning "out of logical replication worker slots" > and continues. Whereas in case of "pg_conflict_detection" replication > slot creation laun

Re: Logging parallel worker draught

Hi, thank you for the review and sorry for the delayed answer. On 12/28/24 05:17, Sami Imseih wrote:> Thinking about this further, it seems to me this logging only makes sense > for parallel query and not maintenance commands. This is because > for the latter case, the commands are executed man

Re: Conflict detection for update_deleted in logical replication

On Fri, Jan 3, 2025 at 12:06 PM Masahiko Sawada wrote: > > I have one comment on the 0001 patch: > > + /* > +* The changes made by this and later transactions are still > non-removable > +* to allow for the detection of update_deleted conflicts when > applying > +* c

Re: Conflict detection for update_deleted in logical replication

On Wed, 25 Dec 2024 at 08:13, Zhijie Hou (Fujitsu) wrote: > > On Monday, December 23, 2024 2:15 PM Kuroda, Hayato/黒田 隼人 > wrote: > > > > Dear Hou, > > > > Thanks for updating the patch. Few comments: > > Thanks for the comments! > > > 02. ErrorOnReservedSlotName() > > > > Currently the function