Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Michael Paquier
On Tue, Dec 03, 2024 at 08:49:41AM +0100, Alvaro Herrera wrote: > That cannot be backpatched, though. Extra code churn is always annoying across stable branches, so I'd rather avoid it if we can. -- Michael signature.asc Description: PGP signature

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Alvaro Herrera
On 2024-Dec-03, Peter Smith wrote: > Perhaps the patch can use foreach_ptr macro instead of foreach? That cannot be backpatched, though. -- Álvaro Herrera 48°01'N 7°57'E — https://www.EnterpriseDB.com/ "Thou shalt not follow the NULL pointer, for chaos and madness await thee at

RE: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-02 Thread Hayato Kuroda (Fujitsu)
Dear Nisha, Thanks for updating the patch! > Fixed. It is reasonable to align with other timeout parameters by > using milliseconds as the unit. It looks you just replaced to GUC_UNIT_MS, but the documentation and postgresql.conf.sample has not been changed yet. They should follow codes. Anyway,

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-12-02 Thread Shlok Kyal
On Tue, 3 Dec 2024 at 10:21, Zhijie Hou (Fujitsu) wrote: > > On Friday, November 29, 2024 9:08 PM Shlok Kyal > wrote: > > > > > > Thanks for reviewing the patch. I have fixed the issue and updated the > > patch. > > Thanks for updating the patch. I have reviewed and have few suggestions. > > Pl

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-02 Thread Peter Smith
Hi Nisha, here are a couple of review comments for patch v52-0001. == Commit Message Add check if slot is already acquired, then mark it invalidate directly. ~ /slot/the slot/ "mark it invalidate" ? Maybe you meant: "then invalidate it directly", or "then mark it 'invalidated' directly",

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Peter Smith
On Tue, Dec 3, 2024 at 5:47 PM Michael Paquier wrote: > > On Mon, Dec 02, 2024 at 03:29:56PM +0530, Amit Kapila wrote: > > We can look at it from a different angle which is that the > > FreePublication(s) relies on how the knowledge of Publication > > structure is built. So, it doesn't look weird

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Michael Paquier
On Mon, Dec 02, 2024 at 03:29:56PM +0530, Amit Kapila wrote: > We can look at it from a different angle which is that the > FreePublication(s) relies on how the knowledge of Publication > structure is built. So, it doesn't look weird if we think from that > angle. OK, I can live with that on all t

Re: speedup ALTER TABLE ADD CHECK CONSTRAINT.

2024-12-02 Thread jian he
On Mon, Dec 2, 2024 at 12:06 AM Sergei Kornilov wrote: > > Hello! > Thanks for the patch, but I think using SPI is still not allowed here: > https://www.postgresql.org/message-id/9878.1511970441%40sss.pgh.pa.us > > > if it uses SPI for sub-operations of ALTER TABLE I think that is sufficient > >

Re: [18] Unintentional behavior change in commit e9931bfb75

2024-12-02 Thread Tom Lane
Jeff Davis writes: > On Mon, 2024-12-02 at 17:25 -0500, Tom Lane wrote: >> Well, also for compatibility with our SQL parser's understanding >> of identifier lowercasing. > But why only for single-byte encodings? And why not for ICU? I think the not-for-utf8 exclusion was mostly because that was

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Michael Paquier
On Tue, Dec 03, 2024 at 09:50:42AM +0530, Amit Kapila wrote: > But that suits the current design more. We allocate PGOutputData and > other contexts in that structure in a "Logical decoding context". A > few of its members (publications, publication_names) residing in > totally unrelated contexts s

Re: [18] Unintentional behavior change in commit e9931bfb75

2024-12-02 Thread Jeff Davis
On Mon, 2024-12-02 at 17:25 -0500, Tom Lane wrote: > Well, also for compatibility with our SQL parser's understanding > of identifier lowercasing. But why only for single-byte encodings? And why not for ICU? > > Should I put the special case back? > > I think so. Done. I put the special case ba

RE: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-12-02 Thread Zhijie Hou (Fujitsu)
On Friday, November 29, 2024 9:08 PM Shlok Kyal wrote: > > > Thanks for reviewing the patch. I have fixed the issue and updated the patch. Thanks for updating the patch. I have reviewed and have few suggestions. Please check the attached diff which includes: 1) Comments in CheckCmdReplicaIde

Re: on_error table, saving error info to a table

2024-12-02 Thread jian he
On Tue, Nov 5, 2024 at 6:30 PM Nishant Sharma wrote: > > Thanks for the v2 patch! > > I see v1 review comments got addressed in v2 along with some > further improvements. > > 1) v2 Patch again needs re-base. > > 2) I think we need not worry whether table name is unique or not, > table name can be

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Amit Kapila
On Tue, Dec 3, 2024 at 2:12 AM Alvaro Herrera wrote: > > On 2024-Dec-02, Amit Kapila wrote: > > > Even, if we want a new context for some localized handling, we should > > add that in PGOutputData rather than a local context as the proposed > > patch is doing at the very least for HEAD. > > I don'

Re: [PoC] Reducing planning time when tables have many partitions

2024-12-02 Thread Ashutosh Bapat
On Mon, Dec 2, 2024 at 2:22 PM Yuya Watari wrote: > > 4. Discussion > > First of all, tables 1, 2 and the figure attached to this email show > that likely and unlikely do not have the effect I expected. Rather, > tables 3, 4, 5 and 6 imply that they can have a negative effect on > queries A and B

Re: Make tuple deformation faster

2024-12-02 Thread David Rowley
On Mon, 2 Dec 2024 at 23:24, David Rowley wrote: > I spent all day today reviewing and fixing up a few missing comments > for the v5 patch series. I'm quite happy with these now. If nobody > else wants to look or test, I plan on pushing these tomorrow (Tuesday > UTC+13). If anyone wants me to dela

Re: Doc: typo in config.sgml

2024-12-02 Thread Bruce Momjian
On Mon, Dec 2, 2024 at 09:33:39PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > Now that we have a warning about non-emittable characters in the PDF > > build, do you want me to put back the Latin1 characters in the SGML > > files or leave them as HTML entities? > > I think going forward we

Re: Doc: typo in config.sgml

2024-12-02 Thread Tom Lane
Bruce Momjian writes: > Now that we have a warning about non-emittable characters in the PDF > build, do you want me to put back the Latin1 characters in the SGML > files or leave them as HTML entities? I think going forward we're going to be putting in people's names in UTF8 --- I was certainly

Re: Doc: typo in config.sgml

2024-12-02 Thread Bruce Momjian
On Tue, Nov 5, 2024 at 10:08:17AM +0100, Peter Eisentraut wrote: > On 02.11.24 14:18, Bruce Momjian wrote: > > On Sat, Nov 2, 2024 at 12:02:12PM +0900, Tatsuo Ishii wrote: > > > > Yes, we _allow_ LATIN1 characters in the SGML docs, but I replaced the > > > > LATIN1 characters we had with HTML ent

Re: Doc: typo in config.sgml

2024-12-02 Thread Bruce Momjian
On Tue, Nov 26, 2024 at 02:04:15PM -0500, Bruce Momjian wrote: > On Tue, Nov 26, 2024 at 12:41:37PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > On Tue, Nov 26, 2024 at 11:43:02AM -0500, Tom Lane wrote: > > >> I don't think this patch is doing anything I want at all. > > > > > Gee, I ki

Re: Using read stream in autoprewarm

2024-12-02 Thread Kirill Reshke
On Fri, 29 Nov 2024 at 16:19, Nazir Bilal Yavuz wrote: > v4 is attached. Hi! I feel like we are ready to mark this as RFC, WDYT? -- Best regards, Kirill Reshke

Re: RISC-V animals sporadically produce weird memory-related failures

2024-12-02 Thread Alexander Lakhin
02.12.2024 18:25, Tom Turelinckx wrote: These crashes are hardly related to code changes, so maybe there are platform-specific issues still... I naively assumed that because llvm and clang are available in Trixie on riscv64 that I could simply install them and enable --with-llvm on copperhead,

RE: Rework subscription-related code for psql and pg_dump

2024-12-02 Thread Hayato Kuroda (Fujitsu)
Dear Michael, > I've looked at that again and let this as-is, then applied all this > cleanup. I missed your e-mail but it's ok to keep the output style for two_phase. Thanks a lot for working on it! Best regards, Hayato Kuroda FUJITSU LIMITED

Re: Remove useless casts to (void *)

2024-12-02 Thread Tom Lane
Thomas Munro writes: > Why do we bother with a "Pointer" type? The idiomatic way to do > byte-oriented pointer arithmetic is to cast to char *, or uint8_t* > etc, which doesn't require the reader to go and figure out what stride > that non-standard type name implies. I think getting rid of Point

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 3:56 PM Peter Geoghegan wrote: > I took what you wrote, and repurposed it to prove my old theory about > GiST index-only scans being broken due to the lack of an appropriate > interlock against concurrent TID recycling. See the attached patch. I've moved discussion of this

Re: Why doesn't GiST VACUUM require a super-exclusive lock, like nbtree VACUUM?

2024-12-02 Thread Peter Geoghegan
On Wed, Nov 3, 2021 at 7:33 PM Peter Geoghegan wrote: > But what about index-only scans, which GiST also supports? I think > that the rules are different there, even though index-only scans use > an MVCC snapshot. (Reviving this old thread after 3 years) I was reminded of this old thread during

Re: code contributions for 2024, WIP version

2024-12-02 Thread Michael Paquier
On Mon, Dec 02, 2024 at 04:10:22PM -0500, Robert Haas wrote: > Donning my asbestos underwear, I remain yours faithfully, Thanks for taking the time to compile all that. That's really nice. -- Michael signature.asc Description: PGP signature

Re: Consider pipeline implicit transaction as a transaction block

2024-12-02 Thread Michael Paquier
On Mon, Dec 02, 2024 at 09:14:11AM -0500, Robert Haas wrote: > On Thu, Nov 28, 2024 at 2:53 AM Anthonin Bonnefoy > wrote: >> Sorry about that. I didn't have a strong need for this to be >> backpatched and should have made this clearer. > > FWIW, I don't think you did anything wrong. To me, the th

Re: meson missing test dependencies

2024-12-02 Thread Michael Paquier
On Mon, Dec 02, 2024 at 01:50:57PM +0300, Nazir Bilal Yavuz wrote: > On Mon, 2 Dec 2024 at 13:11, Peter Eisentraut wrote: >> Is there any reason this was not done yet? > > This looks useful. I am not sure why this was not done before. Do you think that it would be possible to automate the additi

Re: Rework subscription-related code for psql and pg_dump

2024-12-02 Thread Michael Paquier
On Sat, Nov 30, 2024 at 12:54:19PM +0900, Michael Paquier wrote: > - if (strcmp(subinfo->subtwophasestate, two_phase_disabled) != 0) > + if (subinfo->subtwophasestate != LOGICALREP_TWOPHASE_STATE_DISABLED) > appendPQExpBufferStr(query, ", two_phase = on"); > > I'm not feeling strongly

Re: Remove useless casts to (void *)

2024-12-02 Thread Thomas Munro
On Tue, Oct 29, 2024 at 9:06 PM Peter Eisentraut wrote: > There are a bunch of (void *) casts in the code that don't make sense to > me. I think some of these were once necessary because char * was used > in place of void * for some function arguments. Some other places that sprang to my eye rec

Re: Showing applied extended statistics in explain Part 2

2024-12-02 Thread Tom Lane
Ilia Evdokimov writes: > On 19.11.2024 00:38, Tomas Vondra wrote: >> On 11/18/24 22:15, Tomas Vondra wrote: >>> So I think the correct solution is to not pass any expressions with >>> RestrictInfo to deparse_expression(). Either by stripping the nodes, or >>> by not adding them at all. >>> >>> Th

Re: Remove useless casts to (void *)

2024-12-02 Thread Andres Freund
On 2024-12-02 17:42:33 -0500, Tom Lane wrote: > I'll proceed with improving that (in master only, don't think we need it in > back branches, at least not today) unless somebody pushes back soon. +1

Re: Remove useless casts to (void *)

2024-12-02 Thread Tom Lane
Andres Freund writes: > On 2024-12-02 17:11:30 -0500, Tom Lane wrote: >> I'm inclined to propose adding something like >> CPPFLAGS="$CPPFLAGS -D_POSIX_C_SOURCE=200112L" >> to src/template/solaris. Not sure if we have a corresponding >> mechanism for meson, though. > elif host_system == 'sunos' >

Re: [18] Unintentional behavior change in commit e9931bfb75

2024-12-02 Thread Tom Lane
Jeff Davis writes: > The behavior is commented (commit 176d5bae1d) in formatting.c: >* ... When using the default >* collation, we apply the traditional Postgres behavior that >* forces ASCII-style treatment of I/i, but in non-default >* collations you get exactly what the collat

Re: Remove useless casts to (void *)

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 17:11:30 -0500, Tom Lane wrote: > I'm inclined to propose adding something like > > CPPFLAGS="$CPPFLAGS -D_POSIX_C_SOURCE=200112L" > > to src/template/solaris. Not sure if we have a corresponding > mechanism for meson, though. elif host_system == 'sunos' portname = 'solaris

Re: Remove useless casts to (void *)

2024-12-02 Thread Tom Lane
Thomas Munro writes: > I wouldn't change that. illumos is selecting the old pre-standard > declaration here, but it knows the standard one: > https://github.com/illumos/illumos-gate/blob/27ecbff00d8c86a2647d6fe325cacb220d712115/usr/src/uts/common/sys/shm.h#L129 > https://illumos.org/man/2/shmdt

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 12:15 PM Peter Geoghegan wrote: > The underlying reason why nbtree can discriminate like this is that it > "knows" that plain index scans will always visit the heap proper. If a > TID points to an LP_UNUSED item, then it is considered dead to the > scan (even though in gener

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Tom Lane
Andres Freund writes: > I think the problematic scenario involves tuples that *nobody* can see. During > the bitmap index scan we don't know that though. Thus the tid gets inserted > into the bitmap. Then, before we visit the heap, a concurrent vacuum removes > the tuple from the indexes and then

Re: Showing applied extended statistics in explain Part 2

2024-12-02 Thread Ilia Evdokimov
On 19.11.2024 00:38, Tomas Vondra wrote: On 11/18/24 22:15, Tomas Vondra wrote: ... So I think the correct solution is to not pass any expressions with RestrictInfo to deparse_expression(). Either by stripping the nodes, or by not adding them at all. The patch tries to do the stripping by ma

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 12:02 PM Tom Lane wrote: > Andres Freund writes: > > I think the problematic scenario involves tuples that *nobody* can see. > > During > > the bitmap index scan we don't know that though. Thus the tid gets inserted > > into the bitmap. Then, before we visit the heap, a co

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Tom Lane
Peter Geoghegan writes: > On Mon, Dec 2, 2024 at 12:02 PM Tom Lane wrote: >> Yup. I am saying that that qualifies as too-aggressive setting of the >> all-visible bit. I'm not sure what rule we should adopt instead of >> the current one, but I'd much rather slow down page freezing than >> instit

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 11:32 AM Andres Freund wrote: > On 2024-12-02 16:08:02 +0100, Matthias van de Meent wrote: > > Concurrency timeline: > > > > Session 1. amgetbitmap() gets snapshot of index contents, containing > > references to dead tuples on heap P1. > > IIUC, an unstanted important aspect

Incorrect result of bitmap heap scan.

2024-12-02 Thread Konstantin Knizhnik
Hi hackers, Attached script reproduces the problem with incorrect results of `select count(*)` (it returns larger number of records than really available in the table). It is not always reproduced, so you may need to repeat it multiple times - at my system it failed 3 times from 10. The prob

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 11:43:42 -0500, Tom Lane wrote: > Peter Geoghegan writes: > > This theory seems very believable. > > I'm not convinced. I think there are two assumptions underlying > bitmap scan: > > 1. Regardless of index contents, it's not okay for vacuum to remove > tuples that an open tr

Re: Add “FOR UPDATE NOWAIT” lock details to the log.

2024-12-02 Thread Fujii Masao
On 2024/10/18 19:07, Seino Yuki wrote: The choice between adding a new GUC or extending the existing one (e.g., log_lock_waits) is debatable, but I prefer the latter. I'm considering extending log_lock_waits to accept a value like "fail". If set to "on" (the current behavior), detailed logs ar

Re: Count and log pages set all-frozen by vacuum

2024-12-02 Thread Robert Haas
On Fri, Nov 29, 2024 at 2:46 PM Masahiko Sawada wrote: > Finally, in case where we have: > > visibility map: 1 pages set all-visible, 1 pages set all-frozen. > > We can understand that 1 pages newly became all-frozen, but have > no idea how many pages became all-visible but not all-fro

[18] Unintentional behavior change in commit e9931bfb75

2024-12-02 Thread Jeff Davis
Commit e9931bfb75 (version 18) contained an unexpected behavior change. LOWER('I') returns: In the locale tr_TR.iso88599 (single byte encoding): 17 18 defaulti ı specified ı ı In the locale tr_TR.utf8:

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 3:55 PM Andres Freund wrote: > I suspect one contributor to this avoiding attention till now was that the > optimization is fairly narrow: > > /* > * We can potentially skip fetching heap pages if we > do not need >

Re: Remove useless casts to (void *)

2024-12-02 Thread Tom Lane
Peter Eisentraut writes: > Committed, thanks. Now that we have a more-or-less full set of buildfarm results on this, I checked for new warnings, and found two: pg_shmem.c: In function 'PGSharedMemoryIsInUse': pg_shmem.c:323:33: warning: passing argument 1 of 'shmdt' from incompatible pointer ty

Re: Make tuple deformation faster

2024-12-02 Thread Victor Yegorov
пн, 2 дек. 2024 г. в 13:24, David Rowley : > I ended up adjusting the code here so that TupleDescSize() returns the > full size and TupleDescAttrAddress() manually calculates the offset to > start the FormData_pg_attribute array. That allows > TupleDescFullSize() to be deleted. I changed how Tup

code contributions for 2024, WIP version

2024-12-02 Thread Robert Haas
Hi, As many of you are probably aware, I have been doing an annual blog post on who contributes to PostgreSQL development for some years now. It includes information on lines of code committed to PostgreSQL, and also emails sent to the list. This year, I got a jump on analyzing the commit log, and

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
On 02.12.2024 17:46, Ilia Evdokimov wrote: In my opinion, the patches are semantically correct. However, not all dead code has been removed - I'm referring to pgstat_update_snapshot(). Also, the tests need to be fixed. Thank you, I'll fix it -- Regards, Alena Rybakina Postgres Professional

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
Hi! Thank you for your review! On 30.11.2024 07:48, Kirill Reshke wrote: Hello! After a brief glance, I think this patch set is good. But there isn't any more time in the current CF to commit this :(. So I moved to the next CF. I agree with you. Thank you!) I also like the 0001 commit message.

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 3:56 PM Peter Geoghegan wrote: > I took what you wrote, and repurposed it to prove my old theory about > GiST index-only scans being broken due to the lack of an appropriate > interlock against concurrent TID recycling. See the attached patch. BTW, if you change the test ca

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 2:43 PM Andres Freund wrote: > Attached is an isolationtest that reliably shows wrong query results. Nice approach with the cursor! I took what you wrote, and repurposed it to prove my old theory about GiST index-only scans being broken due to the lack of an appropriate in

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 13:39:43 -0500, Peter Geoghegan wrote: > I guess it's natural to suspect more recent work -- commit 7c70996e is > about 6 years old. But I the race condition that I suspect is at play > here is very narrow. FWIW, the test I just posted shows the issue down to 11 (although for 11

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Alvaro Herrera
On 2024-Dec-02, Amit Kapila wrote: > We already have PGOutputData->cachectx which could be used for it. > I think we should be able to reset such a context when we are > revalidating the publications. I don't see that context being reset anywhere, so I have a hard time imagining that this would w

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-02 Thread Alvaro Herrera
On 2024-Dec-02, Michael Paquier wrote: > I am slightly concerned about the current design of GetPublication() > in the long-term, TBH. LoadPublications() has hidden the leak behind > two layers of routines in the WAL sender, and that's easy to miss once > you call anything that loads a Publicatio

Re: RISC-V animals sporadically produce weird memory-related failures

2024-12-02 Thread Tom Turelinckx
Hi Alexander, On Mon, Dec 2, 2024, at 2:00 PM, Alexander Lakhin wrote: > These crashes are hardly related to code changes, so maybe there are > platform-specific issues still... I naively assumed that because llvm and clang are available in Trixie on riscv64 that I could simply install them and

Re: Vacuum statistics

2024-12-02 Thread Alena Rybakina
On 02.12.2024 11:27, Alexander Korotkov wrote: Hi, Alena! On Wed, Nov 13, 2024 at 6:21 PM Alena Rybakina wrote: Updated 0001-v13 attached, as well as the diff between v12 and v13. Thank you) And I agree with your changes. And included them in patches. Thank you for the updated patchset. S

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 11:52 AM Andres Freund wrote: > I think the problematic scenario involves tuples that *nobody* can see. During > the bitmap index scan we don't know that though. Right, exactly. FWIW, this same issue is why it is safe for nbtree to drop its pin early during plain index sca

Re: RISC-V animals sporadically produce weird memory-related failures

2024-12-02 Thread Thomas Munro
On Tue, Dec 3, 2024 at 7:00 AM Alexander Lakhin wrote: > A build made with clang-19 without llvm passed `make check` successfully. We heard in another thread[1] that we'd need to use the JITLink API for RISCV, instead of the RuntimeDyld API we're using. I have a newer patch to use JITLink on all

Re: meson missing test dependencies

2024-12-02 Thread Peter Eisentraut
On 02.12.24 11:50, Nazir Bilal Yavuz wrote: I applied your patches and the cube example worked. But when I edited 'test_json_parser_perf.c' and 'test_json_parser_incremental.c', meson did not rebuild. I used the 'meson test -C build --suite setup --suite test_json_parser' command to test test_jso

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 12:02:39 -0500, Tom Lane wrote: > Andres Freund writes: > > I think the problematic scenario involves tuples that *nobody* can see. > > During > > the bitmap index scan we don't know that though. Thus the tid gets inserted > > into the bitmap. Then, before we visit the heap, a

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 11:43 AM Tom Lane wrote: > Whether that's true or not, it seems like it'd be worth bisecting > to see if we can finger a commit where the behavior changed (and > the same goes for the question of why-isnt-it-an-IOS-scan). However, > the reproducer seems to have quite a low

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 11:31:48 -0500, Andres Freund wrote: > I think it'd be good if we added a test that shows the failure mechanism so > that we don't re-introduce this in the future. Evidently this failure isn't > immediately obvious... Attached is an isolationtest that reliably shows wrong query

Re: More CppAsString2() in psql's describe.c

2024-12-02 Thread Daniel Gustafsson
> On 2 Dec 2024, at 08:52, Tom Lane wrote: > But isn't there a way to improve the macro so this'd lead to an error? That sounds like a pretty decent improvement in general. I experimented with quick hack using a typeof check on the passed symbol which catches symbolname typos. It might be total

Re: Vacuum statistics

2024-12-02 Thread Ilia Evdokimov
In my opinion, the patches are semantically correct. However, not all dead code has been removed - I'm referring to pgstat_update_snapshot(). Also, the tests need to be fixed. -- Best regards, Ilia Evdokimov, Tantor Labs LLC.

Re: sunsetting md5 password support

2024-12-02 Thread Nathan Bossart
On Wed, Nov 20, 2024 at 08:17:07PM -0500, Greg Sabino Mullane wrote: > Sounds good to me. I think my hesitation was more that the hint was > overpromising help, so big +1 to more detail and keeping it. Committed. If anyone wants to try putting together a patch that expands the "migrating to SCRAM

Re: Remove useless casts to (void *)

2024-12-02 Thread Thomas Munro
On Tue, Dec 3, 2024 at 7:06 AM Tom Lane wrote: > This is from hake[1], which is running OpenIndiana/illumos. > That platform shows a couple of other strange warnings, so maybe > re-eliminating these two is not worth worrying about, but > nonetheless the casts to void * were doing something here.

Re: Proposal for Updating CRC32C with AVX-512 Algorithm.

2024-12-02 Thread Nathan Bossart
On Mon, Nov 25, 2024 at 08:54:48PM +, Devulapalli, Raghuveer wrote: > As Nathan suggested, we moved this to a separate thread. The latest set > of patches here need to applied on top of patches in that thread. Raghuveer, would you mind rebasing this patch set now that the SSE4.2 patch is commi

Re: Changing shared_buffers without restart

2024-12-02 Thread Dmitry Dolgov
> On Fri, Nov 29, 2024 at 05:47:27PM GMT, Dmitry Dolgov wrote: > > On Fri, Nov 29, 2024 at 01:56:30AM GMT, Matthias van de Meent wrote: > > > > I mean, we can do the following to get a nice contiguous empty address > > space no other mmap(NULL)s will get put into: > > > > /* reserve size bytes

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-02 Thread Wolfgang Walther
Eric Hanson: a) Transaction ("local") Sandbox: - SET LOCAL ROLE alice NO RESET; - SET LOCAL ROLE alice WITHOUT RESET; - BEGIN AS ROLE alice; Transaction-level sandboxes have the benefit that a pooler can simply start a new sandboxed transaction for each request and never have to worry about re

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Andres Freund
Hi, On 2024-12-02 16:08:02 +0100, Matthias van de Meent wrote: > Concurrency timeline: > > Session 1. amgetbitmap() gets snapshot of index contents, containing > references to dead tuples on heap P1. IIUC, an unstanted important aspect here is that these dead tuples are *not* visible to S1. Othe

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 12:11 PM Tom Lane wrote: > > Freezing a page, and setting a page all-visible are orthogonal. > > Sorry, sloppy wording on my part. Freezing doesn't affect the contents of the visibility map in any way that seems relevant. The executor only cares about the all-visible bit (a

Re: Consider pipeline implicit transaction as a transaction block

2024-12-02 Thread Robert Haas
On Thu, Nov 28, 2024 at 2:53 AM Anthonin Bonnefoy wrote: > On Thu, Nov 28, 2024 at 12:26 AM Michael Paquier wrote: > > I don't mind being more careful here based on your concerns, so I'll > > go remove that in the stable branches. > > Sorry about that. I didn't have a strong need for this to be >

Re: revamp row-security tracking

2024-12-02 Thread Nathan Bossart
On Fri, Nov 29, 2024 at 10:01:41AM +, Dean Rasheed wrote: > On Thu, 21 Nov 2024 at 18:00, Nathan Bossart wrote: >> The attached patch accomplishes this by establishing a global queue of >> row-security "nest levels" that the aforementioned higher-level callers can >> use. > > I'm not convince

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-02 Thread Joe Conway
On 12/2/24 08:41, Eric Hanson wrote: Hi all, I'd like to revisit a previously discussed feature [1] that PostgreSQL could benefit from a "role sandbox", a feature that would build on SET [LOCAL] ROLE, and prevent or restrict RESET ROLE. Rationale:  Connection pooling is widely used to optimi

Re: CREATE SCHEMA ... CREATE DOMAIN support

2024-12-02 Thread Vik Fearing
On 02/12/2024 17:56, Tom Lane wrote: Vik Fearing writes: On 02/12/2024 03:15, Tom Lane wrote: Also, if SQL intended to constrain the search path for unqualified identifiers to be only the new schema, they'd hardly need a concept of at all. I looked up the original paper (MUN-051) that intr

Re: CREATE SCHEMA ... CREATE DOMAIN support

2024-12-02 Thread Tom Lane
Vik Fearing writes: > On 02/12/2024 03:15, Tom Lane wrote: >> Also, if SQL intended to constrain the search path for unqualified >> identifiers to be only the new schema, they'd hardly need a concept >> of at all. > I looked up the original paper (MUN-051) that introduced the path specification

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-02 Thread Michał Kłeczek
> On 2 Dec 2024, at 18:39, Joe Conway wrote: > > I am very much in favor of functionality of this sort being built in to the > core database. Very similar functionality is available in an extension I > wrote years ago (without the SQL grammar support) -- see > https://github.com/pgaudit/set_u

Re: optimize file transfer in pg_upgrade

2024-12-02 Thread Nathan Bossart
Here is a rebased patch set for cfbot. I'm planning to spend some time getting these patches into a more reviewable state in the near future. -- nathan >From 81fe66e0f0aa4f958a8707df669f60756c89bb85 Mon Sep 17 00:00:00 2001 From: Nathan Bossart Date: Tue, 5 Nov 2024 15:59:51 -0600 Subject: [PAT

Re: CREATE SCHEMA ... CREATE DOMAIN support

2024-12-02 Thread Vik Fearing
On 02/12/2024 03:15, Tom Lane wrote: Michael Paquier writes: If I'm parsing the spec right, the doc mentions in its 5)~6) of the syntax rules in CREATE SCHEMA that non-schema-qualified objects should use the new schema name defined in the CREATE SCHEMA query. So that pretty much settles the

Re: Incorrect result of bitmap heap scan.

2024-12-02 Thread Peter Geoghegan
On Mon, Dec 2, 2024 at 10:15 AM Matthias van de Meent wrote: > The running theory is that bitmap executor nodes incorrectly assume > that the rows contained in the bitmap all are still present in the > index, and thus assume they're allowed to only check the visibility > map to see if the referenc

Re: Using read stream in autoprewarm

2024-12-02 Thread Nazir Bilal Yavuz
Hi, On Mon, 2 Dec 2024 at 16:30, Andrey M. Borodin wrote: > > > On 2 Dec 2024, at 16:16, Kirill Reshke wrote: > > > > I feel like we are ready to mark this as RFC, WDYT? > > +1 Done. -- Regards, Nazir Bilal Yavuz Microsoft

Re: Consider the number of columns in the sort cost model

2024-12-02 Thread Dmitry Dolgov
Hi folks, Just wanted to mention, that looks like some CFBot test are failing, something around level_tracking in pgss.

Proposal: Role Sandboxing for Secure Impersonation

2024-12-02 Thread Eric Hanson
Hi all, I'd like to revisit a previously discussed feature [1] that PostgreSQL could benefit from a "role sandbox", a feature that would build on SET [LOCAL] ROLE, and prevent or restrict RESET ROLE. Rationale: Connection pooling is widely used to optimize database performance by reducing use of

Re: Partition-wise join with whole row vars

2024-12-02 Thread Dmitry Dolgov
> On Tue, Oct 08, 2024 at 09:24:15AM GMT, Alexander Pyhalov wrote: > > Attaching rebased patches. Just to let you know, looks like CFBot tests are red again, but this time there are some unexpected differences in some test query plan.

Re: Drop back the redundant "Lock" suffix from LWLock wait event names

2024-12-02 Thread Alvaro Herrera
On 2024-Dec-02, Bertrand Drouvot wrote: > Hi hackers, > > da952b415f unintentionally added back the "Lock" suffix into the LWLock wait > event names: > > - "added back" because the "Lock" suffix was removed in 14a9101091 > - "unintentionally" because there is nothing in the thread [2] that expli

Re: Using read stream in autoprewarm

2024-12-02 Thread Andrey M. Borodin
> On 2 Dec 2024, at 16:16, Kirill Reshke wrote: > > I feel like we are ready to mark this as RFC, WDYT? +1 Best regards, Andrey Borodin.

Re: RISC-V animals sporadically produce weird memory-related failures

2024-12-02 Thread Alexander Lakhin
Hello Tom, 17.11.2024 20:28, Tom Turelinckx wrote: I have now done just that, but on a new HiFive Premier P550 board [2]. It is running Ubuntu 24.04 LTS with a board-specific kernel, currently 6.6.21-9-premier (2024-11-09). The buildfarm client is executing within a Debian Trixie container cr

Re: Re: Added prosupport function for estimating numeric generate_series rows

2024-12-02 Thread Dean Rasheed
On Sat, 30 Nov 2024 at 00:38, tsinghualucky...@foxmail.com wrote: > > Dear Dean Rasheed, I have reviewed the v4 patch and it is very thoughtful and > reasonable, with a very clever attention to detail (plus I am very happy that > we can get rid of the goto, which I was not a big fan of). > > Thi

RE: Conflict detection for update_deleted in logical replication

2024-12-02 Thread Zhijie Hou (Fujitsu)
On Friday, November 29, 2024 6:35 PM Kuroda, Hayato/黒田 隼人 wrote: > > Dear Hou, > > Thanks for updating the patch! Here are my comments mainly for 0001. Thanks for the comments! > > 02. maybe_advance_nonremovable_xid > > ``` > +case RCI_REQUEST_PUBLISHER_STATUS: > +reques

Re: Proposals for EXPLAIN: rename ANALYZE to EXECUTE and extend VERBOSE

2024-12-02 Thread Michael Christofides
> v4 patch attached Thank you Guillaume, and nice work! I tried to see if there was anywhere else in the documentation that would need updating, but it looks like you covered everywhere already. > I'm with Robert in that I've not found the buffer counts to be all that useful most of the time. I

Re: More CppAsString2() in psql's describe.c

2024-12-02 Thread Peter Eisentraut
On 02.12.24 08:52, Tom Lane wrote: (Moreover, the current structure assumes that the C character literal syntax used by the PROKIND_* and other symbols happens to be the same as the SQL string literal syntax required in those queries, which is just an accident.) So? There isn't much about C syn

Re: Proper object locking for GRANT/REVOKE

2024-12-02 Thread Peter Eisentraut
On 25.11.24 02:24, Noah Misch wrote: commit d31bbfb wrote: --- a/src/backend/catalog/aclchk.c +++ b/src/backend/catalog/aclchk.c @@ -659,147 +659,77 @@ ExecGrantStmt_oids(InternalGrant *istmt) * objectNamesToOids * * Turn a list of object names of a given type into an Oid list. - * - *

Re: Sample rate added to pg_stat_statements

2024-12-02 Thread Ilia Evdokimov
On 26.11.2024 01:15, Ilia Evdokimov wrote: On 22.11.2024 09:08, Alexander Korotkov wrote: On Wed, Nov 20, 2024 at 12:07 AM Michael Paquier wrote: On Tue, Nov 19, 2024 at 09:39:21AM -0500, Greg Sabino Mullane wrote: Oh, and a +1 in general to the patch, OP, although it would also be nice to

fixing tsearch locale support

2024-12-02 Thread Peter Eisentraut
Infamously, the tsearch locale support in src/backend/tsearch/ts_locale.c still depends on libc environment variable locale settings and is not caught up with pg_locale_t, collations, ICU, and all that newer stuff. This is used in the tsearch facilities themselves, but also in other modules su

Re: meson missing test dependencies

2024-12-02 Thread Nazir Bilal Yavuz
Hi, On Mon, 2 Dec 2024 at 13:11, Peter Eisentraut wrote: > > I have noticed that under meson many tests don't have dependencies on > the build artifacts that they are testing. As an example among many, if > you make a source code change in contrib/cube/cube.c (see patch 0001 for > a demo) and th

  1   2   >