Re: Use streaming read API in pgstattuple.

2024-12-05 Thread Kirill Reshke
Hm, CI fails[0] This is most likely the reason why `select pgstattuple('test_hashidx');` was omitted in pgstattuple tests: we are not guaranteed about the number of bucket_pages in the empty index. Maybe we should populate base relation and check for only non-volatile fields like live_items and ver

Re: Popcount optimization using SVE for ARM

2024-12-05 Thread Kirill Reshke
On Fri, 6 Dec 2024 at 10:54, devanga.susmi...@fujitsu.com < devanga.susmi...@fujitsu.com> wrote: > Hello, This email is to discuss the contribution of the speed-up > popcount and popcount mask feature we have developed for the ARM > architecture using SVE intrinsics. > The current method for pop

Re: confusing / inefficient "need_transcoding" handling in copy

2024-12-05 Thread Sutou Kouhei
Hi Michael, ping. (Do you think that this patch is still needed?) Thanks, -- kou In <20240214.114608.2091541942684063981@clear-code.com> "Re: confusing / inefficient "need_transcoding" handling in copy" on Wed, 14 Feb 2024 11:46:08 +0900 (JST), Sutou Kouhei wrote: > Hi, > > In >

Re: Considering fractional paths in Append node

2024-12-05 Thread Andrei Lepikhov
On 11/2/24 01:18, Nikita Malakhov wrote: I've corrected failing test and created a patch at Commitfest: https://commitfest.postgresql.org/51/5361/ commitfest.postgresql.org/51/5361/> I have played around with this feature, which looks promising for such a tiny change. It provides a 'bottom bound

Re: Parallel heap vacuum

2024-12-05 Thread Peter Smith
Hi Sawada-San, I started to look at patch v4-0001 in this thread. It is quite a big patch so this is a WIP, and these below are just the comments I have so far. == src/backend/access/heap/vacuumlazy.c 1.1. +/* + * Relation statistics collected during heap scanning and need to be shared amon

Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

2024-12-05 Thread Thomas Munro
Some better new: 1. _chsize_s does in fact seem to work in an msvcrt.dll build. Thanks to Andres for testing that for me on Windows with a standalone 2 line program on ucrt and msvcrt runtimes. 2. MinGW always has _chsize_s, but it does more or less what I had been proposing already if it can't

Re: Use streaming read API in pgstattuple.

2024-12-05 Thread Kirill Reshke
On Fri, 29 Nov 2024 at 22:05, Matheus Alcantara wrote: > Just for reference; On pg_prewarm() for example this loop is > implemented as: > > for (block = first_block; block <= last_block; ++block) > { > Buffer buf; > ... > buf = read_stream_next_buffer(stream, NULL); >

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-12-05 Thread Shlok Kyal
On Fri, 6 Dec 2024 at 02:44, Peter Smith wrote: > > On Thu, Dec 5, 2024 at 8:49 PM Amit Kapila wrote: > > > > On Thu, Dec 5, 2024 at 9:32 AM Peter Smith wrote: > > > > > > On Thu, Dec 5, 2024 at 2:41 PM Amit Kapila > > > wrote: > > > > > > > > On Thu, Dec 5, 2024 at 7:34 AM Peter Smith > > >

Re: Introduce XID age and inactive timeout based replication slot invalidation

2024-12-05 Thread vignesh C
On Thu, 5 Dec 2024 at 06:44, Peter Smith wrote: > > On Wed, Dec 4, 2024 at 9:27 PM vignesh C wrote: > > > ... > > > > Currently, replication slots are invalidated based on the > > replication_slot_inactive_timeout only during a checkpoint. This means > > that if the checkpoint_timeout is set to a

Re: Why we need to check for local buffers in BufferIsExclusiveLocked and BufferIsDirty?

2024-12-05 Thread Srinath Reddy Sadipiralla
> On Thu, 05 Dec 2024 21:11:42 +0530 Andres Freund > wrote --- > Hi, > On 2024-12-05 18:38:16 +0530, Srinath Reddy Sadipiralla wrote: >> Why we need to check for local buffers in BufferIsExclusiveLocked and >> BufferIsDirty?,these 2 functions are called onl

Re: Remove dependence on integer wrapping

2024-12-05 Thread Nathan Bossart
On Thu, Dec 05, 2024 at 08:00:12PM -0500, Joseph Koshakow wrote: > On Thu, Dec 5, 2024 at 5:50 PM Nathan Bossart > wrote: >> Good point. Here is a v27 patch that extracts the bug fix portions of the >> v26 patch. If/when this is committed, I think we should close the >> commitfest entry. > > I

Re: Sort functions with specialized comparators

2024-12-05 Thread John Naylor
On Fri, Dec 6, 2024 at 1:32 AM Andrey M. Borodin wrote: > > > On 5 Dec 2024, at 15:16, John Naylor wrote: > > I believe src/port/qsort.c was meant to be just for the standard sort > > interface as found in a C library. We do have one globally visible > > special sort here: > > src/backend/utils/

Re: MinGW compiler warnings in ecpg tests

2024-12-05 Thread Thomas Munro
On Fri, Dec 6, 2024 at 4:13 AM Tom Lane wrote: > Thomas Munro writes: > > Yeah. This warning is visible on CI, and on fairywren since its MSYS2 > > upgrade a couple of months ago. Old MinGW didn't like %lld (I think > > perhaps the printf from msvcrt.dll from 1996 didn't like it and MinGW > > k

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread jian he
per above discussion: for TupleDescInitEntry, TupleDescInitBuiltinEntry, I didn't change the signature of it. but i added: if (attdim != 0) elog(ERROR, "attdim should be zero"); otherwise it may have "unused parameter" warnings? for the same reason in TypeCreate, I added: if (typ

Re: Assert failure on running a completed portal again

2024-12-05 Thread Tom Lane
I wrote: > Don't like this much. I'm not sure I believe any part of this > approach to deciding whether the portal is "run once". In any > case, a proper fix for this needs to include actually documenting > what that field means and does. After looking at this further, I think this whole "run_on

Re: Remove dependence on integer wrapping

2024-12-05 Thread Joseph Koshakow
On Thu, Dec 5, 2024 at 5:50 PM Nathan Bossart wrote: > Good point. Here is a v27 patch that extracts the bug fix portions of the > v26 patch. If/when this is committed, I think we should close the > commitfest entry. I looked through this patch and it looks good to me, assuming cfbot is happy.

Re: 64 bit numbers vs format strings

2024-12-05 Thread Thomas Munro
On Fri, Dec 6, 2024 at 1:15 PM Melanie Plageman wrote: > So, will this fix the issue that when I do: > > uint64 somevar = 0; > errmsg("hello %lu", somevar) > > locally on my x86-64 linux machine running ubuntu with whatever gcc-11 > comes out of the box, it compiles sans warnings, but in the > min

Re: Count and log pages set all-frozen by vacuum

2024-12-05 Thread Melanie Plageman
On Mon, Dec 2, 2024 at 9:28 AM Robert Haas wrote: > > On Fri, Nov 29, 2024 at 2:46 PM Masahiko Sawada wrote: > > Finally, in case where we have: > > > > visibility map: 1 pages set all-visible, 1 pages set all-frozen. > > > > We can understand that 1 pages newly became all-frozen, but

Re: 64 bit numbers vs format strings

2024-12-05 Thread Melanie Plageman
On Thu, Dec 5, 2024 at 5:12 PM Thomas Munro wrote: > > Having learned some things about gettext based on clues[1] from Peter > E, I decided to see what it would take to expunge all (long long) and > similar casts now that we're using the standard types with system > support. > > The short version

Re: Assert failure on running a completed portal again

2024-12-05 Thread Tom Lane
Yugo Nagata writes: > I notice that the following Assert in PortalRun fails when a same portal is > executed more than once by an Execute message whose "max number of rows" > is specified to zero, that is, "no limit". > /* Set run_once flag. Shouldn't be clear if previously set. */ >

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Tom Lane
Bruce Momjian writes: > How about if we set attndims & typndims to zero in PG 18, and mention > that these fields are deprecated in the release notes. Then, in a few > years, we can remove the columns since interfaces hopefully would have > removed the useless references to the columns. You have

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Bruce Momjian
On Thu, Dec 5, 2024 at 12:03:30PM -0500, Bruce Momjian wrote: > On Thu, Dec 5, 2024 at 12:00:30PM -0500, Tom Lane wrote: > > Bruce Momjian writes: > > > On Thu, Dec 5, 2024 at 11:33:22AM -0500, Tom Lane wrote: > > >> I don't think we can do this as presented. Maybe we could do > > >> something

Re: Remove dependence on integer wrapping

2024-12-05 Thread Nathan Bossart
On Sat, Aug 24, 2024 at 08:44:40AM -0400, Joseph Koshakow wrote: > FWIW, Matthew's patch actually does resolve a bug with `to_timestamp` > and `to_date`. It converts the following incorrect queries > > test=# SELECT to_timestamp('2147483647,999', 'Y,YYY'); > to_timestamp > --

Re: Parallel heap vacuum

2024-12-05 Thread Peter Smith
Hi Sawada-San, FYI, the patch 0001 fails to build stand-alone vacuumlazy.c: In function ‘parallel_heap_vacuum_gather_scan_stats’: vacuumlazy.c:3739:21: error: ‘LVRelScanStats’ has no member named ‘vacuumed_pages’ vacrel->scan_stats->vacuumed_pages += ss->vacuumed_pages; ^

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Jelte Fennema-Nio
On Thu, 5 Dec 2024 at 16:35, Eric Hanson wrote: > When pgbouncer is in transaction mode, the pipeline doesn't stop when > the transaction ends? Mayhaps I have the common misunderstanding. When PgBouncer is in transaction mode, the server connection will only be unlinked, when PgBouncer receives

64 bit numbers vs format strings

2024-12-05 Thread Thomas Munro
Hi, Having learned some things about gettext based on clues[1] from Peter E, I decided to see what it would take to expunge all (long long) and similar casts now that we're using the standard types with system support. The short version is tha given uint64 x: Old: errmsg("hello %llu", (unsig

Re: Proposal to add a new URL data type.

2024-12-05 Thread Ziga
On 05/12/2024 15:59, Peter Eisentraut wrote: On 05.12.24 15:01, Alexander Borisov wrote: Postgres users often store URLs in the database.  As an example, they provide links to their pages on the web, analyze users posts and get links for further storage and analysis.  Naturally, there is a need

Re: Assert failure on running a completed portal again

2024-12-05 Thread Yugo Nagata
On Fri, 6 Dec 2024 06:25:49 +0900 Yugo Nagata wrote: > Hi, > > I notice that the following Assert in PortalRun fails when a same portal is > executed more than once by an Execute message whose "max number of rows" > is specified to zero, that is, "no limit". > > /* Set run_once flag.

Assert failure on running a completed portal again

2024-12-05 Thread Yugo Nagata
Hi, I notice that the following Assert in PortalRun fails when a same portal is executed more than once by an Execute message whose "max number of rows" is specified to zero, that is, "no limit". /* Set run_once flag. Shouldn't be clear if previously set. */ Assert(!portal->run

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-12-05 Thread Peter Smith
On Thu, Dec 5, 2024 at 8:49 PM Amit Kapila wrote: > > On Thu, Dec 5, 2024 at 9:32 AM Peter Smith wrote: > > > > On Thu, Dec 5, 2024 at 2:41 PM Amit Kapila wrote: > > > > > > On Thu, Dec 5, 2024 at 7:34 AM Peter Smith wrote: > > > > > > > > IIUC, these errors are intended for when there is *any*

Re: deferred writing of two-phase state files adds fragility

2024-12-05 Thread Andres Freund
Hi, On 2024-12-05 11:21:12 -0500, Robert Haas wrote: > On Wed, Dec 4, 2024 at 6:36 PM Andres Freund wrote: > > How did this corruption actually come about? Did it actually really just > > affect that single WAL segment? Somehow that doesn't seem too likely. > > I don't know and might not be abl

Re: Using Expanded Objects other than Arrays from plpgsql

2024-12-05 Thread Tom Lane
Michel Pelletier writes: > Here's a WIP patch for a pgexpanded example in src/test/modules. The > object is very simple and starts with an integer and increments that value > every time it is expanded. I added some regression tests that test two sql > functions that replicate the expansion issue

Re: [Bug] Heap Use After Free in Window Aggregate Execution

2024-12-05 Thread David Rowley
On Fri, 6 Dec 2024 at 06:08, Alvaro Herrera wrote: > David, what do you think about this? The code in question was added by > your commits 9d9c02ccd1ae and a8583272218a, so I think you're the one > best prepared to handle this. Thanks for alerting me to this. I missed the original report. I wil

Re: meson missing test dependencies

2024-12-05 Thread Andres Freund
Hi, On 2024-12-05 20:08:24 +0100, Peter Eisentraut wrote: > On 03.12.24 17:01, Andres Freund wrote: > > On 2024-12-02 11:10:56 +0100, Peter Eisentraut wrote: > > That's unfortunately a very partial fix - because we insist on tests being > > run > > against a temporary install, we don't just need

Re: meson missing test dependencies

2024-12-05 Thread Peter Eisentraut
On 03.12.24 17:01, Andres Freund wrote: Hi, On 2024-12-02 11:10:56 +0100, Peter Eisentraut wrote: I have noticed that under meson many tests don't have dependencies on the build artifacts that they are testing. As an example among many, if you make a source code change in contrib/cube/cube.c (

Re: Sort functions with specialized comparators

2024-12-05 Thread Andrey M. Borodin
> On 5 Dec 2024, at 15:16, John Naylor wrote: > > I tried on an older Intel chip and got similar results, so we'll go > with your original comparator: Ack. > I believe src/port/qsort.c was meant to be just for the standard sort > interface as found in a C library. We do have one globally visi

Re: postgres_fdw: Provide better emulation of READ COMMITTED behavior

2024-12-05 Thread Robert Haas
On Thu, Dec 5, 2024 at 4:41 AM Etsuro Fujita wrote: > Comments welcome! Maybe I am missing something, though. I have a hard time seeing how this would work if cursors are in use on the main server. Say I do this: DECLARE foo CURSOR FOR SELECT * FROM ft1 UNION ALL SELECT * FROM ft2; ...fetch som

Re: Potential null pointer dereference in postgres.c

2024-12-05 Thread Tom Lane
Karina Litskevich writes: > When backporting 66e94448 to older versions it was forgotten to check > malloc() result. In 16+ versions guc_malloc() is used to allocate > memory and it checks if the result pointer is NULL, so there is no > need to check it after guc_malloc(). Versions before 16 have

Re: [Bug] Heap Use After Free in Window Aggregate Execution

2024-12-05 Thread Alvaro Herrera
Hello, Adding David Rowley in CC. On 2024-Nov-13, Jayesh Dehankar wrote: > We have discovered a bug in PostgreSQL v16.3 related to a top-level > window aggregate with a partition-by clause. The issue occurs when the > run condition fails, causing the window aggregate status to change > from WIND

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Bruce Momjian
On Thu, Dec 5, 2024 at 12:00:30PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > On Thu, Dec 5, 2024 at 11:33:22AM -0500, Tom Lane wrote: > >> I don't think we can do this as presented. Maybe we could do > >> something like constrain the stored value to be only 0 or 1, > >> but how much doe

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Tom Lane
Bruce Momjian writes: > On Thu, Dec 5, 2024 at 11:33:22AM -0500, Tom Lane wrote: >> I don't think we can do this as presented. Maybe we could do >> something like constrain the stored value to be only 0 or 1, >> but how much does that improve matters? > Well, then the question becomes will we r

Potential null pointer dereference in postgres.c

2024-12-05 Thread Karina Litskevich
Hi hackers, When backporting 66e94448 to older versions it was forgotten to check malloc() result. In 16+ versions guc_malloc() is used to allocate memory and it checks if the result pointer is NULL, so there is no need to check it after guc_malloc(). Versions before 16 have no guc_malloc(), and m

Re: RFC/PoC: GUC option to enable tuple queue autoflush for parallel workers

2024-12-05 Thread Francesco Degrassi
Hello, I hope bumping up this is not frowned upon. Any chance we can get any feedback? Thanks and best regards Francesco On Thu, 26 Sept 2024 at 16:15, Francesco Degrassi wrote: > > Hi all. A brief overview of our use case follows. > > We are developing a foreign data wrapper which employs para

Re: Possible integer overflow in bringetbitmap()

2024-12-05 Thread James Hunter
On Wed, Dec 4, 2024 at 10:13 PM Michael Paquier wrote: > > totalpages is signed, and BlockNumber is unsigned. Hence in theory > you could always fall into a trap once totalpages gets higher than > (2^31 - 1), no? This is not going to be a problem in practice even if > the number of pages per ran

Re: code contributions for 2024, WIP version

2024-12-05 Thread Robert Haas
On Thu, Dec 5, 2024 at 11:19 AM Alvaro Herrera wrote: > Just give credit to all lines for all authors, would be my approach. Is > that unfair? Perhaps, but I'd rather err on the side of giving too much > credit, than on not giving enough. I'm not against somebody putting that together, but I do

Re: code contributions for 2024, WIP version

2024-12-05 Thread Bruce Momjian
On Thu, Dec 5, 2024 at 10:39:38AM -0500, Tom Kincaid wrote: > While I know you said "you will do you" when it comes to your annual > blog, there are a number of similar efforts -- top of mind is the > analysis done (as I understand it) by Daniel Gustafsson and Claire > Giordano [1]

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Bruce Momjian
On Thu, Dec 5, 2024 at 11:33:22AM -0500, Tom Lane wrote: > Alvaro Herrera writes: > > On 2024-Dec-05, Kirill Reshke wrote: > >> I see no open items in this thread. Are there any issues about v1? > > > Should we leave TupleDescInitEntry()'s API alone, to avoid breaking the > > compilation of ever

Re: Windows pg_basebackup unable to create >2GB pg_wal.tar tarballs ("could not close file: Invalid argument" when creating pg_wal.tar of size ~ 2^31 bytes)

2024-12-05 Thread Robert Haas
On Wed, Dec 4, 2024 at 11:23 PM Thomas Munro wrote: > Wow, what a lot of variations we have to handle with no coverage. I'm > coming around to your proposal Robert. We decide that it is OK to > back-patch freely, under a policy along the lines "we try to keep > stable branches working on the lat

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Tom Lane
Alvaro Herrera writes: > On 2024-Dec-05, Kirill Reshke wrote: >> I see no open items in this thread. Are there any issues about v1? > Should we leave TupleDescInitEntry()'s API alone, to avoid breaking the > compilation of every extension in the world? I fear the howls of pain from extension aut

Re: [PERF] Improve Cardinality Estimation for Joins with GROUP BY Having Single Clause

2024-12-05 Thread Ravi
This issue occurs when the Aggregate node has more rows than the other node in the Join. The join selectivity is determined by the jselectivity factor, which is calculated based on the maximum number of distinct rows between the two nodes, as defined in the function eqjoinsel_inner. However, sin

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Jacob Champion
On Thu, Dec 5, 2024 at 12:47 AM Wolfgang Walther wrote: > > If we want something like this, we'd want to allow > > users to re-trigger SCRAM authentication. Which clearly requires a > > protocol change. > > Yes. This. Re-authenticating without re-connecting. The ability to reauthenticate would be

Re: deferred writing of two-phase state files adds fragility

2024-12-05 Thread Robert Haas
On Wed, Dec 4, 2024 at 6:36 PM Andres Freund wrote: > Is 2PC really that special in that regard? If the WAL that contains the > checkpoint record itself gets corrupted, you're also in a world of hurt, once > you shut down? Or, to a slightly lower degree, if there's any corrupted > record between

Re: code contributions for 2024, WIP version

2024-12-05 Thread Alvaro Herrera
On 2024-Dec-05, Robert Haas wrote: > On Thu, Dec 5, 2024 at 7:46 AM Alvaro Herrera wrote: > > No corrections here -- I noticed nothing wrong with the commits I am > > involved with, in a quick read. I did notice that for patches with > > multiple authors, only the first one is listed. For insta

Vacuum scans all-frozen pages with checksums enabled

2024-12-05 Thread Melanie Plageman
Hi, I investigated what I thought was an issue with my patch to amortize aggressive vacuum [1] and found some interesting behavior also present on master. It seems vacuum will scan many all-frozen pages due to an interaction between the opportunistic freeze heuristic and SKIP_PAGES_THRESHOLD. This

Re: Why we need to check for local buffers in BufferIsExclusiveLocked and BufferIsDirty?

2024-12-05 Thread Andres Freund
Hi, On 2024-12-05 18:38:16 +0530, Srinath Reddy Sadipiralla wrote: > Why we need to check for local buffers in BufferIsExclusiveLocked and > BufferIsDirty?,these 2 functions are called only from > XlogRegisterBuffer,AFAIK which will be called only for permanent > relations.Please correct me if i a

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-05 Thread Euler Taveira
On Thu, Dec 5, 2024, at 1:31 AM, Zhijie Hou (Fujitsu) wrote: > No problem. Here is the patch for the HEAD. This patch introduces a new memory > context within PGOutputData, specifically for allocating memory for > publication_names. The new memory context is nested under the logical decoding > cont

Re: code contributions for 2024, WIP version

2024-12-05 Thread Tom Kincaid
> > > While I know you said "you will do you" when it comes to your annual > blog, there are a number of similar efforts -- top of mind is the > analysis done (as I understand it) by Daniel Gustafsson and Claire > Giordano [1], as well as ongoing/recurring analysis done by the > contributor committ

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Eric Hanson
On Wed, Dec 4, 2024 at 4:54 PM Jelte Fennema-Nio wrote: > > > I wouldn't oppose a command that does an absolutely irrevocable SET > > ROLE -- i.e. once you execute it, it is as if you logged in as the > > target role originally, and the only way to get your privileges back > > is a new connection.

Re: code contributions for 2024, WIP version

2024-12-05 Thread Robert Haas
On Thu, Dec 5, 2024 at 7:46 AM Alvaro Herrera wrote: > No corrections here -- I noticed nothing wrong with the commits I am > involved with, in a quick read. I did notice that for patches with > multiple authors, only the first one is listed. For instance, > 53c2a97a926's author ("Improve perfor

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Robert Haas
On Wed, Dec 4, 2024 at 5:54 PM Jelte Fennema-Nio wrote: > On Wed, 4 Dec 2024 at 22:05, Robert Haas wrote: > > I do think the protocol change is better. > > Fully agreed. But I now also know the herculean amount of effort > that's necessary for that to happen, and I personally don't have the > ban

Re: doc: Mention clock synchronization recommendation for hot_standby_feedback

2024-12-05 Thread Andres Freund
On 2024-12-05 12:06:41 -0300, Euler Taveira wrote: > Is it worth a WARNING message if there is a "huge" time difference > between the servers? We already have the reply time in the message so > it is a matter of defining the "huge" interval plus a roundtrip. We also > need to avoid spamming the log

Re: MinGW compiler warnings in ecpg tests

2024-12-05 Thread Tom Lane
Thomas Munro writes: > Yeah. This warning is visible on CI, and on fairywren since its MSYS2 > upgrade a couple of months ago. Old MinGW didn't like %lld (I think > perhaps the printf from msvcrt.dll from 1996 didn't like it and MinGW > knew that), but new MinGW doesn't like %I64d (that's intere

Re: doc: Mention clock synchronization recommendation for hot_standby_feedback

2024-12-05 Thread Euler Taveira
On Thu, Dec 5, 2024, at 6:43 AM, Jakub Wartak wrote: > One of our customers ran into a very odd case, where hot standby feedback > backend_xmin propagation stopped working due to major (hours/days) clock time > shifts on hypervisor-managed VMs. This happens (and is fully reproducible) > e.g. in

Re: Proposal to add a new URL data type.

2024-12-05 Thread Peter Eisentraut
On 05.12.24 15:01, Alexander Borisov wrote: Postgres users often store URLs in the database.  As an example, they provide links to their pages on the web, analyze users posts and get links for further storage and analysis.  Naturally, there is a need to compare, sort, build indexes, get statistic

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Alvaro Herrera
On 2024-Dec-05, Kirill Reshke wrote: > On Fri, 20 Sept 2024 at 07:11, jian he wrote: > > attached patch removes attndims and typndims entirely. > > LGTM? > I see no open items in this thread. Are there any issues about v1? Should we leave TupleDescInitEntry()'s API alone, to avoid breaking the

Re: EphemeralNamedRelation and materialized view

2024-12-05 Thread Yugo NAGATA
On Wed, 20 Nov 2024 12:43:16 -0500 Tom Lane wrote: > Yugo NAGATA writes: > >> You could even argue that case 2 isn't good enough either, > >> and we should be delivering a specific error message saying > >> that an ENR can't be used in a view/matview. To do that, > >> we'd likely need to pass d

Re: generic plans and "initial" pruning

2024-12-05 Thread Tomas Vondra
On 12/5/24 12:28, Amit Langote wrote: > On Thu, Dec 5, 2024 at 3:53 PM Amit Langote wrote: >> On Thu, Dec 5, 2024 at 2:20 AM Tomas Vondra wrote: >>> Sure, changing the APIs is allowed, I'm just wondering if maybe there >>> might be a way to not have this issue, or at least notice the missing >

Re: shared-memory based stats collector - v70

2024-12-05 Thread Bertrand Drouvot
Hi, On Thu, Dec 05, 2024 at 05:13:13PM +0900, Michael Paquier wrote: > On Thu, Dec 05, 2024 at 07:37:27AM +, Bertrand Drouvot wrote: > > That said, I think that's worth to update the comment a bit (like in the > > attached?) as I think that answers a legitimate question someone could have > >

Re: generic plans and "initial" pruning

2024-12-05 Thread Tomas Vondra
On 12/5/24 07:53, Amit Langote wrote: > On Thu, Dec 5, 2024 at 2:20 AM Tomas Vondra wrote: >> ... >> What if an extension doesn't do that? What weirdness will happen? >>> >>> The QueryDesc.planstate won't contain a PlanState tree for starters >>> and other state information that InitP

Re: code contributions for 2024, WIP version

2024-12-05 Thread Andrey M. Borodin
> On 5 Dec 2024, at 17:46, Alvaro Herrera wrote: > > We have plenty of patches with > multiple authors, after all. +1, thanks for raising this. A lot of stuff is actually joint work. It’s much more fun to develop something in a group of co-authors. Best regards, Andrey Borodin.

Re: attndims, typndims still not enforced, but make the value within a sane threshold

2024-12-05 Thread Kirill Reshke
On Fri, 20 Sept 2024 at 07:11, jian he wrote: > attached patch removes attndims and typndims entirely. LGTM? I see no open items in this thread. Are there any issues about v1? -- Best regards, Kirill Reshke

Re: Pass ParseState as down to utility functions.

2024-12-05 Thread Kirill Reshke
Looks like v4 fails on windows, PFA v5. Sorry for the noise, I hope Cirrus CI will like this version. -- Best regards, Kirill Reshke v5-0001-print-out-error-position-for-some-DDL-command.patch Description: Binary data

Why we need to check for local buffers in BufferIsExclusiveLocked and BufferIsDirty?

2024-12-05 Thread Srinath Reddy Sadipiralla
Hi, Why we need to check for local buffers in BufferIsExclusiveLocked and BufferIsDirty?,these 2 functions are called only from XlogRegisterBuffer,AFAIK which will be called only for permanent relations.Please correct me if i am wrong. Regards, Srinath Reddy Sadipiralla Member of Technical Sta

Re: code contributions for 2024, WIP version

2024-12-05 Thread Alvaro Herrera
Hello Robert, On 2024-Dec-02, Robert Haas wrote: > As many of you are probably aware, I have been doing an annual blog > post on who contributes to PostgreSQL development for some years now. > It includes information on lines of code committed to PostgreSQL, and > also emails sent to the list. Th

Re: CREATE TABLE NOT VALID for check and foreign key

2024-12-05 Thread jian he
turns out this has already been discussed at [1] NOT VALID only applies to table_constraint. Amit Langote actually wrote the doc at[1], so I copied some text from there. in create_table.sgml, i placed right below INITIALLY IMMEDIATE INITIALLY DEFERRED so it looks like this: NOT VALID

Re: Pass ParseState as down to utility functions.

2024-12-05 Thread Kirill Reshke
On Thu, 5 Dec 2024 at 11:45, Michael Paquier wrote: > > On Wed, Dec 04, 2024 at 03:31:47PM +0500, Kirill Reshke wrote: > > On Sat, 30 Nov 2024 at 17:37, jian he wrote: > >> So I guess bundling it into a single patch should be fine? > > > > Ok. I created CF entry for this patch. > > > > [0] https:

Re: MinGW compiler warnings in ecpg tests

2024-12-05 Thread Thomas Munro
On Fri, Nov 10, 2023 at 9:00 PM Hayato Kuroda (Fujitsu) wrote: > Sorry for reviving the old thread. While trying to build postgres on msys2 by > meson, > I faced the same warning. The OS is Windows 10. Yeah. This warning is visible on CI, and on fairywren since its MSYS2 upgrade a couple of mon

Re: CREATE TABLE NOT VALID for check and foreign key

2024-12-05 Thread Kirill Reshke
On Thu, 5 Dec 2024 at 14:36, Alvaro Herrera wrote: > Maybe it would have been wise to forbid NOT VALID when used with CREATE > TABLE. But we didn't. Should we do that now? Maybe we can just > document that you can specify it but it doesn't do anything. +1 on that -- Best regards, Kirill Reshk

Re: generic plans and "initial" pruning

2024-12-05 Thread Amit Langote
On Thu, Dec 5, 2024 at 3:53 PM Amit Langote wrote: > On Thu, Dec 5, 2024 at 2:20 AM Tomas Vondra wrote: > > Sure, changing the APIs is allowed, I'm just wondering if maybe there > > might be a way to not have this issue, or at least notice the missing > > call early. > > > > I haven't tried, woul

CREATE TABLE NOT VALID for check and foreign key

2024-12-05 Thread jian he
hi. I found for foreign keys, check constraints, you specify it as NOT VALID, it will not be marked as NOT VALID in the CREATE TABLE statement. CREATE TABLE s6(id bigint , CONSTRAINT con1 check(id > 1) not valid); src2=# \d s6 Table "public.s6" Column | Type | Collation | Nullab

Re: NOT ENFORCED constraint feature

2024-12-05 Thread Alvaro Herrera
On 2024-Dec-03, Peter Eisentraut wrote: > The handling of merging check constraints seems incomplete. What > should be the behavior of this: > > => create table p1 (a int check (a > 0) not enforced); > CREATE TABLE > => create table c1 (a int check (a > 0) enforced) inherits (p1); > CREATE TABLE

Re: SQL Property Graph Queries (SQL/PGQ)

2024-12-05 Thread Ashutosh Bapat
On Fri, Nov 22, 2024 at 7:29 PM Ashutosh Bapat wrote: > > On Tue, Nov 19, 2024 at 10:08 PM Vik Fearing wrote: > > > > > > On 05/11/2024 16:41, Ashutosh Bapat wrote: > > > > On Wed, Aug 28, 2024 at 3:48 PM Ashutosh Bapat > > wrote: > > > > Patches 0001 - 0006 are same as the previous set. > > 000

Re: RFC: Additional Directory for Extensions

2024-12-05 Thread Peter Eisentraut
On 11.11.24 08:16, Peter Eisentraut wrote: I implemented a patch along the lines Craig had suggested.  It's a new GUC variable that is a path for extension control files.  It's called extension_control_path, and it works exactly the same way as dynamic_library_path.  Except that the magic token

Re: Track the amount of time waiting due to cost_delay

2024-12-05 Thread Bertrand Drouvot
Hi, On Thu, Dec 05, 2024 at 05:51:11PM +0900, Masahiro Ikeda wrote: > Hi, > > I recently encountered a case where having this feature would have been very > helpful. Oh great, thanks for the feedback! > Thank you for developing it! I have a few questions and comments. > > Here are questions: >

Re: CSN snapshots in hot standby

2024-12-05 Thread John Naylor
On Tue, Dec 3, 2024 at 9:25 PM Heikki Linnakangas wrote: > > On 20/11/2024 15:33, John Naylor wrote: > I did find one weird thing that makes a big difference: I originally > used AllocSetContextCreate(..., ALLOCSET_DEFAULT_SIZES) for the radix > tree's memory context. With that, XidInMVCCSnapshot(

Re: Sort functions with specialized comparators

2024-12-05 Thread John Naylor
On Wed, Dec 4, 2024 at 2:47 PM Andrey M. Borodin wrote: > sort_int32_cmp Time: 543.690 ms > sort_int32_cmp_2 Time: 609.019 ms > sort_int32_cmp_4 Time: 612.219 ms > > So, I'd stick with sort_int32_cmp. But, perhaps, on Intel we might have > different results. I tried on an older Intel chip and go

Re: Conflict detection for update_deleted in logical replication

2024-12-05 Thread Amit Kapila
On Wed, Dec 4, 2024 at 4:29 PM Amit Kapila wrote: > > 1. > + if (can_advance_nonremovable_xid(&data, last_recv_timestamp)) > + maybe_advance_nonremovable_xid(&data); > > In can_advance_nonremovable_xid(), we determine whether to advance the > oldest xid based on 'last_recv_timestamp' and then agai

Re: Disallow UPDATE/DELETE on table with unpublished generated column as REPLICA IDENTITY

2024-12-05 Thread Amit Kapila
On Thu, Dec 5, 2024 at 9:32 AM Peter Smith wrote: > > On Thu, Dec 5, 2024 at 2:41 PM Amit Kapila wrote: > > > > On Thu, Dec 5, 2024 at 7:34 AM Peter Smith wrote: > > > > > > IIUC, these errors are intended for when there is *any* unpublished > > > generated column found in the RI, and the RI mig

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Jelte Fennema-Nio
On Thu, 5 Dec 2024 at 09:47, Wolfgang Walther wrote: > Right, I should have clarified: My proposal wasn't mean to be taken > literally as an SQL command. Passwords should not be sent as plain text, > no question. This needs to happen on the protocol level. Thanks for clarifying. > I don't want t

doc: Mention clock synchronization recommendation for hot_standby_feedback

2024-12-05 Thread Jakub Wartak
One of our customers ran into a very odd case, where hot standby feedback backend_xmin propagation stopped working due to major (hours/days) clock time shifts on hypervisor-managed VMs. This happens (and is fully reproducible) e.g. in scenarios where standby connects and its own VM is having time f

postgres_fdw: Provide better emulation of READ COMMITTED behavior

2024-12-05 Thread Etsuro Fujita
Hi, postgres_fdw uses REPEATABLE READ isolation level for the remote transaction when the local transaction has READ COMMITTED isolation level, for the reason described in the comments for begin_remote_xact() in connection.c: /* * Start remote transaction or subtransaction, if needed. * * Note

Re: SQL:2011 application time

2024-12-05 Thread vignesh C
On Wed, 4 Dec 2024 at 16:45, Peter Eisentraut wrote: > > On 26.11.24 13:18, Peter Eisentraut wrote: > > I think this is the right idea, but after digging around a bit more, I > > think more could/should be done. > > > > After these changes, the difference between > > get_equal_strategy_number_for_

Re: CREATE TABLE NOT VALID for check and foreign key

2024-12-05 Thread Alvaro Herrera
Hello, On 2024-Dec-05, jian he wrote: > I found for foreign keys, check constraints, > you specify it as NOT VALID, it will not be marked as NOT VALID in the > CREATE TABLE statement. Uhmm, okay. > reading transformCheckConstraints, transformFKConstraints comments > appearingly this is intentio

Re: Memory leak in WAL sender with pgoutput (v10~)

2024-12-05 Thread Masahiko Sawada
On Thu, Dec 5, 2024 at 1:32 PM Zhijie Hou (Fujitsu) wrote: > > On Wednesday, December 4, 2024 7:39 PM Michael Paquier > wrote: > > > > On Wed, Dec 04, 2024 at 06:42:55AM +, Zhijie Hou (Fujitsu) wrote: > > > I can try to write a patch if no one else is working on this. > > > > If you have som

Re: Make pg_stat_io view count IOs as bytes instead of blocks

2024-12-05 Thread Bertrand Drouvot
Hi, On Wed, Dec 04, 2024 at 02:49:11PM +0300, Nazir Bilal Yavuz wrote: > On Thu, 28 Nov 2024 at 16:39, Bertrand Drouvot > wrote: > > > You are right, no need to have this check; it can not be less than 0. > I completely removed the function now. Thanks! Yeah I think that makes sense. > > What

Re: Track the amount of time waiting due to cost_delay

2024-12-05 Thread Masahiro Ikeda
Hi, I recently encountered a case where having this feature would have been very helpful. Thank you for developing it! I have a few questions and comments. Here are questions: After this patch is merged, are you considering adding delayed_time information to the logs output by log_autovacuum_

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Wolfgang Walther
Jelte Fennema-Nio: I am extremely skeptical of something like SET ROLE WITH . Totally agreed on the security concerns here. We don't want to provide passwords in a SQL command. For the same reasons explained by Robert, we also tell people not to set user passwords using SQL, but to use the \pas

Re: Proposal: Role Sandboxing for Secure Impersonation

2024-12-05 Thread Wolfgang Walther
Michał Kłeczek: PostgREST does not know alice's password as it performs JWT based authentication. Yes, that's why I proposed an extension to support JWTs in the next sentence. Then PostgREST would not need to do any auth at all anymore. Best, Wolfgang

[BUG] Assert always fails while checking whether local buffer is dirty of is exclusively locked

2024-12-05 Thread Daniil Davydov
Hi, Commit 00d7fb5e2e39198387ae00af8dd18b787b6a4d63 adds BufferIsExclusiveLocked and BufferIsDirty functions. They take into account that they can also be called for temporary tables, but they both have assert, that will always fail for temporary tables : *** Assert(LWLockHeldByMeInMode(BufferDescr

Re: shared-memory based stats collector - v70

2024-12-05 Thread Michael Paquier
On Thu, Dec 05, 2024 at 07:37:27AM +, Bertrand Drouvot wrote: > That said, I think that's worth to update the comment a bit (like in the > attached?) as I think that answers a legitimate question someone could have > while > reading this code. > > - /* we may have some "dropped"

  1   2   >