Re: testing dist tarballs

2023-05-26 Thread Peter Eisentraut
On 24.05.23 23:24, Andres Freund wrote: First thing I noticed that 'make dist' doesn't work in a vpath, failing in a somewhat obscure way (likely because in a vpath build the the copy from the source dir doesn't include GNUMakefile). Do we expect it to work? I don't think so. Separately, it's

BF animal dikkop reported a failure in 035_standby_logical_decoding

2023-05-26 Thread Yu Shi (Fujitsu)
Hi hackers, I saw a buildfarm failure on "dikkop"[1]. It failed in 035_standby_logical_decoding.pl, because the slots row_removal_inactiveslot and row_removal_activeslot are not invalidated after vacuum. regress_log_035_standby_logical_decoding: ``` [12:15:05.943](4.442s) not ok 22 - inactiveslot

Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

2023-05-26 Thread Peter Smith
On Thu, May 25, 2023 at 6:59 PM Melih Mutlu wrote: > > Hi, > > > Peter Smith , 24 May 2023 Çar, 05:59 tarihinde şunu > yazdı: >> >> Hi, and thanks for the patch! It is an interesting idea. >> >> I have not yet fully read this thread, so below are only my first >> impressions after looking at patc

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Michael Paquier
On Thu, May 25, 2023 at 06:50:31PM -0700, Peter Geoghegan wrote: > It's possible that Bertand would have done it this way to begin with > were it not for the admittedly pretty bad nbtree convention around > P_NEW. It would be nice to get rid of P_NEW in the near future, too -- > I gather that there

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2023-05-26 Thread Daniel Gustafsson
> On 26 May 2023, at 04:08, Michael Paquier wrote: > On Thu, May 25, 2023 at 10:16:27AM +0200, Daniel Gustafsson wrote: >> The changes to the Windows buildsystem worry me a bit, but they don't move >> the >> goalposts in either direction wrt to LibreSSL on Windows so for the purpose >> of >> th

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Alvaro Herrera
On 2023-May-25, Peter Geoghegan wrote: > Attached patch completely removes the changes to _bt_getbuf()'s > signature from 61b313e4. I suppose you're not thinking of applying this to current master, but instead just leave it for when pg17 opens, right? I mean, clearly it seems far too invasive to

pg_get_indexdef() modification to use TxnSnapshot

2023-05-26 Thread shveta malik
I have attempted to convert pg_get_indexdef() to use systable_beginscan() based on transaction-snapshot rather than using SearchSysCache(). The latter does not have any old info and thus provides only the latest info as per the committed txns, which could result in errors in some scenarios. One suc

Re: pg_get_indexdef() modification to use TxnSnapshot

2023-05-26 Thread vignesh C
On Fri, 26 May 2023 at 15:25, shveta malik wrote: > > I have attempted to convert pg_get_indexdef() to use > systable_beginscan() based on transaction-snapshot rather than using > SearchSysCache(). The latter does not have any old info and thus > provides only the latest info as per the committed

Re: PG 16 draft release notes ready

2023-05-26 Thread Alvaro Herrera
On 2023-May-25, Laurenz Albe wrote: > @@ -1335,7 +1335,7 @@ Author: Peter Eisentraut > > > > -Add Windows process the system collations (Jose Santamaria Flecha) > +Add Windows to process the system collations (Jose Santamaria Flecha) > ADD THIS? > > Hmm, not sure this describes the ch

Re: ERROR: no relation entry for relid 6

2023-05-26 Thread Richard Guo
On Fri, May 26, 2023 at 6:06 AM Tom Lane wrote: > 1. The test case you give upthread only needs to ignore commute_below_l, > that is it still passes without the lines > > +join_plus_commute = bms_add_members(join_plus_commute, > + > removable_sjinfo->commute_above_r); > > Based on what decons

Re: Adding SHOW CREATE TABLE

2023-05-26 Thread Andrew Dunstan
On 2023-05-25 Th 09:23, Jelte Fennema wrote: On Mon, 22 May 2023 at 13:52, Andrew Dunstan wrote: A performant server side set of functions would be written in C and follow the patterns in ruleutils.c. We have lots of DDL ruleutils in our Citus codebase: https://github.com/citusdata/citus/blo

Re: running logical replication as the subscription owner

2023-05-26 Thread Masahiko Sawada
On Thu, May 25, 2023 at 5:41 PM Amit Kapila wrote: > > On Thu, May 25, 2023 at 12:33 PM Masahiko Sawada > wrote: > > > > On Tue, May 23, 2023 at 8:21 PM Amit Kapila wrote: > > > > > > On Mon, May 22, 2023 at 6:06 PM Masahiko Sawada > > > wrote: > > > > > > > > Thank you for updating the patch

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
Hi, On 4/21/23 8:07 PM, Nathan Bossart wrote: Attached is a proof-of-concept/work-in-progress patch set that adds functions for "vectors" repreѕented with one-dimensional float8 arrays. These functions may be used in a variety of applications, but I am proposing them with the AI/ML use-cases in

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
On 5/25/23 1:48 PM, Oliver Rice wrote: A nice side effect of using the float8[] to represent vectors is that it allows for vectors of different sizes to coexist in the same column. We most frequently see (pgvector) vector columns being used for storing ML embeddings. Given that different mode

Re: vector search support

2023-05-26 Thread Jonathan S. Katz
On 4/26/23 9:31 AM, Giuseppe Broccolo wrote: Hi Nathan, I find the patches really interesting. Personally, as Data/MLOps Engineer, I'm involved in a project where we use embedding techniques to generate vectors from documents, and use clustering and kNN searches to find similar documents basi

Is NEW.ctid usable as table_tuple_satisfies_snapshot?

2023-05-26 Thread Kaiting Chen
I need to implement a trigger that will behave similarly to a foreign key constraint. The trigger itself will be created with: CREATE CONSTRAINT TRIGGER ... AFTER INSERT OR UPDATE OF ... ON foo I'd like to skip execution of the trigger logic if, by the time that the trigger is executed, the NEW

Re: Is NEW.ctid usable as table_tuple_satisfies_snapshot?

2023-05-26 Thread David G. Johnston
On Fri, May 26, 2023 at 8:04 AM Kaiting Chen wrote: > I need to implement a trigger that will behave similarly to a foreign key > constraint. The trigger itself will be created with: > > CREATE CONSTRAINT TRIGGER ... AFTER INSERT OR UPDATE OF ... ON foo > > I'd like to skip execution of the tri

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2023-05-26 Thread Jacob Champion
On Thu, May 25, 2023 at 7:09 PM Michael Paquier wrote: > Please find attached an updated patch only for the removal of 1.0.1. > Thanks for the review. Nice! Sorry about the new complications with LibreSSL. :( > - # Functions introduced in OpenSSL 1.0.2. LibreSSL does not have > + # Function in

Re: ERROR: no relation entry for relid 6

2023-05-26 Thread Tom Lane
Richard Guo writes: > On Fri, May 26, 2023 at 6:06 AM Tom Lane wrote: >> Based on what deconstruct_distribute_oj_quals is doing, it seems >> likely to me that there are cases that require ignoring >> commute_above_r, but I've not tried to devise one. It'd be good to >> have one to include in the

Re: Is NEW.ctid usable as table_tuple_satisfies_snapshot?

2023-05-26 Thread Kaiting Chen
On Fri, May 26, 2023 at 11:34 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Fri, May 26, 2023 at 8:04 AM Kaiting Chen wrote: > >> I need to implement a trigger that will behave similarly to a foreign key >> constraint. The trigger itself will be created with: >> >> CREATE CONST

Re: Order changes in PG16 since ICU introduction

2023-05-26 Thread Daniel Verite
Jeff Davis wrote: > > #1 > > > > postgres=# create database test1 locale='fr_FR.UTF-8'; > > NOTICE: using standard form "fr-FR" for ICU locale "fr_FR.UTF-8" > > ERROR: new ICU locale (fr-FR) is incompatible with the ICU locale of > > I don't see a problem here. If you specify LOCALE to

Re: Is NEW.ctid usable as table_tuple_satisfies_snapshot?

2023-05-26 Thread Tom Lane
Kaiting Chen writes: > On Fri, May 26, 2023 at 11:34 AM David G. Johnston < > david.g.johns...@gmail.com> wrote: >> On Fri, May 26, 2023 at 8:04 AM Kaiting Chen wrote: >>> 2. If I lookup the row by its ctid, will the visibility map be consulted. >> No, but that doesn't seem to be material anyway

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Peter Geoghegan
On Fri, May 26, 2023 at 12:46 AM Michael Paquier wrote: > Nice cleanup overall. Thanks. To be clear, when I said "it would be nice to get rid of P_NEW", what I meant was that it would be nice to go much further than what I've done in the patch by getting rid of the general idea of P_NEW. So the

Re: Implement generalized sub routine find_in_log for tap test

2023-05-26 Thread vignesh C
On Thu, 25 May 2023 at 23:04, Dagfinn Ilmari Mannsåker wrote: > > vignesh C writes: > > > Hi, > > > > The recovery tap test has 4 implementations of find_in_log sub routine > > for various uses, I felt we can generalize these and have a single > > function for the same. The attached patch is an a

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Peter Geoghegan
On Fri, May 26, 2023 at 1:56 AM Alvaro Herrera wrote: > I suppose you're not thinking of applying this to current master, but > instead just leave it for when pg17 opens, right? I mean, clearly it > seems far too invasive to put it in after beta1. I was planning on targeting 16 with this. Althou

Re: pg_collation.collversion for C.UTF-8

2023-05-26 Thread Jeff Davis
On Thu, 2023-05-25 at 14:48 -0400, Tom Lane wrote: > Jeff Davis writes: > > What should we do with locales like C.UTF-8 in both libc and ICU? > > I vote for passing those to the existing C-specific code paths, Great, this would be a big step toward solving the ICU usability issues in this threa

Re: Is NEW.ctid usable as table_tuple_satisfies_snapshot?

2023-05-26 Thread Kaiting Chen
On Fri, May 26, 2023 at 12:49 PM Tom Lane wrote: > > Just to clarify, there's no way for SELECT FROM foo WHERE ctid = NEW.ctid > > to return a row that ordinary wouldn't be visible right? There's no magic > > going on with the qual on ctid that skips a visibility check right? > > No, a ctid test

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Peter Geoghegan
On Fri, May 26, 2023 at 10:28 AM Peter Geoghegan wrote: > I've added several defensive assertions that make it hard to get the > details wrong. These will catch the issue much earlier than the main > "heapRel != NULL" assertion in _bt_allocbuf(). So, the rules are > reasonably straightforward and

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Andres Freund
Hi, On 2023-05-25 18:50:31 -0700, Peter Geoghegan wrote: > Commit 61b313e4, which prepared index access method code for the > logical decoding on standbys work, made quite a few mechanical > changes. Many routines were revised in order to make sure that heaprel > was available in _bt_getbuf()'s P_

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Peter Geoghegan
On Fri, May 26, 2023 at 2:49 PM Andres Freund wrote: > What do we gain by not passing down the heap relation to those places? Just clearer code, free from noisey changes. Easier when backpatching, too. > If you're concerned about the efficiency of passing down the parameters, > I doubt it will m

Re: Docs: Encourage strong server verification with SCRAM

2023-05-26 Thread Jacob Champion
On Thu, May 25, 2023 at 6:10 PM Jonathan S. Katz wrote: > I read through the proposal and like this much better. Great! > I rewrote this to just focus on server spoofing that can occur with > SCRAM authentication and did some wordsmithing. I was torn on keeping in > the part of offline analysis

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Michael Paquier
On Fri, May 26, 2023 at 10:56:53AM +0200, Alvaro Herrera wrote: > I suppose you're not thinking of applying this to current master, but > instead just leave it for when pg17 opens, right? I mean, clearly it > seems far too invasive to put it in after beta1. On the other hand, > it's painful to kn

Fix search_path for all maintenance commands

2023-05-26 Thread Jeff Davis
Maintenance commands (ANALYZE, CLUSTER, REFRESH MATERIALIZED VIEW, REINDEX, and VACUUM) currently run as the table owner, and as a SECURITY_RESTRICTED_OPERATION. I propose that we also fix the search_path to "pg_catalog, pg_temp" when running maintenance commands, for two reasons: 1. Make the beh

Re: Cleaning up nbtree after logical decoding on standby work

2023-05-26 Thread Peter Geoghegan
On Fri, May 26, 2023 at 4:05 PM Michael Paquier wrote: > On Fri, May 26, 2023 at 10:56:53AM +0200, Alvaro Herrera wrote: > > I can't make up my mind about this. What do others think? > > When I looked at the patch yesterday, my impression was that this > would be material for v17 as it is refacto

Re: Implement generalized sub routine find_in_log for tap test

2023-05-26 Thread vignesh C
On Fri, 26 May 2023 at 04:09, Michael Paquier wrote: > > On Thu, May 25, 2023 at 06:34:20PM +0100, Dagfinn Ilmari Mannsåker wrote: > > However, none of the other functions in ::Utils know anything about node > > objects, which makes me think it should be a method on the node itself > > (i.e. in Po

Re: Cutting support for OpenSSL 1.0.1 and 1.0.2 in 17~?

2023-05-26 Thread Michael Paquier
On Fri, May 26, 2023 at 09:10:17AM -0700, Jacob Champion wrote: > Can X509_get_signature_nid be moved to the required section up above? > As it is now, anyone configuring with -Dssl=auto can still pick up a > 1.0.1 build, which Meson accepts, and then the build fails downstream. > If we require the