Re: 16: Collation versioning and dependency helpers

2022-12-06 Thread Jeff Davis
On Tue, 2022-12-06 at 10:53 -0800, Andres Freund wrote: > Perhaps worth posting a new version? Or are the remaining patches > abandoned in > favor of the other threads? Marked what is there as committed, and the remainder is abandoned in favor of other threads. Thanks, Jeff Davis

Re: 16: Collation versioning and dependency helpers

2022-12-06 Thread Andres Freund
Hi, On 2022-10-31 16:36:54 -0700, Jeff Davis wrote: > Committed these two small changes. FWIW, as it stands cfbot can't apply the remaining changes: http://cfbot.cputube.org/patch_40_3977.log Perhaps worth posting a new version? Or are the remaining patches abandoned in favor of the other thread

Re: 16: Collation versioning and dependency helpers

2022-10-31 Thread Jeff Davis
On Sun, 2022-10-30 at 19:10 +1300, Thomas Munro wrote: > FWIW we did this (plus a lot more) in the per-index version tracking > feature reverted from 14. Thank you. I will catch up on that patch/thread. > >   0002: Enable pg_collation_actual_version() to work on the default > > Makes sense. > >

Re: 16: Collation versioning and dependency helpers

2022-10-29 Thread Thomas Munro
On Sun, Oct 30, 2022 at 5:41 PM Jeff Davis wrote: > We haven't fully solved the changing collation-provider problem. An > upgrade of the OS may change the version of libc or icu, and that might > affect the collation, which could leave you with various corrupt > database objects including: > > *

16: Collation versioning and dependency helpers

2022-10-29 Thread Jeff Davis
Motivation: We haven't fully solved the changing collation-provider problem. An upgrade of the OS may change the version of libc or icu, and that might affect the collation, which could leave you with various corrupt database objects including: * indexes * constraints * range types or mult

Re: Collation versioning

2021-04-15 Thread Thomas Munro
On Mon, Mar 15, 2021 at 2:25 PM Thomas Munro wrote: > FYI I have added this as an open item for PostgreSQL 14. My default > action will be to document this limitation, if we can't come up with > something better in time. Here is a short doc update to explain the situation on Windows and close th

Re: Collation versioning

2021-03-14 Thread Thomas Munro
On Fri, Nov 6, 2020 at 10:56 AM Thomas Munro wrote: > On Wed, Nov 4, 2020 at 9:11 PM Michael Paquier wrote: > > On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote: > > > We could create a static table with the conversion based on what was > > > discussed for commit a1691

Have collation versioning to ignore hash and similar AM

2020-11-25 Thread Julien Rouhaud
Hello, Collation versioning has been committed, but there are still at least 2 cases that aren't perfectly handled: - AMs which don't rely on stable collation ordering (hash, bloom...) - expressions which don't rely on a stable collation ordering (e.g. md5()) Handling expression

Re: Collation versioning

2020-11-08 Thread Thomas Munro
On Fri, Oct 4, 2019 at 1:25 AM Thomas Munro wrote: > Ok. Here's one like that. Also, a WIP patch for FreeBSD. Here's an updated patch for FreeBSD, which I'll sit on for a bit longer. It needs bleeding edge 13-CURRENT (due out Q1ish). From b9cb5562457c214c48c0a6334b8ed3264f50e3d6 Mon Sep 17 00:

Re: Collation versioning

2020-11-05 Thread Thomas Munro
On Wed, Nov 4, 2020 at 9:11 PM Michael Paquier wrote: > On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote: > > We could create a static table with the conversion based on what was > > discussed for commit a169155, please find attached a spreadsheet with the > > compariso

Re: Collation versioning

2020-11-04 Thread Laurenz Albe
On Tue, 2020-11-03 at 23:14 +0100, Christoph Berg wrote: > Re: Thomas Munro > > for all I know, "en" variants might change > > independently (I doubt it in practice, but in theory it's wrong). > > Long before the glibc 2.28 incident, the same collation change > had already happened twice, namely b

Re: Collation versioning

2020-11-04 Thread Julien Rouhaud
On Wed, Nov 4, 2020 at 4:11 PM Michael Paquier wrote: > > On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote: > > We could create a static table with the conversion based on what was > > discussed for commit a169155, please find attached a spreadsheet with the > > compari

Re: Collation versioning

2020-11-04 Thread Michael Paquier
On Wed, Nov 04, 2020 at 08:44:15AM +0100, Juan José Santamaría Flecha wrote: > We could create a static table with the conversion based on what was > discussed for commit a169155, please find attached a spreadsheet with the > comparison. This would require maintenance as new LCIDs are released [1]

Re: Collation versioning

2020-11-03 Thread Juan José Santamaría Flecha
On Tue, Nov 3, 2020 at 10:49 PM Thomas Munro wrote: > > So we have: > > 1. Windows locale names, like "English_United States.1252". Windows > still returns these from setlocale(), so they finish up in datcollate, > and yet some relevant APIs don't accept them, at least on some > machines. > > 2

Re: Collation versioning

2020-11-03 Thread Thomas Munro
On Wed, Nov 4, 2020 at 2:56 PM David Rowley wrote: > initdb works fine. I ran vcregress upgradecheck and it passes. > > With my default locale of English.New Zealand.1252 I get zero rows from: > > select * from pg_depend where coalesce(refobjversion,'') <> ''; > > if I initdb with --lc-collate=en-

Re: Collation versioning

2020-11-03 Thread David Rowley
On Wed, 4 Nov 2020 at 14:21, Thomas Munro wrote: > > On Wed, Nov 4, 2020 at 10:56 AM Thomas Munro wrote: > > On Wed, Nov 4, 2020 at 10:52 AM Tom Lane wrote: > > > Thomas Munro writes: > > > > We want the same algorithm that Windows uses internally to resolve the > > > > old style name to a coll

Re: Collation versioning

2020-11-03 Thread Tom Lane
Thomas Munro writes: > I can't bring myself to commit that, it's not really in the spirit of > this data integrity feature, and it's not our business to second guess > the relationship between different locale naming schemes through fuzzy > logic. Instead, I propose to just neuter the feature if

Re: Collation versioning

2020-11-03 Thread Thomas Munro
on Sep 17 00:00:00 2001 From: Thomas Munro Date: Wed, 4 Nov 2020 13:49:05 +1300 Subject: [PATCH] Tolerate version lookup failure for old-style Windows locale names. Accept that we can't get versions for such locale names for now, leaving it up to the user to provide the modern language tag form

Re: Collation versioning

2020-11-03 Thread Christoph Berg
Re: Thomas Munro > for all I know, "en" variants might change > independently (I doubt it in practice, but in theory it's wrong). Long before the glibc 2.28 incident, the same collation change had already happened twice, namely between RedHat 5 -> 6 -> 7, for de_DE.UTF-8 only. de_AT.UTF-8 and all

Re: Collation versioning

2020-11-03 Thread Thomas Munro
On Wed, Nov 4, 2020 at 10:52 AM Tom Lane wrote: > Thomas Munro writes: > > We want the same algorithm that Windows uses internally to resolve the > > old style name to a collation; in other words we probably want > > something more like the code path that they took away in VS2015 :-(. > > Yeah.

Re: Collation versioning

2020-11-03 Thread Tom Lane
Thomas Munro writes: > On Tue, Nov 3, 2020 at 4:38 PM Thomas Munro wrote: >> On Tue, Nov 3, 2020 at 1:51 PM David Rowley wrote: >>> FWIW, the attached does fix the issue for me. It basically just calls >>> the function that converts the windows-type "English_New Zealand.1252" >>> locale name st

Re: Collation versioning

2020-11-03 Thread Thomas Munro
On Tue, Nov 3, 2020 at 4:38 PM Thomas Munro wrote: > On Tue, Nov 3, 2020 at 1:51 PM David Rowley wrote: > > On Tue, 3 Nov 2020 at 12:29, David Rowley wrote: > > > Running low on ideas for now, so thought I'd post this in case it > > > someone thinks of something else. > > > > FWIW, the attached

Re: Collation versioning

2020-11-03 Thread Juan José Santamaría Flecha
On Tue, Nov 3, 2020 at 4:39 AM Thomas Munro wrote: > On Tue, Nov 3, 2020 at 1:51 PM David Rowley wrote: > > > It would be good if this could also be tested on Visual Studio version > > 12 as I see IsoLocaleName() does something else for anything before > > 15. I only have 10 and 17 installed an

Re: Collation versioning

2020-11-02 Thread Thomas Munro
On Tue, Nov 3, 2020 at 1:51 PM David Rowley wrote: > On Tue, 3 Nov 2020 at 12:29, David Rowley wrote: > > Running low on ideas for now, so thought I'd post this in case it > > someone thinks of something else. > > FWIW, the attached does fix the issue for me. It basically just calls > the functi

Re: Collation versioning

2020-11-02 Thread David Rowley
On Tue, 3 Nov 2020 at 12:29, David Rowley wrote: > Running low on ideas for now, so thought I'd post this in case it > someone thinks of something else. FWIW, the attached does fix the issue for me. It basically just calls the function that converts the windows-type "English_New Zealand.1252" lo

Re: Collation versioning

2020-11-02 Thread David Rowley
On Tue, 3 Nov 2020 at 09:43, Thomas Munro wrote: > Fortunately David Rowley is able to repro this on his Windows box (it > fails even with strings that are succeeding on the other BF machines), > so we have something to work with. The name mangling that is done in > get_iso_localename() looks pre

Re: Collation versioning

2020-11-02 Thread Thomas Munro
On Tue, Nov 3, 2020 at 6:51 AM Tom Lane wrote: > Thomas Munro writes: > > Hmm, a failure from dory (WIndows) during pg_upgrade: > > > performing post-bootstrap initialization ... 2020-11-02 08:08:22.213 > > EST [5392] FATAL: could not get collation version for locale > > "English_United States.1

Re: Collation versioning

2020-11-02 Thread Tom Lane
Thomas Munro writes: > Hmm, a failure from dory (WIndows) during pg_upgrade: > performing post-bootstrap initialization ... 2020-11-02 08:08:22.213 > EST [5392] FATAL: could not get collation version for locale > "English_United States.1252": error code 87 > 87 means invalid parameter. I'm sur

Re: Collation versioning

2020-11-02 Thread Thomas Munro
On Tue, Nov 3, 2020 at 2:08 AM Julien Rouhaud wrote: > On Mon, Nov 2, 2020 at 9:04 PM Thomas Munro wrote: > > Ok, moved and renamed, and finally pushed. Thanks for all the help! > > \o/ Thanks a lot Thomas! Hmm, a failure from dory (WIndows) during pg_upgrade: performing post-bootstrap initia

Re: Collation versioning

2020-11-02 Thread Julien Rouhaud
On Mon, Nov 2, 2020 at 9:04 PM Thomas Munro wrote: > > On Mon, Nov 2, 2020 at 10:28 PM Julien Rouhaud wrote: > > + /* Remember not to complain about this collation again. */ > > + context->checked_colls = lappend_oid(context->checked_colls, > > +otherOb

Re: Collation versioning

2020-11-02 Thread Thomas Munro
On Mon, Nov 2, 2020 at 10:28 PM Julien Rouhaud wrote: > + /* Remember not to complain about this collation again. */ > + context->checked_colls = lappend_oid(context->checked_colls, > +otherObject->objectId); > > It's maybe not immediately obvious that i

Re: Collation versioning

2020-11-02 Thread Julien Rouhaud
On Mon, Nov 2, 2020 at 3:56 PM Thomas Munro wrote: > > On Mon, Nov 2, 2020 at 7:59 PM Julien Rouhaud wrote: > > Note that v34 now fails when run on a without that don't have > > defined(__GLIBC__) (e.g. macos). The failure are in > > collate.icu.utf8.sql, of the form: > > > > - icuidx06_d_en_fr_

Re: Collation versioning

2020-11-01 Thread Julien Rouhaud
On Sat, Oct 31, 2020 at 10:28 AM Thomas Munro wrote: > > On Fri, Oct 30, 2020 at 1:20 AM Thomas Munro wrote: > > 4. I didn't really like the use of '' for unknown. I figured out how > > to use NULL for that. > > Hrmph. That logic didn't work for the pattern ops case. I fixed > that, by partia

Re: Collation versioning

2020-10-26 Thread Julien Rouhaud
On Sun, Oct 25, 2020 at 7:13 PM Julien Rouhaud wrote: > > On Sun, Oct 25, 2020 at 10:36 AM Thomas Munro wrote: > > > > On Fri, Oct 23, 2020 at 2:07 AM Julien Rouhaud wrote: > > > While reviewing the changes, I found a couple of minor issues > > > (inherited from the previous versions). It's mis

Re: Collation versioning

2020-10-25 Thread Julien Rouhaud
On Sun, Oct 25, 2020 at 10:36 AM Thomas Munro wrote: > > On Fri, Oct 23, 2020 at 2:07 AM Julien Rouhaud wrote: > > While reviewing the changes, I found a couple of minor issues > > (inherited from the previous versions). It's missing a > > free_objects_addresses in recordDependencyOnCollations,

Re: Collation versioning

2020-10-22 Thread Julien Rouhaud
On Thu, Oct 22, 2020 at 8:00 PM Thomas Munro wrote: > > On Thu, Sep 24, 2020 at 9:49 PM Julien Rouhaud wrote: > > On Sun, Sep 20, 2020 at 10:24:26AM +0800, Julien Rouhaud wrote: > > > On the other hand the *_pattern_ops are entirely hardcoded, and I > > > don't think that we'll ever have an exten

Re: Collation versioning

2020-09-19 Thread Julien Rouhaud
On Sun, Sep 20, 2020 at 6:36 AM Thomas Munro wrote: > > On Thu, Sep 17, 2020 at 5:41 AM Julien Rouhaud wrote: > > On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut > > wrote: > > > I'm confused now. I think we had mostly agreed on the v28 patch set, > > > without this additional AM flag. There

Re: Collation versioning

2020-09-19 Thread Thomas Munro
On Thu, Sep 17, 2020 at 5:41 AM Julien Rouhaud wrote: > On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut > wrote: > > I'm confused now. I think we had mostly agreed on the v28 patch set, > > without this additional AM flag. There was still some discussion on > > what the AM flag's precise seman

Re: Collation versioning

2020-09-16 Thread Julien Rouhaud
On Tue, Sep 15, 2020 at 2:26 PM Peter Eisentraut wrote: > > On 2020-09-08 16:45, Julien Rouhaud wrote: > > I usually agree with that approach, I'm just afraid that getting a > > consensus on > > the best way to do that will induce a lot of discussions, while this is > > probably a corner case due

Re: Collation versioning

2020-09-15 Thread Peter Eisentraut
On 2020-09-08 16:45, Julien Rouhaud wrote: I usually agree with that approach, I'm just afraid that getting a consensus on the best way to do that will induce a lot of discussions, while this is probably a corner case due to general usage of hash and bloom indexes. Anyway, in order to make progr

Re: Collation versioning

2020-09-10 Thread Julien Rouhaud
On Thu, Sep 10, 2020 at 6:52 PM Peter Eisentraut wrote: > > On 2020-09-08 21:33, Christoph Berg wrote: > > IOW, I think we should aim at simply tracking the version, and leave > > it to the admin (with the help of supplied SQL queries) to either > > rebuild indexes or waive them. > > It's certainl

Re: Collation versioning

2020-09-10 Thread Peter Eisentraut
On 2020-09-08 21:33, Christoph Berg wrote: IOW, I think we should aim at simply tracking the version, and leave it to the admin (with the help of supplied SQL queries) to either rebuild indexes or waive them. It's certainly safer to track dependency for all indexes and then carefully create ex

Re: Collation versioning

2020-09-08 Thread Julien Rouhaud
On Tue, Sep 08, 2020 at 09:33:26PM +0200, Christoph Berg wrote: > Re: Julien Rouhaud > > Here's the rationale for this new flag, extracted from the patch's commit > > message: > > > > Add a new amnostablecollorder flag in IndexAmRoutine. > > > > This flag indicates if the access method does not r

Re: Collation versioning

2020-09-08 Thread Christoph Berg
Re: Julien Rouhaud > Here's the rationale for this new flag, extracted from the patch's commit > message: > > Add a new amnostablecollorder flag in IndexAmRoutine. > > This flag indicates if the access method does not rely on a stable collation > ordering for deterministic collation, i.e. would n

Re: Collation versioning

2020-09-06 Thread Michael Paquier
On Fri, Aug 14, 2020 at 11:02:35AM +0200, Julien Rouhaud wrote: > On Fri, Aug 14, 2020 at 04:37:46PM +0900, Michael Paquier wrote: >> + /* >> +* XXX For deterministic transaction, se should only track the >> version >> +* if the AM relies on a stable ordering. >> +*/ >

Re: Collation versioning

2020-08-14 Thread Michael Paquier
On Fri, Aug 14, 2020 at 02:21:58PM +1200, Thomas Munro wrote: > Thanks Julien. I'm planning to do a bit more testing and review, and > then hopefully commit this next week. If anyone else has objections > to this design, now would be a good time to speak up. The design to use pg_depend for the v

Re: Collation versioning

2020-08-13 Thread Thomas Munro
On Thu, Aug 13, 2020 at 9:52 PM Julien Rouhaud wrote: > Here's a rebased v27 that removes the current approach to ignore indexes that > don't rely on a stable ordering. I'll start a new thread on that matter once > the infrastructure pieces will be committed. Thanks Julien. I'm planning to do a

Re: Collation versioning

2020-07-09 Thread Thomas Munro
On Thu, Jul 9, 2020 at 11:13 PM Julien Rouhaud wrote: > On Thu, Jul 9, 2020 at 10:00 AM Peter Eisentraut > wrote: > > In order not to derail this patch set I think it would be okay for now > > to just include all index AMs in dependency tracking and invent a > > mechanism later that excludes hash

Re: Collation versioning

2020-07-09 Thread Julien Rouhaud
On Thu, Jul 9, 2020 at 10:00 AM Peter Eisentraut wrote: > > On 2020-07-08 08:26, Michael Paquier wrote: > > On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote: > >> I still wish I had a better idea than this: > >> > >> +/* > >> + * Returns whether the given index access method depend on

Re: Collation versioning

2020-07-09 Thread Peter Eisentraut
On 2020-07-08 08:26, Michael Paquier wrote: On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote: I still wish I had a better idea than this: +/* + * Returns whether the given index access method depend on a stable collation + * order. + */ +static bool +index_depends_stable_coll_order(

Re: Collation versioning

2020-07-07 Thread Michael Paquier
On Wed, Jul 08, 2020 at 06:12:51PM +1200, Thomas Munro wrote: > I still wish I had a better idea than this: > > +/* > + * Returns whether the given index access method depend on a stable collation > + * order. > + */ > +static bool > +index_depends_stable_coll_order(Oid amoid) > +{ > + retur

Re: Collation versioning

2020-03-19 Thread Julien Rouhaud
On Thu, Mar 19, 2020 at 12:31:54PM +0900, Michael Paquier wrote: > On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote: > > On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote: > > AFAICT it was only missing a call to index_update_collation_versions() in > > ReindexRelationCon

Re: Collation versioning

2020-03-18 Thread Michael Paquier
On Wed, Mar 18, 2020 at 04:35:43PM +0100, Julien Rouhaud wrote: > On Wed, Mar 18, 2020 at 09:56:40AM +0100, Julien Rouhaud wrote: > AFAICT it was only missing a call to index_update_collation_versions() in > ReindexRelationConcurrently. I added regression tests to make sure that > REINDEX, REINDEX

Re: Collation versioning

2020-03-18 Thread Michael Paquier
On Wed, Mar 18, 2020 at 09:29:55PM +0100, Peter Eisentraut wrote: > OK, I have committed the regcollation patch, and some surrounding cleanup of > the reg* types documentation. Thanks, Peter. -- Michael signature.asc Description: PGP signature

Re: Collation versioning

2020-03-18 Thread Julien Rouhaud
On Wed, Mar 18, 2020 at 09:29:55PM +0100, Peter Eisentraut wrote: > On 2020-03-17 18:43, Julien Rouhaud wrote: > > On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote: > > > Re: Peter Eisentraut > > > 2020-03-17 > > > > Did we discuss the regcollation type? In the current patch set, it

Re: Collation versioning

2020-03-18 Thread Peter Eisentraut
On 2020-03-17 18:43, Julien Rouhaud wrote: On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote: Re: Peter Eisentraut 2020-03-17 Did we discuss the regcollation type? In the current patch set, it's only used in two places in a new regression test, where it can easily be replaced by

Re: Collation versioning

2020-03-18 Thread Julien Rouhaud
On Wed, Mar 18, 2020 at 04:55:25PM +0900, Michael Paquier wrote: > On Tue, Mar 17, 2020 at 11:42:34AM +0100, Julien Rouhaud wrote: > > On Tue, Mar 17, 2020 at 08:19:30AM +0100, Julien Rouhaud wrote: > > It would be good to be careful about the indentation. Certain parts > of 0003 don't respect the

Re: Collation versioning

2020-03-18 Thread Michael Paquier
On Tue, Mar 17, 2020 at 11:42:34AM +0100, Julien Rouhaud wrote: > On Tue, Mar 17, 2020 at 08:19:30AM +0100, Julien Rouhaud wrote: >> On Tue, Mar 17, 2020 at 03:37:49PM +0900, Michael Paquier wrote: >>> It seems to me that you could add an extra test with a catalog that >>> does not exist, making su

Re: Collation versioning

2020-03-17 Thread Michael Paquier
On Tue, Mar 17, 2020 at 06:43:51PM +0100, Julien Rouhaud wrote: > On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote: >> Not sure if that's the case there, but reg* typecasts are very handy >> when used interactively in ad-hoc queries. > > +1. I'm obviously biased, but I find it quite

Re: Collation versioning

2020-03-17 Thread Julien Rouhaud
On Tue, Mar 17, 2020 at 05:31:47PM +0100, Christoph Berg wrote: > Re: Peter Eisentraut 2020-03-17 > > > Did we discuss the regcollation type? In the current patch set, it's only > > used in two places in a new regression test, where it can easily be replaced > > by a join. Do we need it? I or

Re: Collation versioning

2020-03-17 Thread Christoph Berg
Re: Peter Eisentraut 2020-03-17 > Did we discuss the regcollation type? In the current patch set, it's only > used in two places in a new regression test, where it can easily be replaced > by a join. Do we need it? > > I realize we've been adding new reg* types lately; I'm not sure what the >

Re: Collation versioning

2020-03-17 Thread Peter Eisentraut
Did we discuss the regcollation type? In the current patch set, it's only used in two places in a new regression test, where it can easily be replaced by a join. Do we need it? I realize we've been adding new reg* types lately; I'm not sure what the current idea is on that. -- Peter Eisent

Re: Collation versioning

2020-03-17 Thread opolofdez
njhjo Enviado desde mi Redmi 4AEl Julien Rouhaud , 16 mar. 2020 3:05 p. m. escribió:On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote: > On Thu, Mar 12, 2020 at 03:00:26PM +0100, Julien Rouhaud wrote: > > And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj

Re: Collation versioning

2020-03-17 Thread Julien Rouhaud
On Tue, Mar 17, 2020 at 03:37:49PM +0900, Michael Paquier wrote: > On Mon, Mar 16, 2020 at 03:05:20PM +0100, Julien Rouhaud wrote: > > On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote: > >> This comes from a regression test doing a sanity check to look for > >> catalogs which have a

Re: Collation versioning

2020-03-16 Thread Michael Paquier
On Mon, Mar 16, 2020 at 03:05:20PM +0100, Julien Rouhaud wrote: > On Mon, Mar 16, 2020 at 04:57:38PM +0900, Michael Paquier wrote: >> This comes from a regression test doing a sanity check to look for >> catalogs which have a toastable column but no toast tables. As an >> exception, it should be d

Re: Collation versioning

2020-03-16 Thread Michael Paquier
On Thu, Mar 12, 2020 at 03:00:26PM +0100, Julien Rouhaud wrote: > And v15 due to conflict with b08dee24a5 (Add pg_dump support for ALTER obj > DEPENDS ON EXTENSION). I have looked at patches 0001~0003 in the set for now. 0001 looks clean to me. In patch 0002, you have the following addition: @@

Re: Collation versioning

2020-02-26 Thread Thomas Munro
On Thu, Feb 27, 2020 at 3:29 AM Julien Rouhaud wrote: > [v10] Thanks. I'll do some more testing and review soon. It'd be really cool to get this into PG13. FYI cfbot said: +++ /home/travis/build/postgresql-cfbot/postgresql/src/test/regress/results/collate.icu.utf8.out 2020-02-26 14:45:52.114

Re: Collation versioning

2020-02-18 Thread Julien Rouhaud
On Mon, Feb 17, 2020 at 5:58 AM Thomas Munro wrote: > > On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud wrote: > > Hearing no complaints on the suggestions, I'm attaching v8 to address that: > > > > - pg_dump is now using a binary_upgrade_set_index_coll_version() function > > rather than plain D

Re: Collation versioning

2020-02-16 Thread Thomas Munro
On Thu, Feb 13, 2020 at 8:13 AM Julien Rouhaud wrote: > Hearing no complaints on the suggestions, I'm attaching v8 to address that: > > - pg_dump is now using a binary_upgrade_set_index_coll_version() function > rather than plain DDL > - the additional DDL is now of the form: > ALTER INDEX nam

Re: Collation versioning

2020-02-12 Thread Thomas Munro
On Thu, Feb 13, 2020 at 9:16 AM Julien Rouhaud wrote: > On Wed, Feb 12, 2020 at 08:55:06PM +0100, Laurenz Albe wrote: > > I didn't study the patch in detail, but do I get it right that there will > > be no > > warnings about version incompatibilities with libc collations? > > No, libc is also be

Re: Collation versioning

2020-02-12 Thread Julien Rouhaud
On Wed, Feb 12, 2020 at 08:55:06PM +0100, Laurenz Albe wrote: > On Wed, 2020-02-12 at 20:13 +0100, Julien Rouhaud wrote: > > On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote: > > > Note that I didn't change any syntax (or switched to native functions > > > for the binary pg_dump) as i

Re: Collation versioning

2020-02-12 Thread Laurenz Albe
On Wed, 2020-02-12 at 20:13 +0100, Julien Rouhaud wrote: > On Wed, Feb 05, 2020 at 05:17:25PM +0100, Julien Rouhaud wrote: > > Note that I didn't change any syntax (or switched to native functions > > for the binary pg_dump) as it's still not clear to me what exactly > > should be implemented. > >

Re: Collation versioning

2020-01-29 Thread Julien Rouhaud
le previously argued that those command should be available even in non binary upgrade and I'm not clear on whether this is wanted or not. Do you have any thoughts on that? > The documentation snippet for this patch talks about upgrades from PG12 > to later. But what about upgrades on

Re: Collation versioning

2020-01-28 Thread Peter Eisentraut
atch talks about upgrades from PG12 to later. But what about upgrades on platforms where we currently don't have collation versioning but might introduce it later (FreeBSD, Windows)? This needs to be generalized. For 0006 ("Add a new ALTER INDEX name DEPENDS ON COLLATION name CU

Re: Collation versioning

2019-12-12 Thread Julien Rouhaud
Hello Thomas, Thanks for looking at it! On Thu, Dec 12, 2019 at 5:01 AM Thomas Munro wrote: > > We create duplicate pg_depend records: > > [...] > > I wondered if that was harmless That's the assumed behavior of recordMultipleDependencies: /* * Record the Dependency. Note we don't bother to c

Re: Collation versioning

2019-12-11 Thread Thomas Munro
On Thu, Dec 12, 2019 at 6:32 PM Tom Lane wrote: > Thomas Munro writes: > > Erm, but I shouldn't have to reindex my hash indexes (at least not > > until someone invents collation-based equality and therefore > > necessarily also collation-based hashing). How can we exclude that? > > Um, we alread

Re: Collation versioning

2019-12-11 Thread Tom Lane
Thomas Munro writes: > Erm, but I shouldn't have to reindex my hash indexes (at least not > until someone invents collation-based equality and therefore > necessarily also collation-based hashing). How can we exclude that? Um, we already invented that with nondeterministic collations, no?

Re: Collation versioning

2019-12-11 Thread Thomas Munro
On Thu, Dec 12, 2019 at 5:00 PM Thomas Munro wrote: > Then, as a special case, there is the collation of the actual indexed > value, because that will implicitly be used as input to the btree ops > that would be collation sensitive. [...] Erm, but I shouldn't have to reindex my hash indexes (at

Re: Collation versioning

2019-12-11 Thread Thomas Munro
On Thu, Dec 12, 2019 at 2:45 AM Julien Rouhaud wrote: > Hearing no objection in [1], attached v5 with following changes: > > - fix the spurious warnings by doing the version check in > get_relation_info and vacuum_open_relation, and add a flag in > RelationData to mark the check as already being d

Re: Collation versioning

2019-12-10 Thread Julien Rouhaud
On Fri, Nov 8, 2019 at 2:24 AM Thomas Munro wrote: > > On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud wrote: > > > > I didn't do anything for the spurious warning when running a reindex, > > and kept original approach of pg_depend catalog. > > I see three options: > > 1. Change all or some of in

Re: Collation versioning

2019-12-03 Thread Julien Rouhaud
On Mon, Dec 2, 2019 at 2:00 PM Robert Haas wrote: > > ALTER INDEX idx_name DEPENDS ON COLLATION blah VERSION blah; > -- I care about collations and I know which one and which version. > > ALTER INDEX idx_name DEPENDS ON SOME COLLATION; > -- I care about collations but I don't know which one. This

Re: Collation versioning

2019-12-02 Thread Robert Haas
On Thu, Nov 28, 2019 at 8:08 AM Julien Rouhaud wrote: > What we could do is storing an empty string if the compatibility is > unknown, and detect it in index_check_collation_version() to report a > slightly different message. I'm assuming that not knowing the > compatibility would be system-wide

Re: Collation versioning

2019-11-28 Thread Julien Rouhaud
On Thu, Nov 28, 2019 at 5:50 AM Michael Paquier wrote: > > On Wed, Nov 27, 2019 at 04:09:57PM -0500, Robert Haas wrote: > > Yeah, I like #3 too. If we're going to the trouble to build all of > > this mechanism, it seems worth it to build the additional machinery to > > be precise about the differe

Re: Collation versioning

2019-11-27 Thread Michael Paquier
On Wed, Nov 27, 2019 at 04:09:57PM -0500, Robert Haas wrote: > Yeah, I like #3 too. If we're going to the trouble to build all of > this mechanism, it seems worth it to build the additional machinery to > be precise about the difference between "looks like a problem" and "we > don't know". Indeed,

Re: Collation versioning

2019-11-27 Thread Robert Haas
On Tue, Nov 26, 2019 at 3:44 PM Thomas Munro wrote: > I suppose another reason to use such a switch would be if there is a > change in the versioning scheme; for example, as of today in master we > are using the glibc version, but a future glibc release might offer an > interface to query the CLDR

Re: Collation versioning

2019-11-26 Thread Thomas Munro
On Fri, Nov 8, 2019 at 5:40 PM Laurenz Albe wrote: > On Fri, 2019-11-08 at 15:04 +1300, Thomas Munro wrote: > > 3. We don't know if pre-13 indexes are corrupted or not, and we'll > > record that with a special value just as in proposal #1, except that > > we could show a different hint for that s

Re: Collation versioning

2019-11-24 Thread Julien Rouhaud
On Tue, Nov 12, 2019 at 10:16 PM Thomas Munro wrote: > > On Wed, Nov 13, 2019 at 3:27 AM Julien Rouhaud wrote: > > On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro > > wrote: > > > That's because the 0003 patch only calls recordDependencyOnVersion() > > > for simple attribute references. When > >

Re: Collation versioning

2019-11-12 Thread Thomas Munro
On Wed, Nov 13, 2019 at 3:27 AM Julien Rouhaud wrote: > On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro wrote: > > That's because the 0003 patch only calls recordDependencyOnVersion() > > for simple attribute references. When > > recordDependencyOnSingleRelExpr() is called by index_create() to > >

Re: Collation versioning

2019-11-12 Thread Julien Rouhaud
On Sun, Nov 10, 2019 at 10:08 AM Thomas Munro wrote: > > Some more thoughts: > > 1. If you create an index on an expression that includes a COLLATE or > a partial index that has one in the WHERE clause, you get bogus > warnings: > > postgres=# create table t (v text); > CREATE TABLE > postgres=#

Re: Collation versioning

2019-11-10 Thread Thomas Munro
Some more thoughts: 1. If you create an index on an expression that includes a COLLATE or a partial index that has one in the WHERE clause, you get bogus warnings: postgres=# create table t (v text); CREATE TABLE postgres=# create index on t(v) where v > 'hello' collate "en_NZ"; WARNING: index

Re: Collation versioning

2019-11-08 Thread Julien Rouhaud
On Fri, Nov 8, 2019 at 10:20 AM Christoph Berg wrote: > > Re: Laurenz Albe 2019-11-08 > <3c3b9ff84d21acf3188558928249d04db84ea2e9.ca...@cybertec.at> > > #3 is the best proposal, but there is still the need to run > > ALTER INDEX on all affected indexes to keep PostgreSQL from nagging. > > Perhaps

Re: Collation versioning

2019-11-08 Thread Julien Rouhaud
On Fri, Nov 8, 2019 at 2:24 AM Thomas Munro wrote: > > On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud wrote: > > Attached 4th patch handles default collation. I went with an > > ignore_systempin flag in recordMultipleDependencies. > > Thanks for working on this! I haven't looked closely or test

Re: Collation versioning

2019-11-08 Thread Christoph Berg
Re: Laurenz Albe 2019-11-08 <3c3b9ff84d21acf3188558928249d04db84ea2e9.ca...@cybertec.at> > #3 is the best proposal, but there is still the need to run > ALTER INDEX on all affected indexes to keep PostgreSQL from nagging. > Perhaps the situation could be improved with a pg_upgrade option > --i-kno

Re: Collation versioning

2019-11-07 Thread Laurenz Albe
On Fri, 2019-11-08 at 15:04 +1300, Thomas Munro wrote: > So we have three proposals: > > 1. Assume that pre-13 indexes that depend on collations are > potentially corrupted and complain until they are reindexed. This > could be done by having pg_upgrade run ALTER INDEX ... DEPENDS ON > COLLATION

Re: Collation versioning

2019-11-07 Thread Thomas Munro
On Fri, Nov 8, 2019 at 2:37 PM Michael Paquier wrote: > On Fri, Nov 08, 2019 at 02:23:54PM +1300, Thomas Munro wrote: > > Right, so this is basically a policy decision: do we assume that all > > pre-13 indexes that depend on collations are potentially corrupted, or > > assume that they are not? T

Re: Collation versioning

2019-11-07 Thread Thomas Munro
On Thu, Nov 7, 2019 at 1:27 AM Peter Eisentraut wrote: > As I was working on that lately, I came to the conclusion that we should > get *this* patch done first. Cool. Let's aim to get this into 13! > > * Some have expressed doubt that pg_depend is the right place for > > this; let's see if any

Re: Collation versioning

2019-11-07 Thread Michael Paquier
On Fri, Nov 08, 2019 at 02:23:54PM +1300, Thomas Munro wrote: > Right, so this is basically a policy decision: do we assume that all > pre-13 indexes that depend on collations are potentially corrupted, or > assume that they are not? The "correct" thing to do would be to > assume they are potentia

Re: Collation versioning

2019-11-07 Thread Thomas Munro
On Thu, Nov 7, 2019 at 10:20 PM Julien Rouhaud wrote: > Attached 4th patch handles default collation. I went with an > ignore_systempin flag in recordMultipleDependencies. Thanks for working on this! I haven't looked closely or tested yet, but this seems reasonable. Obviously it assumes that t

Re: Collation versioning

2019-11-07 Thread Julien Rouhaud
On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud wrote: > > On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro wrote: > > > > Unfortunately I haven't had time to work on it seriously, but here's a > > quick rebase to get the proof-of-concept back into working shape. Thanks! I also removed the test for R

Re: Collation versioning

2019-11-07 Thread Julien Rouhaud
On Mon, Nov 4, 2019 at 9:59 PM Thomas Munro wrote: > > On Tue, Nov 5, 2019 at 4:18 AM Julien Rouhaud wrote: > > On Mon, Nov 4, 2019 at 11:13 AM Julien Rouhaud wrote: > > > On Mon, Nov 4, 2019 at 4:58 AM Thomas Munro > > > wrote: > > > > Here are some problems to think about: > > > > > > > > *

  1   2   >