Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
Here's the prompt response I got from the maintainer of FreeBSD package, girgen@ Forwarded Message Subject: Re: pg_upgrading to 10.1 breaks some indices maybe due to changing providers for ICU Date: Fri, 22 Dec 2017 17:44:41 +0100 From: Palle Girgensohn To: rihad

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
Here's the prompt response I got from the maintainer of FreeBSD package, girgen@ Forwarded Message Subject: Re: pg_upgrading to 10.1 breaks some indices maybe due to changing providers for ICU Date: Fri, 22 Dec 2017 17:44:41 +0100 From: Palle Girgensohn To: rihad

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Peter Geoghegan
On Fri, Dec 22, 2017 at 4:16 PM, rihad wrote: > You mean they would have to bring back ICU support as an option? Ok, I'll > try to report the problem. The FreeBSD package previously contained a hacked-in ICU library, meaning that the version of Postgres shipped with FreeBSD was technically a fork

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 08:07 PM, Peter Geoghegan wrote: Please report this as a bug to the freebsd package maintainer. You mean they would have to bring back ICU support as an option? Ok, I'll try to report the problem.

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Peter Geoghegan
Please report this as a bug to the freebsd package maintainer. -- Peter Geoghegan (Sent from my phone)

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 07:43 PM, Peter Geoghegan wrote: I have no idea. Maybe something to do with not actually using the same collation provider? The freebsd package contains modifications to make icu work, or at least did. Possibly, you just stopped using the same collation provider, without even re

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Peter Geoghegan
I have no idea. Maybe something to do with not actually using the same collation provider? The freebsd package contains modifications to make icu work, or at least did. Possibly, you just stopped using the same collation provider, without even realizing it, because they switched over to a new prov

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
I wonder why the errors don't show up under 9.6.6 and only started appearing after the pg_upgrade to 10.1? Both severs naturally used the same OS environment.

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 07:01 PM, rihad wrote: [snip] foo=# select bt_index_check('index_users_on_email_and_type'); ERROR:  item order invariant violated for index "index_users_on_email_and_type" DETAIL:  Lower index tid=(3,25) (points to index tid=(26,1)) higher index tid=(3,26) (points to index tid=(27

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 05:32 PM, Peter Geoghegan wrote: On Fri, Dec 22, 2017 at 11:56 AM, rihad wrote: I forgot to mention the real problem: the mentioned unique constraint didn't work and allowed duplicate rows to get inserted into the table until the duplicates were manually removed the the index was

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Laurenz Albe
rihad wrote: > On 12/22/2017 05:09 PM, Laurenz Albe wrote: > > rihad wrote: > > > Hi there, this is a reproducible error. We recently pg_upgraded our > > > production database to 10.1 from 9.6.6. The upgrade runs fine with the > > > suggestion to analyze all data. > > > > [...] > > > > > Trying t

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 05:09 PM, Laurenz Albe wrote: rihad wrote: Hi there, this is a reproducible error. We recently pg_upgraded our production database to 10.1 from 9.6.6. The upgrade runs fine with the suggestion to analyze all data. [...] Trying to find data using the specified indices fails to fi

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
On 12/22/2017 05:09 PM, Magnus Hagander wrote: What platform are you on FreeBSD 10.3-RELEASE-p26 amd64 , how was PostgreSQL installed pkg install. Additionally, 10.1 needed for pg_upgrade to run was installed using portmaster -m PREFIX=/10.1, because FreeBSD doesn't allow more than one maj

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Peter Geoghegan
On Fri, Dec 22, 2017 at 11:56 AM, rihad wrote: > I forgot to mention the real problem: the mentioned unique constraint didn't > work and allowed duplicate rows to get inserted into the table until the > duplicates were manually removed the the index was rebuilt. You should run amcheck functions o

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Laurenz Albe
rihad wrote: > Hi there, this is a reproducible error. We recently pg_upgraded our > production database to 10.1 from 9.6.6. The upgrade runs fine with the > suggestion to analyze all data. [...] > Trying to find data using the specified indices fails to find matching rows: [...] > reindexing

Re: pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread Magnus Hagander
On Fri, Dec 22, 2017 at 12:53 PM, rihad wrote: > Hi there, this is a reproducible error. We recently pg_upgraded our > production database to 10.1 from 9.6.6. The upgrade runs fine with the > suggestion to analyze all data. > > [rihad@postgres-10-test]$ cat analyze_new_cluster.sh > #!/bin/sh > >

pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
I forgot to mention the real problem: the mentioned unique constraint didn't work and allowed duplicate rows to get inserted into the table until the duplicates were manually removed the the index was rebuilt.

pg_upgrading to 10.1 corrupts (varchar,varchar) UNIQUE indices

2017-12-22 Thread rihad
Hi there, this is a reproducible error. We recently pg_upgraded our production database to 10.1 from 9.6.6. The upgrade runs fine with the suggestion to analyze all data. [rihad@postgres-10-test]$ cat analyze_new_cluster.sh #!/bin/sh echo 'This script will generate minimal optimizer statistics