Re: postgres 9.5 DB corruption

2019-07-25 Thread Adrian Klaver
On 7/25/19 10:24 AM, Thomas Tignor wrote: Hi Adrian, Thanks for responding. Below is the schema data for the tables where we always see corruption. You'll notice they have triggers for a postgres extension called Slony-I which provides replication service. It's not clear if/how that's a factor

Re: postgres 9.5 DB corruption

2019-07-25 Thread Thomas Tignor
Hi Adrian,Thanks for responding. Below is the schema data for the tables where we always see corruption. You'll notice they have triggers for a postgres extension called Slony-I which provides replication service. It's not clear if/how that's a factor, though. ams=# \d ams.alert_instance      

Re: postgres 9.5 DB corruption

2019-07-24 Thread Adrian Klaver
On 7/24/19 7:38 AM, Thomas Tignor wrote: Hello postgres community, Writing again to see if there are insights on this issue. We have had infrequent but recurring corruption since upgrading from postgres 9.1 to postgres 9.5. We are presently on 9.5.16. Our DB-facing app continually performs a

Re: postgres 9.5 DB corruption

2019-07-24 Thread Adrian Klaver
On 7/24/19 7:38 AM, Thomas Tignor wrote: Hello postgres community, Writing again to see if there are insights on this issue. We have had infrequent but recurring corruption since upgrading from postgres 9.1 to postgres 9.5. We are presently on 9.5.16. Our DB-facing app continually performs a

postgres 9.5 DB corruption

2019-07-24 Thread Thomas Tignor
Hello postgres community, Writing again to see if there are insights on this issue. We have had infrequent but recurring corruption since upgrading from postgres 9.1 to postgres 9.5. We are presently on 9.5.16. Our DB-facing app continually performs a mixture of DML, primarily inserts and update

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-26 Thread Thomas Tignor
Hello again Brad, and hello Tom. Thanks for writing, and Tom I'll add some extra thanks for the many google searches over the years which have been answered by your posts. :) It seems there's been some confusion on the data flow so I'll try to explain. The original "source" I referred to was our

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-26 Thread Tom Lane
"Brad Nicholson" writes: > Thomas Tignor wrote on 03/25/2019 08:25:49 PM: >> Thanks for writing. As I mentioned to Vijay, the "source" is a JVM >> using the postgres v42.0.0 JDBC driver. I do not believe we have any >> explicit encoding set, and so I expect the client encoding is >> SQL_ASCII. Th

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-26 Thread Brad Nicholson
Thomas Tignor wrote on 03/25/2019 08:25:49 PM: > > Hi Brad, > Thanks for writing. As I mentioned to Vijay, the "source" is a JVM > using the postgres v42.0.0 JDBC driver. I do not believe we have any > explicit encoding set, and so I expect the client encoding is > SQL_ASCII. The DB is most defi

Re: postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-26 Thread Thomas Tignor
Hi Laurenz. Thanks for writing. I can tell you that while the error message usually identifies just one byte (though sometimes up to three), inspection of the data has frequently shown several bytes impacted. It often seems that the corruption begins at one point in the row and continues to the

Re: postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-26 Thread Laurenz Albe
Thomas Tignor wrote: > We are experiencing intermittent DB corruption in postgres 9.5.14. We are > trying to > identify and eliminate all sources. We are using two independent services for > data > replication, Slony-I v2.2.6 and a custom service developed in-house. Both are > based > on COPY op

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-25 Thread Thomas Tignor
Hi Brad,Thanks for writing. As I mentioned to Vijay, the "source" is a JVM using the postgres v42.0.0 JDBC driver. I do not believe we have any explicit encoding set, and so I expect the client encoding is SQL_ASCII. The DB is most definitely UTF8. Our log shows no issue with the input data we'v

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-25 Thread Thomas Tignor
Hi Vijay,Thanks for writing. To your questions: i think the experts will chime in soon, but why do you think this as db corruption and not just a bad input? https://github.com/postgres/postgres/blob/master/src/pl/plperl/expected/plperl_lc_1.out tptignor: Our JVM logs data before the insert and

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-25 Thread Brad Nicholson
Vijaykumar Jain wrote on 03/25/2019 03:07:19 PM: > but why do you think this as db corruption and not just a bad input? > https://urldefense.proofpoint.com/v2/url? > u=https-3A__github.com_postgres_postgres_blob_master_src_pl_plperl_expected_plperl-5Flc-5F1.out&d=DwIFaQ&c=jf_iaSHvJObTbx- > siA

Re: [External] postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-25 Thread Vijaykumar Jain
i think the experts will chime in soon, but why do you think this as db corruption and not just a bad input? https://github.com/postgres/postgres/blob/master/src/pl/plperl/expected/plperl_lc_1.out or it may also be encoding issue. https://pganalyze.com/docs/log-insights/app-errors/U137 can you d

postgres 9.5 DB corruption: invalid byte sequence for encoding "UTF8"

2019-03-25 Thread Thomas Tignor
Hoping someone may be able to offer some guidance on this recurring problem. I am providing this "problem report" to the general list as I understand the bugs list requires a set of reproduction steps we do not yet have. Please advise if I have the process wrong. I have tried to provide all know