Re: locate DB corruption

2018-08-31 Thread Dave Peticolas
On Fri, Aug 31, 2018 at 5:19 PM Adrian Klaver wrote: > On 08/31/2018 08:51 AM, Dave Peticolas wrote: > > On Fri, Aug 31, 2018 at 8:14 AM Adrian Klaver > > wrote: > > > > On 08/31/2018 08:02 AM, Dave Peticolas wrote: > > > Hello, I'm running into the fol

Re: using a plpgsql function argument as a table column.

2018-08-31 Thread Shaun Savage
I fixed it by changed that data. all the years were in one row so I created a separate row for each year. create table tvalues as select t.key, year, value from values t   join lateral (values     (2021,t.y2021),     (2020,t.y2020),     (2019,t.y2019),     (2018,t.y2018),     (2017,t.y2017),    

Re: using a plpgsql function argument as a table column.

2018-08-31 Thread ss
I changed the data from the years in one row to each row has a year create table tvalues as select t.key, year, value from values t   join lateral (values     (2021,t.y2021),     (2020,t.y2020),     (2019,t.y2019),     (2018,t.y2018),     (2017,t.y2017),     (2016,t.y2016),     (2015,t.y2015),   

Re: locate DB corruption

2018-08-31 Thread Adrian Klaver
On 08/31/2018 08:51 AM, Dave Peticolas wrote: On Fri, Aug 31, 2018 at 8:14 AM Adrian Klaver > wrote: On 08/31/2018 08:02 AM, Dave Peticolas wrote: > Hello, I'm running into the following error running a large query on a > database restored from

Re: Autovacuum degrades all other operations by keeping all buffers dirty?

2018-08-31 Thread Andres Freund
On 2018-08-31 19:31:47 -0300, Alvaro Herrera wrote: > On 2018-Aug-31, David Pacheco wrote: > > > From reading the 9.6.3 source, it looks like the autovacuum process > > itself is single-threaded, and it reads pages essentially linearly > > from the relation (possibly skipping some). When the auto

Re: Autovacuum degrades all other operations by keeping all buffers dirty?

2018-08-31 Thread Alvaro Herrera
On 2018-Aug-31, David Pacheco wrote: > From reading the 9.6.3 source, it looks like the autovacuum process > itself is single-threaded, and it reads pages essentially linearly > from the relation (possibly skipping some). When the autovacuum > process needs to modify a page, it doesn't write it d

Autovacuum degrades all other operations by keeping all buffers dirty?

2018-08-31 Thread David Pacheco
Hello, We've been struggling with some major performance issues related to autovacuum. (I know this is a common problem.) For a while, we believed this was primarily related to I/O contention, but recent observations from our monitoring make me wonder if there's a deeper issue here, so I'm lookin

Re: locate DB corruption

2018-08-31 Thread Dave Peticolas
On Fri, Aug 31, 2018 at 8:14 AM Adrian Klaver wrote: > On 08/31/2018 08:02 AM, Dave Peticolas wrote: > > Hello, I'm running into the following error running a large query on a > > database restored from WAL replay: > > > > could not access status of transaction 330569126 > > DETAIL: Could not ope

Re: locate DB corruption

2018-08-31 Thread Adrian Klaver
On 08/31/2018 08:02 AM, Dave Peticolas wrote: Hello, I'm running into the following error running a large query on a database restored from WAL replay: could not access status of transaction 330569126 DETAIL: Could not open file "pg_clog/0C68": No such file or directory Postgres version? Wh

locate DB corruption

2018-08-31 Thread Dave Peticolas
Hello, I'm running into the following error running a large query on a database restored from WAL replay: could not access status of transaction 330569126 DETAIL: Could not open file "pg_clog/0C68": No such file or directory Searches of this mailing list seem to indicate this means a database row

Re: pg_upgrade fails saying function unaccent(text) doesn't exist

2018-08-31 Thread Adrian Klaver
On 08/30/2018 09:03 AM, Gunnlaugur Thor Briem wrote: \dFd unaccent List of text search dictionaries  Schema |   Name   | Description +--+-  public | unaccent | (1 row) \dFd+ unaccent     List of text search dictionaries  Schema |   Name   |    Temp

Re: very slow largeobject transfers through JDBC

2018-08-31 Thread Mate Varga
I see -- we could try that, though we're mostly using an ORM (Hibernate) to do this. Thanks! On Fri, Aug 31, 2018 at 3:57 PM Dmitry Igrishin wrote: > пт, 31 авг. 2018 г. в 16:35, Mate Varga : > > > > Hi, > > > > we're fetching binary data from pg_largeobject table. The data is not > very large,

Re: pg_upgrade fails saying function unaccent(text) doesn't exist

2018-08-31 Thread Adrian Klaver
On 08/30/2018 09:03 AM, Gunnlaugur Thor Briem wrote: \dFd unaccent List of text search dictionaries  Schema |   Name   | Description +--+-  public | unaccent | (1 row) \dFd+ unaccent     List of text search dictionaries  Schema |   Name   |    Temp

Re: very slow largeobject transfers through JDBC

2018-08-31 Thread Dmitry Igrishin
пт, 31 авг. 2018 г. в 16:35, Mate Varga : > > Hi, > > we're fetching binary data from pg_largeobject table. The data is not very > large, but we ended up storing it there. If I'm copying the data to a file > from the psql console, then it takes X time (e.g. a second), fetching it > through the J

very slow largeobject transfers through JDBC

2018-08-31 Thread Mate Varga
Hi, we're fetching binary data from pg_largeobject table. The data is not very large, but we ended up storing it there. If I'm copying the data to a file from the psql console, then it takes X time (e.g. a second), fetching it through the JDBC driver takes at least 10x more. We don't see this diff

Re: Ways to deal with large amount of columns;

2018-08-31 Thread Brent Wood
You need to normalise your data model... the basic dataset you describe will probably fit in one table with three columns, but that might be simplistic... All your aggregate queries are pretty basic SQL statements, which could be represented as views or made simpler to invoke by turning them int

Re: Ways to deal with large amount of columns;

2018-08-31 Thread Ben Madin
Hi - this is a spreadsheet model, not a database model, and could be modelled with three columns. The aggregate functions are an analytic issue, not a data issue. cheers Ben On 30 August 2018 at 17:13, a <372660...@qq.com> wrote: > Hi all: > > I need to make a table contains projected monthly