postgres materialized view refresh in parallel

2020-08-18 Thread Ayub M
Created materialized view in parallel by setting max_parallel_workers_per_gather to 4. I could see parallel sessions kicking off and creating the mview much faster than serial execution (10mins - parallel vs over an hour - serial). Now I want the refresh of the mview to happen taking around same t

Re: "Go" (lang) standard driver

2020-08-18 Thread David Gallagher
I can recommend sqlx: https://github.com/jmoiron/sqlx On Tue, Aug 18, 2020 at 4:56 PM Scottix wrote: > I have been using pgx and seems to be working well. > > I recommend if your just starting to use the v4 version. > > On Tue, Aug 18, 2020 at 5:53 AM Stephen Frost wrote: > >> Greetings, >> >>

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 1:31 PM Matthias van de Meent wrote: > Would this extra ordering not effectively be an extra tiebreaker in > the ordering, applied before the TID? I do not know the full > implications of that, but I believe that would not result in the > limitations that you are mentioning

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Matthias van de Meent
On Tue, 18 Aug 2020 at 22:00, Peter Geoghegan wrote: > > On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent > wrote: > > Given that the above could work, the current btree tuple ordering is > > not optimized for opclass-equal but datum image-distinct values: > > ordering of opclass-equal valu

Re: "Go" (lang) standard driver

2020-08-18 Thread Scottix
I have been using pgx and seems to be working well. I recommend if your just starting to use the v4 version. On Tue, Aug 18, 2020 at 5:53 AM Stephen Frost wrote: > Greetings, > > * Olivier Gautherot (ogauthe...@gautherot.net) wrote: > > Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit > : >

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Matthias van de Meent
On Tue, 18 Aug 2020 at 18:44, Peter Geoghegan wrote: > > On Mon, Aug 17, 2020 at 11:44 PM Matthias van de Meent > wrote: > > But, if the ordering of operator-class equal tuples is already > > system-defined, could the physical ordering of index tuples in a btree > > (with deduplication enabled fo

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Adrian Klaver
On 8/18/20 1:19 PM, Jason Myers wrote: > On Tue, Aug 18, 2020 at 3:49 PM Adrian Klaver mailto:adrian.kla...@aklaver.com>> wrote: > Have you tried with: > BEGIN; > CREATE TABLE some_table SELECT some_data FROM other_table LIMIT 1 WITH > NO DATA; > COMMIT; > > The above gets you the table

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Jason Myers
> On Tue, Aug 18, 2020 at 3:49 PM Adrian Klaver wrote: > > Have you tried with: > BEGIN; > CREATE TABLE some_table SELECT some_data FROM other_table LIMIT 1 WITH > NO DATA; > COMMIT; > > The above gets you the table structure, but no data. > > BEGIN; > INSERT into some_table SELECT * FROM other_t

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent wrote: > Given that the above could work, the current btree tuple ordering is > not optimized for opclass-equal but datum image-distinct values: > ordering of opclass-equal values is currently determined only by tid, > with as an example curre

Re: Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Adrian Klaver
On 8/18/20 12:35 PM, Jason Myers wrote: Postgres 12.4 I was directed in slack to mention here that we're being impacted by Postgres leaving orphaned pages in /base/ after a crash while a CREATE TABLE is being run in transaction. The issue is the same as the reproduction steps listed here [1]

Orphaned relations after crash/sigkill during CREATE TABLE

2020-08-18 Thread Jason Myers
Postgres 12.4 I was directed in slack to mention here that we're being impacted by Postgres leaving orphaned pages in /base/ after a crash while a CREATE TABLE is being run in transaction. The issue is the same as the reproduction steps listed here [1], that is: - Start a CREATE TABLE transactio

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 11:52 AM Matthias van de Meent wrote: > Deduplication does not need to destroy semantic differences? 'equal' > can (in my book) mean: > - 'opclass-equal', that is the opclass returns true for an equality check > - 'binary equal' or 'datum-equal' (? maybe incorrect term), th

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Tue, Aug 18, 2020 at 9:44 AM Peter Geoghegan wrote: > If we wanted to fix this for numeric, we'd have to invent a new > numeric datatype (called numeric2, say). That probably isn't as hard > as it sounds, since it could be part of the same B-Tree operator > family as numeric. It could also be i

Re: Index tuple deduplication limitations in pg13

2020-08-18 Thread Peter Geoghegan
On Mon, Aug 17, 2020 at 11:44 PM Matthias van de Meent wrote: > But, if the ordering of operator-class equal tuples is already > system-defined, could the physical ordering of index tuples in a btree > (with deduplication enabled for "unsafe" opclasses) be updated from > [index_columns, tid] to [i

Re: import XML

2020-08-18 Thread David G. Johnston
On Tue, Aug 18, 2020 at 8:27 AM PASCAL CROZET < pascal.cro...@qualis-consulting.com> wrote: > I want to import XML file into PG database table. > I've find functions to get the XML content of a cell after imported an XML > file with the pg_get_file function. > But, I want to explode the XML conten

import XML

2020-08-18 Thread PASCAL CROZET
Hi all, I want to import XML file into PG database table. I've find functions to get the XML content of a cell after imported an XML file with the pg_get_file function. But, I want to explode the XML content to colums. How can I do this ? PG 10 under Ubuntu 18 _

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Stephen writes: > * Adam Sjøgren (a...@koldfront.dk) wrote: >> >> We have 60 processes (workers) running on different machines accessing >> the database, that all grab jobs from a queue and update rows in a table >> after doing some calculations (which vary in time from <1s to perhaps a >> minute

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Tom Lane
Stephen Frost writes: > * Adam Sjøgren (a...@koldfront.dk) wrote: >> Sometimes new database logins slow down, from usually taking <0.05s to >> taking minutes. This is for psql as a normal user using Kerberos, for >> psql as the postgres superuser, for the web-application logging into the >> databa

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Srinivasa T N
On Tue, Aug 18, 2020, 8:04 PM Stephen Frost wrote: > Greetings, > > * Adam Sjøgren (a...@koldfront.dk) wrote: > > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > > sometimes exhibits a behaviour I can't quite understand: simply logging > > into the database starts to take mi

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Laurenz writes: > On Tue, 2020-08-18 at 14:53 +0200, Adam Sjøgren wrote: >> >> We have 60 processes (workers) running on different machines accessing >> the database, that all grab jobs from a queue and update rows in a table >> after doing some calculations (which vary in time from <1s to perhaps

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Stephen Frost
Greetings, * Adam Sjøgren (a...@koldfront.dk) wrote: > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > sometimes exhibits a behaviour I can't quite understand: simply logging > into the database starts to take minutes to complete. > > We have 60 processes (workers) running o

Re: Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Laurenz Albe
On Tue, 2020-08-18 at 14:53 +0200, Adam Sjøgren wrote: > We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which > sometimes exhibits a behaviour I can't quite understand: simply logging > into the database starts to take minutes to complete. > > We have 60 processes (workers) running

Database logins taking longer and longer, showing up as "authentication" in ps(1)

2020-08-18 Thread Adam Sjøgren
Hi, We have a PostgreSQL 11.3¹ running on an Ubuntu 16.04 server, which sometimes exhibits a behaviour I can't quite understand: simply logging into the database starts to take minutes to complete. We have 60 processes (workers) running on different machines accessing the database, that all gr

Re: Point in time recovery

2020-08-18 Thread Stephen Frost
Greetings, Please don't top-post. * Ron (ronljohnso...@gmail.com) wrote: > Search the log file for errors? Sadly, this doesn't actually work if you're really doing PITR- you need to look for the specific message saying "recovery stopping before commit" or similar, eg: 2020-08-18 12:55:31.240 UT

Re: "Go" (lang) standard driver

2020-08-18 Thread Stephen Frost
Greetings, * Olivier Gautherot (ogauthe...@gautherot.net) wrote: > Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit : > > -- Forwarded message - > > From: Tony Shelver > > Date: Tue, 18 Aug 2020 at 09:33 > > Subject: Re: "Go" (lang) standard driver > > To: Edson Richter > > >

Re: Point in time recovery

2020-08-18 Thread Ron
Search the log file for errors? Query the database(s) to verify that the latest data s there? On 8/18/20 5:10 AM, Daulat Ram wrote: Hello Team, I want to know the best way to ensure/verify that the Point in time recovery has done successfully after the crash and the restore. Thanks, --

Point in time recovery

2020-08-18 Thread Daulat Ram
Hello Team, I want to know the best way to ensure/verify that the Point in time recovery has done successfully after the crash and the restore. Thanks,

BUG? Slave don't reconnect to the master

2020-08-18 Thread Олег Самойлов
Hi all. I found some strange behaviour of postgres, which I recognise as a bug. First of all, let me explain situation. I created a "test bed" (not sure how to call it right), to test high availability clusters based on Pacemaker and PostgreSQL. The test bed consist of 12 virtual machines (on

Re: passing linux user to PG server as a variable ?

2020-08-18 Thread Daniel Verite
David Gauthier wrote: > I can avoid the error by just throwing a namespace in there... > atletx7-reg036:/home/dgauthie[ 214 ] --> setenv PGOPTIONS "-c > 'os.user=$USER' " > But once in, "show os.user" is undefined. It's documented to work [1], but you need to remove these single quotes. F

Re: "Go" (lang) standard driver

2020-08-18 Thread Olivier Gautherot
Hi Edson, Le mar. 18 août 2020 à 09:36, Tony Shelver a écrit : > -- Forwarded message - > From: Tony Shelver > Date: Tue, 18 Aug 2020 at 09:33 > Subject: Re: "Go" (lang) standard driver > To: Edson Richter > > > A quick Google search found https://github.com/lib/pq. Has 6.1K

Fwd: "Go" (lang) standard driver

2020-08-18 Thread Tony Shelver
-- Forwarded message - From: Tony Shelver Date: Tue, 18 Aug 2020 at 09:33 Subject: Re: "Go" (lang) standard driver To: Edson Richter A quick Google search found https://github.com/lib/pq. Has 6.1K stars so I would guess fairly well used. On Tue, 18 Aug 2020 at 05:56, Edson Ri