Re: psql: FATAL: the database system is starting up

2019-06-02 Thread Tom K
before this happened: https://microdevsys.com/wp/postgresql-pull-backup/ Thx, TK On Sun, Jun 2, 2019 at 4:48 PM Adrian Klaver wrote: > On 6/2/19 11:14 AM, Tom K wrote: > > > > > Nope. wal_level was set to replica, not logical. Unless you mean > > > > > >

Re: psql: FATAL: the database system is starting up

2019-06-02 Thread Tom K
On Sun, Jun 2, 2019 at 11:47 AM Adrian Klaver wrote: > On 6/1/19 8:07 PM, Tom K wrote: > > > > > https://www.postgresql.org/docs/10/app-postgres.html > > Single-User Mode > > ... > > > > and see if that at least gets the server started.

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 8:53 PM Adrian Klaver wrote: > On 6/1/19 5:32 PM, Tom K wrote: > > > > > > > Trying what we did above but on the second node: > > > > Was this node the primary? > > To me the below looks like there are replication slots set up tha

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
trust hostreplication all 127.0.0.1/32trust host replication all ::1/128 trust host replication replicator 127.0.0.1/32 md5 host replication replicator 192.168.0.108/0 md5 host replication replicator 192.168.0.124/0 md5 host replicat

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 8:25 PM Tom K wrote: > So the best bet will be trying to get through this error then: > > [ PSQL02 ] > PANIC:replication check point has wrong magic 0 instead of 307747550 > > > > > On Sat, Jun 1, 2019 at 8:21 PM Tom K wrote: > >>

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
So the best bet will be trying to get through this error then: [ PSQL02 ] PANIC:replication check point has wrong magic 0 instead of 307747550 On Sat, Jun 1, 2019 at 8:21 PM Tom K wrote: > > > On Sat, Jun 1, 2019 at 7:34 PM Adrian Klaver > wrote: > >> On 6/1/19

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 7:34 PM Adrian Klaver wrote: > On 6/1/19 4:22 PM, Tom K wrote: > > > > > > > > > Looks like this crash was far more catastrophic then I thought. By the > > looks of things, thinking on psql02 would be my best bet. > > >

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 7:12 PM Adrian Klaver wrote: > On 6/1/19 3:56 PM, Tom K wrote: > > > > > > > > > postgres=# select oid, datname from pg_database ; > >oid | datname > > ---+--- > > 13806 | postgres > > 1 | tem

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 6:36 PM Adrian Klaver wrote: > On 6/1/19 3:14 PM, Tom K wrote: > > > > > > > ** Correction. There is postgres, template1 and template2 but none of > > the other databases we had. > > In a psql session do: > > select oid, datna

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 6:39 PM Adrian Klaver wrote: > On 6/1/19 3:14 PM, Tom K wrote: > > > > > ** Correction. There is postgres, template1 and template2 but none of > > the other databases we had. > > Just noticed, is that really template2 or is it actually t

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 6:05 PM Tom K wrote: > > > On Sat, Jun 1, 2019 at 5:51 PM Adrian Klaver > wrote: > >> On 6/1/19 2:31 PM, Tom K wrote: >> > >> > >> >> > >> > Spoke too soon. There's no tables when it's started witho

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 5:51 PM Adrian Klaver wrote: > On 6/1/19 2:31 PM, Tom K wrote: > > > > > > > > > Spoke too soon. There's no tables when it's started without the > > recovery.conf file. > > Where there any errors in the start up? &

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 5:30 PM Tom K wrote: > > > On Sat, Jun 1, 2019 at 4:52 PM Adrian Klaver > wrote: > >> On 6/1/19 12:42 PM, Tom K wrote: >> > >> > >> >> > >> > Of note are the characters f2W below. I see nothing in the

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 4:52 PM Adrian Klaver wrote: > On 6/1/19 12:42 PM, Tom K wrote: > > > > > > > > > Of note are the characters f2W below. I see nothing in the postgres > > source code to indicate this is any recognizable postgres message. A >

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 4:11 PM Adrian Klaver wrote: > On 6/1/19 12:32 PM, Tom K wrote: > > > > > > > > > What if you move the recovery.conf file out? > > > > > > Will try. > > > > > > > > The below looks like miss

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 3:32 PM Tom K wrote: > > > On Sat, Jun 1, 2019 at 9:55 AM Adrian Klaver > wrote: > >> On 5/31/19 7:53 PM, Tom K wrote: >> > >> >> > There are two places to connect with the Patroni community: on >> github, >>

Re: psql: FATAL: the database system is starting up

2019-06-01 Thread Tom K
On Sat, Jun 1, 2019 at 9:55 AM Adrian Klaver wrote: > On 5/31/19 7:53 PM, Tom K wrote: > > > > > There are two places to connect with the Patroni community: on > github, > > via Issues and PRs, and on channel #patroni in the PostgreSQL Slack. > If > &g

Re: psql: FATAL: the database system is starting up

2019-05-31 Thread Tom K
On Wed, May 29, 2019 at 10:28 AM Adrian Klaver wrote: > On 5/28/19 6:59 PM, Tom K wrote: > > > > > > On Tue, May 28, 2019 at 9:53 AM Adrian Klaver > <mailto:adrian.kla...@aklaver.com>> wrote: > > > > > > > Correct. Master election occurs th

Re: psql: FATAL: the database system is starting up

2019-05-28 Thread Tom K
On Tue, May 28, 2019 at 9:53 AM Adrian Klaver wrote: > On 5/27/19 9:59 PM, Tom K wrote: > > Hey Guy's, > > > > > > I'm running Patroni w/ PostgreSQL 10, ETCD, Haproxy and Keepalived on 3 > > RHEL 7.6 VM's. Every now and then the underlying