Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Guillaume Lelarge
ter and replica databases: >>> - autovacuum_vacuum_scale_factor >>> - autovacuum_analyze_scale_factor >>> >>> Can the above settings be configured differently on the master and >>> replica databases in a streaming replication setup? I just tested t

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Franjo Stipanovic
17, 2025 at 4:34 PM Guillaume Lelarge wrote: > Hi, > > Le ven. 17 janv. 2025 à 15:38, Franjo Stipanovic a > écrit : > >> Question regarding autovacuum settings in a PostgreSQL streaming >> replication setup. Specifically, I am curious about whether it is possible

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Guillaume Lelarge
Hi, Le ven. 17 janv. 2025 à 15:38, Franjo Stipanovic a écrit : > Question regarding autovacuum settings in a PostgreSQL streaming > replication setup. Specifically, I am curious about whether it is possible > (and advisable) to have different values for the following settings on the

Re: Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Tom Lane
Franjo Stipanovic writes: > Question regarding autovacuum settings in a PostgreSQL streaming > replication setup. Specifically, I am curious about whether it is possible > (and advisable) to have different values for the following settings on the > master and repli

Different Autovacuum Settings on Master and Replica in Streaming Replication

2025-01-17 Thread Franjo Stipanovic
Question regarding autovacuum settings in a PostgreSQL streaming replication setup. Specifically, I am curious about whether it is possible (and advisable) to have different values for the following settings on the master and replica databases: - autovacuum_vacuum_scale_factor

Re: Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tim Gerber
Thanks for the thoughts, Tomas. I suspected storage only because data was missing. Everything I've looked at is not showing any indication of this. Here's the tuple data for each. Appears to look normal to me. ISSUE table: PRIMARY: issuedb=#=# SELECT ctid, xmin, xmax, issue_id as issue_id

Re: Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tomas Vondra
On 1/15/25 19:50, Tim Gerber wrote: > Hi All, > >... > > Archive mode is on.   I ran pg_amcheck and everything came back clean. I > know synchronous replication is an option, but it feels like something > else is going on and would like to get to the bottom of it. > > Any ideas on what could be ca

Data Out of Sync with Physical Streaming Replication

2025-01-15 Thread Tim Gerber
Hi All, We have run into an issue where the data for a primary and standby Postgres instances are out of sync. We are using streaming physical replication. We have encountered this issue several times over the last year on different Postgres Clusters. Postgres indicates everything is in sync but

Re: Streaming replication problem with collation

2024-12-20 Thread Tom Lane
Ekaterina Amez Gonzalez writes: > I tried what was suggested: reindexing and running "refresh collation" > alter after that and everything seems to work ok so this looks like an easy > wat to migrate from one server to another. Plus I feel more comfortable > using stre

Streaming replication problem with collation

2024-12-20 Thread Ekaterina Amez Gonzalez
different glibc version between systems before I started my tests, but I tried it anyway. I first upgraded CentOS version to 15 and then made a streaming replication to the other server (Rocky). After that I encountered the next warning when connecting to new hot standby database: WARNING: datab

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-25 Thread Subhash Udata
to use a switchover first? > You can upgrade the standby, then switchover to it. > (You could even don't switchover back, when the old primary > would be upgraded and synchonized). > > > > * After upgrading the old primary server to version 15.9, I > wan

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-25 Thread Ilya Anfimov
r upgrading the old primary server to version 15.9, I want to > configure it as a standby server and set up streaming replication > with the new primary server, which will still be running version > 15.0. > * Is it possible to establish stream

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 4:58 PM Adrian Klaver wrote: > On 11/24/24 13:00, Ron Johnson wrote: > > On Sun, Nov 24, 2024 at 2:55 PM Christophe Pettus > > wrote: > > > > > On Nov 24, 2024, at 09:15, Ron Johnson > > wrote: > > >

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Adrian Klaver
to receive data from the old primary server even after promotion until the connection string is altered. This might result in data inconsistency issues, preventing the logical subscribers from being able to continue replication from the new primary server. " https://www.postgresql.org/docs

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 2:55 PM Christophe Pettus wrote: > > On Nov 24, 2024, at 09:15, Ron Johnson wrote: > > > > Doesn't the existence of a replication slot force PG to retain WAL files > when replication is broken? > > It does. I don't recall if the OP said that they were using a persistent

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 09:15, Ron Johnson wrote: > > Doesn't the existence of a replication slot force PG to retain WAL files when > replication is broken? It does. I don't recall if the OP said that they were using a persistent replication slot or not; it's not as common with binary replic

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
Thank you, everyone, for your valuable clarifications and insights. Your guidance has been incredibly helpful in addressing my concerns and understanding the best approach to proceed. I truly appreciate the time and effort you’ve taken to assist me. On Sun, 24 Nov 2024 at 22:36, Christophe Pettus

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 09:03, Subhash Udata wrote: > When we shut down the standby, upgrade it, and then start it back up, will > the replication automatically resume from the primary to the standby? Assuming that the standby has access to any WAL generated during the shutdown (either still i

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 12:06 PM Christophe Pettus wrote: > > On Nov 24, 2024, at 09:03, Subhash Udata wrote: > > When we shut down the standby, upgrade it, and then start it back up, > will the replication automatically resume from the primary to the standby? > > Assuming that the standby has a

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Christophe Pettus
> On Nov 24, 2024, at 08:51, Subhash Udata wrote: > However, my concern lies in the fact that we are working with production > servers, where downtime is not acceptable. There is no way to upgrade community PostgreSQL, either to a new minor version or a new major version, with absolute zero

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
Thank you for your valuable suggestion! I have a question regarding the process: When we shut down the standby, upgrade it, and then start it back up, will the replication automatically resume from the primary to the standby? Looking forward to your clarification. 2) What do you mean by resource

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 11:52 AM Subhash Udata wrote: > I understand your point and appreciate the clarification. > > I have reviewed the references and now have a better understanding of the > minor upgrade process. > > However, my concern lies in the fact that we are working with production > s

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
> > > *Upgrade and Replication Compatibility*: > > > > > > * My plan is to perform a failover, promote the standby > server > > > (currently 15.0) to primary, and then upgrade the old > primary > > > server to versi

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
On Sun, Nov 24, 2024 at 11:41 AM Adrian Klaver wrote: > On 11/24/24 08:36, Subhash Udata wrote: > > The reason to upgrade from 15.0 to 15.9 is this > > https://www.postgresql.org/support/security/CVE-2024-10979/ > > > > > > Here it is

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Ron Johnson
you start it back up, replication from the still-15.0 primary will catch back up to the now-patched Secondary. Fail over to the Secondary (now new-Primary), and then patch old-Primary to 15.10. > >1. > - After upgrading the old primary server to version 15.9, I want to >

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Adrian Klaver
.9, I want >         to configure it as a standby server and set up streaming >         replication with the new primary server, which will still be >         running version 15.0. >       * Is it possible to establish streaming replication between

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
rrently 15.0) to primary, and then upgrade the old primary > > server to version 15.9. > > * After upgrading the old primary server to version 15.9, I want > > to configure it as a standby server and set up streaming > > replication with the new pr

Re: Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Adrian Klaver
server to version 15.9, I want to configure it as a standby server and set up streaming replication with the new primary server, which will still be running version 15.0. * Is it possible to establish streaming replication between these two versions (*15.0* as

Questions on Upgrading PostgreSQL from 15.0 to 15.9 and Setting Up Streaming Replication

2024-11-24 Thread Subhash Udata
as a standby server and set up streaming replication with the new primary server, which will still be running version 15.0. - Is it possible to establish streaming replication between these two versions (*15.0* as primary and *15.9* as standby)? 2. *Efficient Replication

Re: Streaming replication issue post upgrade from version 11 to 14 on windows 2016 Server

2024-08-06 Thread Muhammad Ikram
Hi Vamsi, We have been doing these steps to setup streaming replication --Primay-Server CREATE ROLE repl REPLICATION LOGIN PASSWORD 'your_password'; postgresql.conf listen_addresses = '*' wal_level = replica archive_mode = on archive_command = 'copy %p \\path_to_arc

Streaming replication issue post upgrade from version 11 to 14 on windows 2016 Server

2024-08-06 Thread Vamsi Chava
Hi Team, we have upgraded postgresql version 11 to 14 on windows server 2016, post upgrade we configured streaming replication. the data is not getting replicated. but compared to version 11, 14 is very different in configuring replication. Tried from documentation from postgres site. no luck

Question regarding automatically paused streaming replication

2024-06-10 Thread Ian van der Linde
Hi everyone I run a PostgreSQL 14 environment with one primary and two standby instances with streaming replication. I recently had a case where the streaming replication on one of the standby instances paused (seemingly automatically): 2024-06-01 01:11:38.482 CEST [4575476] @ app=LOG

pg_basebackup "Permission denied" error when initiating streaming replication

2024-05-07 Thread Frans Simmelvuo (Papula-Nevinpat)
Hi, I'm trying to initiate streaming replication between two Postgres 10 servers on Windows. The source cluster is of size ~450gb, and the cluster consists of the primary data folder, and two separated tablespaces. One of the tablespaces is configured with a symbolic link inside Windows.

Re: Failing streaming replication on PostgreSQL 14

2024-04-22 Thread Nicolas Seinlet
Hi, facing the same situation again, but this time, analyzing the wal with xxd shows a different pattern. I had no blocks of . The output of pg_waldump is: pg_waldump: fatal: error in WAL record at 11C/93F9FF70: invalid magic number in log segment 0001011C0093, offset 16384

Re: Failing streaming replication on PostgreSQL 14

2024-04-16 Thread Nicolas Seinlet
Hello, > What exactly is "cyphered ZFS"? Can you reproduce the problem with some > other filesystem? If it's something very unusual, it might well be a > bug in the filesystem. The filesystem is openzfs with native aes-256-gcm encryption: https://openzfs.github.io/openzfs-docs/man/master/7/zfspro

Re: Failing streaming replication on PostgreSQL 14

2024-04-15 Thread Alvaro Herrera
On 2024-Apr-15, Nicolas Seinlet wrote: > I'm using the Ubuntu/cyphered ZFS/PostgreSQL combination. I'm using > Ubuntu LTS (20.04 22.04) and provided ZFS/PostgreSQL with LTS > (PostgreSQL 12 on Ubuntu 20.04 and 14 on 22.04). What exactly is "cyphered ZFS"? Can you reproduce the problem with some

Re: Failing streaming replication on PostgreSQL 14

2024-04-15 Thread Nicolas Seinlet
On Monday, April 15th, 2024 at 14:36, Ron Johnson wrote: > On Mon, Apr 15, 2024 at 2:53 AM Nicolas Seinlet wrote: > > > Hello everyone, > > > > Since I moved some clusters from PostgreSQL 12 to 14, I noticed random > > failures in streaming replication. I

Re: Failing streaming replication on PostgreSQL 14

2024-04-15 Thread Ron Johnson
On Mon, Apr 15, 2024 at 2:53 AM Nicolas Seinlet wrote: > Hello everyone, > > Since I moved some clusters from PostgreSQL 12 to 14, I noticed random > failures in streaming replication. I say "random" mostly because I haven't > got the source of the issue. > &g

Failing streaming replication on PostgreSQL 14

2024-04-14 Thread Nicolas Seinlet
Hello everyone, Since I moved some clusters from PostgreSQL 12 to 14, I noticed random failures in streaming replication. I say "random" mostly because I haven't got the source of the issue. I'm using the Ubuntu/cyphered ZFS/PostgreSQL combination. I'm using Ubun

RE: Postgres 13 streaming replication standby not sending password, 'fe_sendauth: no password supplied'

2024-01-11 Thread Keaney, Will
On Wed, Jan 10, 2024 at 23:22 Ron Johnson wrote: >> On Wed, Jan 10, 2024 at 5:51 PM Keaney, Will >> wrote: >> Hello, >> >> I'm building a new 2-node Postgreql 13 streaming replication cluster. I'm >> able to clone the primary to the standby u

Re: Postgres 13 streaming replication standby not sending password, 'fe_sendauth: no password supplied'

2024-01-11 Thread Laurenz Albe
On Wed, 2024-01-10 at 22:51 +, Keaney, Will wrote: > However, the standby is unable to authenticate to the primary to begin > recovery during startup. > It logs an error, "FATAL: could not connect to the primary server: > fe_sendauth: no password supplied". > > pg_hba.conf on the primary: >

Re: Postgres 13 streaming replication standby not sending password, 'fe_sendauth: no password supplied'

2024-01-10 Thread Ron Johnson
On Wed, Jan 10, 2024 at 5:51 PM Keaney, Will wrote: > Hello, > > I'm building a new 2-node Postgreql 13 streaming replication cluster. I'm > able to clone the primary to the standby using pg_basebackup. > However, the standby is unable to authenticate to the primary t

Postgres 13 streaming replication standby not sending password, 'fe_sendauth: no password supplied'

2024-01-10 Thread Keaney, Will
Hello, I'm building a new 2-node Postgreql 13 streaming replication cluster. I'm able to clone the primary to the standby using pg_basebackup. However, the standby is unable to authenticate to the primary to begin recovery during startup. It logs an error, "FATAL: could not

Re: setting up streaming replication, part 2

2023-10-25 Thread Guillaume Lelarge
Hi, Le mer. 25 oct. 2023 à 02:29, Brad White a écrit : > I have the replication server set up and the streaming is working. > The latest data show up, as desired. > > 3 minor issues. > > 1) I also have the WAL files being copied to a common location from the > primary server with > archive

Re: setting up streaming replication, part 2

2023-10-25 Thread Brad White
From: Ron Sent: Wednesday, October 25, 2023 4:35:59 AM To: pgsql-generallists.postgresql.org Subject: Re: setting up streaming replication, part 2 On 10/24/23 23:47, Brad White wrote: On Tue, Oct 24, 2023, 9:02 PM Ron mailto:ronljohnso...@gmail.com>> wrote: On 10/24/23 19:29, Brad White

Re: setting up streaming replication, part 2

2023-10-25 Thread Ron
On 10/24/23 23:47, Brad White wrote: On Tue, Oct 24, 2023, 9:02 PM Ron wrote: On 10/24/23 19:29, Brad White wrote: > I have the replication server set up and the streaming is working. > The latest data show up, as desired. > > 3 minor issues. > > 2) I have the conn

Re: setting up streaming replication, part 2

2023-10-24 Thread Brad White
On Tue, Oct 24, 2023, 9:02 PM Ron wrote: > On 10/24/23 19:29, Brad White wrote: > > I have the replication server set up and the streaming is working. > > The latest data show up, as desired. > > > > 3 minor issues. > > > > > 2) I have the connection string set for the replication server to > con

Re: setting up streaming replication, part 2

2023-10-24 Thread Ron
py %p "DISKSTATION\\AccessData\\Prod\\WALfiles\\%f"' It's not clear to me how the replication server finds those files. It doesn't, because streaming replication continuous streams WAL records. I also have the cleanup set to go, but it doesn't seem to be cle

setting up streaming replication, part 2

2023-10-24 Thread Brad White
I have the replication server set up and the streaming is working. The latest data show up, as desired. 3 minor issues. 1) I also have the WAL files being copied to a common location from the primary server with archive_command = 'copy %p "DISKSTATION\\AccessData\\Prod\\WALfiles\\%f"'

Re: setting up streaming replication

2023-10-24 Thread Ron
automatically resumes if the secondary must temporarily be taken down. -- Wait. Are you saying that once I get streaming replication set up, it quits working when I reboot the servers once a week? Maybe with temporary slots, but definitely not with permanent slots. I just stopped a

Re: setting up streaming replication

2023-10-24 Thread Christophe Pettus
> On Oct 24, 2023, at 11:31, Brad White wrote: > Are you saying that once I get streaming replication set up, it quits working > when I reboot the servers once a week? Not unless the downtime is sufficiently long that the replica can't find the WAL information it needs. You

Re: setting up streaming replication

2023-10-24 Thread Brad White
es if the > secondary must temporarily be taken down. > > -- > Wait. Are you saying that once I get streaming replication set up, it quits working when I reboot the servers once a week? Thanks, Brad.

Re: setting up streaming replication

2023-10-24 Thread Ron
=basebackup -D $PGDATA --progress \   --checkpoint=fast -v \       --write-recovery-conf --wal-method=stream \       --create-slot --slot=pgstandby1 --compress=server-zstd pg_ctl start -wt90 -l ${MajVer}/pgstart_standby.log Source: https://www.tecmint.com/configure-po

Re: setting up streaming replication

2023-10-24 Thread David G. Johnston
On Monday, October 23, 2023, Brad White wrote: > I'm stumped. > > Using this command to set up the slave and replication on PG v 15: > > You must, by some means: Have PostgreSQL binaries installed on the machine that is to become the replica. Have the output of pg_basebackup present on the repli

Re: setting up streaming replication

2023-10-24 Thread b55white
here slot_name = 'pgstandby1';" > /usr/bin/rm -r $PGDATA/* > pg_basebackup --dbname=service=basebackup -D $PGDATA --progress \ > --checkpoint=fast -v \ > --write-recovery-conf --wal-method=stream \ > --create-slot --slot=pgstandby1 --compress=server-zstd > pg_ctl

Re: setting up streaming replication

2023-10-23 Thread Ron
e-slot --slot=pgstandby1 --compress=server-zstd pg_ctl start -wt90 -l ${MajVer}/pgstart_standby.log Source: https://www.tecmint.com/configure-postgresql-streaming-replication-in-centos-8/ -- Born in Arizona, moved to Babylonia.

setting up streaming replication

2023-10-23 Thread Brad White
I'm stumped. Using this command to set up the slave and replication on PG v 15: "C:\Program Files\PostgreSQL\15\bin\pg_basebackup" -h 192.168.1.126 -U pgrep_user -p 5433 -D "C:\Program Files\PostgreSQL\15\data" -Fp -Xs -R If I have PG running on the remote server and the data directory is intact,

Logical Streaming Replication stuck at specific LSN

2023-08-20 Thread Kristjan Mustkivi
Hello, I am doing a major version upgrade from Postgres standard version 11.15 (Debian 11.15-1.pgdg90+1) to Cybertec PGEE version 14.8_EE_1.1.5 (Ubuntu 14.8ee1.1.5-1.cybertec22.04+1). While I am waiting for the support case to be processed, I was hoping to get advice from the Community about how t

deb’s pg_upgradecluster(1) vs streaming replication

2023-06-17 Thread James Cloos
[ hmm. i thought i sent this here, not to pgsql-hackers. don't know what happened. resending... ] Has anyone recently tried updating a streaming replication cluster using debian’s pg_upgradecluster(1) on each node? Did things work well? My last attempt (11 to 13, as I recall) had issue

Re: synchronous streaming replication

2022-10-25 Thread j.emerlik
andby_streaming_delay=30" ? > > * On the other hand I wonder if the application should not be developed > in such a way to support replication > >of PostgreSQL configured as a streaming synchronous replication > cluster with "synchronous_commit=remote_apply" ? >

Re: synchronous streaming replication

2022-10-23 Thread Laurenz Albe
ed. >  * So I'm wondering if, in this configuration, it can work properly at all > without setting, for example, >"max_standby_streaming_delay=30" ? >  * On the other hand I wonder if the application should not be developed in > such a way to support replication

synchronous streaming replication

2022-10-22 Thread Joseph Kennedy
f the application should not be developed in such a way to support replication of PostgreSQL configured as a streaming synchronous replication cluster with "synchronous_commit=remote_apply" ? - Or perhaps "synchronous streaming replication" is a bad choice, maybe

Re: [(catch-ext)] Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Ivan N. Ivanov
Yes, pgbackrest seems the best solution for faster backup and restore. We can close the discussion here for now, replaying million WAL files is just slow and this is normal. On Thu, Aug 18, 2022 at 12:10 PM Vijaykumar Jain < vijaykumarjain.git...@gmail.com> wrote: > Sorry for top posting, from p

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Vijaykumar Jain
Sorry for top posting, from phone. But pgbackrest exactly helped with that. With compression and parallel process in backup, the backup and restore was quick. I used this, where I took a backup and immediately did a restore so less wals to replay, else wal replay is indeed slow. On Thu, Aug 18, 2

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-18 Thread Ivan N. Ivanov
Thank you, people. The big problem in my case, which I have not mentioned, is that I think the network is a bottleneck, because I am running pg_basebackup through internet from local country to Amazon instance in Germany and the speed in copying is around 50 MB/sec max, that is why it takes 2 days

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Vijaykumar Jain
backup. > > On 8/17/22 15:06, Ivan N. Ivanov wrote: > > I have a large database (~25 TB) and I want to set up streaming > > replication for the first time. > > > > My problem is that after completion of the pg_basebackup (which > completed > > for 2 days with -

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Ron
pg_backrest will certainly backup your data faster. It might be able to be used as a seed instead of pg_basebackup. On 8/17/22 15:06, Ivan N. Ivanov wrote: I have a large database (~25 TB) and I want to set up streaming replication for the first time. My problem is that after completion of

Re: [(catch-ext)] Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Ivan N. Ivanov
Thank you for your answer! I have found this tool and I will try it tomorrow to see if this "read-ahead" feature will speed up the process. On Wed, Aug 17, 2022 at 11:09 PM Christophe Pettus wrote: > > > > On Aug 17, 2022, at 13:06, Ivan N. Ivanov > wrote: > > > > How to speed up recovering of

Re: Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Christophe Pettus
> On Aug 17, 2022, at 13:06, Ivan N. Ivanov wrote: > > How to speed up recovering of WAL files? Since you are running on your own hardware, you might take a look at: https://github.com/TritonDataCenter/pg_prefaulter

Setting up streaming replication on large database (20+ TB) for the first time

2022-08-17 Thread Ivan N. Ivanov
I have a large database (~25 TB) and I want to set up streaming replication for the first time. My problem is that after completion of the pg_basebackup (which completed for 2 days with --wal-method=none) now PG is replaying the WAL files from the WAL archive directory but it can not keep up. The

Re: Streaming Replication, can't select row in backup without specifying collation

2022-07-25 Thread Adrian Klaver
On 7/25/22 15:36, Kelly Burkhart wrote: I have a primary and backup database running 12.10 and synced with streaming replication.  I have a simple table that is returning different results for the same query in the primary vs backup database. On the primary DB: > select tag::bytea, * f

Streaming Replication, can't select row in backup without specifying collation

2022-07-25 Thread Kelly Burkhart
I have a primary and backup database running 12.10 and synced with streaming replication. I have a simple table that is returning different results for the same query in the primary vs backup database. On the primary DB: > select tag::bytea, * from sentinel where

Re: message log merge (streaming replication)

2022-06-09 Thread Kyotaro Horiguchi
At Thu, 9 Jun 2022 16:26:24 +0900, Ian Lawrence Barwick wrote in > 2022年6月9日(木) 14:32 Peter Adlersburg : > > > > Dear fellow DBA's, > > > > > > While troubleshooting one of our production replication clusters (phys. > > streaming replication using

Re: message log merge (streaming replication)

2022-06-09 Thread Ian Lawrence Barwick
2022年6月9日(木) 14:32 Peter Adlersburg : > > Dear fellow DBA's, > > > While troubleshooting one of our production replication clusters (phys. > streaming replication using the patroni framework) > I stumbled over a - at least for me - strange phenomenon in the postgres

Fwd: message log merge (streaming replication)

2022-06-08 Thread Peter Adlersburg
Dear fellow DBA's, While troubleshooting one of our production replication clusters (phys. streaming replication using the patroni framework) I stumbled over a - at least for me - strange phenomenon in the postgres logs of the two cluster members: *** node-01 *** [postgres@db-node-01

Re: Query on WAL Optimization and Streaming Replication

2022-03-17 Thread Shukla, Pranjal
Ok, Then what will increase the size of the WAL? Sent from my iPhone > On 17-Mar-2022, at 8:08 PM, Laurenz Albe wrote: > On Thu, 2022-03-17 at 14:05 +, Shukla, Pranjal wrote: >> From the configuration we have, does it mean that the primary will retain 32 >> WAL's >> of 1 GB each and then s

Re: Query on WAL Optimization and Streaming Replication

2022-03-17 Thread Shukla, Pranjal
Hi Laurenze, From the configuration we have, does it mean that the primary will retain 32 WAL's of 1 GB each and then start evicting the first WAL as soon as the last one gets filled? In layman's term, 32GB is huge amount of data and I don't think that much changes during upgrades. In fact the t

Re: Query on WAL Optimization and Streaming Replication

2022-03-17 Thread Laurenz Albe
On Thu, 2022-03-17 at 14:05 +, Shukla, Pranjal wrote: > From the configuration we have, does it mean that the primary will retain 32 > WAL's > of 1 GB each and then start evicting the first WAL as soon as the last one > gets filled? > In layman's term, 32GB is huge amount of data and I don't

Re: Query on WAL Optimization and Streaming Replication

2022-03-17 Thread Laurenz Albe
On Thu, 2022-03-17 at 12:36 +, Shukla, Pranjal wrote: > uring upgrades of our application, we generally shutdown all Secondary servers > which are getting stream replicated from Primary Servers. This is to maintain > a copy of database on other servers should > we wish to revert (of course we t

Query on WAL Optimization and Streaming Replication

2022-03-17 Thread Shukla, Pranjal
During upgrades of our application, we generally shutdown all Secondary servers which are getting stream replicated from Primary Servers. This is to maintain a copy of database on other servers should we wish to revert (of course we take DB Backups too before starting the activity). After the ap

Re: Pause streaming replication

2021-11-10 Thread Laurenz Albe
On Wed, 2021-11-10 at 20:36 -0500, Rita wrote: > On Wed, Nov 10, 2021 at 7:24 PM Ben Chobot wrote: > >Rita wrote on 11/10/21 1:25 PM: > > > Hello. > > > > > > I am testing alerting on my primary and standby setup. I have async > > > replication working but I would like to temporarily pause it so

Re: Pause streaming replication

2021-11-10 Thread Michael Paquier
On Wed, Nov 10, 2021 at 08:36:45PM -0500, Rita wrote: > Yes, I have read the manual and seen this. It pauses the replication > (select pg_is_wal_replay_paused()). But on the primary, when I look at > pg_stat_replication, it still says 'streaming' in the state column. My > question was how do I get

Re: Pause streaming replication

2021-11-10 Thread Ben Chobot
ion, and my suggestion doesn't help you there. But I would say that, in my experience with monitoring streaming replication, I haven't cared so much about if the replica is streaming vs. rebuilding, but rather how far behind it has fallen when it is supposedly streaming. Pausing repli

Re: Pause streaming replication

2021-11-10 Thread Rita
Yes, I have read the manual and seen this. It pauses the replication (select pg_is_wal_replay_paused()). But on the primary, when I look at pg_stat_replication, it still says 'streaming' in the state column. My question was how do I get it from 'streaming' to anything else? ( https://www.postgres

Re: Pause streaming replication

2021-11-10 Thread Ben Chobot
Rita wrote on 11/10/21 1:25 PM: Hello. I am testing alerting on my primary and standby setup. I have async replication working but I would like to temporarily pause it so the value of 'state' isn't streaming. (select * from pg_stat_replication). How can I do that? By reading the fine manua

Pause streaming replication

2021-11-10 Thread Rita
Hello. I am testing alerting on my primary and standby setup. I have async replication working but I would like to temporarily pause it so the value of 'state' isn't streaming. (select * from pg_stat_replication). How can I do that? -- --- Get your facts first, then you can distort them as you

Re: Streaming replication versus Logical replication

2021-11-04 Thread Christophe Pettus
> On Nov 4, 2021, at 12:16, Alanoly Andrews wrote: > > Thanks, Ninad, for the response. > So, am I to understand that when there is a long-running query on the > subscriber, the vacuumed data updates from the publisher are held over on the > subscriber until the query completes? If so, where

RE: Streaming replication versus Logical replication

2021-11-04 Thread Alanoly Andrews
disk space (either on the publisher or on the subscriber)? Regards. Alanoly Andrews. From: Ninad Shah [mailto:nshah.postg...@gmail.com] Sent: Thursday, November 4, 2021 2:20 PM To: Alanoly Andrews Cc: pgsql-general@lists.postgresql.org Subject: Re: Streaming replication versus Logical replication

Re: Streaming replication versus Logical replication

2021-11-04 Thread Ninad Shah
administration, they are different entities. In case the subscriber has long-running queries, unlike streaming replication, it does not affect synchronisation operations. Regards, Ninad Shah On Thu, 4 Nov 2021 at 21:16, Alanoly Andrews wrote: > We are currently running some long-runn

Streaming replication versus Logical replication

2021-11-04 Thread Alanoly Andrews
We are currently running some long-running SELECT queries on the replication database in a streaming replication pair. Some of these queries can run for 1 hour or more. To avoid errors related to "data no more being available" on the replication due to vacuuming of old data on t

Re: streaming replication different versions

2021-10-06 Thread Marc Millas
mer. 6 oct. 2021 à 13:46, Thomas Kellerer a écrit : > >> Marc Millas schrieb am 06.10.2021 um 13:43: >> > on release 10, I remember reading something like: streaming >> replication is NOW upward compatible. >> > which could be understood as: its possible to have

Re: streaming replication different versions

2021-10-06 Thread Guillaume Lelarge
Le mer. 6 oct. 2021 à 13:46, Thomas Kellerer a écrit : > Marc Millas schrieb am 06.10.2021 um 13:43: > > on release 10, I remember reading something like: streaming replication > is NOW upward compatible. > > which could be understood as: its possible to have a master in rel 1

Re: streaming replication different versions

2021-10-06 Thread Thomas Kellerer
Marc Millas schrieb am 06.10.2021 um 13:43: > on release 10,  I remember reading something like: streaming replication is > NOW upward compatible. > which could be understood as: its possible to have a master in rel 10 and a > slave in rel 11. No, that's not possible. For stre

streaming replication different versions

2021-10-06 Thread Marc Millas
Hi, on release 10, I remember reading something like: streaming replication is NOW upward compatible. which could be understood as: its possible to have a master in rel 10 and a slave in rel 11. Can someone confirm (or not !) this ? if true, is the following scheme possible: a master and a

Re: Cluster fencing tool/software for PostgreSQL cluster with streaming replication

2021-08-25 Thread Jehan-Guillaume de Rorthais
On Mon, 16 Aug 2021 11:31:23 +0100 Vikas Sharma wrote: > Hi, > > I am planning for an enterprise grade PostgreSQL cluster and so looking for > the tool/softwares which will do the cluster management or fencing to avoid > split brain. Look at Pacemaker, pro support and devel by both RedHat and S

Re: Cluster fencing tool/software for PostgreSQL cluster with streaming replication

2021-08-16 Thread Bruce Momjian
t or fencing > > to avoid split brain. > > > > Please, could you let me know if there are some tools/software which can > > do that so it can be used in a production environment. > > > > PostgreSQL12 with streaming replication + repmgr and pgpool for > >

Re: Cluster fencing tool/software for PostgreSQL cluster with streaming replication

2021-08-16 Thread Mladen Gogala
to avoid split brain. Please, could you let me know if there are some tools/software which can do that so it can be used in a production environment. PostgreSQL12 with streaming replication + repmgr and pgpool for connection management and load balancing. Regards Vikas S -- Mladen

Cluster fencing tool/software for PostgreSQL cluster with streaming replication

2021-08-16 Thread Vikas Sharma
. PostgreSQL12 with streaming replication + repmgr and pgpool for connection management and load balancing. Regards Vikas S

Re: Streaming replication: PANIC on tertiary when secondary promoted

2021-06-17 Thread Alexey Bashtanov
On 16/06/2021 20:31, Alexey Bashtanov wrote: I had it "latest" as well. I'll try to reproduce it again tomorrow. replica -v -d "dbname=postgres port=5432" -U postgres I cannot quite reproduce it artificially. One more piece of detail: in the chain serverA->serverB->serverC->serverD when serv

Re: Streaming replication: PANIC on tertiary when secondary promoted

2021-06-16 Thread Alexey Bashtanov
@db:~/playground$ psql -p 5433 psql (14beta1) Type "help" for help. -- create a replica slot for cascading streaming replication postgres=# select pg_create_physical_replication_slot('cascading_replica'); pg_create_physical_replication_slot ---

  1   2   3   >