o
> "NOT in replication list" tables ? that can help reduce this kind of
> error.
>
> Thanks,
>
> James
>
>
>>
>>
>>
>> *From:* khan Affan
>> *Sent:* Wednesday, July 31, 2024 1:42 PM
>> *To:* James Pang
>> *Cc:* pgsql-
From:* khan Affan
> *Sent:* Wednesday, July 31, 2024 1:42 PM
> *To:* James Pang
> *Cc:* pgsql-performance@lists.postgresql.org
> *Subject:* Re: logical replication out of memory
>
>
>
> Hi Pang
>
> The text column is exceptionally large, Your server must be out of memory,
Hi Pang
The text column is exceptionally large, Your server must be out of memory,
Such a process ran out of memory while handling a large text column update.
I suggest using an S3 bucket for such files, Consider increasing the
memory-related configuration parameters, like work_mem, maintenance_
chaolpan) 於 2024年7月31日週三 上午10:28寫道：
>
>
>
>
> *From:* James Pang
> *Sent:* Wednesday, July 31, 2024 10:18 AM
> *To:* pgsql-performance@lists.postgresql.org
> *Subject:* logical replication out of memory
>
>
>
>
>
>
>
> 2024-07-31 00:01:02.795
>
2024-07-31 00:01:02.795
UTC:10.240.6.139(33068):repl13801@pgpodb:[3603770]:[10-1]:pgreps_13801ERROR:
out of memory
2024-07-31 00:01:02.795
UTC:10.240.6.139(33068):repl13801@pgpodb:[3603770]:[11-1]:pgreps_13801DETAIL:
Cannot enlarge string buffer containing 378355896 bytes by 756711422 more
bytes.
On 16/07/22, Rory Campbell-Lange (r...@campbell-lange.net) wrote:
> I'd be grateful for some comments on the advisability of using a large
> number of concurrent logical replication publications/subscriptions.
> Below I've set out the current environment and a suggested design.
On 17/07/22, Rick Otten (rottenwindf...@gmail.com) wrote:
> On Sat, Jul 16, 2022 at 12:07 PM Rory Campbell-Lange <
> r...@campbell-lange.net> wrote:
>
> > I'd be grateful for some comments on the advisability of using a large
> > number of concurrent logical replic
On Sat, Jul 16, 2022 at 12:07 PM Rory Campbell-Lange <
r...@campbell-lange.net> wrote:
> I'd be grateful for some comments on the advisability of using a large
> number of concurrent logical replication publications/subscriptions.
> Below I've set out the current en
I'd be grateful for some comments on the advisability of using a large
number of concurrent logical replication publications/subscriptions.
Below I've set out the current environment and a suggested design.
Apologies for the length of this email.
We presently have many hundreds of small
On Thu, Aug 5, 2021 at 12:57 AM Nikhil Shetty
wrote:
> Hi,
>
> Thank you for the suggestion.
>
> We tried by dropping indexes and it worked faster compared to what we saw
> earlier. We wanted to know if anybody has done any other changes that helps
> speed-up initial data load without dropping in
On Fri, 6 Aug 2021 at 00:15, Nikhil Shetty wrote:
> Hi Vijaykumar,
>
> Thanks for the details.
> In this method you are saying the pg_basebackup will make the initial load
> faster ?
>
We intend to bring only a few tables. Using pg_basebackup will clone an
> entire instance.
>
yeah. In that case
ical_create_subscriber.pl>
> Once you attain some reasonable sync state, you can drop the pglogical
> extension, and check if things continue fine.
> I have done something similar when upgrading from 9.6 to 11 using
> pglogical and then dropping the extension and it was smooth,
>
to the simple steps i am posting here that demonstrates
> pg_dump/pg_restore instead.
>
> If you cannot pause the app, then, you could look into how you
> could use pg_replication_origin_advance
> <https://www.postgresql.org/docs/13/functions-admin.html#PG-REPLICATION-ORIGIN-ADVANCE>
2 : Create Publication on the Primary CREATE PUBLICATION
FOR ALL TABLES;
Step 3 : Create Logical Replication Slot on the Primary SELECT * FROM
pg_create_logical_replication_slot('', 'pgoutput'); Step 4
: Create Subscription but do not enable the Subscription
CREATE SUBSCRIPT
ne.
I have done something similar when upgrading from 9.6 to 11 using pglogical
and then dropping the extension and it was smooth,
maybe you need to try this out and share if things works fine.
and
The 1-2-3 for PostgreSQL Logical Replication Using an RDS Snapshot -
Percona Database Performance B
On Thu, Aug 5, 2021 at 12:57 AM Nikhil Shetty
wrote:
> Hi,
>
> Thank you for the suggestion.
>
> We tried by dropping indexes and it worked faster compared to what we saw
> earlier. We wanted to know if anybody has done any other changes that helps
> speed-up initial data load without dropping in
the destination
>> first if possible. After that, you can add the indexes.
>>
>> Regards.
>>
>>
>> Nikhil Shetty , 4 Ağu 2021 Çar, 18:07 tarihinde
>> şunu yazdı:
>>
>>> Hi Team,
>>>
>>> We have a highly transactional system as th
;
>> We have a highly transactional system as the source of logical
>> replication and the database size is 500GB+. We are replicating all tables
>> from source using logical replication.
>>
>> For two tables the initial data load is very slow and it never completes
>&
> On Aug 4, 2021, at 08:06, Nikhil Shetty wrote:
>
> How can we increase the speed of the initial data load without dropping the
> indexes on destination?
You can do the usual steps of increasing checkpoint_timeout and max_wal_size
(since incoming logical replication changes ar
Team,
>>
>> We have a highly transactional system as the source of logical
>> replication and the database size is 500GB+. We are replicating all tables
>> from source using logical replication.
>>
>> For two tables the initial data load is very slow and it neve
f logical replication
> and the database size is 500GB+. We are replicating all tables from source
> using logical replication.
>
> For two tables the initial data load is very slow and it never completes
> even after 24hrs+
> Table size is under 100GB and index size is around 400GB.
>
&
Hi Team,
We have a highly transactional system as the source of logical replication
and the database size is 500GB+. We are replicating all tables from source
using logical replication.
For two tables the initial data load is very slow and it never completes
even after 24hrs+
Table size is under
perfect :-)
thanks
Marc MILLAS
Senior Architect
+33607850334
www.mokadb.com
On Fri, May 21, 2021 at 3:21 PM Andrew Dunstan wrote:
>
> On 5/21/21 8:41 AM, Marc Millas wrote:
> > Hi,
> >
> > my POC in postgres 12.(important ?)
> >
> > if I setup 2 postgres clusters, and create a publication in
On 5/21/21 8:41 AM, Marc Millas wrote:
> Hi,
>
> my POC in postgres 12.(important ?)
>
> if I setup 2 postgres clusters, and create a publication in one and a
> subscription in the other,
> and do on the pub an update which does not change the data (updating
> an existing record with same data) t
Hi,
my POC in postgres 12.(important ?)
if I setup 2 postgres clusters, and create a publication in one and a
subscription in the other,
and do on the pub an update which does not change the data (updating an
existing record with same data) then this (useless) update go through
replication.(ie co
Hi all,
*My top-level query is*: I'm using logical replication under pg 9.6 to do a
kind of change data capture and I'm seeing occasional extended periods of
significant lag. I'm not sure what conceptual model I'm missing in order to
understand why this happens.
*The de
th the “replica identity”, giving you an opportunity to go build the
index without losing all of the work.
> Will the logical replication flow be buffered by the replication slots
> during index creation and get in sync afterwards or will it conflict due to
> locking issues?
>
Em sex., 29 de nov. de 2019 às 17:06, Florian Philippon <
florian.philip...@doctolib.com> escreveu:
> Hello community!
>
> Hi Florian
> We are currently testing PostgreSQL 11's built-in logical replication. We
> are trying to initialize a subscriber (from scratch)
Hello community!
We are currently testing PostgreSQL 11's built-in logical replication. We
are trying to initialize a subscriber (from scratch) from a publisher with
a large database (+6TB) with around 220 tables.
We tweaked the configuration parameters below, both on publisher and
subsc
29 matches
Mail list logo
