Re: Fwd: increase insert into local table from remote oracle table preformance

main ideas are: - inserting directly to the right partition: perform as many inserts as pg partitions found in main_table_hist, like INSERT INTO 14/08/2018_value1 select * from remote_oracle_hist where day=to_date('14/08/2018','DD/MM/') and value='value1' please check execution plan (in O

Re: Calculating how much redo log space has been used

Hi, On 2018-08-14 18:51:34 +, Reza Taheri wrote: > Also, it looks like the full_page_writes parameter is the only thing > that can help reduce the log usage size There's also wal_compression. > Another requirement is a very short, 6-minute checkpoint time, which > means we will likely write

Calculating how much redo log space has been used

Hello PostgreSQL community, I am helping with a benchmarking exercise using PGSQL (I chair the TPC subcommittee that has released a benchmark using PGSQL). A requirement of the benchmark is having enough log space allocated for 8 hours of running without ne

Re: Bi-modal streaming replication throughput

Hi, On 2018-08-14 10:46:45 -0700, Andres Freund wrote: > On 2018-08-14 15:18:55 +0200, Alexis Lê-Quôc wrote: > > + 30.25%26.78% postgres postgres [.] mdnblocks > > This I've likely fixed ~two years back: > > http://archives.postgresql.org/message-id/72a98a639574d2e25ed946528485

Re: Bi-modal streaming replication throughput

Hi, On 2018-08-14 15:18:55 +0200, Alexis Lê-Quôc wrote: > We run > a cluster of > large, SSD-backed, i3.16xl (64 cores visible to Linux, ~500GB of RAM, with > 8GB of shared_buffers, fast NVMe drives) nodes > , each > running PG 9.3 > on linux > in a vanilla streaming asynchronous replication setup

Re: Bi-modal streaming replication throughput

On Tue, Aug 14, 2018 at 9:18 AM, Alexis Lê-Quôc wrote: > each running PG 9.3 on linux That is the oldest version which is still supported. There have been a lot of improvements since then, including to performance. You should see if an upgrade solves the problem. If not, at least you will

Bi-modal streaming replication throughput

Hi, I have been puzzled by very different replication performance (meaning 50-100x slower) between identical replicas (both in “hardware” and configuration) once the amount of data to replicate increases. I’ve gone down a number of dead ends and am missing something ( likely obvious ) that I hope

Re: Fwd: increase insert into local table from remote oracle table preformance

Hi, I'll try to answer all your question so that you will have more information about the situation : I have one main table that is called main_table_hist. The "main_table _hist" is partitioned by range (date column) and includes data that is considered as "history data" . I'm trying to copy the d