On Mon, Jul 6, 2020 at 5:24 PM Daniel Gustafsson wrote:
> > On 12 Jun 2020, at 19:28, Robert Haas wrote:
>
> > I am sure that nobody is arguing that the patch has to be beneficial
> > in all cases in order to justify applying it. However, there are
> > several good arguments against proceding wi
> On 12 Jun 2020, at 19:28, Robert Haas wrote:
> I am sure that nobody is arguing that the patch has to be beneficial
> in all cases in order to justify applying it. However, there are
> several good arguments against proceding with this patch:
This thread has stalled with no resolution to the r
On Thu, Jun 11, 2020 at 1:41 PM Hamid Akhtar wrote:
> As far I understand, parallel backup is not a mandatory performance feature,
> rather, one at user's discretion. This IMHO indicates that it will benefit
> some users and it may not others.
>
> IMHO, parallel backup has obvious performance be
As far I understand, parallel backup is not a mandatory performance
feature, rather, one at user's discretion. This IMHO indicates that it will
benefit some users and it may not others.
Taking a backup is an I/O intensive workload. So by parallelizing it
through multiple worker threads/processes,
On Thu, May 21, 2020 at 2:06 AM Rushabh Lathia wrote:
> Yes. My colleague Suraj tried this and here are the pg_stat_activity output
> files.
>
> Captured wait events after every 3 seconds during the backup for -
> 1: parallel backup for 100GB data with 4 workers
> (pg_stat_activity_normal_backup
On Thu, May 21, 2020 at 11:36 AM Rushabh Lathia
wrote:
>
> On Thu, May 21, 2020 at 10:47 AM Ahsan Hadi wrote:
>>
During an offlist discussion with Robert, he pointed out that current
basebackup's code doesn't account for the wait event for the reading
of files which can change
On Mon, May 4, 2020 at 6:22 PM Rushabh Lathia
wrote:
>
>
> On Thu, Apr 30, 2020 at 4:15 PM Amit Kapila
> wrote:
>
>> On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
>> wrote:
>> >
>> > Hi,
>> >
>> > We at EnterpriseDB did some performance testing around this parallel
>> backup to check how this i
On Thu, Apr 30, 2020 at 4:15 PM Amit Kapila wrote:
> On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
> wrote:
> >
> > Hi,
> >
> > We at EnterpriseDB did some performance testing around this parallel
> backup to check how this is beneficial and below are the results. In this
> testing, we run the b
On 2020-04-30 2:18 a.m., Sumanta Mukherjee wrote:
Hi,
Would it be possible to put in the absolute numbers of the perf
so that it is easier to understand the amount of improvement with
and without the patch and different loads and workers.
Here is the parameters used to record the perf data on
On Thu, Apr 30, 2020 at 4:15 PM Amit Kapila wrote:
>
> On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
> wrote:
> >
> > Hi,
> >
> > We at EnterpriseDB did some performance testing around this parallel backup
> > to check how this is beneficial and below are the results. In this testing,
> > we ru
On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
wrote:
>
> Hi,
>
> We at EnterpriseDB did some performance testing around this parallel backup
> to check how this is beneficial and below are the results. In this testing,
> we run the backup -
> 1) Without Asif’s patch
> 2) With Asif’s patch and co
Hi,
Would it be possible to put in the absolute numbers of the perf
so that it is easier to understand the amount of improvement with
and without the patch and different loads and workers.
I am also unsure why the swapper is taking such a huge percentage of the
absolute time
in the base run of ju
On Mon, Apr 27, 2020 at 10:23 PM David Zhang wrote:
>
> Hi,
>
> Here is the parallel backup performance test results with and without
> the patch "parallel_backup_v15" on AWS cloud environment. Two
> "t2.xlarge" machines were used: one for Postgres server and the other
> one for pg_basebackup with
Hi,
Here is the parallel backup performance test results with and without
the patch "parallel_backup_v15" on AWS cloud environment. Two
"t2.xlarge" machines were used: one for Postgres server and the other
one for pg_basebackup with the same machine configuration showing below.
Machine confi
On Thu, Apr 23, 2020 at 1:47 PM Asif Rehman wrote:
>
>
> On Thu, Apr 23, 2020 at 11:43 AM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>>
>>
>> On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman
>> wrote:
>>
>>>
>>> Hi Dipesh,
>>>
>>> The rebased and updated patch is attache
On Thu, Apr 23, 2020 at 11:43 AM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>
> On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman
> wrote:
>
>>
>> Hi Dipesh,
>>
>> The rebased and updated patch is attached. Its rebased to (9f2c4ede).
>>
>
> Make is failing for v15 patch.
>
>
On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman wrote:
>
> Hi Dipesh,
>
> The rebased and updated patch is attached. Its rebased to (9f2c4ede).
>
Make is failing for v15 patch.
gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Werror=vla -Wendif-labels
-Wmissi
On Wed, Apr 22, 2020 at 10:18 AM Asif Rehman wrote:
> I don't foresee memory to be a challenge here. Assuming a database containing
> 10240
> relation files (that max reach to 10 TB of size), the list will occupy
> approximately 102MB
> of space in memory. This obviously can be reduced, but it d
Hi Dipesh,
The rebased and updated patch is attached. Its rebased to (9f2c4ede).
> +typedef struct
> +{
> ...
> +} BackupFile;
> +
> +typedef struct
> +{
> ...
> +} BackupState;
>
> These structures need comments.
>
Done.
>
> +list_wal_files_opt_list:
> + SCONST SCONST
>
Hi Asif,
I am reviewing your recent patch and found the patch is not applicable on
latest master.
Could you please resolve the conflicts and update a new patch?
Thanks,
Dipesh
EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 21, 2020 at 5:26 PM Ahsan Hadi wrote:
>
> On Tue, Apr 21, 2020 at 4:50 PM Amit Kapila wrote:
>>
>> On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila wrote:
>> >
>> > On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
>> > >
>> > > I did some tests a while back, and here are the results. T
On Tue, Apr 21, 2020 at 4:50 PM Amit Kapila wrote:
> On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila
> wrote:
> >
> > On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman
> wrote:
> > >
> > > I did some tests a while back, and here are the results. The tests
> were done to simulate
> > > a live database env
On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila wrote:
>
> On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
> >
> > I did some tests a while back, and here are the results. The tests were
> > done to simulate
> > a live database environment using pgbench.
> >
> > machine configuration used for thi
On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
>
> I did some tests a while back, and here are the results. The tests were done
> to simulate
> a live database environment using pgbench.
>
> machine configuration used for this test:
> Instance Type:t2.xlarge
> Volume Type :io1
> Memo
On Tue, 21 Apr 2020 at 2:36 PM, Jeevan Ladhe
wrote:
> Hi Asif,
>
> On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman
> wrote:
>
>> Hi,
>>
>> I did some tests a while back, and here are the results. The tests were
>> done to simulate
>> a live database environment using pgbench.
>>
>> machine configura
Hi Asif,
On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
> Hi,
>
> I did some tests a while back, and here are the results. The tests were
> done to simulate
> a live database environment using pgbench.
>
> machine configuration used for this test:
> Instance Type:t2.xlarge
> Volume Type
Hi,
I did some tests a while back, and here are the results. The tests were
done to simulate
a live database environment using pgbench.
machine configuration used for this test:
Instance Type:t2.xlarge
Volume Type :io1
Memory (MiB) :16384
vCPU # :4
Architecture:
On Tue, Apr 14, 2020 at 8:07 PM Asif Rehman wrote:
>
> I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
>
>
Have we done any performance testing with this patch to see the benefits?
If so, can you point me to the results? If not, then can we perform some
tests on
On Tue, Apr 14, 2020 at 5:33 PM Asif Rehman wrote:
>
>
> On Wed, Apr 8, 2020 at 6:53 PM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>>
>>
>> On Tue, Apr 7, 2020 at 9:44 PM Asif Rehman
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>>>
On Tue, Apr 14, 2020 at 7:37 PM Asif Rehman wrote:
>
>
> On Tue, Apr 14, 2020 at 6:32 PM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>> Hi Asif
>>
>> Getting the following error on Parallel backup when --no-manifest option
>> is used.
>>
>> [edb@localhost bin]$
>> [edb@localhost
On Wed, Apr 15, 2020 at 4:49 AM Ahsan Hadi wrote:
> Fair enough. Some of this is also due to backup related features i.e backup
> manifest, progress reporting that got committed to master towards the tail
> end of PG-13. Rushing to get parallel backup feature compatible with these
> features al
Hi Asif,
In below scenarios backup verification failed for tablespace, when backup
taken with parallel option.
without parallel for the same scenario pg_verifybackup is passed without
any error.
[edb@localhost bin]$ mkdir /tmp/test_bkp/tblsp1
[edb@localhost bin]$ ./psql postgres -p 5432 -c "creat
On Wed, 15 Apr 2020 at 1:49 AM, Robert Haas wrote:
> On Tue, Apr 14, 2020 at 10:37 AM Asif Rehman
> wrote:
> > I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
>
> +typedef struct
> +{
> ...
> +} BackupFile;
> +
> +typedef struct
> +{
> ...
> +} BackupState;
>
>
On Tue, Apr 14, 2020 at 10:37 AM Asif Rehman wrote:
> I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
+typedef struct
+{
...
+} BackupFile;
+
+typedef struct
+{
...
+} BackupState;
These structures need comments.
+list_wal_files_opt_list:
+
Hi Asif
Getting the following error on Parallel backup when --no-manifest option is
used.
[edb@localhost bin]$
[edb@localhost bin]$
[edb@localhost bin]$ ./pg_basebackup -v -j 5 -D
/home/edb/Desktop/backup/ --no-manifest
pg_basebackup: initiating base backup, waiting for checkpoint to complete
On Tue, Apr 7, 2020 at 9:44 PM Asif Rehman wrote:
> Hi,
>
> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>
> I have added the shared state as previously described. The new grammar
> changes
> are as follows:
>
> START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
> - This will gene
rebased and updated to current master (d025cf88ba). v12 is attahced.
Also, changed the grammar for LIST_WAL_FILES and SEND_FILE to:
- LIST_WAL_FILES 'startptr' 'endptr'
- SEND_FILE 'FILE' [NOVERIFY_CHECKSUMS]
On Wed, Apr 8, 2020 at 10:48 AM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterpris
Hi Asif,
Thanks for new patches.
Patches need to be rebased on head. Getting a failure while applying the
0003 patch.
edb@localhost postgresql]$ git apply
v11/0003-Parallel-Backup-Backend-Replication-commands.patch
error: patch failed: src/backend/storage/ipc/ipci.c:147
error: src/backend/storage
On Tue, Apr 7, 2020 at 1:25 PM Asif Rehman wrote:
> I will, however parallel backup is already quite a large patch. So I think we
> should first
> agree on the current work before adding a backup manifest and
> progress-reporting support.
It's going to be needed for commit, but it may make sens
On Fri, Apr 3, 2020 at 4:46 AM Asif Rehman wrote:
> Non-parallel backup already does the early error checking. I only intended
> to make parallel behave the same as non-parallel here. So, I agree with
> you that the behavior of parallel backup should be consistent with the
> non-parallel one. Ple
On Tue, Apr 7, 2020 at 10:03 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Tue, Apr 7, 2020 at 10:14 PM Asif Rehman
> wrote:
>
>> Hi,
>>
>> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>>
>> I have added the shared state as previously described. The new gram
On Tue, Apr 7, 2020 at 10:14 PM Asif Rehman wrote:
> Hi,
>
> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>
> I have added the shared state as previously described. The new grammar
> changes
> are as follows:
>
> START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
> - This will gen
Hi,
Thanks, Kashif and Rajkumar. I have fixed the reported issues.
I have added the shared state as previously described. The new grammar
changes
are as follows:
START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
- This will generate a unique backupid using pg_strong_random(16) and
hex-encoded
On Fri, Apr 3, 2020 at 3:01 PM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> Hi Asif
>
> When a non-existent slot is used with tablespace then correct error is
> displayed but then the backup folder is not cleaned and leaves a corrupt
> backup.
>
> Steps
> ===
>
> edb@localhost bi
Asif,
After recent backup manifest addition, patches needed to rebase and
reconsideration of a few things like making sure that parallel backup
creates
a manifest file correctly or not etc.
--
Jeevan Chalke
Associate Database Architect & Team Lead, Product Development
EnterpriseDB Corporation
Th
Hi Asif
When a non-existent slot is used with tablespace then correct error is
displayed but then the backup folder is not cleaned and leaves a corrupt
backup.
Steps
===
edb@localhost bin]$
[edb@localhost bin]$ mkdir /home/edb/tbl1
[edb@localhost bin]$ mkdir /home/edb/tbl_res
[edb@localhost
On Thu, Apr 2, 2020 at 8:45 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman
> wrote:
> >> Why would you need to do that? As long as the process where
> >> STOP_BACKUP can do the check, that seems good enough.
> >
> > Yes, but the user will get the error only after the STOP_BA
On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman wrote:
>> Why would you need to do that? As long as the process where
>> STOP_BACKUP can do the check, that seems good enough.
>
> Yes, but the user will get the error only after the STOP_BACKUP, not while
> the backup is
> in progress. So if the backup
On Thu, Apr 2, 2020 at 4:47 PM Robert Haas wrote:
> On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman
> wrote:
> > Yes, we are fetching a single file. However, SEND_FILES is still capable
> of fetching multiple files in one
> > go, that's why the name.
>
> I don't see why it should work that way. If w
On Thu, Apr 2, 2020 at 9:46 AM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> Does it fail to clean up the backup folder in all cases where the
>> backup failed, or just in this case?
>>
> The cleanup is done in the cases I have seen so far with base
> pg_basebackup functionality (not
On Thu, Apr 2, 2020 at 6:23 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan
> wrote:
> > Thanks alot Robert,
> > In this case the backup folder was not being emptied as the backup was
> failed, the cleanup should be done in this case too.
>
> Does it fail to clean up the ba
On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan
wrote:
> Thanks alot Robert,
> In this case the backup folder was not being emptied as the backup was
> failed, the cleanup should be done in this case too.
Does it fail to clean up the backup folder in all cases where the
backup failed, or just in t
On Thu, Apr 2, 2020 at 4:48 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>> The backup failed with errors "error: could not connect to server: could
>> not look up local user ID 1000: Too many open files" when the
>> max_wal_s
On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> The backup failed with errors "error: could not connect to server: could
> not look up local user ID 1000: Too many open files" when the
> max_wal_senders was set to 2000.
> The errors generated for the worke
On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman wrote:
> Yes, we are fetching a single file. However, SEND_FILES is still capable of
> fetching multiple files in one
> go, that's why the name.
I don't see why it should work that way. If we're fetching individual
files, why have an unused capability
Hi Asif
The backup failed with errors "error: could not connect to server: could
not look up local user ID 1000: Too many open files" when the
max_wal_senders was set to 2000.
The errors generated for the workers starting from backup worke=1017.
Please note that the backup directory was also not c
Hi Asif,
My colleague Kashif Zeeshan reported an issue off-list, posting here,
please take a look.
When executing two backups at the same time, getting FATAL error due to
max_wal_senders and instead of exit Backup got completed
And when tried to start the server from the backup cluster, getting
On Mon, Mar 30, 2020 at 3:44 PM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Thanks Asif,
>
> I have re-verified reported issue. expect standby backup, others are fixed.
>
Yes As Asif mentioned he is working on the standby issue and adding
bandwidth throttling functional
Thanks Asif,
I have re-verified reported issue. expect standby backup, others are fixed.
Thanks & Regards,
Rajkumar Raghuwanshi
On Fri, Mar 27, 2020 at 11:04 PM Asif Rehman wrote:
>
>
> On Wed, Mar 25, 2020 at 12:22 PM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>
On Wed, Mar 25, 2020 at 12:22 PM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Hi Asif,
>
> While testing further I observed parallel backup is not able to take
> backup of standby server.
>
> mkdir /tmp/archive_dir
> echo "archive_mode='on'">> data/postgresql.conf
> echo
Hi Asif,
While testing further I observed parallel backup is not able to take backup
of standby server.
mkdir /tmp/archive_dir
echo "archive_mode='on'">> data/postgresql.conf
echo "archive_command='cp %p /tmp/archive_dir/%f'">> data/postgresql.conf
./pg_ctl -D data -l logs start
./pg_basebackup
Hi Asif,
In another scenarios, bkp data is corrupted for tablespace. again this is
not reproducible everytime,
but If I am running the same set of commands I am getting the same error.
[edb@localhost bin]$ ./pg_ctl -D data -l logfile start
waiting for server to start done
server started
[edb@
Hi Asif,
On testing further, I found when taking backup with -R, pg_basebackup
crashed
this crash is not consistently reproducible.
[edb@localhost bin]$ ./psql postgres -p 5432 -c "create table test (a
text);"
CREATE TABLE
[edb@localhost bin]$ ./psql postgres -p 5432 -c "insert into test values
(
Hi Asif,
> Thanks Rajkumar. I have fixed the above issues and have rebased the patch
> to the latest master (b7f64c64).
> (V9 of the patches are attached).
>
I had a further review of the patches and here are my few observations:
1.
+/*
+ * stop_backup() - ends an online backup
+ *
+ * The func
On Mon, Mar 16, 2020 at 11:52 AM Asif Rehman wrote:
>
>
> On Mon, Mar 16, 2020 at 11:08 AM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>> Thanks for the patches.
>>
>> I have verified reported issues with new patches, issues are fixed now.
>>
>> I got another observat
On Mon, Mar 16, 2020 at 11:08 AM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Thanks for the patches.
>
> I have verified reported issues with new patches, issues are fixed now.
>
> I got another observation where If a new slot name given without -C
> option, it leads to
Thanks for the patches.
I have verified reported issues with new patches, issues are fixed now.
I got another observation where If a new slot name given without -C option,
it leads to server crash error.
[edb@localhost bin]$ ./pg_basebackup -p 5432 -j 4 -D /tmp/bkp --slot
test_bkp_slot
pg_baseba
Hi Asif
I have started testing this feature. I have applied v6 patch on commit
a069218163704c44a8996e7e98e765c56e2b9c8e (30 Jan).
I got few observations, please take a look.
*--if backup failed, backup directory is not getting removed.*
[edb@localhost bin]$ ./pg_basebackup -p 5432 --jobs=9 -D /tm
Hi,
I have created a commitfest entry.
https://commitfest.postgresql.org/27/2472/
On Mon, Feb 17, 2020 at 1:39 PM Asif Rehman wrote:
> Thanks Jeevan. Here is the documentation patch.
>
> On Mon, Feb 10, 2020 at 6:49 PM Jeevan Chalke <
> jeevan.cha...@enterprisedb.com> wrote:
>
>> Hi Asif,
>>
>
Thanks Jeevan. Here is the documentation patch.
On Mon, Feb 10, 2020 at 6:49 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
> Hi Asif,
>
> On Thu, Jan 30, 2020 at 7:10 PM Asif Rehman
> wrote:
>
>>
>> Here are the the updated patches, taking care of the issues pointed
>> earlier. This
Hi Asif,
On Thu, Jan 30, 2020 at 7:10 PM Asif Rehman wrote:
>
> Here are the the updated patches, taking care of the issues pointed
> earlier. This patch adds the following commands (with specified option):
>
> START_BACKUP [LABEL ''] [FAST]
> STOP_BACKUP [NOWAIT]
> LIST_TABLESPACES [PROGRESS]
>
On Thu, Dec 19, 2019 at 10:47 PM Robert Haas wrote:
> On Thu, Dec 12, 2019 at 10:20 AM Asif Rehman
> wrote:
> > I have updated the patches (v7 attached) and have taken care of all
> issues pointed by Jeevan, additionally
> > ran the pgindent on each patch. Furthermore, Command names have been
>
On Thu, Dec 12, 2019 at 10:20 AM Asif Rehman wrote:
> I have updated the patches (v7 attached) and have taken care of all issues
> pointed by Jeevan, additionally
> ran the pgindent on each patch. Furthermore, Command names have been renamed
> as suggested and I
> have simplified the SendFiles f
On Thu, Nov 28, 2019 at 12:57 AM Robert Haas wrote:
> On Wed, Nov 27, 2019 at 3:38 AM Jeevan Chalke
> wrote:
> > I am still not sure why we need SEND_BACKUP_FILELIST as a separate
> command.
> > Can't we return the file list with START_BACKUP itself?
>
> I had the same thought, but I think it's
On Wed, Nov 27, 2019 at 1:38 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Wed, Nov 13, 2019 at 7:04 PM Asif Rehman
> wrote:
>
>>
>> Sorry, I sent the wrong patches. Please see the correct version of the
>> patches (_v6).
>>
>
> Review comments on these patches:
>
> 1.
> +
On Wed, Nov 27, 2019 at 3:38 AM Jeevan Chalke
wrote:
> I am still not sure why we need SEND_BACKUP_FILELIST as a separate command.
> Can't we return the file list with START_BACKUP itself?
I had the same thought, but I think it's better to keep them separate.
Somebody might want to use the SEND_B
On Wed, Nov 13, 2019 at 7:04 PM Asif Rehman wrote:
>
> Sorry, I sent the wrong patches. Please see the correct version of the
> patches (_v6).
>
Review comments on these patches:
1.
+XLogRecPtrwal_location;
Looking at the other field names in basebackup_options structure, let's use
wal
On Fri, Nov 1, 2019 at 8:53 PM Robert Haas wrote:
> On Wed, Oct 30, 2019 at 10:16 AM Asif Rehman
> wrote:
> > 'startptr' is used by sendFile() during checksum verification. Since
> > SendBackupFiles() is using sendFIle we have to set a valid WAL location.
>
> Ugh, global variables.
>
> Why are S
On Wed, Oct 30, 2019 at 10:16 AM Asif Rehman wrote:
> 'startptr' is used by sendFile() during checksum verification. Since
> SendBackupFiles() is using sendFIle we have to set a valid WAL location.
Ugh, global variables.
Why are START_BACKUP, SEND_BACKUP_FILELIST, SEND_BACKUP_FILES, and
STOP_BAC
On Mon, Oct 28, 2019 at 8:29 PM Robert Haas wrote:
> On Mon, Oct 28, 2019 at 10:03 AM Asif Rehman
> wrote:
> > I have updated the patch to include the changes suggested by Jeevan.
> This patch also implements the thread workers instead of
> > processes and fetches a single file at a time. The ta
On Mon, Oct 28, 2019 at 10:03 AM Asif Rehman wrote:
> I have updated the patch to include the changes suggested by Jeevan. This
> patch also implements the thread workers instead of
> processes and fetches a single file at a time. The tar format has been
> disabled for first version of parallel
On Thu, Oct 24, 2019 at 3:21 PM Ibrar Ahmed wrote:
>
>
> On Fri, Oct 18, 2019 at 4:12 PM Jeevan Chalke <
> jeevan.cha...@enterprisedb.com> wrote:
>
>>
>>
>> On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman
>> wrote:
>>
>>>
>>> Attached are the updated patches.
>>>
>>
>> I had a quick look over these
On Fri, Oct 18, 2019 at 4:12 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman
> wrote:
>
>>
>> Attached are the updated patches.
>>
>
> I had a quick look over these changes and they look good overall.
> However, here are my few review c
On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman wrote:
>
> Attached are the updated patches.
>
I had a quick look over these changes and they look good overall.
However, here are my few review comments I caught while glancing the patches
0002 and 0003.
--- 0002 patch
1.
Can lsn option be renamed
On Thu, Oct 17, 2019 at 1:33 AM Jeevan Ladhe
wrote:
> I quickly tried to have a look at your 0001-refactor patch.
> Here are some comments:
>
> 1. The patch fails to compile.
>
> Sorry if I am missing something, but am not able to understand why in new
> function collectTablespaces() you have add
I quickly tried to have a look at your 0001-refactor patch.
Here are some comments:
1. The patch fails to compile.
Sorry if I am missing something, but am not able to understand why in new
function collectTablespaces() you have added an extra parameter NULL while
calling sendTablespace(), it fail
On Mon, Oct 7, 2019 at 6:35 PM Asif Rehman wrote:
>
>
> On Mon, Oct 7, 2019 at 6:05 PM Robert Haas wrote:
>
>> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman
>> wrote:
>> > Sure. Though the backup manifest patch calculates and includes the
>> checksum of backup files and is done
>> > while the file
On Mon, Oct 7, 2019 at 9:43 AM Ibrar Ahmed wrote:
> What about have an API to get the single file or list of files? We will use a
> single file in
> our application and other tools can get the benefit of list of files.
That sounds a bit speculative to me. Who is to say that anyone will
find that
On Mon, Oct 7, 2019 at 6:06 PM Robert Haas wrote:
> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> > Sure. Though the backup manifest patch calculates and includes the
> checksum of backup files and is done
> > while the file is being transferred to the frontend-end. The manifest
> file its
On Mon, Oct 7, 2019 at 6:05 PM Robert Haas wrote:
> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> > Sure. Though the backup manifest patch calculates and includes the
> checksum of backup files and is done
> > while the file is being transferred to the frontend-end. The manifest
> file its
On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> Sure. Though the backup manifest patch calculates and includes the checksum
> of backup files and is done
> while the file is being transferred to the frontend-end. The manifest file
> itself is copied at the
> very end of the backup. In parall
On Mon, Oct 7, 2019 at 1:52 PM Rushabh Lathia
wrote:
> Thanks Asif for the patch. I am opting this for a review. Patch is
> bit big, so here are very initial comments to make the review process
> easier.
>
Thanks Rushabh for reviewing the patch.
> 1) Patch seems doing lot of code shuffling,
Thanks Asif for the patch. I am opting this for a review. Patch is
bit big, so here are very initial comments to make the review process
easier.
1) Patch seems doing lot of code shuffling, I think it would be easy
to review if you can break the clean up patch separately.
Example:
a: setup_throt
On Fri, Oct 4, 2019 at 7:02 AM Asif Rehman wrote:
> Based on my understanding your main concern is that the files won't be
> distributed fairly i.e one worker might get a big file and take more time
> while others get done early with smaller files? In this approach I have
> created a list of fi
On Thu, Oct 3, 2019 at 6:40 PM Robert Haas wrote:
> On Fri, Sep 27, 2019 at 12:00 PM Asif Rehman
> wrote:
> >> > - SEND_FILES_CONTENTS (file1, file2,...) - returns the files in given
> list.
> >> > pg_basebackup will then send back a list of filenames in this
> command. This commands will be sen
On Fri, Sep 27, 2019 at 12:00 PM Asif Rehman wrote:
>> > - SEND_FILES_CONTENTS (file1, file2,...) - returns the files in given list.
>> > pg_basebackup will then send back a list of filenames in this command.
>> > This commands will be send by each worker and that worker will be getting
>> > the
Hi Asif,
I was looking at the patch and tried comipling it. However, got few errors
and warnings.
Fixed those in the attached patch.
On Fri, Sep 27, 2019 at 9:30 PM Asif Rehman wrote:
> Hi Robert,
>
> Thanks for the feedback. Please see the comments below:
>
> On Tue, Sep 24, 2019 at 10:53 PM
Hi Robert,
Thanks for the feedback. Please see the comments below:
On Tue, Sep 24, 2019 at 10:53 PM Robert Haas wrote:
> On Wed, Aug 21, 2019 at 9:53 AM Asif Rehman
> wrote:
> > - BASE_BACKUP [PARALLEL] - returns a list of files in PGDATA
> > If the parallel option is there, then it will only
On Wed, Aug 21, 2019 at 9:53 AM Asif Rehman wrote:
> - BASE_BACKUP [PARALLEL] - returns a list of files in PGDATA
> If the parallel option is there, then it will only do pg_start_backup, scans
> PGDATA and sends a list of file names.
So IIUC, this would mean that BASE_BACKUP without PARALLEL ret
Greetings,
* Ahsan Hadi (ahsan.h...@gmail.com) wrote:
> On Fri, 23 Aug 2019 at 10:26 PM, Stephen Frost wrote:
> > I would expect you to quickly want to support compression on the server
> > side, before the data is sent across the network, and possibly
> > encryption, and so it'd likely make sens
1 - 100 of 106 matches
Mail list logo
