Hi Jakub, Thanks for reviewing it. I'm resending the chain of patches here, with 0005 already merged + your suggestions applied.
Here is a summary of what I've fixed: > I've forgotten to support incremental mode (which means that I ended up issuing posix_fadvise SEQUENTIAL even in incremental mode 1. Fixed. Only issue posix_fadvise SEQUENTIAL for full backups and only issue posix_fadvise WILLNEED for incremental backups. > there's bug that when m_io_c = 0 you still issue posix_fadvise()s from 2nd callsite (no way to turn it off) and also with e.g. m_io_c = 1 we seem to issue an additional one. 2. Fixed this by only calling prefetch_next_incremental_run when m_io_c > 1. Also moved the call to happen after read_file_data_into_buffer instead of before. Both changes together make the prefetching logic a little more honest regarding I/O queue depth, i.e: - When m_io_c = 0 or m_io_c = 1: no prefetching at all, queue depth is constantly1 (the synchronous block read). - When m_io_c > 1 (e.g. 2): prefetch 2 initially > read 1 + prefetch 1 > read 1 + prefetch 1... on average, there will always be 2 I/O simultaneously (assuming fadvise calls are honored). > I'm not sure f that merging of adjacent blocks shouldn't be also capped up a io_combine_limit GUC 3. Agree that capping is good there. Though I think maintenance_io_concurrency can be reused for this purpose instead? I think a new GUC for this might be too specific. I currently have it implemented using m_io_c. Let me know your thoughts on this if you think otherwise. > I would rename pf_index to just prefetch_idx, so it would easier to read. 4. Agreed. Fixed. Thanks, Gustavo Oliveira
v22092026-0001-pg_basebackup-rename-the-blackhole-backup-.patch
Description: Binary data
v22092026-0002-pg_basebackup-add-new-client-blackhole-ben.patch
Description: Binary data
v22092026-0003-pg_basebackup-report-average-data-transfer.patch
Description: Binary data
v22092026-0004-basebackup-bump-SINK_BUFFER_LENGTH-to-256k.patch
Description: Binary data
v22092026-0005-basebackup-issue-posix_fadvise-for-more-ef.patch
Description: Binary data
v22092026-0006-pg_basebackup-elimiate-usage-of-libc-to-co.patch
Description: Binary data
v22092026-0007-pg_basebackup-preallocate-extracted-files-.patch
Description: Binary data
v22092026-0008-libpq-pg_basebackup-add-PQgetCopyDataInter.patch
Description: Binary data
v22092026-0009-pg_basebackup-add-support-for-Direct-I-O-a.patch
Description: Binary data
v22092026-0010-pg_basebackup-preallocate-DIO-writes-also-.patch
Description: Binary data
