On 2016-02-04 16:54:58 +0100, Andres Freund wrote:
> Hi,
>
> Fabien asked me to post a new version of the checkpoint flushing patch
> series. While this isn't entirely ready for commit, I think we're
> getting closer.
>
> I don't want to post a full series right now, but my working state is
> available on
> http://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/checkpoint-flush
> git://git.postgresql.org/git/users/andresfreund/postgres.git checkpoint-flush
I've updated the git tree.
Here's the next two (the most important) patches of the series:
0001: Allow to trigger kernel writeback after a configurable number of writes.
0002: Checkpoint sorting and balancing.
For 0001 I've recently changed:
* Don't schedule writeback after smgrextend() - that defeats linux
delayed allocation mechanism, increasing fragmentation noticeably.
* Add docs for the new GUC variables
* comment polishing
* BackendWritebackContext now isn't dynamically allocated anymore
I think this patch primarily needs:
* review of the docs, not sure if they're easy enough to
understand. Some language polishing might also be needed.
* review of the writeback API, combined with the smgr/md.c changes.
* Currently *_flush_after can be set to a nonzero value, even if there's
no support for flushing on that platform. Imo that's ok, but perhaps
other people's opinion differ.
For 0002 I've recently changed:
* Removed the sort timing information, we've proven sufficiently that
it doesn't take a lot of time.
* Minor comment polishing.
I think this patch primarily needs:
* Benchmarking on FreeBSD/OSX to see whether we should enable the
mmap()/msync(MS_ASYNC) method by default. Unless somebody does so, I'm
inclined to leave it off till then.
Regards,
Andres
>From 58aee659417372f3dda4420d8f2a4f4d41c56d31 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Fri, 19 Feb 2016 12:13:05 -0800
Subject: [PATCH 1/4] Allow to trigger kernel writeback after a configurable
number of writes.
Currently writes to the main data files of postgres all go through the
OS page cache. This means that currently several operating systems can
end up collecting a large number of dirty buffers in their respective
page caches. When these dirty buffers are flushed to storage rapidly,
be it because of fsync(), timeouts, or dirty ratios, latency for other
writes can increase massively. This is the primary reason for regular
massive stalls observed in real world scenarios and artificial
benchmarks; on rotating disks stalls on the order of hundreds of seconds
have been observed.
On linux it is possible to control this by reducing the global dirty
limits significantly, reducing the above problem. But global
configuration is rather problematic because it'll affect other
applications; also PostgreSQL itself doesn't always generally want this
behavior, e.g. for temporary files it's undesirable.
Several operating systems allow some control over the kernel page
cache. Linux has sync_file_range(2), several posix systems have msync(2)
and posix_fadvise(2). sync_file_range(2) is preferable because it
requires no special setup, whereas msync() requires the to-be-flushed
range to be mmap'ed. For the purpose of flushing dirty data
posix_fadvise(2) is the worst alternative, as flushing dirty data is
just a side-effect of POSIX_FADV_DONTNEED, which also removes the pages
from the page cache. Thus the feature is enabled by default only on
linux, but can be enabled on all systems that have any of the above
APIs.
With the infrastructure added, writes made via checkpointer, bgwriter
and normal user backends can be flushed after a configurable number of
writes. Each of these sources of writes controlled by a separate GUC,
checkpointer_flush_after, bgwriter_flush_after and backend_flush_after
respectively; they're separate because the number of flushes that are
good are separate, and because the performance considerations of
controlled flushing for each of these are different.
A later patch will add checkpoint sorting - after that flushes from the
ckeckpoint will almost always be desirable. Bgwriter flushes are most of
the time going to be random, which are slow on lots of storage hardware.
Flushing in backends works well if the storage and bgwriter can keep up,
but if not it can have negative consequences. This patch is likely to
have negative performance consequences without checkpoint sorting, but
unfortunately so has sorting without flush control.
TODO:
* verify msync codepath
* properly detect mmap() && msync(MS_ASYNC) support, use it by default
if available and sync_file_range is *not* available
Discussion: alpine.DEB.2.10.1506011320000.28433@sto
Author: Fabien Coelho and Andres Freund
---
doc/src/sgml/config.sgml | 81 +++++++++++++++
doc/src/sgml/wal.sgml | 13 +++
src/backend/postmaster/bgwriter.c | 8 +-
src/backend/storage/buffer/buf_init.c | 5 +
src/backend/storage/buffer/bufmgr.c | 185 +++++++++++++++++++++++++++++++++-
src/backend/storage/file/copydir.c | 4 +-
src/backend/storage/file/fd.c | 153 +++++++++++++++++++++++++---
src/backend/storage/smgr/md.c | 49 +++++++++
src/backend/storage/smgr/smgr.c | 19 +++-
src/backend/utils/misc/guc.c | 36 +++++++
src/include/storage/buf_internals.h | 31 +++++-
src/include/storage/bufmgr.h | 22 +++-
src/include/storage/fd.h | 3 +-
src/include/storage/smgr.h | 4 +
src/tools/pgindent/typedefs.list | 2 +
15 files changed, 586 insertions(+), 29 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index a09ceb2..3dc6719 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1843,6 +1843,32 @@ include_dir 'conf.d'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="guc-bgwriter-flush-after" xreflabel="bgwriter_flush_after">
+ <term><varname>bgwriter_flush_after</varname> (<type>int</type>)
+ <indexterm>
+ <primary><varname>bgwriter_flush_after</> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Whenever more than <varname>bgwriter_flush_after</varname> bytes have
+ been written by the bgwriter, hint to OS to flush these writes to the
+ underlying storage. Doing so will limit the amount of dirty data in
+ the kernel's page cache, reducing the likelihood of stalls when fsync
+ is issued at the end of a checkpoint, or when the OS writes out data
+ in larger batches in the background. Often that will result in
+ greatly reduced transaction latency, but there also are some cases,
+ especially with workloads that are bigger than <xref
+ linkend="guc-shared-buffers">, but smaller than the OS's page cache,
+ where performance might degrade. This setting may have no effect on
+ some platforms. <literal>0</literal> disables controlled flushing.
+ The default is <literal>256Kb</> on Linux, <literal>0</> otherwise.
+ This parameter can only be set in the <filename>postgresql.conf</>
+ file or on the server command line.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
<para>
@@ -1944,6 +1970,35 @@ include_dir 'conf.d'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="guc-backend-flush-after" xreflabel="backend_flush_after">
+ <term><varname>backend_flush_after</varname> (<type>int</type>)
+ <indexterm>
+ <primary><varname>backend_flush_after</> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Whenever more than <varname>backend_flush_after</varname> bytes have
+ been written by a single backend, hint to OS to flush these writes to
+ the underlying storage. Doing so will limit the amount of dirty data
+ in the kernel's page cache, reducing the likelihood of stalls when
+ fsync is issued at the end of a checkpoint, or when the OS writes out
+ data in larger batches in the background. Often that will result in
+ greatly reduced transaction latency, but there also are some cases,
+ especially with workloads that are bigger than <xref
+ linkend="guc-shared-buffers">, but smaller than the OS's page cache,
+ where performance might degrade. Note that because
+ <varname>backend_flush_after</varname> is per-backend, the total
+ amount of dirty data in the kerne's page cache can be considerably
+ bigger than this setting. This setting may have no effect on some
+ platforms. <literal>0</literal> disables controlled flushing. The
+ default is <literal>256Kb</> on Linux, <literal>0</> otherwise. This
+ parameter can only be set in the <filename>postgresql.conf</> file or
+ on the server command line.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
</sect2>
</sect1>
@@ -2475,6 +2530,32 @@ include_dir 'conf.d'
</listitem>
</varlistentry>
+ <varlistentry id="guc-checkpoint-flush-after" xreflabel="checkpoint_flush_after">
+ <term><varname>checkpoint_flush_after</varname> (<type>int</type>)
+ <indexterm>
+ <primary><varname>checkpoint_flush_after</> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Whenever more than <varname>checkpoint_flush_after</varname> bytes
+ have been written while performing a checkpoint, hint to OS to flush
+ these writes to the underlying storage. Doing so will limit the
+ amount of dirty data in the kernel's page cache, reducing the
+ likelihood of stalls when fsync is issued at the end of a checkpoint,
+ or when the OS writes out data in larger batches in the background.
+ Often that will result in greatly reduced transaction latency, but
+ there also are some cases, especially with workloads that are bigger
+ than <xref linkend="guc-shared-buffers">, but smaller than the OS's
+ page cache, where performance might degrade. This setting may have no
+ effect on some platforms. <literal>0</literal> disables controlled
+ flushing. The default is <literal>256Kb</> on Linux, <literal>0</>
+ otherwise. This parameter can only be set in the
+ <filename>postgresql.conf</> file or on the server command line.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-checkpoint-warning" xreflabel="checkpoint_warning">
<term><varname>checkpoint_warning</varname> (<type>integer</type>)
<indexterm>
diff --git a/doc/src/sgml/wal.sgml b/doc/src/sgml/wal.sgml
index e3941c9..96496b0 100644
--- a/doc/src/sgml/wal.sgml
+++ b/doc/src/sgml/wal.sgml
@@ -546,6 +546,19 @@
</para>
<para>
+ On Linux and POSIX platforms <xref linkend="guc-checkpoint-flush-after">
+ allows to guide the OS that pages written by the checkpoint should be
+ flushed to disk. Otherwise, these pages may be kept in the OS's page
+ cache, inducing a stall when <literal>fsync</> is called later. This
+ setting helps to reduce transaction latency, but it also can an adverse
+ effect on performance; particularly for workloads that are bigger than
+ <xref linkend="guc-shared-buffers">, but smaller than the OS's page cache.
+ It should be beneficial for high write loads on HDD. This feature probably
+ brings no benefit on SSD, as the I/O write latency is small on such
+ hardware, thus it may be disabled.
+ </para>
+
+ <para>
The number of WAL segment files in <filename>pg_xlog</> directory depends on
<varname>min_wal_size</>, <varname>max_wal_size</> and
the amount of WAL generated in previous checkpoint cycles. When old log
diff --git a/src/backend/postmaster/bgwriter.c b/src/backend/postmaster/bgwriter.c
index 4ff4caf..7d0371d 100644
--- a/src/backend/postmaster/bgwriter.c
+++ b/src/backend/postmaster/bgwriter.c
@@ -111,6 +111,7 @@ BackgroundWriterMain(void)
sigjmp_buf local_sigjmp_buf;
MemoryContext bgwriter_context;
bool prev_hibernate;
+ WritebackContext wb_context;
/*
* Properly accept or ignore signals the postmaster might send us.
@@ -164,6 +165,8 @@ BackgroundWriterMain(void)
ALLOCSET_DEFAULT_MAXSIZE);
MemoryContextSwitchTo(bgwriter_context);
+ WritebackContextInit(&wb_context, &bgwriter_flush_after);
+
/*
* If an exception is encountered, processing resumes here.
*
@@ -208,6 +211,9 @@ BackgroundWriterMain(void)
/* Flush any leaked data in the top-level context */
MemoryContextResetAndDeleteChildren(bgwriter_context);
+ /* re-initilialize to avoid repeated errors causing problems */
+ WritebackContextInit(&wb_context, &bgwriter_flush_after);
+
/* Now we can allow interrupts again */
RESUME_INTERRUPTS();
@@ -269,7 +275,7 @@ BackgroundWriterMain(void)
/*
* Do one cycle of dirty-buffer writing.
*/
- can_hibernate = BgBufferSync();
+ can_hibernate = BgBufferSync(&wb_context);
/*
* Send off activity statistics to the stats collector
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index f013a4d..e10071d 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -23,6 +23,7 @@ char *BufferBlocks;
LWLockMinimallyPadded *BufferIOLWLockArray = NULL;
LWLockTranche BufferIOLWLockTranche;
LWLockTranche BufferContentLWLockTranche;
+WritebackContext BackendWritebackContext;
/*
@@ -149,6 +150,10 @@ InitBufferPool(void)
/* Init other shared buffer-management stuff */
StrategyInitialize(!foundDescs);
+
+ /* Initialize per-backend file flush context */
+ WritebackContextInit(&BackendWritebackContext,
+ &backend_flush_after);
}
/*
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index 7141eb8..cdbda0f 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -83,6 +83,14 @@ bool track_io_timing = false;
int effective_io_concurrency = 0;
/*
+ * GUC variables about triggering kernel writeback for buffers written; OS
+ * dependant defaults are set via the GUC mechanism.
+ */
+int checkpoint_flush_after = 0;
+int bgwriter_flush_after = 0;
+int backend_flush_after = 0;
+
+/*
* How many buffers PrefetchBuffer callers should try to stay ahead of their
* ReadBuffer calls by. This is maintained by the assign hook for
* effective_io_concurrency. Zero means "never prefetch". This value is
@@ -399,7 +407,7 @@ static bool PinBuffer(BufferDesc *buf, BufferAccessStrategy strategy);
static void PinBuffer_Locked(BufferDesc *buf);
static void UnpinBuffer(BufferDesc *buf, bool fixOwner);
static void BufferSync(int flags);
-static int SyncOneBuffer(int buf_id, bool skip_recently_used);
+static int SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *flush_context);
static void WaitIO(BufferDesc *buf);
static bool StartBufferIO(BufferDesc *buf, bool forInput);
static void TerminateBufferIO(BufferDesc *buf, bool clear_dirty,
@@ -416,6 +424,7 @@ static void FlushBuffer(BufferDesc *buf, SMgrRelation reln);
static void AtProcExit_Buffers(int code, Datum arg);
static void CheckForBufferLeaks(void);
static int rnode_comparator(const void *p1, const void *p2);
+static int buffertag_comparator(const void *p1, const void *p2);
/*
@@ -818,6 +827,12 @@ ReadBuffer_common(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
MemSet((char *) bufBlock, 0, BLCKSZ);
/* don't set checksum for all-zero page */
smgrextend(smgr, forkNum, blockNum, (char *) bufBlock, false);
+
+ /*
+ * XXX: Note that we're *not* doing a ScheduleBufferTagForWriteback
+ * here. At least on linux doing so defeats 'delayed allocation',
+ * leading to more fragmented files.
+ */
}
else
{
@@ -1084,6 +1099,9 @@ BufferAlloc(SMgrRelation smgr, char relpersistence, ForkNumber forkNum,
FlushBuffer(buf, NULL);
LWLockRelease(BufferDescriptorGetContentLock(buf));
+ ScheduleBufferTagForWriteback(&BackendWritebackContext,
+ &buf->tag);
+
TRACE_POSTGRESQL_BUFFER_WRITE_DIRTY_DONE(forkNum, blockNum,
smgr->smgr_rnode.node.spcNode,
smgr->smgr_rnode.node.dbNode,
@@ -1642,6 +1660,7 @@ BufferSync(int flags)
int num_to_write;
int num_written;
int mask = BM_DIRTY;
+ WritebackContext wb_context;
/* Make sure we can handle the pin inside SyncOneBuffer */
ResourceOwnerEnlargeBuffers(CurrentResourceOwner);
@@ -1694,6 +1713,9 @@ BufferSync(int flags)
if (num_to_write == 0)
return; /* nothing to do */
+
+ WritebackContextInit(&wb_context, &checkpoint_flush_after);
+
TRACE_POSTGRESQL_BUFFER_SYNC_START(NBuffers, num_to_write);
/*
@@ -1725,7 +1747,7 @@ BufferSync(int flags)
*/
if (bufHdr->flags & BM_CHECKPOINT_NEEDED)
{
- if (SyncOneBuffer(buf_id, false) & BUF_WRITTEN)
+ if (SyncOneBuffer(buf_id, false, &wb_context) & BUF_WRITTEN)
{
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
BgWriterStats.m_buf_written_checkpoints++;
@@ -1777,7 +1799,7 @@ BufferSync(int flags)
* bgwriter_lru_maxpages to 0.)
*/
bool
-BgBufferSync(void)
+BgBufferSync(WritebackContext *wb_context)
{
/* info obtained from freelist.c */
int strategy_buf_id;
@@ -2002,7 +2024,8 @@ BgBufferSync(void)
/* Execute the LRU scan */
while (num_to_scan > 0 && reusable_buffers < upcoming_alloc_est)
{
- int buffer_state = SyncOneBuffer(next_to_clean, true);
+ int buffer_state = SyncOneBuffer(next_to_clean, true,
+ wb_context);
if (++next_to_clean >= NBuffers)
{
@@ -2079,10 +2102,11 @@ BgBufferSync(void)
* Note: caller must have done ResourceOwnerEnlargeBuffers.
*/
static int
-SyncOneBuffer(int buf_id, bool skip_recently_used)
+SyncOneBuffer(int buf_id, bool skip_recently_used, WritebackContext *wb_context)
{
BufferDesc *bufHdr = GetBufferDescriptor(buf_id);
int result = 0;
+ BufferTag tag;
ReservePrivateRefCountEntry();
@@ -2123,8 +2147,13 @@ SyncOneBuffer(int buf_id, bool skip_recently_used)
FlushBuffer(bufHdr, NULL);
LWLockRelease(BufferDescriptorGetContentLock(bufHdr));
+
+ tag = bufHdr->tag;
+
UnpinBuffer(bufHdr, true);
+ ScheduleBufferTagForWriteback(wb_context, &tag);
+
return result | BUF_WRITTEN;
}
@@ -3724,3 +3753,149 @@ rnode_comparator(const void *p1, const void *p2)
else
return 0;
}
+
+
+/*
+ * BufferTag comparator.
+ */
+static int
+buffertag_comparator(const void *a, const void *b)
+{
+ const BufferTag *ba = (const BufferTag *) a;
+ const BufferTag *bb = (const BufferTag *) b;
+ int ret;
+
+ ret = rnode_comparator(&ba->rnode, &bb->rnode);
+
+ if (ret != 0)
+ return ret;
+
+ if (ba->forkNum < bb->forkNum)
+ return -1;
+ if (ba->forkNum > bb->forkNum)
+ return 1;
+
+ if (ba->blockNum < bb->blockNum)
+ return -1;
+ if (ba->blockNum > bb->blockNum)
+ return 1;
+
+ return 0;
+}
+
+
+/*
+ * Initialize a writeback context, discarding potential previous state.
+ *
+ * *max_coalesce is a pointer to a variable containing the current maximum
+ * number of writeback requests that will be coalesced into a bigger one. A
+ * value <= 0 means that no writeback control will be performed. max_pending
+ * is a pointer instead of an immediate value, so the coalesce limits can
+ * easily changed by the GUC mechanism, and so calling code does not have to
+ * check the current config variables.
+ */
+void
+WritebackContextInit(WritebackContext *context, int *max_pending)
+{
+ Assert(*max_pending <= WRITEBACK_MAX_PENDING_FLUSHES);
+
+ context->max_pending = max_pending;
+ context->nr_pending = 0;
+}
+
+
+/*
+ * Add buffer to list of pending writeback requests.
+ */
+void
+ScheduleBufferTagForWriteback(WritebackContext *context, BufferTag *tag)
+{
+ PendingWriteback *pending;
+
+ /* nothing to do if flushing is disabled */
+ if (*context->max_pending <= 0 && context->nr_pending <= 0)
+ return;
+
+ Assert(*context->max_pending <= WRITEBACK_MAX_PENDING_FLUSHES);
+
+ pending = &context->pending_writebacks[context->nr_pending++];
+
+ pending->tag = *tag;
+
+ if (context->nr_pending >= *context->max_pending)
+ IssuePendingWritebacks(context);
+}
+
+/*
+ * Issue all pending writeback requests, previously scheduled with
+ * ScheduleBufferTagForWriteback, to the OS.
+ *
+ * Because this is only used to improve the OSs IO scheduling we try to never
+ * error out - it's just a hint.
+ */
+void
+IssuePendingWritebacks(WritebackContext *context)
+{
+ int i;
+
+ if (context->nr_pending == 0)
+ return;
+
+ /*
+ * Executing the writes in-order can make them a lot faster, and allows to
+ * merge writeback requests to consecutive blocks into larger writebacks.
+ */
+ qsort(&context->pending_writebacks, context->nr_pending,
+ sizeof(PendingWriteback), buffertag_comparator);
+
+ /*
+ * Coalesce neighbouring writes, but nothing else. For that we iterate
+ * through the, now sorted, array of pending flushes, and look forward to
+ * find all neighbouring (or identical) writes.
+ */
+ for (i = 0; i < context->nr_pending; i++)
+ {
+ PendingWriteback *cur;
+ PendingWriteback *next;
+ SMgrRelation reln;
+ int ahead;
+ BufferTag tag;
+ Size nblocks = 1;
+
+ cur = &context->pending_writebacks[i];
+ tag = cur->tag;
+
+ /*
+ * Peek ahead, into following writeback requests, to see if they can
+ * be combined with the current one.
+ */
+ for (ahead = 0; i + ahead + 1 < context->nr_pending; ahead++)
+ {
+ next = &context->pending_writebacks[i + ahead + 1];
+
+ /* different file, skip */
+ if (!RelFileNodeEquals(cur->tag.rnode, next->tag.rnode) ||
+ cur->tag.forkNum != cur->tag.forkNum)
+ break;
+
+ /* ok, block flushed twice, skip */
+ if (cur->tag.blockNum == next->tag.blockNum)
+ continue;
+
+ /* only merge consecutive writes */
+ if (cur->tag.blockNum + 1 != next->tag.blockNum)
+ break;
+
+ nblocks++;
+ cur = next;
+ }
+
+ i += ahead;
+
+ /* and finally tell the kernel to write the data to storage */
+ reln = smgropen(tag.rnode, InvalidBackendId);
+ smgrwriteback(reln, tag.forkNum, tag.blockNum, nblocks);
+ }
+
+ context->nr_pending = 0;
+}
diff --git a/src/backend/storage/file/copydir.c b/src/backend/storage/file/copydir.c
index 522f420..a51ee81 100644
--- a/src/backend/storage/file/copydir.c
+++ b/src/backend/storage/file/copydir.c
@@ -190,9 +190,9 @@ copy_file(char *fromfile, char *tofile)
/*
* We fsync the files later but first flush them to avoid spamming the
* cache and hopefully get the kernel to start writing them out before
- * the fsync comes. Ignore any error, since it's only a hint.
+ * the fsync comes.
*/
- (void) pg_flush_data(dstfd, offset, nbytes);
+ pg_flush_data(dstfd, offset, nbytes);
}
if (CloseTransientFile(dstfd))
diff --git a/src/backend/storage/file/fd.c b/src/backend/storage/file/fd.c
index 1b30100..5b8a765 100644
--- a/src/backend/storage/file/fd.c
+++ b/src/backend/storage/file/fd.c
@@ -61,6 +61,9 @@
#include <sys/file.h>
#include <sys/param.h>
#include <sys/stat.h>
+#ifndef WIN32
+#include <sys/mman.h>
+#endif
#include <unistd.h>
#include <fcntl.h>
#ifdef HAVE_SYS_RESOURCE_H
@@ -82,6 +85,8 @@
/* Define PG_FLUSH_DATA_WORKS if we have an implementation for pg_flush_data */
#if defined(HAVE_SYNC_FILE_RANGE)
#define PG_FLUSH_DATA_WORKS 1
+#elif !defined(WIN32) && defined(MS_ASYNC)
+#define PG_FLUSH_DATA_WORKS 1
#elif defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_DONTNEED)
#define PG_FLUSH_DATA_WORKS 1
#endif
@@ -380,29 +385,126 @@ pg_fdatasync(int fd)
}
/*
- * pg_flush_data --- advise OS that the data described won't be needed soon
+ * pg_flush_data --- advise OS that the described dirty data should be flushed
*
- * Not all platforms have sync_file_range or posix_fadvise; treat as no-op
- * if not available. Also, treat as no-op if enableFsync is off; this is
- * because the call isn't free, and some platforms such as Linux will actually
- * block the requestor until the write is scheduled.
+ * An offset of 0 with an amount of 0 means that the entire file should be
+ * flushed.
*/
-int
-pg_flush_data(int fd, off_t offset, off_t amount)
+void
+pg_flush_data(int fd, off_t offset, off_t nbytes)
{
#ifdef PG_FLUSH_DATA_WORKS
- if (enableFsync)
- {
+
+ /*
+ * Right now file flushing is primarily used to avoid making later
+ * fsync()/fdatasync() calls have a less impact. Thus don't trigger
+ * flushes if fsyncs are disabled - that's a decision we might want to
+ * make configurable at some point.
+ */
+ if (!enableFsync)
+ return;
+
#if defined(HAVE_SYNC_FILE_RANGE)
- return sync_file_range(fd, offset, amount, SYNC_FILE_RANGE_WRITE);
+ {
+ int rc = 0;
+
+ /*
+ * sync_file_range(SYNC_FILE_RANGE_WRITE), currently linux specific,
+ * tells the OS that writeback for the passed in blocks should be
+ * started, but that we don't want to wait for completion. Note that
+ * this call might block if too much dirty data exists in the range.
+ * This is the preferrable method on OSs supporting it, as it works
+ * reliably when available (contrast to msync()) and doesn't flush out
+ * clean data (like FADV_DONTNEED).
+ */
+ rc = sync_file_range(fd, offset, nbytes,
+ SYNC_FILE_RANGE_WRITE);
+
+ /* don't error out, this is just a performance optimization */
+ if (rc != 0)
+ {
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not flush dirty data: %m")));
+ }
+ }
+#elif !defined(WIN32) && defined(MS_ASYNC)
+ {
+ int rc = 0;
+ void *p;
+
+ /*
+ * On many OSs msync() on a mmap'ed file triggers writeback. On linux
+ * it only does so when MS_SYNC is specified, but then it does the
+ * writeback synchronously. Luckily all common linux systems have
+ * sync_file_range(). This is preferrable over FADV_DONTNEED because
+ * it doesn't flush out clean data.
+ *
+ * We map the file (mmap()), tell the kernel to sync back the contents
+ * (msync()), and then remove the mapping again (munmap()).
+ */
+ p = mmap(NULL, context->nbytes,
+ PROT_READ | PROT_WRITE, MAP_SHARED,
+ context->fd, context->offset);
+ if (p == MAP_FAILED)
+ {
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not mmap while flushing dirty data in file \"%s\": %m",
+ context->filename ? context->filename : "")));
+ goto out;
+ }
+
+ rc = msync(p, context->nbytes, MS_ASYNC);
+ if (rc != 0)
+ {
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not flush dirty data in file \"%s\": %m",
+ context->filename ? context->filename : "")));
+ /* NB: need to fall through to munmap()! */
+ }
+
+ rc = munmap(p, context->nbytes);
+ if (rc != 0)
+ {
+ /* FATAL error because mapping would remain */
+ ereport(FATAL,
+ (errcode_for_file_access(),
+ errmsg("could not munmap while flushing blocks in file \"%s\": %m",
+ context->filename ? context->filename : "")));
+ }
+ }
#elif defined(USE_POSIX_FADVISE) && defined(POSIX_FADV_DONTNEED)
- return posix_fadvise(fd, offset, amount, POSIX_FADV_DONTNEED);
+ {
+ int rc = 0;
+
+ /*
+ * Signal the kernel that the passed in range should not be cached
+ * anymore. This has the, desired, side effect of writing out dirty
+ * data, and the, undesired, side effect of likely discarding useful
+ * clean cached blocks. For the latter reason this is the least
+ * preferrable method.
+ */
+
+ rc = posix_fadvise(context->fd, context->offset, context->nbytes,
+ POSIX_FADV_DONTNEED);
+
+ /* don't error out, this is just a performance optimization */
+ if (rc != 0)
+ {
+ ereport(WARNING,
+ (errcode_for_file_access(),
+ errmsg("could not flush dirty data in file \"%s\": %m",
+ context->filename ? context->filename : "")));
+ goto out;
+ }
+ }
#else
#error PG_FLUSH_DATA_WORKS should not have been defined
#endif
- }
-#endif
- return 0;
+
+#endif /* PG_FLUSH_DATA_WORKS */
}
@@ -1289,6 +1391,24 @@ FilePrefetch(File file, off_t offset, int amount)
#endif
}
+void
+FileWriteback(File file, off_t offset, int amount)
+{
+ int returnCode;
+
+ Assert(FileIsValid(file));
+
+ DO_DB(elog(LOG, "FileWriteback: %d (%s) " INT64_FORMAT " %d",
+ file, VfdCache[file].fileName,
+ (int64) offset, amount));
+
+ returnCode = FileAccess(file);
+ if (returnCode < 0)
+ return;
+
+ pg_flush_data(VfdCache[file].fd, offset, amount);
+}
+
int
FileRead(File file, char *buffer, int amount)
{
@@ -2655,9 +2775,10 @@ pre_sync_fname(const char *fname, bool isdir, int elevel)
}
/*
- * We ignore errors from pg_flush_data() because this is only a hint.
+ * pg_flush_data() ignores errors, which is ok because this is only a
+ * hint.
*/
- (void) pg_flush_data(fd, 0, 0);
+ pg_flush_data(fd, 0, 0);
(void) CloseTransientFile(fd);
}
diff --git a/src/backend/storage/smgr/md.c b/src/backend/storage/smgr/md.c
index f6b79a9..bb2b465 100644
--- a/src/backend/storage/smgr/md.c
+++ b/src/backend/storage/smgr/md.c
@@ -662,6 +662,55 @@ mdprefetch(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum)
#endif /* USE_PREFETCH */
}
+/*
+ * mdwriteback() -- Tell the kernel to write pages back to storage.
+ *
+ * This accepts a rnage of blocks because flushing several pages at once is
+ * considerably more efficient than doing so individually.
+ */
+void
+mdwriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum, int nblocks)
+{
+ off_t seekpos;
+ MdfdVec *v;
+
+ /*
+ * Issue flush requests in as few requests as possible; have to split at
+ * segment boundaries though, since those are actually separate files.
+ */
+ while (nblocks != 0)
+ {
+ int nflush = nblocks;
+ int segnum_start, segnum_end;
+
+ v = _mdfd_getseg(reln, forknum, blocknum, false, EXTENSION_RETURN_NULL);
+
+ /*
+ * We might be flushing buffers of already removed relations, that's
+ * ok, just ignore that case.
+ */
+ if (!v)
+ return;
+
+ /* compute offset inside the current segment */
+ segnum_start = blocknum / RELSEG_SIZE;
+
+ /* compute number of desired writes within the current segment */
+ segnum_end = (blocknum + nblocks - 1) / RELSEG_SIZE;
+ if (segnum_start != segnum_end)
+ nflush = RELSEG_SIZE - (blocknum % ((BlockNumber) RELSEG_SIZE) );
+
+ Assert(nflush >= 1);
+ Assert(nflush <= nblocks);
+
+ seekpos = (off_t) BLCKSZ * (blocknum % ((BlockNumber) RELSEG_SIZE));
+
+ FileWriteback(v->mdfd_vfd, seekpos, BLCKSZ * nflush);
+
+ nblocks -= nflush;
+ blocknum += nflush;
+ }
+}
/*
* mdread() -- Read the specified block from a relation.
diff --git a/src/backend/storage/smgr/smgr.c b/src/backend/storage/smgr/smgr.c
index 87ff358..2cae5aa 100644
--- a/src/backend/storage/smgr/smgr.c
+++ b/src/backend/storage/smgr/smgr.c
@@ -53,6 +53,8 @@ typedef struct f_smgr
BlockNumber blocknum, char *buffer);
void (*smgr_write) (SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, char *buffer, bool skipFsync);
+ void (*smgr_writeback) (SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks);
BlockNumber (*smgr_nblocks) (SMgrRelation reln, ForkNumber forknum);
void (*smgr_truncate) (SMgrRelation reln, ForkNumber forknum,
BlockNumber nblocks);
@@ -66,8 +68,8 @@ typedef struct f_smgr
static const f_smgr smgrsw[] = {
/* magnetic disk */
{mdinit, NULL, mdclose, mdcreate, mdexists, mdunlink, mdextend,
- mdprefetch, mdread, mdwrite, mdnblocks, mdtruncate, mdimmedsync,
- mdpreckpt, mdsync, mdpostckpt
+ mdprefetch, mdread, mdwrite, mdwriteback, mdnblocks, mdtruncate,
+ mdimmedsync, mdpreckpt, mdsync, mdpostckpt
}
};
@@ -649,6 +651,19 @@ smgrwrite(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
buffer, skipFsync);
}
+
+/*
+ * smgrwriteback() -- Trigger kernel writeback for the supplied range of
+ * blocks.
+ */
+void
+smgrwriteback(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
+ int nblocks)
+{
+ (*(smgrsw[reln->smgr_which].smgr_writeback)) (reln, forknum, blocknum,
+ nblocks);
+}
+
/*
* smgrnblocks() -- Calculate the number of blocks in the
* supplied relation.
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ea5a09a..789efbc 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -2385,6 +2385,42 @@ static struct config_int ConfigureNamesInt[] =
},
{
+ {"checkpoint_flush_after", PGC_SIGHUP, RESOURCES_ASYNCHRONOUS,
+ gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &checkpoint_flush_after,
+ /* see bufmgr.h: OS dependant default */
+ DEFAULT_CHECKPOINT_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"backend_flush_after", PGC_USERSET, WAL_CHECKPOINTS,
+ gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &backend_flush_after,
+ /* see bufmgr.h: OS dependant default */
+ DEFAULT_BACKEND_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
+ NULL, NULL, NULL
+ },
+
+ {
+ {"bgwriter_flush_after", PGC_SIGHUP, WAL_CHECKPOINTS,
+ gettext_noop("Number of pages after which previously performed writes are flushed to disk."),
+ NULL,
+ GUC_UNIT_BLOCKS
+ },
+ &bgwriter_flush_after,
+ /* see bufmgr.h: 16 on Linux, 0 otherwise */
+ DEFAULT_BGWRITER_FLUSH_AFTER, 0, WRITEBACK_MAX_PENDING_FLUSHES,
+ NULL, NULL, NULL
+ },
+
+ {
{"max_worker_processes",
PGC_POSTMASTER,
RESOURCES_ASYNCHRONOUS,
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index cbc4843..fe8b423 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -16,6 +16,7 @@
#define BUFMGR_INTERNALS_H
#include "storage/buf.h"
+#include "storage/bufmgr.h"
#include "storage/latch.h"
#include "storage/lwlock.h"
#include "storage/shmem.h"
@@ -208,16 +209,44 @@ extern PGDLLIMPORT LWLockMinimallyPadded *BufferIOLWLockArray;
#define UnlockBufHdr(bufHdr) SpinLockRelease(&(bufHdr)->buf_hdr_lock)
+/*
+ * The PendingWriteback & WritebackContext structure are used to keep
+ * information about pending flush requests to be issued to the OS.
+ */
+typedef struct PendingWriteback
+{
+ /* could store different types of pending flushes here */
+ BufferTag tag;
+} PendingWriteback;
+
+/* typedef forward declared in bufmgr.h */
+typedef struct WritebackContext
+{
+ /* max number of writeback requests to coalesce */
+ int *max_pending;
+
+ /* current number of pending writeback requests */
+ int nr_pending;
+
+ /* pending requests */
+ PendingWriteback pending_writebacks[WRITEBACK_MAX_PENDING_FLUSHES];
+} WritebackContext;
+
/* in buf_init.c */
extern PGDLLIMPORT BufferDescPadded *BufferDescriptors;
+extern PGDLLIMPORT WritebackContext BackendWritebackContext;
/* in localbuf.c */
extern BufferDesc *LocalBufferDescriptors;
/*
- * Internal routines: only called by bufmgr
+ * Internal buffer management routines
*/
+/* bufmgr.c */
+extern void WritebackContextInit(WritebackContext *context, int *max_coalesce);
+extern void IssuePendingWritebacks(WritebackContext *context);
+extern void ScheduleBufferTagForWriteback(WritebackContext *context, BufferTag *tag);
/* freelist.c */
extern BufferDesc *StrategyGetBuffer(BufferAccessStrategy strategy);
diff --git a/src/include/storage/bufmgr.h b/src/include/storage/bufmgr.h
index 92c4bc5..a4b1b37 100644
--- a/src/include/storage/bufmgr.h
+++ b/src/include/storage/bufmgr.h
@@ -45,16 +45,36 @@ typedef enum
* replay; otherwise same as RBM_NORMAL */
} ReadBufferMode;
+/* forward declared, to avoid having to expose buf_internals.h here */
+struct WritebackContext;
+
/* in globals.c ... this duplicates miscadmin.h */
extern PGDLLIMPORT int NBuffers;
/* in bufmgr.c */
+#define WRITEBACK_MAX_PENDING_FLUSHES 128
+
+/* FIXME: Also default to on for mmap && msync(MS_ASYNC)? */
+#ifdef HAVE_SYNC_FILE_RANGE
+#define DEFAULT_CHECKPOINT_FLUSH_AFTER 32
+#define DEFAULT_BACKEND_FLUSH_AFTER 16
+#define DEFAULT_BGWRITER_FLUSH_AFTER 64
+#else
+#define DEFAULT_CHECKPOINT_FLUSH_AFTER 0
+#define DEFAULT_BACKEND_FLUSH_AFTER 0
+#define DEFAULT_BGWRITER_FLUSH_AFTER 0
+#endif /* HAVE_SYNC_FILE_RANGE */
+
extern bool zero_damaged_pages;
extern int bgwriter_lru_maxpages;
extern double bgwriter_lru_multiplier;
extern bool track_io_timing;
extern int target_prefetch_pages;
+extern int checkpoint_flush_after;
+extern int backend_flush_after;
+extern int bgwriter_flush_after;
+
/* in buf_init.c */
extern PGDLLIMPORT char *BufferBlocks;
@@ -209,7 +229,7 @@ extern bool HoldingBufferPinThatDelaysRecovery(void);
extern void AbortBufferIO(void);
extern void BufmgrCommit(void);
-extern bool BgBufferSync(void);
+extern bool BgBufferSync(struct WritebackContext *wb_context);
extern void AtProcExit_LocalBuffers(void);
diff --git a/src/include/storage/fd.h b/src/include/storage/fd.h
index 4a3fccb..0f67760 100644
--- a/src/include/storage/fd.h
+++ b/src/include/storage/fd.h
@@ -75,6 +75,7 @@ extern int FileSync(File file);
extern off_t FileSeek(File file, off_t offset, int whence);
extern int FileTruncate(File file, off_t offset);
extern char *FilePathName(File file);
+extern void FileWriteback(File file, off_t offset, int amount);
/* Operations that allow use of regular stdio --- USE WITH CAUTION */
extern FILE *AllocateFile(const char *name, const char *mode);
@@ -112,7 +113,7 @@ extern int pg_fsync(int fd);
extern int pg_fsync_no_writethrough(int fd);
extern int pg_fsync_writethrough(int fd);
extern int pg_fdatasync(int fd);
-extern int pg_flush_data(int fd, off_t offset, off_t amount);
+extern void pg_flush_data(int fd, off_t offset, off_t amount);
extern void fsync_fname(char *fname, bool isdir);
extern void SyncDataDirectory(void);
diff --git a/src/include/storage/smgr.h b/src/include/storage/smgr.h
index a7267ea..0483fa3 100644
--- a/src/include/storage/smgr.h
+++ b/src/include/storage/smgr.h
@@ -96,6 +96,8 @@ extern void smgrread(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, char *buffer);
extern void smgrwrite(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, char *buffer, bool skipFsync);
+extern void smgrwriteback(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks);
extern BlockNumber smgrnblocks(SMgrRelation reln, ForkNumber forknum);
extern void smgrtruncate(SMgrRelation reln, ForkNumber forknum,
BlockNumber nblocks);
@@ -122,6 +124,8 @@ extern void mdread(SMgrRelation reln, ForkNumber forknum, BlockNumber blocknum,
char *buffer);
extern void mdwrite(SMgrRelation reln, ForkNumber forknum,
BlockNumber blocknum, char *buffer, bool skipFsync);
+extern void mdwriteback(SMgrRelation reln, ForkNumber forknum,
+ BlockNumber blocknum, int nblocks);
extern BlockNumber mdnblocks(SMgrRelation reln, ForkNumber forknum);
extern void mdtruncate(SMgrRelation reln, ForkNumber forknum,
BlockNumber nblocks);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d96896b..f501f55 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1411,6 +1411,7 @@ Pattern_Type
PendingOperationEntry
PendingRelDelete
PendingUnlinkEntry
+PendingWriteback
PerlInterpreter
Perl_ppaddr_t
Permutation
@@ -2142,6 +2143,7 @@ WriteBytePtr
WriteDataPtr
WriteExtraTocPtr
WriteFunc
+WritebackContext
X509
X509_NAME
X509_NAME_ENTRY
--
2.7.0.229.g701fa7f
>From 73e9eb9fa487aef370c0ffac710e71d0ee431b8d Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Fri, 19 Feb 2016 12:17:51 -0800
Subject: [PATCH 2/4] Checkpoint sorting and balancing.
Up to now checkpoints were written in the order they're in the
BufferDescriptors. That's nearly random in a lot of cases, which
performs badly on rotating media, but even on SSDs it causes slowdowns.
To avoid that, sort checkpoints before writing them out. We currently
sort by tablespace, relfilenode, fork and block number.
Previously that wasn't done out of fear of imbalance between
tablespaces, so additionally balance writes between tablespaces.
Another concern was that the relatively large allocation to sort the
buffers in might fail, preventing checkpoints from happening. Thus
pre-allocate the required memory in shared memory, at server startup.
This particularly makes it more efficient to have checkpoint flushing
enabled, because that'll often result in a lot of writes that can be
coalesced into one flush.
TODO:
* remove debugging output
Discussion: alpine.DEB.2.10.1506011320000.28433@sto
Author: Fabien Coelho and Andres Freund
---
src/backend/storage/buffer/README | 5 -
src/backend/storage/buffer/buf_init.c | 22 ++-
src/backend/storage/buffer/bufmgr.c | 289 +++++++++++++++++++++++++++++-----
src/include/storage/buf_internals.h | 18 +++
src/tools/pgindent/typedefs.list | 2 +
5 files changed, 291 insertions(+), 45 deletions(-)
diff --git a/src/backend/storage/buffer/README b/src/backend/storage/buffer/README
index c4a7668..dc12c8c 100644
--- a/src/backend/storage/buffer/README
+++ b/src/backend/storage/buffer/README
@@ -267,11 +267,6 @@ only needs to take the lock long enough to read the variable value, not
while scanning the buffers. (This is a very substantial improvement in
the contention cost of the writer compared to PG 8.0.)
-During a checkpoint, the writer's strategy must be to write every dirty
-buffer (pinned or not!). We may as well make it start this scan from
-nextVictimBuffer, however, so that the first-to-be-written pages are the
-ones that backends might otherwise have to write for themselves soon.
-
The background writer takes shared content lock on a buffer while writing it
out (and anyone else who flushes buffer contents to disk must do so too).
This ensures that the page image transferred to disk is reasonably consistent.
diff --git a/src/backend/storage/buffer/buf_init.c b/src/backend/storage/buffer/buf_init.c
index e10071d..bfa37f1 100644
--- a/src/backend/storage/buffer/buf_init.c
+++ b/src/backend/storage/buffer/buf_init.c
@@ -24,6 +24,7 @@ LWLockMinimallyPadded *BufferIOLWLockArray = NULL;
LWLockTranche BufferIOLWLockTranche;
LWLockTranche BufferContentLWLockTranche;
WritebackContext BackendWritebackContext;
+CkptSortItem *CkptBufferIds;
/*
@@ -70,7 +71,8 @@ InitBufferPool(void)
{
bool foundBufs,
foundDescs,
- foundIOLocks;
+ foundIOLocks,
+ foundBufCkpt;
/* Align descriptors to a cacheline boundary. */
BufferDescriptors = (BufferDescPadded *)
@@ -104,10 +106,21 @@ InitBufferPool(void)
LWLockRegisterTranche(LWTRANCHE_BUFFER_CONTENT,
&BufferContentLWLockTranche);
- if (foundDescs || foundBufs || foundIOLocks)
+ /*
+ * The array used to sort to-be-checkpointed buffer ids is located in
+ * shared memory, to avoid having to allocate significant amounts of
+ * memory at runtime. As that'd be in the middle of a checkpoint, or when
+ * the checkpointer is restarted, memory allocation failures would be
+ * painful.
+ */
+ CkptBufferIds = (CkptSortItem *)
+ ShmemInitStruct("Checkpoint BufferIds",
+ NBuffers * sizeof(CkptSortItem), &foundBufCkpt);
+
+ if (foundDescs || foundBufs || foundIOLocks || foundBufCkpt)
{
/* should find all of these, or none of them */
- Assert(foundDescs && foundBufs && foundIOLocks);
+ Assert(foundDescs && foundBufs && foundIOLocks && foundBufCkpt);
/* note: this path is only taken in EXEC_BACKEND case */
}
else
@@ -190,5 +203,8 @@ BufferShmemSize(void)
/* to allow aligning the above */
size = add_size(size, PG_CACHE_LINE_SIZE);
+ /* size of checkpoint sort array in bufmgr.c */
+ size = add_size(size, mul_size(NBuffers, sizeof(CkptSortItem)));
+
return size;
}
diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c
index cdbda0f..7a13997 100644
--- a/src/backend/storage/buffer/bufmgr.c
+++ b/src/backend/storage/buffer/bufmgr.c
@@ -37,6 +37,7 @@
#include "catalog/catalog.h"
#include "catalog/storage.h"
#include "executor/instrument.h"
+#include "lib/binaryheap.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
@@ -75,6 +76,34 @@ typedef struct PrivateRefCountEntry
/* 64 bytes, about the size of a cache line on common systems */
#define REFCOUNT_ARRAY_ENTRIES 8
+/*
+ * Status of buffers to checkpoint for a particular tablespace, used
+ * internally in BufferSync.
+ */
+typedef struct CkptTsStatus
+{
+ /* oid of the tablespace */
+ Oid tsId;
+
+ /*
+ * Checkpoint progress for this tablespace. To make progress comparable
+ * between tablespaces the progress is, for each tablespace, measured as a
+ * number between 0 and the total number of to-be-checkpointed pages. Each
+ * page checkpointed in this tablespace increments this space's progress
+ * by progress_slice.
+ */
+ float8 progress;
+ float8 progress_slice;
+
+ /* number of to-be checkpointed pages in this tablespace */
+ int num_to_scan;
+ /* already processed pages in this tablespace */
+ int num_scanned;
+
+ /* current offset in CkptBufferIds for this tablespace */
+ int index;
+} CkptTsStatus;
+
/* GUC variables */
bool zero_damaged_pages = false;
int bgwriter_lru_maxpages = 100;
@@ -425,6 +454,8 @@ static void AtProcExit_Buffers(int code, Datum arg);
static void CheckForBufferLeaks(void);
static int rnode_comparator(const void *p1, const void *p2);
static int buffertag_comparator(const void *p1, const void *p2);
+static int ckpt_buforder_comparator(const void *pa, const void *pb);
+static int ts_ckpt_progress_comparator(Datum a, Datum b, void *arg);
/*
@@ -1657,8 +1688,13 @@ BufferSync(int flags)
{
int buf_id;
int num_to_scan;
- int num_to_write;
+ int num_spaces;
+ int num_processed;
int num_written;
+ CkptTsStatus *per_ts_stat = NULL;
+ Oid last_tsid;
+ binaryheap *ts_heap;
+ int i;
int mask = BM_DIRTY;
WritebackContext wb_context;
@@ -1676,7 +1712,7 @@ BufferSync(int flags)
/*
* Loop over all buffers, and mark the ones that need to be written with
- * BM_CHECKPOINT_NEEDED. Count them as we go (num_to_write), so that we
+ * BM_CHECKPOINT_NEEDED. Count them as we go (num_to_scan), so that we
* can estimate how much work needs to be done.
*
* This allows us to write only those pages that were dirty when the
@@ -1690,7 +1726,7 @@ BufferSync(int flags)
* BM_CHECKPOINT_NEEDED still set. This is OK since any such buffer would
* certainly need to be written for the next checkpoint attempt, too.
*/
- num_to_write = 0;
+ num_to_scan = 0;
for (buf_id = 0; buf_id < NBuffers; buf_id++)
{
BufferDesc *bufHdr = GetBufferDescriptor(buf_id);
@@ -1703,35 +1739,140 @@ BufferSync(int flags)
if ((bufHdr->flags & mask) == mask)
{
+ CkptSortItem *item;
+
bufHdr->flags |= BM_CHECKPOINT_NEEDED;
- num_to_write++;
+
+ item = &CkptBufferIds[num_to_scan++];
+ item->buf_id = buf_id;
+ item->tsId = bufHdr->tag.rnode.spcNode;
+ item->relNode = bufHdr->tag.rnode.relNode;
+ item->forkNum = bufHdr->tag.forkNum;
+ item->blockNum = bufHdr->tag.blockNum;
}
UnlockBufHdr(bufHdr);
}
- if (num_to_write == 0)
+ if (num_to_scan == 0)
return; /* nothing to do */
-
WritebackContextInit(&wb_context, &checkpoint_flush_after);
- TRACE_POSTGRESQL_BUFFER_SYNC_START(NBuffers, num_to_write);
+ TRACE_POSTGRESQL_BUFFER_SYNC_START(NBuffers, num_to_scan);
/*
- * Loop over all buffers again, and write the ones (still) marked with
- * BM_CHECKPOINT_NEEDED. In this loop, we start at the clock sweep point
- * since we might as well dump soon-to-be-recycled buffers first.
- *
- * Note that we don't read the buffer alloc count here --- that should be
- * left untouched till the next BgBufferSync() call.
+ * Sort buffers that need to be written to reduce the likelihood of random
+ * IO. The sorting is also important for the implementation of balancing
+ * writes between tablespaces. Without balancing writes we'd potentially
+ * end up writing to the tablespaces one-by-one; possibly overloading the
+ * underlying system.
*/
- buf_id = StrategySyncStart(NULL, NULL);
- num_to_scan = NBuffers;
+ qsort(CkptBufferIds, num_to_scan, sizeof(CkptSortItem),
+ ckpt_buforder_comparator);
+
+ num_spaces = 0;
+
+ /*
+ * Allocate progress status for each tablespace with buffers that need to
+ * be flushed. This requires the to-be-flushed array to be sorted.
+ */
+ last_tsid = InvalidOid;
+ for (i = 0; i < num_to_scan; i++)
+ {
+ CkptTsStatus *s;
+ Oid cur_tsid;
+
+ cur_tsid = CkptBufferIds[i].tsId;
+
+ /*
+ * Grow array of per-tablespace status structs, everytime a new
+ * tablespace is found.
+ */
+ if (last_tsid == InvalidOid || last_tsid != cur_tsid)
+ {
+ Size sz;
+
+ num_spaces++;
+
+ /*
+ * Not worth adding grow-by-power-of-2 logic here - even with a
+ * few hundred tablespaces this will be fine.
+ */
+ sz = sizeof(CkptTsStatus) * num_spaces;
+
+ if (per_ts_stat == NULL)
+ per_ts_stat = (CkptTsStatus *) palloc(sz);
+ else
+ per_ts_stat = (CkptTsStatus *) repalloc(per_ts_stat, sz);
+
+ s = &per_ts_stat[num_spaces - 1];
+ memset(s, 0, sizeof(*s));
+ s->tsId = cur_tsid;
+
+ /*
+ * The first buffer in this tablespace. As CkptBufferIds is sorted
+ * by tablespace all (s->num_to_scan) buffers in this tablespace
+ * will follow afterwards.
+ */
+ s->index = i;
+
+ /*
+ * progress_slice will be determined once we know how many buffers
+ * are in each tablespace, i.e. after this loop.
+ */
+
+ last_tsid = cur_tsid;
+ }
+ else
+ {
+ s = &per_ts_stat[num_spaces - 1];
+ }
+
+ s->num_to_scan++;
+ }
+
+ Assert(num_spaces > 0);
+
+ /*
+ * Build a min-heap over the write-progress in the individual tablespaces,
+ * and compute how large a portion of the total progress a single
+ * processed buffer is.
+ */
+ ts_heap = binaryheap_allocate(num_spaces,
+ ts_ckpt_progress_comparator,
+ NULL);
+
+ for (i = 0; i < num_spaces; i++)
+ {
+ CkptTsStatus *ts_stat = &per_ts_stat[i];
+
+ ts_stat->progress_slice = (float8) num_to_scan / ts_stat->num_to_scan;
+
+ binaryheap_add_unordered(ts_heap, PointerGetDatum(ts_stat));
+ }
+
+ binaryheap_build(ts_heap);
+
+ /*
+ * Iterate through to-be-checkpointed buffers and write the ones (still)
+ * marked with BM_CHECKPOINT_NEEDED. The writes are balanced between
+ * tablespaces.
+ */
+ num_processed = 0;
num_written = 0;
- while (num_to_scan-- > 0)
+ while (!binaryheap_empty(ts_heap))
{
- BufferDesc *bufHdr = GetBufferDescriptor(buf_id);
+ BufferDesc *bufHdr = NULL;
+ CkptTsStatus *ts_stat = (CkptTsStatus *)
+ DatumGetPointer(binaryheap_first(ts_heap));
+
+ buf_id = CkptBufferIds[ts_stat->index].buf_id;
+ Assert(buf_id != -1);
+
+ bufHdr = GetBufferDescriptor(buf_id);
+
+ num_processed++;
/*
* We don't need to acquire the lock here, because we're only looking
@@ -1752,31 +1893,52 @@ BufferSync(int flags)
TRACE_POSTGRESQL_BUFFER_SYNC_WRITTEN(buf_id);
BgWriterStats.m_buf_written_checkpoints++;
num_written++;
+ }
+ }
- /*
- * We know there are at most num_to_write buffers with
- * BM_CHECKPOINT_NEEDED set; so we can stop scanning if
- * num_written reaches num_to_write.
- *
- * Note that num_written doesn't include buffers written by
- * other backends, or by the bgwriter cleaning scan. That
- * means that the estimate of how much progress we've made is
- * conservative, and also that this test will often fail to
- * trigger. But it seems worth making anyway.
- */
- if (num_written >= num_to_write)
- break;
+ /*
+ * Measure progress independent of actualy having to flush the buffer
+ * - otherwise writing become unbalanced.
+ */
+ ts_stat->progress += ts_stat->progress_slice;
+ ts_stat->num_scanned++;
+ ts_stat->index++;
- /*
- * Sleep to throttle our I/O rate.
- */
- CheckpointWriteDelay(flags, (double) num_written / num_to_write);
- }
+ /* Have all the buffers from the tablespace been processed? */
+ if (ts_stat->num_scanned == ts_stat->num_to_scan)
+ {
+ binaryheap_remove_first(ts_heap);
+ }
+ else
+ {
+ /* update heap with the new progress */
+ binaryheap_replace_first(ts_heap, PointerGetDatum(ts_stat));
}
- if (++buf_id >= NBuffers)
- buf_id = 0;
+ /*
+ * Sleep to throttle our I/O rate.
+ */
+ CheckpointWriteDelay(flags, (double) num_processed / num_to_scan);
+
+/* #define CHECKPOINT_PROGRESS */
+#ifdef CHECKPOINT_PROGRESS
+ /* FIXME: remove before commit */
+ /* delete current content of the line, print progress */
+ fprintf(stderr, "\33[2K\rto_scan: %d, scanned: %d, %%processed: %.2f, %%writeouts: %.2f",
+ num_to_scan, num_processed,
+ (((double) num_processed) / num_to_scan) * 100,
+ ((double) num_written / num_processed) * 100);
+#endif
}
+#ifdef CHECKPOINT_PROGRESS
+ fprintf(stderr, "\n");
+#endif
+
+ /* issue all pending flushes */
+ IssuePendingWritebacks(&wb_context);
+
+ pfree(per_ts_stat);
+ per_ts_stat = NULL;
/*
* Update checkpoint statistics. As noted above, this doesn't include
@@ -3754,6 +3916,59 @@ rnode_comparator(const void *p1, const void *p2)
return 0;
}
+/*
+ * Comparator determining the writeout order in a checkpoint.
+ *
+ * It is important that tablespaces are compared first, the logic balancing
+ * writes between tablespaces relies on it.
+ */
+static int
+ckpt_buforder_comparator(const void *pa, const void *pb)
+{
+ const CkptSortItem *a = (CkptSortItem *) pa;
+ const CkptSortItem *b = (CkptSortItem *) pb;
+
+ /* compare tablespace */
+ if (a->tsId < b->tsId)
+ return -1;
+ else if (a->tsId > b->tsId)
+ return 1;
+ /* compare relation */
+ if (a->relNode < b->relNode)
+ return -1;
+ else if (a->relNode > b->relNode)
+ return 1;
+ /* compare fork */
+ else if (a->forkNum < b->forkNum)
+ return -1;
+ else if (a->forkNum > b->forkNum)
+ return 1;
+ /* compare block number */
+ else if (a->blockNum < b->blockNum)
+ return -1;
+ else /* should not be the same block anyway... */
+ return 1;
+}
+
+/*
+ * Comparator for a Min-Heap over the, per-tablespace, checkpoint completion
+ * progress.
+ */
+static int
+ts_ckpt_progress_comparator(Datum a, Datum b, void *arg)
+{
+ CkptTsStatus *sa = (CkptTsStatus *) a;
+ CkptTsStatus *sb = (CkptTsStatus *) b;
+
+ /* we want a min-heap, so return 1 for the a < b */
+ if (sa->progress < sb->progress)
+ return 1;
+ else if (sa->progress == sb->progress)
+ return 0;
+ else
+ return -1;
+}
+
/*
* BufferTag comparator.
diff --git a/src/include/storage/buf_internals.h b/src/include/storage/buf_internals.h
index fe8b423..de84bc4 100644
--- a/src/include/storage/buf_internals.h
+++ b/src/include/storage/buf_internals.h
@@ -239,6 +239,24 @@ extern PGDLLIMPORT WritebackContext BackendWritebackContext;
/* in localbuf.c */
extern BufferDesc *LocalBufferDescriptors;
+/* in bufmgr.c */
+
+/*
+ * Structure to sort buffers per file on checkpoints.
+ *
+ * This structure is allocated per buffer in shared memory, so it should be
+ * kept as small as possible.
+ */
+typedef struct CkptSortItem
+{
+ Oid tsId;
+ Oid relNode;
+ ForkNumber forkNum;
+ BlockNumber blockNum;
+ int buf_id;
+} CkptSortItem;
+
+extern CkptSortItem *CkptBufferIds;
/*
* Internal buffer management routines
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index f501f55..b850db0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -283,6 +283,8 @@ CheckpointerRequest
CheckpointerShmemStruct
Chromosome
City
+CkptSortItem
+CkptTsStatus
ClientAuthentication_hook_type
ClientData
ClonePtr
--
2.7.0.229.g701fa7f
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
