On Thu, Jul 12, 2018 at 03:35:53PM +0900, Yugo Nagata wrote: > I think it makes sense to remove unnecessary temporary WAL files although > I'm not sure how high the risk of ENOSPC is.
It depends on how close to the partition size limit max_wal_size is set, and how much a system is unstable. Switching on/off a VM where Postgres is located can participate in that, as well as VM snapshots taken without memory (I work a lot on those as you can guess :D). Setting it to 70% of the partition size is what I imagine is the base, but I can imagine as well people setting it at 90% or more. Still the probability is low, which is why I think that it would make sense to just fix the problem on HEAD and move on. > One little thing I noticed is the function name "RemoveXLogTempFiles". > Other similar functions are named as RemoveOldXlogFiles or RemoveXlogFile > (using Xlog not XLog), so it seem to me more consistent to rename this > "RemoveXlogTempFiles" or "RemoveTempXlogFiles" and so on. I see, a lower-case for Xlog instead of XLog. That makes sense. I have used your second suggestion in the attached. I have also changed the thing so as the format of the comment block is better even after indenting. Thoughts? -- Michael
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 022dac6cd7..94526e8942 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -887,6 +887,7 @@ static bool WaitForWALToBecomeAvailable(XLogRecPtr RecPtr, bool randAccess,
static int emode_for_corrupt_record(int emode, XLogRecPtr RecPtr);
static void XLogFileClose(void);
static void PreallocXlogFiles(XLogRecPtr endptr);
+static void RemoveTempXlogFiles(void);
static void RemoveOldXlogFiles(XLogSegNo segno, XLogRecPtr PriorRedoPtr, XLogRecPtr endptr);
static void RemoveXlogFile(const char *segname, XLogRecPtr PriorRedoPtr, XLogRecPtr endptr);
static void UpdateLastRemovedPtr(char *filename);
@@ -3863,6 +3864,35 @@ UpdateLastRemovedPtr(char *filename)
SpinLockRelease(&XLogCtl->info_lck);
}
+/*
+ * Remove all temporary log files in pg_wal
+ *
+ * This is called at the beginning of recovery after a previous crash,
+ * at a point where no other processes write fresh WAL data.
+ */
+static void
+RemoveTempXlogFiles(void)
+{
+ DIR *xldir;
+ struct dirent *xlde;
+
+ elog(DEBUG2, "removing all temporary WAL files");
+
+ xldir = AllocateDir(XLOGDIR);
+ while ((xlde = ReadDir(xldir, XLOGDIR)) != NULL)
+ {
+ char path[MAXPGPATH];
+
+ if (strncmp(xlde->d_name, "xlogtemp.", 9) != 0)
+ continue;
+
+ snprintf(path, MAXPGPATH, XLOGDIR "/%s", xlde->d_name);
+ elog(DEBUG2, "removed temporary WAL file \"%s\"", path);
+ unlink(path);
+ }
+ FreeDir(xldir);
+}
+
/*
* Recycle or remove all log files older or equal to passed segno.
*
@@ -6379,17 +6409,25 @@ StartupXLOG(void)
*/
ValidateXLOGDirectoryStructure();
- /*
- * If we previously crashed, there might be data which we had written,
- * intending to fsync it, but which we had not actually fsync'd yet.
- * Therefore, a power failure in the near future might cause earlier
- * unflushed writes to be lost, even though more recent data written to
- * disk from here on would be persisted. To avoid that, fsync the entire
- * data directory.
+ /*----------
+ * If we previously crashed, perform a couple of actions:
+ * - The pg_wal directory may still include some temporary WAL segments
+ * used when creating a new segment, so perform some clean up to not
+ * bloat this path. This is done first as there is no point to sync this
+ * temporary data.
+ * - There might be data which we had written, intending to fsync it,
+ * but which we had not actually fsync'd yet. Therefore, a power failure
+ * in the near future might cause earlier unflushed writes to be lost,
+ * even though more recent data written to disk from here on would be
+ * persisted. To avoid that, fsync the entire data directory.
+ *---------
*/
if (ControlFile->state != DB_SHUTDOWNED &&
ControlFile->state != DB_SHUTDOWNED_IN_RECOVERY)
+ {
+ RemoveTempXlogFiles();
SyncDataDirectory();
+ }
/*
* Initialize on the assumption we want to recover to the latest timeline
signature.asc
Description: PGP signature
