On Fri, Apr 22, 2022 at 07:17:37PM +0530, Bharath Rupireddy wrote: > Right. We find enough disk space and go to write and suddenly the > write operations fail for some reason or the VM crashes because of a > reason other than disk space. I think the foolproof solution is to > figure out the available disk space before prepadding or compressing > and also use the > write-first-to-temp-file-and-then-rename-it-to-original-file as > proposed in the earlier patches in this thread.
Yes, what would count here is only the amount of free space in a partition. The total amount of space available becomes handy once you begin introducing things like percentage-based quota policies for the disk when archiving. The free amount of space could be used to define a policy based on the maximum number of bytes you need to leave around, as well, but this is not perfect science as this depends of what FSes decide to do underneath. There are a couple of designs possible here. When I had to deal with my upthread case I have chosen one as I had no need to worry only about Linux, it does not mean that this is the best choice that would fit with the long-term community picture. This comes down to how much pg_receivewal should handle automatically, and how it should handle it. > Having said that, is there a portable way that we can find out the > disk space available? I read your response upthread that statvfs isn't > portable to WIN32 platforms. So, we just say that part of the fix we > proposed here (checking disk space before prepadding or compressing) > isn't supported on WIN32 and we just do the temp file thing for WIN32 > alone? Something like GetDiskFreeSpaceA() would do the trick on WIN32. https://docs.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-getdiskfreespacea When it comes to compression, creating a temporary file would only lead to its deletion, mostly, and extra I/O is never free. Anyway, why would you need the extra logic of the temporary file at all? That's basically what the .partial file is as pg_receivewal begins streaming at the beginning of a segment, to the partial file, each time it sees fit to restart a streaming cycle. -- Michael
signature.asc
Description: PGP signature
