Re: logical decoding : exceeded maxAllocatedDescs for .spill files

[Amit Khandekar]

On Fri, 22 Nov 2019 at 09:08, Amit Kapila <amit.kapil...@gmail.com> wrote:
> Have you tried before that fix , if not, can you once try by
> temporarily reverting that fix in your environment and share the
> output of each step?  After you get the error due to EOF, check that
> you have .spill files in pg_replslot/<slot_name>/ and then again try
> to get changes by pg_logical_slot_get_changes().   If you want, you
> can use the test provided in Amit Khandekar's patch.

On my Linux machine, I added elog() in ReorderBufferRestoreChanges(),
just after FileRead() returns 0. This results in error. But the thing is, in
ReorderBufferCommit(), the error is already handled using PG_CATCH :

PG_CATCH();
{
.....
   AbortCurrentTransaction();
.......
   if (using_subtxn)
      RollbackAndReleaseCurrentSubTransaction();
........
........
   /* remove potential on-disk data, and deallocate */
   ReorderBufferCleanupTXN(rb, txn);
}

So ReorderBufferCleanupTXN() removes all the .spill files using unlink().

And on Windows, what should happen is : unlink() should succeed
because the file is opened using FILE_SHARE_DELETE. But the files
should still remain there because these are still open. It is just
marked for deletion until there is no one having opened the file. That
is what is my conclusion from running a sample attached program test.c
.
But what you are seeing is "Permission denied" errors. Not sure why
unlink() is failing.

The thing that is still a problem is : On Windows, if the file remains
open, and later even when the unlink() succeeds, the file will be left
there until it is closed. So subsequent operations will open the same
old file. Not sure what happens if we open a file that is marked for
deletion.

-

Thanks,
-Amit Khandekar
EnterpriseDB Corporation
The Postgres Database Company

//#include <windows.h>
#include <stdio.h>
#include <stdlib.h>

void pgwin32_open(const char *fileName, int fileFlags,...)
{

	HANDLE		h;
	SECURITY_ATTRIBUTES sa;

	sa.nLength = sizeof(sa);
	sa.bInheritHandle = TRUE;
	sa.lpSecurityDescriptor = NULL;
	
	h = CreateFile(fileName,
						   GENERIC_READ,
	/* These flags allow concurrent rename/unlink */
						   (FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE),
						   &sa,
						   OPEN_EXISTING,
						   FILE_ATTRIBUTE_NORMAL ,
						   NULL);

	if (h == INVALID_HANDLE_VALUE)
	{
		printf("File could not be opened. Error %d\n", GetLastError());
		exit(1);
	}

	/* CloseHandle(h); */


	if (_unlink(fileName) != 0)
	{
		printf("File could not be deleted. Error %d\n", GetLastError());
		exit(1);
	}
}

int main(void)
{
	pgwin32_open("c:/temp/amit/file", 0 /* O_RDONLY | PG_BINARY */);
}