Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-22 Thread Michael Paquier
On Mon, Mar 22, 2021 at 09:25:26AM +0200, Andrus wrote: > Errors in pg_wal directory seems not to occur in patched version. Errors in > pg_stat_tmp still occur. Yesterdays log introduces new error message > > using stale statistics instead of current ones because stats collector is > not respondin

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-22 Thread Andrus
Hi! Errors in pg_wal directory seems not to occur in patched version. Errors in pg_stat_tmp still occur. Yesterdays log introduces new error message using stale statistics instead of current ones because stats collector is not responding 2021-03-21 03:02:23 EET stats collector LOG:  cou

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-20 Thread Andrus
Hi! Today got 2 errors in patched version: 2021-03-20 20:31:27 EET stats collector LOG:  could not rename temporary statistics file "pg_stat_tmp/global.tmp" to "pg_stat_tmp/global.stat": Permission denied 2021-03-20 20:51:25 EET stats collector LOG:  could not rename temporary stati

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-19 Thread Andrus
Hi! Have you tested the unpatched builds? No. Andrus.

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-19 Thread Michael Paquier
On Fri, Mar 19, 2021 at 09:00:10AM +0200, Andrus wrote: > I replaced files in 13.1 server with ones from your patched version. There > are no errors in log file now for 8 hours. Yippee. Thanks. Have you tested the unpatched builds? And did you see some errors with them? -- Michael signature.a

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-19 Thread Andrus
Hi! Okay, cool. I am going to send you privately two links to the builds I am going to produce, 13.2 unpatched and 13.2 patched. I replaced files in 13.1 server with ones from your patched version. There are no errors in log file now for 8 hours. Andrus.

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Guy Burgess
Thanks - yes pleas send me a private link and I will try to reproduce it in a test environment and then test the patch. Regards, Guy On 18/03/2021 1:08 pm, Michael Paquier wrote: On Thu, Mar 18, 2021 at 12:25:45PM +1300, Guy Burgess wrote: FWIW, this looks the same issue I am getting (repor

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Michael Paquier
On Thu, Mar 18, 2021 at 12:25:45PM +1300, Guy Burgess wrote: > FWIW, this looks the same issue I am getting (reported last month: > https://www.postgresql.org/message-id/f444a84e-2d29-55f9-51a6-a5dcea3bc253%40burgess.co.nz) Yep. > I get the same Process Monitor output, including BUFFER OVERFLOW e

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 10:45:28AM +0200, Andrus wrote: > In this server hopefully no. Application code contains xml parsing it but > probably those queries are never running in this server. Okay, cool. I am going to send you privately two links to the builds I am going to produce, 13.2 unpatched

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Guy Burgess
FWIW, this looks the same issue I am getting (reported last month: https://www.postgresql.org/message-id/f444a84e-2d29-55f9-51a6-a5dcea3bc253%40burgess.co.nz) I get the same Process Monitor output, including BUFFER OVERFLOW entries.  No sign of any process other than postgres.exe touching the

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Tom Lane
Michael Paquier writes: > On Wed, Mar 17, 2021 at 11:44:48AM -0400, Tom Lane wrote: >> Oh! That's an interesting theory; it'd explain why this broke recently, >> because we didn't use to use that function. But how do you draw that >> conclusion from this stack trace? > All the reports received

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 11:44:48AM -0400, Tom Lane wrote: > Oh! That's an interesting theory; it'd explain why this broke recently, > because we didn't use to use that function. But how do you draw that > conclusion from this stack trace? > > Anyway, if you've diagnosed this correctly, I bet the

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Tom Lane
Andrus writes: > It looks like too small buffer is passed to NtQueryInformationFile . Oh! That's an interesting theory; it'd explain why this broke recently, because we didn't use to use that function. But how do you draw that conclusion from this stack trace? Anyway, if you've diagnosed this

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
Hi! BUFFER OVERFLOW result stack trace is "Frame","Module","Location","Address","Path" "0","FLTMGR.SYS","FltDecodeParameters + 0x1c5d","0xf8019f72555d","C:\Windows\System32\drivers\FLTMGR.SYS" "1","FLTMGR.SYS","FltDecodeParameters + 0x17bc","0xf8019f7250bc","C:\Windows\System32\drivers

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
Hi! Doesn't look like these are error messages. There should be error messages that correspond to the error messages you see in the PostgreSQL log. I excluded SUCCESS and DELETE PENDING results and included pg_wal path and postgres.exe process. Log file contains only few entries, BUFFER OV

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Laurenz Albe
On Wed, 2021-03-17 at 15:26 +0200, Andrus wrote: > > I think what it would be most helpful to run "process monitor", so that you > > get > > a log of the system calls and errors; perhaps that shows some details that > > we cannot get from the error message. > > > > Process monitor shows huge num

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
Hi! I think what it would be most helpful to run "process monitor", so that you get a log of the system calls and errors; perhaps that shows some details that we cannot get from the error message. Process monitor shows huge number of DELETE PENDING results from CreateFile operation: 15:22:3

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Laurenz Albe
On Wed, 2021-03-17 at 01:09 +0200, Andrus wrote: > > Those ten seconds are coming from RemoveXlogFile(), where pgrename() > > loops 100 times for 100ms before giving up. So something holding up > > the file's handle prevents the removal to happen. > > I tried sysinternals > > handle pg_wal

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
>On Wed, Mar 17, 2021 at 09:25:00AM +0200, Andrus wrote: pg_config --configure outputs --enable-thread-safety --enable-nls --with-ldap --with-openssl --with-uuid --with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python Thanks. Do you actually use OpenSSL, Not directly. May

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Michael Paquier
On Wed, Mar 17, 2021 at 09:25:00AM +0200, Andrus wrote: > pg_config --configure outputs > > --enable-thread-safety --enable-nls --with-ldap --with-openssl --with-uuid > --with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python Thanks. Do you actually use OpenSSL, LDAP, uuid-os

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
Hi >I am not completely sure which flags your installation has, but pg_config --configure outputs --enable-thread-safety --enable-nls --with-ldap --with-openssl --with-uuid --with-libxml --with-libxslt --with-icu --with-tcl --with-perl --with-python Andrus.

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-17 Thread Andrus
Hi! I am not completely sure which flags your installation has, but another possibility is that I directly send to you two compiled builds, one with the patch and one without it that you could directly test. I would not send that to the lists as an installation is rather large, but I could just

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-16 Thread Michael Paquier
On Wed, Mar 17, 2021 at 01:09:24AM +0200, Andrus wrote: > Should I try install Visual C++ , compile and replace postgres.exe file in > AMD server. Mostly. That's the annoying part: https://www.postgresql.org/docs/devel/install-windows-full.html It is also possible to compile the code on a first

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-16 Thread Andrus
Hi! >Those ten seconds are coming from RemoveXlogFile(), where pgrename() loops 100 times for 100ms before giving up. So something holding up the file's handle prevents the removal to happen. I tried sysinternals handle pg_wal It shows that only postgres processes have opened files in pg_wa

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-16 Thread Michael Paquier
On Tue, Mar 16, 2021 at 01:59:07PM +0200, Andrus wrote: > I have two Windows 2019 servers. In Intel Xeon Cold 6226R server it occurs > after every 10 seconds. Last logs: > > 2021-03-16 13:48:12 EET checkpointer LOG:  could not rename file > "pg_wal/000100110097": Permission denied

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-16 Thread Andrus
Hi Where you getting the Postgres binaries from? https://www.enterprisedb.com/downloads/postgres-postgresql-downloads If we provide a patch, could you test it? Yes This would require that you do your own build, unfortunately, but having an environment where this is easily reproducible is a

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-16 Thread Michael Paquier
Hi Andrus, On Wed, Mar 10, 2021 at 03:20:47PM +0200, Andrus wrote: > After re-starting postgres service problem persists. Where you getting the Postgres binaries from? If we provide a patch, could you test it? This would require that you do your own build, unfortunately, but having an environme

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-11 Thread Thomas Munro
Hi Andrus, On Thu, Mar 11, 2021 at 2:21 AM Andrus wrote: > Windows Resource manger shows that wal files are used by large number of > postgres processes: > > postgres.exe22656FileC:\Program > Files\PostgreSQL\13\data\pg_wal\0001000A0075 > postgres.exe30788File

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-10 Thread Andrus
Hi! After re-starting postgres service problem persists. Log file contains permission denied errors exactly after every 10 seconds: ... 2021-03-10 15:15:45 EET checkpointer LOG:  could not rename file "pg_wal/0001000A0030": Permission denied 2021-03-10 15:15:55 EET checkpo

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-10 Thread Andrus
Today log file contains 2021-03-10 11:01:58 EET checkpointer LOG:  could not rename file "pg_wal/0001000A003B": Permission denied messages. Resource Manager shows that 0001000A003B is is locked by process 30376 According to pg_stat_activity this is regular postg

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-09 Thread Thomas Munro
On Tue, Mar 9, 2021 at 9:43 PM Andrus wrote: > > Any hints in Windows event viewer? Events occurring at the same time > showing up there. > > Looked into Administrative Events/Custom views and few others. There are no > messages about this. Windowsi perfomance monitor and Filemon show files >

Re: SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-09 Thread Andrus
Any hints in Windows event viewer? Events occurring at the same time showing up there. Looked into Administrative Events/Custom views and few others. There are no messages about this. Windowsi perfomance monitor and Filemon show files opened by process. How to do reverse: log processes and

SV: Log files polluted with permission denied error messages after every 10 seconds

2021-03-08 Thread Niels Jespersen
> > >Fra: Andrus >Sendt: 8. marts 2021 23:24 >Emne: Re: Log files polluted with permission denied error messages after every >10 seconds > >Permission denied error occur in Windows. Debian logs doesnt contain it. >In Windows also pg_database_size('mydb') and similar ones fail frequently >with