Re: [PATCH] xlogreader: do not read a file block twice

2019-02-18 Thread Arthur Zakirov
On 18.02.2019 04:09, Michael Paquier wrote: And done, after doing and extra pass, doing more testing using by own plugins, pg_waldump and more fancy stuff with a primary/standby and pgbench. Thank you Michael. -- Arthur Zakirov Postgres Professional: http://www.postgrespro.com Russian Postgres

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-17 Thread Michael Paquier
On Fri, Feb 15, 2019 at 08:06:16AM +0900, Michael Paquier wrote: > Thanks, I see what you have done. I cannot comment if your shortcut > is actually fully correct based on my knowledge of this code, but > things cannot be in the best conditions without having the WAL reader > handle properly the l

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-14 Thread Michael Paquier
On Thu, Feb 14, 2019 at 11:20:56AM +0300, Arthur Zakirov wrote: > So we read whole page with size XLOG_BLCKSZ. The full code: > https://github.com/postgrespro/pg_probackup/blob/c052651b8c8864733bcabbc2660c387b792229d8/src/parsexlog.c#L1074 > > Here is the little optimization I made. Mainly I just

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-14 Thread Arthur Zakirov
On 14.02.2019 09:51, Michael Paquier wrote: Now I don't actually agree that this qualifies as a bug fix. As things stand, a page may finish by being more than once if what has been read previously equals what is requested, however this does not prevent the code to work correctly. The performanc

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-13 Thread Michael Paquier
On Tue, Feb 12, 2019 at 11:44:14AM +0300, Arthur Zakirov wrote: > Of course. Agree, it may be a non trivial case. Added as a bug fix: > https://commitfest.postgresql.org/22/1994/ I have been looking at the patch, and I agree that the current coding is a bit crazy. If the wanted data has already b

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-13 Thread Arthur Zakirov
On 12.02.2019 20:47, Andrey Lepikhov wrote: I looked at the history of the code changes: --- 7fcbf6a405f (Alvaro Herrera 2013-01-16 16:12:53 -0300 539) reqLen < state->readLen) 1bb2558046c (Heikki Linnakangas 2010-01-27 15:27:51 +000

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-12 Thread Andrey Lepikhov
On 11.02.2019 21:25, Arthur Zakirov wrote: Hello hackers, Grigory noticed that one of our utilities has very slow performance when xlogreader reads zlib archives. We found out that xlogreader sometimes reads a WAL file block twice. zlib has slow performance when you read an archive not in

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-12 Thread Arthur Zakirov
On 12.02.2019 07:23, Michael Paquier wrote: On 02/11/2019 07:25 PM, Arthur Zakirov wrote: Grigory noticed that one of our utilities has very slow performance when xlogreader reads zlib archives. We found out that xlogreader sometimes reads a WAL file block twice. What do you think? I think th

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-11 Thread Michael Paquier
On Mon, Feb 11, 2019 at 07:32:59PM +0300, Grigory Smolkin wrote: > Hm, looks like it could speed up PostgreSQL recovery, but is it > safe? (Please avoid top-posting.) > On 02/11/2019 07:25 PM, Arthur Zakirov wrote: >> Grigory noticed that one of our utilities has very slow performance when >> xlo

Re: [PATCH] xlogreader: do not read a file block twice

2019-02-11 Thread Grigory Smolkin
Hm, looks like it could speed up PostgreSQL recovery, but is it safe? On 02/11/2019 07:25 PM, Arthur Zakirov wrote: Hello hackers, Grigory noticed that one of our utilities has very slow performance when xlogreader reads zlib archives. We found out that xlogreader sometimes reads a WAL file