Re: Map WAL segment files on PMEM as WAL buffers

:55:13 +0900, Takashi Menjo wrote: > > Here is patchset v8. It will have "make check-world" and Cirrus to > > pass. > > This unfortunately does not apply anymore: > http://cfbot.cputube.org/patch_37_3181.log > > Could you rebase? > > - And

Re: Map WAL segment files on PMEM as WAL buffers

tion, in my patchset PMEM_IS_PMEM_FORCE=1 will be given as an environment variable in .cirrus.yml and "wal_pmem_map = on" will be given by initdb. Regards, Takashi -- Takashi Menjo v8-0001-Add-with-libpmem-option-for-PMEM-support.patch Description: Binary data v8-0002-Add-wal_pme

Re: Map WAL segment files on PMEM as WAL buffers

Hi Justin, I can reproduce the error you reported, with PMEM_IS_PMEM_FORCE=1. Moreover, I can reproduce it **on a real PMem device**. So the causes are in my patchset, not in PMem environment. I'll fix it in the next patchset version. Regards, Takashi -- Takashi Menjo

Re: Map WAL segment files on PMEM as WAL buffers

ed. Regards, Takashi On Mon, Jan 17, 2022 at 4:34 PM Justin Pryzby wrote: > > On Thu, Jan 06, 2022 at 10:43:37PM -0600, Justin Pryzby wrote: > > On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote: > > > > But in this case it really doesn't work :( > &g

Re: Map WAL segment files on PMEM as WAL buffers

Linux. Please see [1][2] for details; or 2. Set the environment variable PMEM_IS_PMEM_FORCE=1 to tell libpmem to treat any devices as if they were PMEM. Regards, Takashi [1] https://www.intel.com/content/www/us/en/developer/articles/training/how-to-emulate-persistent-memory-on-an-intel

Re: Map WAL segment files on PMEM as WAL buffers

Rebased. On Fri, Nov 5, 2021 at 3:47 PM Takashi Menjo wrote: > > Hi Daniel, > > The issue you told has been fixed. I attach the v5 patchset to this email. > > The v5 has all the patches in the v4, and in addition, has the > following two new patches: > > - (v5-0002)

Re: Map WAL segment files on PMEM as WAL buffers

-with-libpmem:" $config->{pmem} = 'C:\path\to\pmdk\x64-windows'; - (v5-0006) Compatible to Windows: This patch resolves conflicting mode_t typedefs and libpmem API variants (U or W, like Windows API). Best regards, Takashi On Thu, Nov 4, 2021 at 5:46 PM Takashi Menjo wrote: >

Re: Map WAL segment files on PMEM as WAL buffers

AVE_LIBPMEM at src/include/pg_config.h line 347 at C:/Users/menjo/Documents/git/postgres/src/tools/msvc/Mkvcbuild.pm line 860. Best regards, Takashi On Wed, Nov 3, 2021 at 10:04 PM Daniel Gustafsson wrote: > > > On 28 Oct 2021, at 08:09, Takashi Menjo wrote: > > > Rebased, and

Re: Map WAL segment files on PMEM as WAL buffers

GetXLogBuffer when --enable-cassert. - (0008) Update document This adds a new entry for wal_pmem_map in the section Write Ahead Log -> Settings. Best regards, Takashi On Fri, Oct 8, 2021 at 5:07 PM Takashi Menjo wrote: > > Hello Matthias, > > Thank you for your comment! > &

Re: Map WAL segment files on PMEM as WAL buffers

en, and is mutually independent from other happenings. I will try to find a case where the WARNING happens repeatedly; or I will de-duplicate the messages if it is easier. Best regards, Takashi -- Takashi Menjo

Re: Map WAL segment files on PMEM as WAL buffers

Rebased. -- Takashi Menjo v3-0001-Add-with-libpmem-option-for-PMEM-support.patch Description: Binary data v3-0002-Add-wal_pmem_map-to-GUC.patch Description: Binary data v3-0003-Export-InstallXLogFileSegment.patch Description: Binary data v3-0004-Map-WAL-segment-files-on-PMEM-as-WAL

Re: [PoC] Non-volatile WAL buffer

Hi Tomas, > Hello Takashi-san, > > On 3/5/21 9:08 AM, Takashi Menjo wrote: > > Hi Tomas, > > > > Thank you so much for your report. I have read it with great interest. > > > > Your conclusion sounds reasonable to me. My patchset you call "NTT / &

Re: [PoC] Non-volatile WAL buffer

It does perform about as well as the "NTT / buffer" patch (and much both > perform much better than the experimental patches I shared in January). > > The "NTT / buffer" patch seems much more disruptive - it introduces one > large buffer for WAL, which makes various other tasks more complicated > (i.e. it needs additional complexity to handle WAL archival, etc.). Are > there some advantages of this patch (compared to the other patch)? > > As for the "NTT / segments" patch, I wonder if we can just rework the > code like this (to use mmap etc.) or whether we need to support both > both ways (file I/O and mmap). I don't have much experience with many > other platforms, but it seems quite possible that mmap won't work all > that well on some of them. So my assumption is we'll need to support > both file I/O and mmap to make any of this committable, but I may be wrong. > > > [1] > https://www.postgresql.org/message-id/CAOwnP3Oz4CnKp0-_KU-x5irr9pBqPNkk7pjwZE5Pgo8i1CbFGg%40mail.gmail.com > > -- > Tomas Vondra > EnterpriseDB: http://www.enterprisedb.com > The Enterprise PostgreSQL Company -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

Hi Sawada, I am relieved to hear that the performance problem was solved. And I added a tip about PMEM namespace and partitioning in PG wiki[1]. Regards, [1] https://wiki.postgresql.org/wiki/Persistent_Memory_for_WAL#Configure_and_verify_DAX_hugepage_faults -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

://www.postgresql.org/message-id/caownp3ofofosftmeikqcbmp0ywdjn0kvb4ka_0tj+urq7dt...@mail.gmail.com -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

]; please read the files' prefix "v4-" as "v5-" [2] https://www.postgresql.org/message-id/CAOwnP3O3O1GbHpddUAzT%3DCP3aMpX99%3D1WtBAfsRZYe2Ui53MFQ%40mail.gmail.com [3] https://www.postgresql.org/message-id/CAOwnP3Oz4CnKp0-_KU-x5irr9pBqPNkk7pjwZE5Pgo8i1CbFGg%40mail.gmail.com -- Takashi Menjo postgresql.conf Description: Binary data

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

://www.postgresql.org/message-id/flat/01d4c2a1%2488c6cc40%249a5464c0%24%40lab.ntt.co.jp -- Takashi Menjo v5-0001-Add-configure-option-for-PMDK.patch Description: Binary data v5-0003-Walreceiver-WAL-IO-using-PMDK.patch Description: Binary data v5-0002-Read-write-WAL-files-using

Re: [PoC] Non-volatile WAL buffer

PMEM (2016 partially) [1] and PMDK supports Windows [2]. All the above contents will be updated gradually. Please stay tuned. Regards, [1] https://docs.microsoft.com/en-us/windows-server/storage/storage-spaces/deploy-pmem [2] https://docs.pmem.io/persistent-memory/getting-started-guide/installing-pmdk/installing-pmdk-on-windows -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

/Persistent_Memory_for_WAL Regards, -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

ing [3] https://github.com/meistervonperf/pseudo-pram [4] https://www.kernel.org/doc/html/latest/x86/pat.html [5] https://pgcon.org/2018/schedule/events/1154.en.html [6] https://www.postgresql.org/message-id/CAOwnP3ONd9uXPXKoc5AAfnpCnCyOna1ru6sU=ey_4wfmjak...@mail.gmail.com -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

[1] https://www.snia.org/educational-library/how-can-persistent-memory-make-databases-faster-and-how-could-we-go-ahead-2020 2021年1月26日(火) 18:50 Takashi Menjo : > Dear everyone, Tomas, > > First of all, the "v4" patchset for non-volatile WAL buffer attached to > the previo

Re: [PoC] Non-volatile WAL buffer

d the latest msync patchset v3 attached to the previous to master 411ae64 (on Jan18, 2021) then tested it, and I got no error when pgbench -i -s 500. Please try it if necessary. Best regards, Takashi 2021年1月26日(火) 17:52 Takashi Menjo : > Dear everyone, > > Sorry but I forgot to atta

Re: [PoC] Non-volatile WAL buffer

the machine you used? > > > > > > > From what I know, this is somewhat expected for PMEM devices, for a > > bunch of reasons: > > > > 1) The memory bandwidth is much lower than for DRAM (maybe ~10-20%), so > > it takes fewer processes to saturate it. > > > > 2) Internally, the PMEM has a 256B buffer for writes, used for combining > > etc. With too many processes sending writes, it becomes to look more > > random, which is harmful for throughput. > > > > When combined, this means the performance starts dropping at certain > > number of threads, and the optimal number of threads is rather low > > (something like 5-10). This is very different behavior compared to DRAM. > > Makes sense. > > > > > There's a nice overview and measurements in this paper: > > > > Building blocks for persistent memory / How to get the most out of your > > new memory? > > Alexander van Renen, Lukas Vogel, Viktor Leis, Thomas Neumann & Alfons > > Kemper > > > > https://link.springer.com/article/10.1007/s00778-020-00622-9 > > Thank you. I'll read it. > > > > > > > >> I'm also wondering if WAL is the right usage for PMEM. Per [2] > there's a > > >> huge read-write assymmetry (the writes being way slower), and their > > >> recommendation (in "Observation 3" is) > > >> > > >> The read-write asymmetry of PMem im-plies the necessity of > avoiding > > >> writes as much as possible for PMem. > > >> > > >> So maybe we should not be trying to use PMEM for WAL, which is pretty > > >> write-heavy (and in most cases even write-only). > > > > > > I think using PMEM for WAL is cost-effective but it leverages the only > > > low-latency (sequential) write, but not other abilities such as > > > fine-grained access and low-latency random write. If we want to > > > exploit its all ability we might need some drastic changes to logging > > > protocol while considering storing data on PMEM. > > > > > > > True. I think investigating whether it's sensible to use PMEM for this > > purpose. It may turn out that replacing the DRAM WAL buffers with writes > > directly to PMEM is not economical, and aggregating data in a DRAM > > buffer is better :-( > > Yes. I think it might be interesting to do an analysis of the > bottlenecks of NTT patch by perf etc. If bottlenecks are moved to > other places by removing WALWriteLock during flush, it's probably a > good sign for further performance improvements. IIRC WALWriteLock is > one of the main bottlenecks on OLTP workload, although my memory might > already be out of date. > > Regards, > > -- > Masahiko Sawada > EDB: https://www.enterprisedb.com/ > -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

or your checkpoint configurations might be not appropriate. Could you check your results again? Best regards, Takashi -- Takashi Menjo

Re: [PoC] Non-volatile WAL buffer

cation for that process goes over segments. This causes the performance difference between the two. Best regards, Takashi [1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/ftruncate.html -- Takashi Menjo

RE: [PoC] Non-volatile WAL buffer

, Takashi -- Takashi Menjo NTT Software Innovation Center > -Original Message- > From: Deng, Gang > Sent: Friday, October 9, 2020 3:10 PM > To: Takashi Menjo > Cc: pgsql-hack...@postgresql.org; 'Takashi Menjo' > Subject: RE: [PoC] Non-volatile WAL buffer >

RE: [PoC] Non-volatile WAL buffer

compiler: gcc 9.3.0 - libc: glibc 2.31 - Linux kernel: 5.7 (vanilla) - Filesystem: ext4 (DAX enabled when using Optane PMem) - PMDK: 1.9 - PostgreSQL (Original): 14devel (200f610: Jul 26, 2020) - PostgreSQL (Non-volatile WAL buffer): 14devel (200f610: Jul 26, 2020) + non-volatile WAL buffer patchset

Re: [PoC] Non-volatile WAL buffer

gt; Througput (10^3 TPS) > 13.0 16.9 > > CPU Time % of CopyXlogRecordToWAL > 3.0 1.6 > > CPU Time % of XLogInsertRecord > 23.0 16.4 > > CPU Time % of XLogFlush >

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

Dear hackers, I rebased my old patchset. It would be good to compare this v4 patchset to non-volatile WAL buffer's one [1]. [1] https://www.postgresql.org/message-id/002101d649fb$1f5966e0$5e0c34a0$@hco.ntt.co.jp_1 Regards, Takashi -- Takashi Menjo v4-0001-Add-configure-optio

Re: Remove page-read callback from XLogReaderState.

日(木) 13:53 Kyotaro Horiguchi : > cfbot is complaining as this is no longer applicable. Rebased. > > In v14, some reference to XLogReaderState parameter to read_pages > functions are accidentally replaced by the reference to the global > variable xlogreader. Fixed it, too. > > regards. > > -- > Kyotaro Horiguchi > NTT Open Source Software Center > -- Takashi Menjo

RE: [PoC] Non-volatile WAL buffer

size of the new NVWAL is same as the master's one. Best regards, Takashi -- Takashi Menjo NTT Software Innovation Center > -Original Message- > From: Takashi Menjo > Sent: Wednesday, March 18, 2020 5:59 PM > To: 'PostgreSQL-development' > Cc: 'Robe

RE: [PoC] Non-volatile WAL buffer

Dear Amit, Thank you for your advice. Exactly, it's so to speak "do as the hackers do when in pgsql"... I'm rebasing my branch onto master. I'll submit an updated patchset and performance report later. Best regards, Takashi -- Takashi Menjo NTT Software Innovatio

RE: [PoC] Non-volatile WAL buffer

. Also we can check more easily the features and improvements by using release notes and user manuals. Best regards, Takashi -- Takashi Menjo NTT Software Innovation Center > -Original Message- > From: Amit Langote > Sent: Monday, February 17, 2020 1:39 PM > To: Tak

RE: [PoC] Non-volatile WAL buffer

tions to explain why the performance improved. I'd talk about all the details at the appropriate time and place. (The conference, or here later...) Best regards, Takashi -- Takashi Menjo NTT Software Innovation Center > -Original Message- > From: Takashi Menjo >

RE: [PoC] Non-volatile WAL buffer

for the result report... Best regards, Takashi -- Takashi Menjo NTT Software Innovation Center > -Original Message- > From: Robert Haas > Sent: Wednesday, January 29, 2020 6:00 AM > To: Takashi Menjo > Cc: Heikki Linnakangas ; pgsql-hack...@postgresql.org > Subj

RE: [PoC] Non-volatile WAL buffer

WAL segment, etc, is reasonable or not. If it really is, then I will go to (2). Best regards, Takashi -- Takashi Menjo NTT Software Innovation Center

RE: [PoC] Non-volatile WAL buffer

S properly or working around it is future work. Best regards, Takashi [1] https://www.postgresql.org/message-id/83eafbfd-d9c5-6623-2423-7cab1be3888c%40iki.fi [2] https://www.postgresql.org/message-id/2aec6e2a-6a32-0c39-e4e2-aad854543aa8%40iki.fi [3] https://pmem.io/2018/11/26/bad-blocks.ht

RE: [PoC] Non-volatile WAL buffer

.wiki.kernel.org/ -- Takashi Menjo NTT Software Innovation Center

[PoC] Non-volatile WAL buffer

volatile WAL buffer, comparing with the original one that uses PMEM as "a faster- than-SSD storage." I will talk about the results if accepted. Best regards, Takashi Menjo [1] Persistent Memory (SNIA) https://www.snia.org/PM [2] Persistent Memory Development Kit (pmem.io)

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

w any work in this area. Thank you for your comment. It is reasonable that you are concerned with maintainability. Our patchset still lacks of it. I will consider about that when I submit a next update. (It may take a long time, so please be patient...) Regards, Takashi -- Takashi Menjo - NTT Software Innovation Center

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

Hi, Sorry but I found that the patchset v2 had a bug in managing WAL segment file offset. I fixed it and updated a patchset as v3 (attached). Regards, Takashi -- Takashi Menjo - NTT Software Innovation Center 0001-Add-configure-option-for-PMDK-v3.patch Description: Binary data 0002-Read

RE: static global variable openLogOff in xlog.c seems no longer used

n be fixed. But now I understand it is not a simple thing. Thank you. Regards, Takashi -- Takashi Menjo - NTT Software Innovation Center

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

ump looks working as before. Regards, Takashi -- Takashi Menjo - NTT Software Innovation Center

static global variable openLogOff in xlog.c seems no longer used

"pg_ctl -m immediate stop" then "pg_ctl start" looked OK. Regards, Takashi [1] See commit c24dcd0cfd949bdf245814c4c2b3df828ee7db36. -- Takashi Menjo - NTT Software Innovation Center Remove-openLogOff.patch Description: Binary data

RE: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

s. Sorry but my new patchset still uses PMDK, because PMDK is supported on Linux _and Windows_, and I think someone may want to test this patchset on Windows... Regards, Takashi -- Takashi Menjo - NTT Software Innovation Center 0001-Add-configure-option-for-PMDK-v2.patch Description: Bi