Re: Map WAL segment files on PMEM as WAL buffers

2022-08-01 Thread Jacob Champion
As discussed in [1], we're taking this opportunity to return some patchsets that don't appear to be getting enough reviewer interest. This is not a rejection, since we don't necessarily think there's anything unacceptable about the entry, but it differs from a standard "Returned with Feedback" in

Re: Map WAL segment files on PMEM as WAL buffers

2022-03-23 Thread Takashi Menjo
Hi Andres, Thank you for your report. I rebased and made patchset v9 attached to this email. Note that v9-0009 and v9-0010 are for those who want to pass their own Cirrus CI. Regards, Takashi On Tue, Mar 22, 2022 at 9:44 AM Andres Freund wrote: > > Hi, > > On 2022-01-20 14:55:13 +0900, Takashi

Re: Map WAL segment files on PMEM as WAL buffers

2022-03-21 Thread Andres Freund
Hi, On 2022-01-20 14:55:13 +0900, Takashi Menjo wrote: > Here is patchset v8. It will have "make check-world" and Cirrus to > pass. This unfortunately does not apply anymore: http://cfbot.cputube.org/patch_37_3181.log Could you rebase? - Andres

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-19 Thread Takashi Menjo
Hi Justin, Here is patchset v8. It will have "make check-world" and Cirrus to pass. Would you try this one? The v8 squashes some patches in v7 into related ones, and adds the following patches: - v8-0003: Add wal_pmem_map to postgresql.conf.sample. It also helps v8-0011. - v8-0009: Fix wrong ha

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-18 Thread Takashi Menjo
Hi Justin, I can reproduce the error you reported, with PMEM_IS_PMEM_FORCE=1. Moreover, I can reproduce it **on a real PMem device**. So the causes are in my patchset, not in PMem environment. I'll fix it in the next patchset version. Regards, Takashi -- Takashi Menjo

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-18 Thread Takashi Menjo
Hi Justin, Thanks for your help. I'm making an additional patch for Cirrus CI. I'm also trying to reproduce the "make check-world" error you reported, on my Linux environment that has neither a real PMem nor an emulated one, with PMEM_IS_PMEM_FORCE=1. I'll keep you updated. Regards, Takashi On

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-16 Thread Justin Pryzby
On Thu, Jan 06, 2022 at 10:43:37PM -0600, Justin Pryzby wrote: > On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote: > > > But in this case it really doesn't work :( > > > > > > running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL: > > > file not on PMEM: path "pg_wal

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-06 Thread Justin Pryzby
On Fri, Jan 07, 2022 at 12:50:01PM +0900, Takashi Menjo wrote: > > But in this case it really doesn't work :( > > > > running bootstrap script ... 2022-01-05 23:17:30.244 CST [12088] FATAL: > > file not on PMEM: path "pg_wal/00010001" > > Do you have a real PMEM device such as NV

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-06 Thread Takashi Menjo
Hi Justin, Thank you for your build test and comments. The v7 patchset attached to this email fixes the issues you reported. > The cfbot showed issues compiling on linux and windows. > http://cfbot.cputube.org/takashi-menjo.html > > https://cirrus-ci.com/task/6125740327436288 > [02:30:06.538] In

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-05 Thread Justin Pryzby
On Thu, Jan 06, 2022 at 05:52:01PM +1300, Thomas Munro wrote: > On Thu, Jan 6, 2022 at 5:00 PM Justin Pryzby wrote: > > I'm attaching a probable fix. Unfortunately, for patches like this, most of > > the functionality isn't exercised unless the library is installed and > > compilation and runtime

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-05 Thread Thomas Munro
On Thu, Jan 6, 2022 at 5:00 PM Justin Pryzby wrote: > I'm attaching a probable fix. Unfortunately, for patches like this, most of > the functionality isn't exercised unless the library is installed and > compilation and runtime are enabled by default. By the way, you could add a separate patch m

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-05 Thread Justin Pryzby
The cfbot showed issues compiling on linux and windows. http://cfbot.cputube.org/takashi-menjo.html https://cirrus-ci.com/task/6125740327436288 [02:30:06.538] In file included from xlog.c:38: [02:30:06.538] ../../../../src/include/access/xlogpmem.h:32:42: error: unknown type name ‘tli’ [02:30:06.

Re: Map WAL segment files on PMEM as WAL buffers

2022-01-05 Thread Takashi Menjo
Rebased. On Fri, Nov 5, 2021 at 3:47 PM Takashi Menjo wrote: > > Hi Daniel, > > The issue you told has been fixed. I attach the v5 patchset to this email. > > The v5 has all the patches in the v4, and in addition, has the > following two new patches: > > - (v5-0002) Support build with MSVC on Wi

Re: Map WAL segment files on PMEM as WAL buffers

2021-11-04 Thread Takashi Menjo
Hi Daniel, The issue you told has been fixed. I attach the v5 patchset to this email. The v5 has all the patches in the v4, and in addition, has the following two new patches: - (v5-0002) Support build with MSVC on Windows: Please have src\tools\msvc\config.pl as follows to "configure --with-li

Re: Map WAL segment files on PMEM as WAL buffers

2021-11-04 Thread Takashi Menjo
Hello Daniel, Thank you for your comment. I had the following error message with MSVC on Windows. It looks the same as what you told me. I'll fix it. | > cd src\tools\msvc | > build | (..snipped..) | Copying pg_config_os.h... | Generating configuration headers... | undefined symbol: HAVE_LIBPMEM

Re: Map WAL segment files on PMEM as WAL buffers

2021-11-03 Thread Daniel Gustafsson
> On 28 Oct 2021, at 08:09, Takashi Menjo wrote: > Rebased, and added the patches below into the patchset. Looks like the 0001 patch needs to be updated to support Windows and MSVC. See src/tools/msvc/Mkvcbuild.pm and Solution.pm et.al for inspiration on how to add the MSVC equivalent of --with

Re: Map WAL segment files on PMEM as WAL buffers

2021-10-27 Thread Takashi Menjo
Hi, Rebased, and added the patches below into the patchset. - (0006) Let wal_pmem_map be constant unless --with-libpmem wal_pmem_map never changes from false in that case, so let it be constant. Thanks, Matthias! - (0007) Ensure WAL mappings before assertion This fixes SIGSEGV abortion in GetXL

Re: Map WAL segment files on PMEM as WAL buffers

2021-10-08 Thread Takashi Menjo
Hello Matthias, Thank you for your comment! > > [ v3-0002-Add-wal_pmem_map-to-GUC.patch ] > > +extern bool wal_pmem_map; > > A lot of the new code in these patches is gated behind this one flag, > but the flag should never be true on !pmem systems. Could you instead > replace it with something li

Re: Map WAL segment files on PMEM as WAL buffers

2021-10-07 Thread Matthias van de Meent
On Wed, 30 Jun 2021 at 06:53, Takashi Menjo wrote: > > Rebased. Thanks for these patches! I recently took a dive into the WAL subsystem, and got to this patchset through the previous ones linked from [0]. This patchset seems straightforward to understand, so thanks! I've looked over the patches

Re: Map WAL segment files on PMEM as WAL buffers

2021-06-29 Thread Takashi Menjo
Rebased. -- Takashi Menjo v3-0001-Add-with-libpmem-option-for-PMEM-support.patch Description: Binary data v3-0002-Add-wal_pmem_map-to-GUC.patch Description: Binary data v3-0003-Export-InstallXLogFileSegment.patch Description: Binary data v3-0004-Map-WAL-segment-files-on-PMEM-as-WAL-buffe