Re: pg_preadv() and pg_pwritev()

2021-01-19 Thread James Hilliard
On Thu, Jan 14, 2021 at 6:45 PM Tom Lane wrote: > > Sergey Shinderuk writes: > > On 15.01.2021 01:13, Tom Lane wrote: > >> Also, after re-reading [1] I am not at all excited about trying to > >> remove the -isysroot switches from our *FLAGS. What I propose to do > >> is keep that, but improve ou

Re: pg_preadv() and pg_pwritev()

2021-01-15 Thread Tom Lane
Sergey Shinderuk writes: > On 15.01.2021 04:45, Tom Lane wrote: >> Hence, I propose the attached. This works as far as I can tell >> to fix the problem you're seeing. > Yes, it works fine. Thank you very much. Great. Pushed with a little more polishing. regards, tom la

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 04:45, Tom Lane wrote: Hence, I propose the attached. This works as far as I can tell to fix the problem you're seeing. Yes, it works fine. Thank you very much.

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 05:04, Tom Lane wrote: on her machine there's no detail at all: % xcrun --verbose --no-cache --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk The same on my machine. I get details for --find, but not for --show-sdk-path. So I'm not sure what to make

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 04:53, Sergey Shinderuk wrote: I see that "xcrun --sdk macosx --show-sdk-path" really calls "/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk macosx -version Path" under the hood. ... and caches the result. xcodebuild not called without --no-cache. So it stil

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > I see that "xcrun --sdk macosx --show-sdk-path" really calls > "/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk > macosx -version Path" under the hood. Hmm. I found something odd on my wife's Mac: although on my other machines, I get something like

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 01:13, Tom Lane wrote: than relying entirely on xcodebuild. Maybe there's a case for trying "xcrun --sdk macosx --show-sdk-path" in between; in my tests that seemed noticeably faster than invoking xcodebuild, and I've not yet seen a case where it gave a different answer. I see

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > On 15.01.2021 01:13, Tom Lane wrote: >> Also, after re-reading [1] I am not at all excited about trying to >> remove the -isysroot switches from our *FLAGS. What I propose to do >> is keep that, but improve our mechanism for choosing a default value >> for PG_SYSROOT.

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 14.01.2021 21:05, Tom Lane wrote: After considerable playing around, I'm guessing that the reason -no_weak_imports doesn't help is that it rejects calls that are marked as weak references on the *calling* side. Since AC_CHECK_FUNCS doesn't bother to #include the relevant header file, the comp

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 01:13, Tom Lane wrote: I borrowed my wife's Mac, which is still on Catalina and up to now never had Xcode on it, and found some very interesting things. Step 1: download/install Xcode 12.3, open it, agree to license, wait for it to finish "installing components". At this point, /L

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
I borrowed my wife's Mac, which is still on Catalina and up to now never had Xcode on it, and found some very interesting things. Step 1: download/install Xcode 12.3, open it, agree to license, wait for it to finish "installing components". At this point, /Library/Developer/CommandLineTools doesn

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
I wrote: > It seems like the more productive approach would be to try to identify > the right sysroot to use. I wonder if there is some less messy way > to find out the compiler's default sysroot than to scrape it out of > -v output. This is, of course, not terribly well documented by Apple. But

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > On 14.01.2021 18:42, Tom Lane wrote: >>> I noticed that "cc" invoked from command line uses: >>> -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk >> Hm, how did you determine that exactly? > % cc -v tmp.c > ... > -isysroot /Library/Developer/CommandLi

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 14.01.2021 18:42, Tom Lane wrote: I noticed that "cc" invoked from command line uses: -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk Hm, how did you determine that exactly? % echo 'int main(void){}' >tmp.c % cc -v tmp.c Apple clang version 12.0.0 (clang-1200.0.32.28) T

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > I noticed that "cc" invoked from command line uses: > -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk Hm, how did you determine that exactly? regards, tom lane

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
The symptoms sound consistent with using bleeding-edge Xcode on a Catalina machine ... please report exact OS and Xcode versions. macOS 10.15.7 (19H2) Xcode 12.3 (12C33) macOS SDK 11.1 (20C63) Everything is fine if I run "configure" with PG_SYSROOT=/Library/Developer/CommandLineTools/SDKs/Mac

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
I wrote: > So I'm a little confused as to why this test is failing to fail > with (I assume) newer Xcode. Can we see the relevant part of > config.log on your machine? After further digging I believe I understand what's happening, and it's a bit surprising we've not been bit by it before. If the

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Thu, Jan 14, 2021 at 9:26 AM Tom Lane wrote: > * You need to remove pread.o and pwrite.o from the hard-wired > part of the list in src/port/Makefile, else they get built > whether needed or not. Right, done. > * I don't much like this in fd.h: > > @@ -46,6 +46,7 @@ > #include > > > +struct

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 14, 2021 at 5:13 AM Tom Lane wrote: >> Looks like we need to be more careful about not including pread.c >> in the build unless it actually has code to contribute. > I did it that way because it made it easy to test different > combinations of the replacements

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Thu, Jan 14, 2021 at 5:13 AM Tom Lane wrote: > I have a different complaint, using Big Sur and Xcode 12.3: > > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: > file: libpgport.a(pread.o) has no symbols > /Applications/Xcode.app/Contents/Developer

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Hmmm ... I can further report that on Catalina + Xcode 12.0, everything seems fine. configure correctly detects that preadv and pwritev aren't there: configure:15161: checking for preadv configure:15161: ccache gcc -o conftest -Wall -Wmissing-prototypes -Wpointer-ar\ ith -Wdeclaration-after-state

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
I wrote: > Sergey Shinderuk writes: note: 'pwritev' has been marked as being introduced in macOS 11.0 here, but the deployment target is macOS 10.15.0 > The symptoms sound consistent with using bleeding-edge Xcode on a > Catalina machine ... please report exact OS and Xcode versions. I

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Sergey Shinderuk writes: > On 13.01.2021 12:56, Thomas Munro wrote: >> On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk >> wrote: >>> note: 'pwritev' has been marked as being introduced in macOS 11.0 here, >>> but the deployment target is macOS 10.15.0 >> Hrm... So why did "configure" think yo

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
On 13.01.2021 12:56, Thomas Munro wrote: On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk wrote: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include/sys/uio.h:104:9: note: 'pwritev' has been marked as being introduced in macOS 11.0 her

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk wrote: > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include/sys/uio.h:104:9: > note: 'pwritev' has been marked as being introduced in macOS 11.0 here, > but the deployment target is macOS

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
On 11.01.2021 05:59, Thomas Munro wrote: Since only sifaka has managed to return a result so far (nice CPU), I had plenty of time to notice that macOS Big Sur has introduced preadv/pwritev. They were missing on Catalina[1]. [1] https://cirrus-ci.com/task/6002157537198080 Hi, Thomas! Indeed,

Re: pg_preadv() and pg_pwritev()

2021-01-12 Thread Thomas Munro
On Mon, Jan 11, 2021 at 3:59 PM Thomas Munro wrote: > On Mon, Jan 11, 2021 at 3:34 PM Thomas Munro wrote: > > I pushed it with that name, and a couple more cosmetic changes. I'll > > keep an eye on the build farm. > > Since only sifaka has managed to return a result so far (nice CPU), I > had pl

Re: pg_preadv() and pg_pwritev()

2021-01-10 Thread Thomas Munro
On Mon, Jan 11, 2021 at 3:34 PM Thomas Munro wrote: > On Wed, Dec 23, 2020 at 12:06 AM Thomas Munro wrote: > > On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > > > Can we come up with a better name than 'uio'? I find that a not exactly > > > meaningful name. > > > > Ok, let's try port/pg_

Re: pg_preadv() and pg_pwritev()

2021-01-10 Thread Thomas Munro
On Wed, Dec 23, 2020 at 12:06 AM Thomas Munro wrote: > On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > > Can we come up with a better name than 'uio'? I find that a not exactly > > meaningful name. > > Ok, let's try port/pg_iovec.h. I pushed it with that name, and a couple more cosmetic

Re: pg_preadv() and pg_pwritev()

2020-12-22 Thread Thomas Munro
ven won't prove anything except that the code > > > compiles. Is it worth fixing at least one code path to make > > > use of pg_preadv and pg_pwritev, so we can make sure this code > > > is tested before there's layers of other new code on top? > > >

Re: pg_preadv() and pg_pwritev()

2020-12-21 Thread Thomas Munro
On Mon, Dec 21, 2020 at 8:25 PM Jakub Wartak wrote: > > > I'm drawing a blank on trivial candidate uses for preadv(), without > > > infrastructure from later patches. > > > > Can't immediately think of something either. > > This might be not that trivial , but maybe acquire_sample_rows() from > a

RE: pg_preadv() and pg_pwritev()

2020-12-20 Thread Jakub Wartak
> > I'm drawing a blank on trivial candidate uses for preadv(), without > > infrastructure from later patches. > > Can't immediately think of something either. This might be not that trivial , but maybe acquire_sample_rows() from analyze.c ? Please note however there's patch https://www.post

Re: pg_preadv() and pg_pwritev()

2020-12-20 Thread Andres Freund
he value of the define at configure time? That way only pread.c / pwrite.c would need it. > > 3. The patch as given won't prove anything except that the code > > compiles. Is it worth fixing at least one code path to make > > use of pg_preadv and pg_pwritev, so we can make su

Re: pg_preadv() and pg_pwritev()

2020-12-20 Thread Thomas Munro
On Sun, Dec 20, 2020 at 8:07 PM Tom Lane wrote: > One minor thought is that in > > + struct iovec iov[Min(IOV_MAX, 1024)]; /* cap stack space */ > > it seems like pretty much every use of IOV_MAX would want some > similar cap. Should we centralize that idea with, say, > > #define

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Tom Lane
Thomas Munro writes: > OK, here's a patch to zero-fill fresh WAL segments with pwritev(). > I'm drawing a blank on trivial candidate uses for preadv(), without > infrastructure from later patches. This looks OK to me. I tried it on prairiedog (has writev and pwrite but not pwritev) as well as ga

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Thomas Munro
pwritev(). I'm drawing a blank on trivial candidate uses for preadv(), without infrastructure from later patches. From d7178f296642e177bc57fabe93abec395cbaac5f Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Thu, 26 Nov 2020 15:48:31 +1300 Subject: [PATCH v2 1/2] Add pg_preadv() and pg_pwritev(). Provide synchr

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Tom Lane
fallback definitions. 3. The patch as given won't prove anything except that the code compiles. Is it worth fixing at least one code path to make use of pg_preadv and pg_pwritev, so we can make sure this code is tested before there's layers of other new code on top? regards, tom lane

pg_preadv() and pg_pwritev()

2020-12-19 Thread Thomas Munro
63cb0 Mon Sep 17 00:00:00 2001 From: Thomas Munro Date: Thu, 26 Nov 2020 15:48:31 +1300 Subject: [PATCH 07/11] Add pg_preadv() and pg_pwritev(). Provide synchronous scatter/gather I/O routines. These map to preadv(), pwritev() with various fallbacks for systems that don't have them. -