Re: pg_preadv() and pg_pwritev()

2021-01-19 Thread James Hilliard
On Thu, Jan 14, 2021 at 6:45 PM Tom Lane wrote: > > Sergey Shinderuk writes: > > On 15.01.2021 01:13, Tom Lane wrote: > >> Also, after re-reading [1] I am not at all excited about trying to > >> remove the -isysroot switches from our *FLAGS. What I propose to do > >> is keep that, but improve ou

Re: pg_preadv() and pg_pwritev()

2021-01-15 Thread Tom Lane
Sergey Shinderuk writes: > On 15.01.2021 04:45, Tom Lane wrote: >> Hence, I propose the attached. This works as far as I can tell >> to fix the problem you're seeing. > Yes, it works fine. Thank you very much. Great. Pushed with a little more polishing. regards, tom la

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 04:45, Tom Lane wrote: Hence, I propose the attached. This works as far as I can tell to fix the problem you're seeing. Yes, it works fine. Thank you very much.

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 05:04, Tom Lane wrote: on her machine there's no detail at all: % xcrun --verbose --no-cache --show-sdk-path /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk The same on my machine. I get details for --find, but not for --show-sdk-path. So I'm not sure what to make

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 04:53, Sergey Shinderuk wrote: I see that "xcrun --sdk macosx --show-sdk-path" really calls "/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk macosx -version Path" under the hood. ... and caches the result. xcodebuild not called without --no-cache. So it stil

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > I see that "xcrun --sdk macosx --show-sdk-path" really calls > "/Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -sdk > macosx -version Path" under the hood. Hmm. I found something odd on my wife's Mac: although on my other machines, I get something like

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 01:13, Tom Lane wrote: than relying entirely on xcodebuild. Maybe there's a case for trying "xcrun --sdk macosx --show-sdk-path" in between; in my tests that seemed noticeably faster than invoking xcodebuild, and I've not yet seen a case where it gave a different answer. I see

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > On 15.01.2021 01:13, Tom Lane wrote: >> Also, after re-reading [1] I am not at all excited about trying to >> remove the -isysroot switches from our *FLAGS. What I propose to do >> is keep that, but improve our mechanism for choosing a default value >> for PG_SYSROOT.

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 14.01.2021 21:05, Tom Lane wrote: After considerable playing around, I'm guessing that the reason -no_weak_imports doesn't help is that it rejects calls that are marked as weak references on the *calling* side. Since AC_CHECK_FUNCS doesn't bother to #include the relevant header file, the comp

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 15.01.2021 01:13, Tom Lane wrote: I borrowed my wife's Mac, which is still on Catalina and up to now never had Xcode on it, and found some very interesting things. Step 1: download/install Xcode 12.3, open it, agree to license, wait for it to finish "installing components". At this point, /L

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
I borrowed my wife's Mac, which is still on Catalina and up to now never had Xcode on it, and found some very interesting things. Step 1: download/install Xcode 12.3, open it, agree to license, wait for it to finish "installing components". At this point, /Library/Developer/CommandLineTools doesn

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
I wrote: > It seems like the more productive approach would be to try to identify > the right sysroot to use. I wonder if there is some less messy way > to find out the compiler's default sysroot than to scrape it out of > -v output. This is, of course, not terribly well documented by Apple. But

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > On 14.01.2021 18:42, Tom Lane wrote: >>> I noticed that "cc" invoked from command line uses: >>> -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk >> Hm, how did you determine that exactly? > % cc -v tmp.c > ... > -isysroot /Library/Developer/CommandLi

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Sergey Shinderuk
On 14.01.2021 18:42, Tom Lane wrote: I noticed that "cc" invoked from command line uses: -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk Hm, how did you determine that exactly? % echo 'int main(void){}' >tmp.c % cc -v tmp.c Apple clang version 12.0.0 (clang-1200.0.32.28) T

Re: pg_preadv() and pg_pwritev()

2021-01-14 Thread Tom Lane
Sergey Shinderuk writes: > I noticed that "cc" invoked from command line uses: > -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk Hm, how did you determine that exactly? regards, tom lane

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
The symptoms sound consistent with using bleeding-edge Xcode on a Catalina machine ... please report exact OS and Xcode versions. macOS 10.15.7 (19H2) Xcode 12.3 (12C33) macOS SDK 11.1 (20C63) Everything is fine if I run "configure" with PG_SYSROOT=/Library/Developer/CommandLineTools/SDKs/Mac

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
I wrote: > So I'm a little confused as to why this test is failing to fail > with (I assume) newer Xcode. Can we see the relevant part of > config.log on your machine? After further digging I believe I understand what's happening, and it's a bit surprising we've not been bit by it before. If the

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Thu, Jan 14, 2021 at 9:26 AM Tom Lane wrote: > * You need to remove pread.o and pwrite.o from the hard-wired > part of the list in src/port/Makefile, else they get built > whether needed or not. Right, done. > * I don't much like this in fd.h: > > @@ -46,6 +46,7 @@ > #include > > > +struct

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Thomas Munro writes: > On Thu, Jan 14, 2021 at 5:13 AM Tom Lane wrote: >> Looks like we need to be more careful about not including pread.c >> in the build unless it actually has code to contribute. > I did it that way because it made it easy to test different > combinations of the replacements

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Thu, Jan 14, 2021 at 5:13 AM Tom Lane wrote: > I have a different complaint, using Big Sur and Xcode 12.3: > > /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: > file: libpgport.a(pread.o) has no symbols > /Applications/Xcode.app/Contents/Developer

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Hmmm ... I can further report that on Catalina + Xcode 12.0, everything seems fine. configure correctly detects that preadv and pwritev aren't there: configure:15161: checking for preadv configure:15161: ccache gcc -o conftest -Wall -Wmissing-prototypes -Wpointer-ar\ ith -Wdeclaration-after-state

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
I wrote: > Sergey Shinderuk writes: note: 'pwritev' has been marked as being introduced in macOS 11.0 here, but the deployment target is macOS 10.15.0 > The symptoms sound consistent with using bleeding-edge Xcode on a > Catalina machine ... please report exact OS and Xcode versions. I

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Tom Lane
Sergey Shinderuk writes: > On 13.01.2021 12:56, Thomas Munro wrote: >> On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk >> wrote: >>> note: 'pwritev' has been marked as being introduced in macOS 11.0 here, >>> but the deployment target is macOS 10.15.0 >> Hrm... So why did "configure" think yo

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
On 13.01.2021 12:56, Thomas Munro wrote: On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk wrote: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include/sys/uio.h:104:9: note: 'pwritev' has been marked as being introduced in macOS 11.0 her

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Thomas Munro
On Wed, Jan 13, 2021 at 10:40 PM Sergey Shinderuk wrote: > /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk/usr/include/sys/uio.h:104:9: > note: 'pwritev' has been marked as being introduced in macOS 11.0 here, > but the deployment target is macOS

Re: pg_preadv() and pg_pwritev()

2021-01-13 Thread Sergey Shinderuk
On 11.01.2021 05:59, Thomas Munro wrote: Since only sifaka has managed to return a result so far (nice CPU), I had plenty of time to notice that macOS Big Sur has introduced preadv/pwritev. They were missing on Catalina[1]. [1] https://cirrus-ci.com/task/6002157537198080 Hi, Thomas! Indeed,

Re: pg_preadv() and pg_pwritev()

2021-01-12 Thread Thomas Munro
On Mon, Jan 11, 2021 at 3:59 PM Thomas Munro wrote: > On Mon, Jan 11, 2021 at 3:34 PM Thomas Munro wrote: > > I pushed it with that name, and a couple more cosmetic changes. I'll > > keep an eye on the build farm. > > Since only sifaka has managed to return a result so far (nice CPU), I > had pl

Re: pg_preadv() and pg_pwritev()

2021-01-10 Thread Thomas Munro
On Mon, Jan 11, 2021 at 3:34 PM Thomas Munro wrote: > On Wed, Dec 23, 2020 at 12:06 AM Thomas Munro wrote: > > On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > > > Can we come up with a better name than 'uio'? I find that a not exactly > > > meaningful name. > > > > Ok, let's try port/pg_

Re: pg_preadv() and pg_pwritev()

2021-01-10 Thread Thomas Munro
On Wed, Dec 23, 2020 at 12:06 AM Thomas Munro wrote: > On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > > Can we come up with a better name than 'uio'? I find that a not exactly > > meaningful name. > > Ok, let's try port/pg_iovec.h. I pushed it with that name, and a couple more cosmetic

Re: pg_preadv() and pg_pwritev()

2020-12-22 Thread Thomas Munro
On Mon, Dec 21, 2020 at 11:40 AM Andres Freund wrote: > On 2020-12-20 16:26:42 +1300, Thomas Munro wrote: > > > 1. port.h cannot assume that has already been included; > > > nor do I want to fix that by including there. Do we > > > really need to define a fallback value of IOV_MAX? If so, > >

Re: pg_preadv() and pg_pwritev()

2020-12-21 Thread Thomas Munro
On Mon, Dec 21, 2020 at 8:25 PM Jakub Wartak wrote: > > > I'm drawing a blank on trivial candidate uses for preadv(), without > > > infrastructure from later patches. > > > > Can't immediately think of something either. > > This might be not that trivial , but maybe acquire_sample_rows() from > a

RE: pg_preadv() and pg_pwritev()

2020-12-20 Thread Jakub Wartak
> > I'm drawing a blank on trivial candidate uses for preadv(), without > > infrastructure from later patches. > > Can't immediately think of something either. This might be not that trivial , but maybe acquire_sample_rows() from analyze.c ? Please note however there's patch https://www.post

Re: pg_preadv() and pg_pwritev()

2020-12-20 Thread Andres Freund
Hi, On 2020-12-20 16:26:42 +1300, Thomas Munro wrote: > > 1. port.h cannot assume that has already been included; > > nor do I want to fix that by including there. Do we > > really need to define a fallback value of IOV_MAX? If so, > > maybe the answer is to put the replacement struct iovec an

Re: pg_preadv() and pg_pwritev()

2020-12-20 Thread Thomas Munro
On Sun, Dec 20, 2020 at 8:07 PM Tom Lane wrote: > One minor thought is that in > > + struct iovec iov[Min(IOV_MAX, 1024)]; /* cap stack space */ > > it seems like pretty much every use of IOV_MAX would want some > similar cap. Should we centralize that idea with, say, > > #define

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Tom Lane
Thomas Munro writes: > OK, here's a patch to zero-fill fresh WAL segments with pwritev(). > I'm drawing a blank on trivial candidate uses for preadv(), without > infrastructure from later patches. This looks OK to me. I tried it on prairiedog (has writev and pwrite but not pwritev) as well as ga

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Thomas Munro
On Sun, Dec 20, 2020 at 12:34 PM Tom Lane wrote: > Thomas Munro writes: > > I want to be able to do synchronous vectored file I/O, so I made > > wrapper macros for preadv() and pwritev() with fallbacks for systems > > that don't have them. Following the precedent of the pg_pread() and > > pg_pwr

Re: pg_preadv() and pg_pwritev()

2020-12-19 Thread Tom Lane
Thomas Munro writes: > I want to be able to do synchronous vectored file I/O, so I made > wrapper macros for preadv() and pwritev() with fallbacks for systems > that don't have them. Following the precedent of the pg_pread() and > pg_pwrite() macros, the "pg_" prefix reflects a subtle contract >