Re: LLVM 16 (opaque pointers)

2023-10-24 Thread Mark Wong
On Mon, Oct 23, 2023 at 10:47:24PM -0400, Tom Lane wrote: > Andres Freund writes: > > FC 29 is well out of support, so I don't think it makes sense to invest any > > further time in this. Personally, I don't think it's useful to have years > > old > > fedora in the buildfarm... > > +1. It's goo

Re: LLVM 16 (opaque pointers)

2023-10-23 Thread Tom Lane
Andres Freund writes: > FC 29 is well out of support, so I don't think it makes sense to invest any > further time in this. Personally, I don't think it's useful to have years old > fedora in the buildfarm... +1. It's good to test old LTS distros, but Fedora releases have a short shelf life by d

Re: LLVM 16 (opaque pointers)

2023-10-23 Thread Andres Freund
Hi, On 2023-10-24 10:17:22 +1300, Thomas Munro wrote: > This POWER animal fails, unexpectedly to me: > > > bonito: 7.0.1 Fedora 29 > > We could try to chase that down, or we could rejoice that at least it > works on current release. It must begin working somewhere between 7 > and 11, but when I

Re: LLVM 16 (opaque pointers)

2023-10-23 Thread Mark Wong
On Tue, Oct 24, 2023 at 10:17:22AM +1300, Thomas Munro wrote: > On Tue, Oct 24, 2023 at 4:27 AM Mark Wong wrote: > > I haven't gotten around to disabling llvm on any of my animals with llvm > > < 7 yet. Do you still want to hold on that? > > Yes, please disable --with-llvm on s390x and POWER ani

Re: LLVM 16 (opaque pointers)

2023-10-23 Thread Thomas Munro
On Tue, Oct 24, 2023 at 4:27 AM Mark Wong wrote: > I haven't gotten around to disabling llvm on any of my animals with llvm > < 7 yet. Do you still want to hold on that? Yes, please disable --with-llvm on s390x and POWER animals with LLVM < 7 (see below). Also, you have a bunch of machines with

Re: LLVM 16 (opaque pointers)

2023-10-23 Thread Mark Wong
On Mon, Oct 23, 2023 at 01:15:04PM +1300, Thomas Munro wrote: > On Sun, Oct 22, 2023 at 2:44 PM Thomas Munro wrote: > > On Sat, Oct 21, 2023 at 2:45 PM Tom Lane wrote: > > > Thomas Munro writes: > > > > (It'd be nice if the > > > > build farm logged "$LLVM_CONFIG --version" somewhere.) > > > > >

Re: LLVM 16 (opaque pointers)

2023-10-22 Thread Thomas Munro
On Sun, Oct 22, 2023 at 2:44 PM Thomas Munro wrote: > On Sat, Oct 21, 2023 at 2:45 PM Tom Lane wrote: > > Thomas Munro writes: > > > (It'd be nice if the > > > build farm logged "$LLVM_CONFIG --version" somewhere.) > > > > It's not really the buildfarm script's responsibility to do that, > > but

Re: LLVM 16 (opaque pointers)

2023-10-21 Thread Thomas Munro
On Sat, Oct 21, 2023 at 2:45 PM Tom Lane wrote: > Thomas Munro writes: > > (It'd be nice if the > > build farm logged "$LLVM_CONFIG --version" somewhere.) > > It's not really the buildfarm script's responsibility to do that, > but feel free to make configure do so. Done, copying the example of h

Re: LLVM 16 (opaque pointers)

2023-10-21 Thread Thomas Munro
On Sat, Oct 21, 2023 at 7:08 PM Andres Freund wrote: > I've attached a patch revision that I spent the last couple hours working > on. It's very very roughly based on a patch Tom Stellard had written (which I > think a few rpm packages use). But instead of encoding details about specific > layout

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Andres Freund
Hi, On 2023-10-19 06:20:26 +1300, Thomas Munro wrote: > Interestingly, a new problem just showed up on the the RHEL9 s390x > machine "lora", where a previously reported problem [1] apparently > re-appeared. It complains about incompatible layout, previously > blamed on mismatch between clang and

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Tom Lane
Thomas Munro writes: > (It'd be nice if the > build farm logged "$LLVM_CONFIG --version" somewhere.) It's not really the buildfarm script's responsibility to do that, but feel free to make configure do so. regards, tom lane

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Thomas Munro
On Sat, Oct 21, 2023 at 12:02 PM Thomas Munro wrote: > On Sat, Oct 21, 2023 at 11:12 AM Andres Freund wrote: > > I'm quite sure that jiting did pass on ppc64 at some point. > > Yeah, I remember debugging it on EDB's POWER machine. First off, we > know that LLVM < 7 doesn't work for us on POWER,

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Andres Freund
Hi, On 2023-10-21 12:02:51 +1300, Thomas Munro wrote: > On Sat, Oct 21, 2023 at 11:12 AM Andres Freund wrote: > > > I doubt I can get anywhere near an s390x though, and we definitely had > > > pre-existing problems on that arch. > > > > IMO an LLVM bug, rather than a postgres bug, but I guess it'

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Thomas Munro
On Sat, Oct 21, 2023 at 11:12 AM Andres Freund wrote: > On 2023-10-21 10:48:47 +1300, Thomas Munro wrote: > > On Thu, Oct 19, 2023 at 6:20 AM Thomas Munro wrote: > > I see that Mark has also just enabled --with-llvm on some POWER Linux > > animals, and they have failed in various ways. The failu

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Andres Freund
Hi, On 2023-10-21 10:48:47 +1300, Thomas Munro wrote: > On Thu, Oct 19, 2023 at 6:20 AM Thomas Munro wrote: > > Interestingly, a new problem just showed up on the the RHEL9 s390x > > machine "lora", where a previously reported problem [1] apparently > > re-appeared. It complains about incompatib

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Tom Lane
Thomas Munro writes: > I doubt I can get anywhere near an s390x though, and we definitely had > pre-existing problems on that arch. Yeah. Too bad there's no s390x in the gcc compile farm. (I'm wondering how straight a line to draw between that fact and llvm's evident shortcomings on s390x.) I'm

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Mark Wong
On Sat, Oct 21, 2023 at 10:48:47AM +1300, Thomas Munro wrote: > On Thu, Oct 19, 2023 at 6:20 AM Thomas Munro wrote: > > Interestingly, a new problem just showed up on the the RHEL9 s390x > > machine "lora", where a previously reported problem [1] apparently > > re-appeared. It complains about inc

Re: LLVM 16 (opaque pointers)

2023-10-20 Thread Thomas Munro
On Thu, Oct 19, 2023 at 6:20 AM Thomas Munro wrote: > Interestingly, a new problem just showed up on the the RHEL9 s390x > machine "lora", where a previously reported problem [1] apparently > re-appeared. It complains about incompatible layout, previously > blamed on mismatch between clang and LL

Re: LLVM 16 (opaque pointers)

2023-10-18 Thread Devrim Gündüz
Hi, On Thu, 2023-10-19 at 06:20 +1300, Thomas Munro wrote: > Pushed, after digging up some old LLVM skeletons to test, and those > "old LLVM" animals are turning green now.  I went ahead and pushed the > much smaller and simpler patch for LLVM 17. Thank you! I can confirm that RPMs built fine o

Re: LLVM 16 (opaque pointers)

2023-10-18 Thread Andres Freund
Hi, On 2023-10-19 06:20:26 +1300, Thomas Munro wrote: > On Thu, Oct 19, 2023 at 1:06 AM Thomas Munro wrote: > > On Thu, Oct 19, 2023 at 12:02 AM Thomas Munro > > wrote: > > > I pushed the first patch, for LLVM 16, and the build farm told me that > > > some old LLVM versions don't like it. The

Re: LLVM 16 (opaque pointers)

2023-10-18 Thread Thomas Munro
On Thu, Oct 19, 2023 at 1:06 AM Thomas Munro wrote: > On Thu, Oct 19, 2023 at 12:02 AM Thomas Munro wrote: > > I pushed the first patch, for LLVM 16, and the build farm told me that > > some old LLVM versions don't like it. The problem seems to be the > > function LLVMGlobalGetValueType(). I ca

Re: LLVM 16 (opaque pointers)

2023-10-18 Thread Thomas Munro
On Thu, Oct 19, 2023 at 12:02 AM Thomas Munro wrote: > I pushed the first patch, for LLVM 16, and the build farm told me that > some old LLVM versions don't like it. The problem seems to be the > function LLVMGlobalGetValueType(). I can see that that was only added > to the C API in 2018, so it

Re: LLVM 16 (opaque pointers)

2023-10-18 Thread Thomas Munro
I pushed the first patch, for LLVM 16, and the build farm told me that some old LLVM versions don't like it. The problem seems to be the function LLVMGlobalGetValueType(). I can see that that was only added to the C API in 2018, so it looks like I may need to back-port that (trivial) wrapper into

Re: LLVM 16 (opaque pointers)

2023-10-16 Thread Ronan Dunklau
Le vendredi 13 octobre 2023, 22:32:13 CEST Thomas Munro a écrit : > On Wed, Oct 11, 2023 at 10:31 PM Ronan Dunklau wrote: > > Le mercredi 11 octobre 2023, 10:59:50 CEST Thomas Munro a écrit : > > > The back-patch to 12 was a little trickier than anticipated, but after > > > taking a break and try

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Thomas Munro
On Sat, Oct 14, 2023 at 3:56 AM Andres Freund wrote: > On 2023-10-13 16:44:13 +0200, Dmitry Dolgov wrote: > > Here is what I had in mind (only this part in the second patch was changed). > > Makes sense to me. I think we'll likely eventually want to use a custom > pipeline anyway, and I think we s

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Thomas Munro
On Wed, Oct 11, 2023 at 10:31 PM Ronan Dunklau wrote: > Le mercredi 11 octobre 2023, 10:59:50 CEST Thomas Munro a écrit : > > The back-patch to 12 was a little trickier than anticipated, but after > > taking a break and trying again I now have PG 12...17 patches that > > I've tested against LLVM 1

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Andres Freund
On 2023-10-13 16:44:13 +0200, Dmitry Dolgov wrote: > Here is what I had in mind (only this part in the second patch was changed). Makes sense to me. I think we'll likely eventually want to use a custom pipeline anyway, and I think we should consider using an optimization level inbetween "not at al

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Andres Freund
Hi, On 2023-10-13 11:06:21 +0200, Dmitry Dolgov wrote: > > On Thu, Oct 12, 2023 at 04:31:20PM -0700, Andres Freund wrote: > > I also don't think we should add the mem2reg pass outside of -O0 - running > > it > > after a real optimization pipeline doesn't seem useful and might even make > > the >

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Dmitry Dolgov
> On Fri, Oct 13, 2023 at 11:06:21AM +0200, Dmitry Dolgov wrote: > > On Thu, Oct 12, 2023 at 04:31:20PM -0700, Andres Freund wrote: > > I don't think the "function(no-op-function),no-op-module" bit does something > > particularly useful? > > Right, looks like leftovers after verifying which passes

Re: LLVM 16 (opaque pointers)

2023-10-13 Thread Dmitry Dolgov
> On Thu, Oct 12, 2023 at 04:31:20PM -0700, Andres Freund wrote: > Hi, > > On 2023-10-11 21:59:50 +1300, Thomas Munro wrote: > > +#else > > + LLVMPassBuilderOptionsRef options; > > + LLVMErrorRef err; > > + int compile_optlevel; > > + char *passes; > > + > > + if

Re: LLVM 16 (opaque pointers)

2023-10-12 Thread Andres Freund
Hi, On 2023-10-11 21:59:50 +1300, Thomas Munro wrote: > +#else > + LLVMPassBuilderOptionsRef options; > + LLVMErrorRef err; > + int compile_optlevel; > + char *passes; > + > + if (context->base.flags & PGJIT_OPT3) > + compile_optlevel = 3;

Re: LLVM 16 (opaque pointers)

2023-10-11 Thread Ronan Dunklau
Le mercredi 11 octobre 2023, 10:59:50 CEST Thomas Munro a écrit : > The back-patch to 12 was a little trickier than anticipated, but after > taking a break and trying again I now have PG 12...17 patches that > I've tested against LLVM 10...18 (that's 54 combinations), in every > case only with the

Re: LLVM 16 (opaque pointers)

2023-10-11 Thread Thomas Munro
On Thu, Sep 21, 2023 at 12:47 PM Thomas Munro wrote: > On Thu, Sep 21, 2023 at 12:24 PM Devrim Gündüz wrote: > > RHEL releases new LLVM version along with their new minor releases every > > 6 month, and we have to build older versions with new LLVM each time. > > From RHEL point of view, it would

Re: LLVM 16 (opaque pointers)

2023-09-20 Thread Thomas Munro
On Thu, Sep 21, 2023 at 12:24 PM Devrim Gündüz wrote: > On Thu, 2023-09-21 at 08:22 +1200, Thomas Munro wrote: > > So far I've tested LLVM versions 10, 15, 16, 17, 18 (= their main > > branch) against PostgreSQL versions 14, 15, 16. I've attached the > > versions that apply to master and 16, and

Re: LLVM 16 (opaque pointers)

2023-09-20 Thread Devrim Gündüz
Hi Thomas, On Thu, 2023-09-21 at 08:22 +1200, Thomas Munro wrote: > So far I've tested LLVM versions 10, 15, 16, 17, 18 (= their main > branch) against PostgreSQL versions 14, 15, 16.  I've attached the > versions that apply to master and 16, and pushed back-patches to 14 > and 15 to public bran

Re: LLVM 16 (opaque pointers)

2023-09-20 Thread Thomas Munro
Belated thanks Dmitry, Ronan, Andres for your feedback. Here's a new version, also including Dmitry's patch for 17 which it is now also time to push. It required a bit more trivial #if magic to be conditional, as Dmitry already mentioned. I just noticed that Dmitry had the LLVMPassBuilderOptions

Re: LLVM 16 (opaque pointers)

2023-08-15 Thread Fabien COELHO
[...] Further changes are already needed for their "main" branch (LLVM 17-to-be), so this won't quite be enough to shut seawasp up. For information, the physical server which was hosting my 2 bf animals (seawasp and moonjelly) has given up rebooting after a power cut a few weeks/months ago,

Re: LLVM 16 (opaque pointers)

2023-08-11 Thread Andres Freund
Hi, On 2023-05-21 15:01:41 +1200, Thomas Munro wrote: > *For anyone working with this type of IR generation code and > questioning their sanity, I can pass on some excellent advice I got > from Andres: build LLVM yourself with assertions enabled, as they > catch some classes of silly mistake that

Re: LLVM 16 (opaque pointers)

2023-08-11 Thread Andres Freund
Hi, On 2023-08-10 16:56:54 +0200, Ronan Dunklau wrote: > I tried my hand at backporting it to previous versions, and not knowing > anything about it made me indeed question my sanity. It's quite easy for PG > 15, 14, 13. PG 12 is nothing insurmontable either, but PG 11 is a bit hairier > most not

Re: LLVM 16 (opaque pointers)

2023-08-10 Thread Ronan Dunklau
Le dimanche 21 mai 2023, 05:01:41 CEST Thomas Munro a écrit : > Hi, > > Here is a draft version of the long awaited patch to support LLVM 16. > It's mostly mechanical donkeywork, but it took some time: this donkey > found it quite hard to understand the mighty getelementptr > instruction[1] and th

Re: LLVM 16 (opaque pointers)

2023-06-04 Thread Dmitry Dolgov
> On Mon, May 22, 2023 at 03:38:44PM +1200, Thomas Munro wrote: > Further changes are already needed for their "main" branch (LLVM > 17-to-be), so this won't quite be enough to shut seawasp up. At a > glance, we will need to change from the "old pass manager" API that > has recently been vaporised

Re: LLVM 16 (opaque pointers)

2023-05-21 Thread Thomas Munro
Oh, one important thing I forgot to mention: that patch is for LLVM 16 only, and I developed it with a local build of their "release/16.x" branch on a FreeBSD box, and also tested with a released package for 16 on a Debian box. Further changes are already needed for their "main" branch (LLVM 17-to

LLVM 16 (opaque pointers)

2023-05-20 Thread Thomas Munro
Hi, Here is a draft version of the long awaited patch to support LLVM 16. It's mostly mechanical donkeywork, but it took some time: this donkey found it quite hard to understand the mighty getelementptr instruction[1] and the code generation well enough to figure out all the right types, and small