Re: Make tuple deformation faster

2024-12-23 Thread David Rowley
On Tue, 24 Dec 2024 at 11:19, David Rowley wrote: > The attached adjusts that Assert code so that a fresh CompactAttribute > is populated instead of modifying the TupleDesc's one. I'm not sure > if populate_compact_attribute_internal() is exactly the nicest way to > do this. I'll think a bit hard

Re: Make tuple deformation faster

2024-12-23 Thread David Rowley
On Tue, 24 Dec 2024 at 02:00, Alexander Lakhin wrote: > Please look at the following query, which triggers (sometimes not on a > first run) an assert added with 5983a4cff: > regression=# SELECT COUNT(*) FROM > (SELECT (aclexplode(proacl)).* FROM pg_proc) a, > (SELECT oid FROM pg_proc UNION ALL SE

Re: Make tuple deformation faster

2024-12-23 Thread David Rowley
On Tue, 24 Dec 2024 at 02:00, Alexander Lakhin wrote: > regression=# SELECT COUNT(*) FROM > (SELECT (aclexplode(proacl)).* FROM pg_proc) a, > (SELECT oid FROM pg_proc UNION ALL SELECT oid FROM pg_proc) b; > WARNING: terminating connection because of crash of another server process > ... > TRAP: f

Re: Make tuple deformation faster

2024-12-23 Thread Alexander Lakhin
Hello David, 20.12.2024 12:31, David Rowley wrote: The attcacheoff removal is now pushed. I've attached the two remaining patches. Please look at the following query, which triggers (sometimes not on a first run) an assert added with 5983a4cff: regression=# SELECT COUNT(*) FROM (SELECT (aclexp

Re: Make tuple deformation faster

2024-12-20 Thread David Rowley
On Fri, 20 Dec 2024 at 23:04, David Rowley wrote: > I've now pushed the 0001 patch and the 0002 patch as one commit. I'm > going to give the buildfarm a bit of time then push the commit to > remove pg_attribute.attcacheoff. I'm planning on letting that settle > overnight then if all is well push

Re: Make tuple deformation faster

2024-12-20 Thread David Rowley
On Thu, 5 Dec 2024 at 13:09, Andres Freund wrote: > On 2024-12-05 11:52:01 +1300, David Rowley wrote: > > On Thu, 5 Dec 2024 at 03:51, Andres Freund wrote: > > > Possibly stupid idea: Could we instead store the attributes *before* the > > > main > > > TupleDescData, with increasing "distance" fo

Re: Make tuple deformation faster

2024-12-04 Thread Andres Freund
Hi, On 2024-12-05 11:52:01 +1300, David Rowley wrote: > On Thu, 5 Dec 2024 at 03:51, Andres Freund wrote: > > Possibly stupid idea: Could we instead store the attributes *before* the > > main > > TupleDescData, with increasing "distance" for increased attnos? That way we > > wouldn't need to ca

Re: Make tuple deformation faster

2024-12-04 Thread Matthias van de Meent
On Wed, 4 Dec 2024 at 23:52, David Rowley wrote: > > On Thu, 5 Dec 2024 at 03:51, Andres Freund wrote: > > Possibly stupid idea: Could we instead store the attributes *before* the > > main > > TupleDescData, with increasing "distance" for increased attnos? That way we > > wouldn't need to calcu

Re: Make tuple deformation faster

2024-12-04 Thread David Rowley
On Thu, 5 Dec 2024 at 03:51, Andres Freund wrote: > Possibly stupid idea: Could we instead store the attributes *before* the main > TupleDescData, with increasing "distance" for increased attnos? That way we > wouldn't need to calculate any variable offsets. Of course the price would be > to have

Re: Make tuple deformation faster

2024-12-04 Thread Andres Freund
Hi, On 2024-12-05 01:42:36 +1300, David Rowley wrote: > On Tue, 3 Dec 2024 at 16:54, David Rowley wrote: > > After making some last-minute cosmetic adjustments, I've just pushed > > the 0001 patch. > > So that commit broke all of the debug_parallel_query = regress > buildfarm animals. Seems we

Re: Make tuple deformation faster

2024-12-04 Thread David Rowley
On Tue, 3 Dec 2024 at 16:54, David Rowley wrote: > After making some last-minute cosmetic adjustments, I've just pushed > the 0001 patch. So that commit broke all of the debug_parallel_query = regress buildfarm animals. This was due to how I'd changed the TupleDescData.attrs array to a pointer wh

Re: Make tuple deformation faster

2024-12-02 Thread David Rowley
On Mon, 2 Dec 2024 at 23:24, David Rowley wrote: > I spent all day today reviewing and fixing up a few missing comments > for the v5 patch series. I'm quite happy with these now. If nobody > else wants to look or test, I plan on pushing these tomorrow (Tuesday > UTC+13). If anyone wants me to dela

Re: Make tuple deformation faster

2024-12-02 Thread Victor Yegorov
пн, 2 дек. 2024 г. в 13:24, David Rowley : > I ended up adjusting the code here so that TupleDescSize() returns the > full size and TupleDescAttrAddress() manually calculates the offset to > start the FormData_pg_attribute array. That allows > TupleDescFullSize() to be deleted. I changed how Tup

Re: Make tuple deformation faster

2024-12-02 Thread David Rowley
On Sat, 30 Nov 2024 at 02:54, Victor Yegorov wrote: > I've been testing this patch for the last week, I have M3 and i7 based MBP > around. Thanks for having a look at this and running the benchmarks. > Construct >sizeof(FormData_pg_attribute) * (src)->natts > is used in 7 places (in various

Re: Make tuple deformation faster

2024-10-03 Thread Andres Freund
Hi, A few weeks ago David and I discussed this patch. We were curious *why* the flags approach was slower. It turns out that, at least on my machine, this is just a compiler optimization issue. Putting a pg_compiler_barrier() just after: > for (; attnum < natts; attnum++) > { > -

Re: Make tuple deformation faster

2024-08-05 Thread David Rowley
On Tue, 16 Jul 2024 at 00:13, Matthias van de Meent wrote: > > On Tue, 2 Jul 2024 at 02:23, David Rowley wrote: > > I'm happy to keep going with this version of the patch > > +1, go for it. I've attached an updated patch series which are a bit more polished than the last set. I've attempted to d

Re: Make tuple deformation faster

2024-07-24 Thread John Naylor
On Mon, Jul 1, 2024 at 5:07 PM David Rowley wrote: > cycles idle >8505168 stalled-cycles-backend:u #0.02% backend cycles > idle > 165442142326 instructions:u#3.35 insn per cycle > #0.00 stall

Re: Make tuple deformation faster

2024-07-15 Thread Matthias van de Meent
On Tue, 2 Jul 2024 at 02:23, David Rowley wrote: > > On Mon, 1 Jul 2024 at 23:42, Matthias van de Meent > wrote: > > > > On Mon, 1 Jul 2024 at 12:49, David Rowley wrote: > > > > > > On Mon, 1 Jul 2024 at 22:07, Matthias van de Meent > > > wrote: > > > > One thing I'm slightly concerned about is

Re: Make tuple deformation faster

2024-07-01 Thread Andy Fan
David Rowley writes: > You can see the branch predictor has done a *much* better job in the > patched code vs master with about 10x fewer misses. This should have > helped contribute to the "insn per cycle" increase. 4.29 is quite > good for postgres. I often see that around 0.5. According to [

Re: Make tuple deformation faster

2024-07-01 Thread David Rowley
On Mon, 1 Jul 2024 at 23:42, Matthias van de Meent wrote: > > On Mon, 1 Jul 2024 at 12:49, David Rowley wrote: > > > > On Mon, 1 Jul 2024 at 22:07, Matthias van de Meent > > wrote: > > > One thing I'm slightly concerned about is that this allocates another > > > 8 bytes for each attribute in the

Re: Make tuple deformation faster

2024-07-01 Thread Matthias van de Meent
On Mon, 1 Jul 2024 at 12:49, David Rowley wrote: > > On Mon, 1 Jul 2024 at 22:07, Matthias van de Meent > wrote: > > One thing I'm slightly concerned about is that this allocates another > > 8 bytes for each attribute in the tuple descriptor. While that's not a > > lot when compared with the ->at

Re: Make tuple deformation faster

2024-07-01 Thread David Rowley
On Mon, 1 Jul 2024 at 22:07, Matthias van de Meent wrote: > Cool, that's similar to, but even better than, my patch from 2021 over at [0]. I'll have a read of that. Thanks for pointing it out. > One thing I'm slightly concerned about is that this allocates another > 8 bytes for each attribute in

Re: Make tuple deformation faster

2024-07-01 Thread David Rowley
On Mon, 1 Jul 2024 at 21:17, Andy Fan wrote: > Yet another a wonderful optimization! I just want to know how did you > find this optimization (CPU cache hit) case and think it worths some > time. because before we invest our time to optimize something, it is > better know that we can get some meas

Re: Make tuple deformation faster

2024-07-01 Thread Matthias van de Meent
On Mon, 1 Jul 2024 at 10:56, David Rowley wrote: > > Currently, TupleDescData contains the descriptor's attributes in a > variable length array of FormData_pg_attribute allocated within the > same allocation as the TupleDescData. According to my IDE, > sizeof(FormData_pg_attribute) == 104 bytes. I

Re: Make tuple deformation faster

2024-07-01 Thread Andy Fan
David Rowley writes: > Currently, TupleDescData contains the descriptor's attributes in a > variable length array of FormData_pg_attribute allocated within the > same allocation as the TupleDescData. According to my IDE, > sizeof(FormData_pg_attribute) == 104 bytes. It's that large mainly due > t