Re: Generating partitioning tuple conversion maps faster

2019-06-18 Thread Michael Paquier
On Mon, Jun 17, 2019 at 12:03:03PM +0200, didier wrote: > cherry-pick apply cleanly and with a 100 columns table it improves > performance nicely (20%). 42f70cd is a performance improvement, and not a bug fix. -- Michael signature.asc Description: PGP signature

Re: Generating partitioning tuple conversion maps faster

2019-06-17 Thread didier
Hi, Any chance for a backport to PG 11? cherry-pick apply cleanly and with a 100 columns table it improves performance nicely (20%). Regards Didier On Sat, Jul 14, 2018 at 1:25 AM David Rowley wrote: > > On 14 July 2018 at 04:57, Heikki Linnakangas wrote: > > Pushed, thanks! > > Thanks for pu

Re: Generating partitioning tuple conversion maps faster

2018-07-13 Thread David Rowley
On 14 July 2018 at 04:57, Heikki Linnakangas wrote: > Pushed, thanks! Thanks for pushing, and thanks again for reviewing it, Alexander. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: Generating partitioning tuple conversion maps faster

2018-07-13 Thread Heikki Linnakangas
On 09/07/18 22:58, David Rowley wrote: On 9 July 2018 at 23:28, Alexander Kuzmenkov wrote: On 07/09/2018 10:13 AM, David Rowley wrote: I've attached v5. v5 looks good to me, I've changed the status to ready. Many thanks for reviewing this. Pushed, thanks! - Heikki

Re: Generating partitioning tuple conversion maps faster

2018-07-09 Thread David Rowley
On 9 July 2018 at 23:28, Alexander Kuzmenkov wrote: > On 07/09/2018 10:13 AM, David Rowley wrote: >> I've attached v5. > > v5 looks good to me, I've changed the status to ready. Many thanks for reviewing this. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Developmen

Re: Generating partitioning tuple conversion maps faster

2018-07-09 Thread Alexander Kuzmenkov
On 07/09/2018 10:13 AM, David Rowley wrote: I've attached v5. v5 looks good to me, I've changed the status to ready. Please feel free to add yourself as an author of this patch in the commitfest app. You've probably contributed about as much as I have to this. Thanks, I'm fine with being cr

Re: Generating partitioning tuple conversion maps faster

2018-07-09 Thread David Rowley
On 7 July 2018 at 01:08, Alexander Kuzmenkov wrote: > Great. I think we can use the same approach for make_inh_translation_list, > as in the attached patch. It show some improvement on test 6. I got the > following tps, median of 11 runs (forgot to turn off fsync though): > > test masterv3

Re: Generating partitioning tuple conversion maps faster

2018-07-06 Thread Alexander Kuzmenkov
On 07/05/2018 02:52 PM, David Rowley wrote: On 30 June 2018 at 05:40, David Rowley wrote: I think your idea to reduce the loops in test 6 from 2000 down to 1001 should be worth it. I'll try the idea out next week. The attached changes things to use your way of picking up the search for the nex

Re: Generating partitioning tuple conversion maps faster

2018-07-05 Thread David Rowley
On 30 June 2018 at 05:40, David Rowley wrote: > I think your idea > to reduce the loops in test 6 from 2000 down to 1001 should be worth > it. I'll try the idea out next week. The attached changes things to use your way of picking up the search for the next column at the column after the last mat

Re: Generating partitioning tuple conversion maps faster

2018-06-29 Thread David Rowley
On 30 June 2018 at 00:22, Alexander Kuzmenkov wrote: > On 06/29/2018 03:25 AM, David Rowley wrote: >> I've attached a patch that uses SearchSysCacheAttName to speed up >> these translations in the planner. > > Good idea. On my desktop, this gives 270 tps dropped vs 610 tps plain (for > updates). I

Re: Generating partitioning tuple conversion maps faster

2018-06-29 Thread Alexander Kuzmenkov
On 06/29/2018 03:25 AM, David Rowley wrote: I've attached a patch that uses SearchSysCacheAttName to speed up these translations in the planner. Good idea. On my desktop, this gives 270 tps dropped vs 610 tps plain (for updates). If you combine it with persistent inner loop index, it's probab

Re: Generating partitioning tuple conversion maps faster

2018-06-28 Thread David Rowley
On 29 June 2018 at 11:10, David Rowley wrote: > On further inspection, the slowdown is coming from the planner in > make_inh_translation_list(). The INSERT path does not hit that since > the planner's job is pretty simple for simple INSERTs. I've attached a patch that uses SearchSysCacheAttName t

Re: Generating partitioning tuple conversion maps faster

2018-06-28 Thread David Rowley
On 29 June 2018 at 02:23, Alexander Kuzmenkov wrote: > I think I found one possible reason for this slowness. Your patch behaves as > expected when there is a dropped output column, but does one extra > comparison when there is a dropped input column. This backwards conversion > is called from Exe

Re: Generating partitioning tuple conversion maps faster

2018-06-28 Thread Alexander Kuzmenkov
On 06/25/2018 08:00 AM, David Rowley wrote: I'd have expected Test 6 to do about 480-500tps. Adding debug to check why it's not revealed that the search is doing what's expected. I'm unsure why it has not improved more. I think I found one possible reason for this slowness. Your patch behaves

Re: Generating partitioning tuple conversion maps faster

2018-06-24 Thread amul sul
On Mon, Jun 25, 2018 at 10:57 AM Amit Langote wrote: > > On 2018/06/25 14:12, amul sul wrote: > > On Mon, Jun 25, 2018 at 10:30 AM David Rowley > > wrote: > >> > > [...] > >> Given the small size of this patch. I think it's a good candidate for > >> the July 'fest. > >> > > > > Just a different t

Re: Generating partitioning tuple conversion maps faster

2018-06-24 Thread Amit Langote
On 2018/06/25 14:12, amul sul wrote: > On Mon, Jun 25, 2018 at 10:30 AM David Rowley > wrote: >> > [...] >> Given the small size of this patch. I think it's a good candidate for >> the July 'fest. >> > > Just a different thought, how about having flag array something like > needed_tuple_conv? >

Re: Generating partitioning tuple conversion maps faster

2018-06-24 Thread amul sul
On Mon, Jun 25, 2018 at 10:30 AM David Rowley wrote: > [...] > Given the small size of this patch. I think it's a good candidate for > the July 'fest. > Just a different thought, how about having flag array something like needed_tuple_conv? While loading partdesc, we could calculate that the col

Generating partitioning tuple conversion maps faster

2018-06-24 Thread David Rowley
Hi, The code in convert_tuples_by_name_map() could be made a bit smarter to speed up the search for the column with the same name by first looking at the same attnum in the other relation rather than starting the search from the first attnum each time. I imagine most people are going to create pa