Re: [HACKERS] Optimization for lazy_scan_heap

2016-10-17 Thread Masahiko Sawada
On Mon, Oct 3, 2016 at 10:59 AM, Michael Paquier wrote: > On Mon, Sep 26, 2016 at 5:26 AM, Rahila Syed wrote: >> Some initial comments on optimize_lazy_scan_heap_v2.patch. > > Seems worth pursuing. Marking as returned with feedback because of > lack of activity and some basic reviews sent. Thank

Re: [HACKERS] Optimization for lazy_scan_heap

2016-10-02 Thread Michael Paquier
On Mon, Sep 26, 2016 at 5:26 AM, Rahila Syed wrote: > Some initial comments on optimize_lazy_scan_heap_v2.patch. Seems worth pursuing. Marking as returned with feedback because of lack of activity and some basic reviews sent. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@po

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-25 Thread Rahila Syed
Hello, Some initial comments on optimize_lazy_scan_heap_v2.patch. >- while (next_unskippable_block < nblocks) >+ while (next_unskippable_block < nblocks && >+ !FORCE_CHECK_PAGE(next_unskippable_block)) Dont we need similar check of !FORCE_CHECK_P

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Masahiko Sawada
On Thu, Sep 8, 2016 at 11:27 PM, Jim Nasby wrote: > On 9/8/16 3:03 AM, Masahiko Sawada wrote: >> >> Current manual vacuum doesn't output how may all_frozen pages we >> skipped according to visibility map. >> That's why I attached 0001 patch which makes the manual vacuum emit >> such information. >

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Robert Haas
On Thu, Sep 8, 2016 at 4:03 AM, Masahiko Sawada wrote: > On Wed, Sep 7, 2016 at 4:11 PM, Simon Riggs wrote: >> On 7 September 2016 at 04:13, Masahiko Sawada wrote: >> >>> Since current HEAD could scan visibility map twice, the execution time >>> of Patched is approximately half of HEAD. >> >> So

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Jim Nasby
On 9/8/16 3:03 AM, Masahiko Sawada wrote: Current manual vacuum doesn't output how may all_frozen pages we skipped according to visibility map. That's why I attached 0001 patch which makes the manual vacuum emit such information. I think we should add that, and info about all-frozen skips, rega

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-08 Thread Masahiko Sawada
On Wed, Sep 7, 2016 at 4:11 PM, Simon Riggs wrote: > On 7 September 2016 at 04:13, Masahiko Sawada wrote: > >> Since current HEAD could scan visibility map twice, the execution time >> of Patched is approximately half of HEAD. > > Sounds good. > > To ensure we are doing exactly same amount of wor

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-07 Thread Robert Haas
On Tue, Sep 6, 2016 at 11:13 PM, Masahiko Sawada wrote: > I understood, thank you. > > I've measured the performance benefit of this patch by following steps. > 1. Create very large table and set all-visible flag to all blocks. > 2. Measure the execution time of vacuum that skips to scan all heap

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-07 Thread Simon Riggs
On 7 September 2016 at 04:13, Masahiko Sawada wrote: > Since current HEAD could scan visibility map twice, the execution time > of Patched is approximately half of HEAD. Sounds good. To ensure we are doing exactly same amount of work as before, did you see the output of VACUUM VEROBOSE? Can we

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-06 Thread Masahiko Sawada
On Wed, Sep 7, 2016 at 1:47 AM, Robert Haas wrote: > On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >>> What performance difference does this make, in a realistic use case? >> >> I have yet to measure performance effect but it would be effect for >> very large frozen table. > > I think if

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-06 Thread Robert Haas
On Mon, Sep 5, 2016 at 8:57 PM, Masahiko Sawada wrote: >> What performance difference does this make, in a realistic use case? > > I have yet to measure performance effect but it would be effect for > very large frozen table. I think if you are proposing this patch because you think that the exis

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Masahiko Sawada
On Mon, Sep 5, 2016 at 6:27 PM, Simon Riggs wrote: > On 4 August 2016 at 05:57, Masahiko Sawada wrote: > >> While reviewing freeze map code, Andres pointed out[1] that >> lazy_scan_heap could accesses visibility map twice and its logic is >> seems a bit tricky. >> As discussed before, it's not ni

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-05 Thread Simon Riggs
On 4 August 2016 at 05:57, Masahiko Sawada wrote: > While reviewing freeze map code, Andres pointed out[1] that > lazy_scan_heap could accesses visibility map twice and its logic is > seems a bit tricky. > As discussed before, it's not nice especially when large relation is > entirely frozen. > >

Re: [HACKERS] Optimization for lazy_scan_heap

2016-08-24 Thread Masahiko Sawada
On Thu, Aug 25, 2016 at 1:41 AM, Anastasia Lubennikova wrote: > The following review has been posted through the commitfest application: > make installcheck-world: tested, passed > Implements feature: not tested > Spec compliant: not tested > Documentation:not tested >

Re: [HACKERS] Optimization for lazy_scan_heap

2016-08-24 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: not tested Spec compliant: not tested Documentation:not tested Hi, I haven't tested the performance yet, but the patch itself looks pret

[HACKERS] Optimization for lazy_scan_heap

2016-08-03 Thread Masahiko Sawada
Hi all, While reviewing freeze map code, Andres pointed out[1] that lazy_scan_heap could accesses visibility map twice and its logic is seems a bit tricky. As discussed before, it's not nice especially when large relation is entirely frozen. I posted the patch for that before but since this is an