Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-02-26 Thread Amit Kapila
On Sat, Feb 27, 2016 at 12:41 AM, Andres Freund wrote: > > Hi, > > On 2016-02-25 12:56:39 +0530, Amit Kapila wrote: > > From past few weeks, we were facing some performance degradation in the > > read-only performance bench marks in high-end machines. My colleague

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-02-26 Thread Amit Kapila
On Tue, Feb 23, 2016 at 7:06 PM, Robert Haas wrote: > On Sun, Feb 21, 2016 at 7:45 PM, Amit Kapila > wrote: > >> I mean, my basic feeling is that I would not accept a 2-3% regression in >>> the single client case to get a 10% speedup in the case where we have 128 >&

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-02-26 Thread Amit Kapila
On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila wrote: > > > Here, we can see that there is a gain of ~15% to ~38% at higher client > count. > > The attached document (perf_write_clogcontrollock_data_v6.ods) contains > data, mainly focussing on single client performance. The d

Re: [HACKERS] Move PinBuffer and UnpinBuffer to atomics

2016-02-28 Thread Amit Kapila
8 sockets and 64 cores, I have seen more regression compared to my previous run in power8 with 2 socket, currently I tested Read only workload for 5 mins Run, When I get time, I will run for longer time and confirm again. > Have you tried by reverting the commits 6150a1b0 and ac1d794, which I

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-02-29 Thread Amit Kapila
On Wed, Feb 24, 2016 at 7:14 PM, Robert Haas wrote: > > On Mon, Feb 22, 2016 at 10:05 AM, Amit Kapila wrote: > >> I wouldn't bother tinkering with it at this point. The value isn't > >> going to be recorded on disk anywhere, so it will be easy to change > &

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-02-29 Thread Amit Kapila
and wait event and will add few examples as well. Let me know if you want to see something else with respect to documentation? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-02-29 Thread Amit Kapila
On Mon, Feb 29, 2016 at 11:10 PM, Robert Haas wrote: > > On Fri, Feb 26, 2016 at 11:37 PM, Amit Kapila wrote: > > On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila > > wrote: > >> > >> Here, we can see that there is a gain of ~15% to ~38% at higher client > &g

Re: [HACKERS] Relation extension scalability

2016-03-01 Thread Amit Kapila
to try by doing extend w.r.t load in system. I mean to say we can batch the extension requests (as we have done in ProcArrayGroupClearXid) and extend accordingly, if that works out then the benefit could be that we don't need any configurable knob for the same. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-01 Thread Amit Kapila
etpid()) % NUM_FREELISTS : 0) > > ... > > // now nentries could be negative in this case > // Assert(FREELIST(hctl, freelist_idx).nentries > 0); > > In which case, do you think entries can go negative? I think the nentries is incremented and decremented in the way as without patch, so I am not getting what can make it go negative. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-01 Thread Amit Kapila
On Mon, Feb 29, 2016 at 11:10 PM, Robert Haas wrote: > > On Fri, Feb 26, 2016 at 11:37 PM, Amit Kapila wrote: > > On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila > > wrote: > >> > >> Here, we can see that there is a gain of ~15% to ~38% at higher client > &g

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-03-01 Thread Amit Kapila
t would tell you whether the > performance is not increasing because the patch doesn't reduce > ProcArrayLock contention, or whether the performance is not increasing > because the patch DOES reduce ProcArrayLock contention but then the > contention shifts to CLogControlLock or elsewhere. > +1 With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-01 Thread Amit Kapila
o matter what freelist was used when element was added to a hash table. > We always try to return free memory to the same freelist number getpid() > % FREELIST_ITEMS and decrease number of elements in a hash table using > corresponding nentries field. > Won't it always use the same freelist to remove and add the entry from freelist as for both cases it will calculate the freelist_idx in same way? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-03 Thread Amit Kapila
On Wed, Feb 24, 2016 at 7:14 PM, Robert Haas wrote: > > On Mon, Feb 22, 2016 at 10:05 AM, Amit Kapila wrote: > >> I wouldn't bother tinkering with it at this point. The value isn't > >> going to be recorded on disk anywhere, so it will be easy to change > &

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-03 Thread Amit Kapila
sure if that applies here, so feel free to go in the way you think is better. [1] - http://www.postgresql.org/message-id/CA+TgmoacVsdcY=qn0do7nok7w2-ssqz3kr2y84bavifckqd...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] ExecGather() + nworkers

2016-03-04 Thread Amit Kapila
On Fri, Mar 4, 2016 at 11:57 AM, Haribabu Kommi wrote: > On Wed, Jan 13, 2016 at 7:19 PM, Amit Kapila > wrote: > >> > > > > Changed the code such that nworkers_launched gets used wherever > > appropriate instead of nworkers. This includes places other th

Re: [HACKERS] ExecGather() + nworkers

2016-03-04 Thread Amit Kapila
On Fri, Mar 4, 2016 at 5:21 PM, Haribabu Kommi wrote: > On Fri, Mar 4, 2016 at 10:33 PM, Amit Kapila > wrote: > > On Fri, Mar 4, 2016 at 11:57 AM, Haribabu Kommi < > kommi.harib...@gmail.com> > > wrote: > >> > >> On Wed, Jan

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-04 Thread Amit Kapila
On Fri, Mar 4, 2016 at 4:01 PM, Alexander Korotkov wrote: > On Fri, Mar 4, 2016 at 7:05 AM, Amit Kapila > wrote: > >> > I think the wait event types should be documented - and the wait >> > events too, perhaps. >> > >> >> As discussed upthread,

Re: [HACKERS] ExecGather() + nworkers

2016-03-04 Thread Amit Kapila
On Fri, Mar 4, 2016 at 11:41 PM, Robert Haas wrote: > On Fri, Mar 4, 2016 at 6:55 AM, Amit Kapila > wrote: > > On Fri, Mar 4, 2016 at 5:21 PM, Haribabu Kommi > > > wrote: > >> > >> On Fri, Mar 4, 2016 at 10:33 PM, Amit Kapila > >> wrote: > &

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-04 Thread Amit Kapila
moment. > > I'm also not keen on all the "A server process is waiting" all the way > down the list. > How about giving the column name as "Wait For" instead of "Description" and then use text like "finding or allocating space in shared memory"? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-04 Thread Amit Kapila
the reloid for relation in each proc and then get the relation descriptor for relation extension lock. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
+ * It's only needed atop a node that doesn't support projection "needed atop a node", seems unclear to me, typo? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
On Sat, Mar 5, 2016 at 4:51 PM, Amit Kapila wrote: > > On Fri, Mar 4, 2016 at 11:31 PM, Tom Lane wrote: > > > > OK, here is a version that I think addresses all of the recent comments: > > > > * Fixed handling of parallel-query fields in new path node types. > &g

Re: [HACKERS] WIP: Upper planner pathification

2016-03-05 Thread Amit Kapila
On Sat, Mar 5, 2016 at 10:11 PM, Tom Lane wrote: > > Amit Kapila writes: > > On Fri, Mar 4, 2016 at 11:31 PM, Tom Lane wrote: > >> (BTW, I found what seemed to be a couple of pre-existing bugs of > >> the same kind, eg create_mergejoin_path was different from the &

Re: [HACKERS] WIP: Upper planner pathification

2016-03-06 Thread Amit Kapila
On Sun, Mar 6, 2016 at 9:02 PM, Tom Lane wrote: > > Amit Kapila writes: > > On Sat, Mar 5, 2016 at 10:11 PM, Tom Lane wrote: > >> Is there some reason why hash and nestloop are safe but merge isn't? > > > I think it is because we consider to pushdown hash a

Re: [HACKERS] WIP: Upper planner pathification

2016-03-06 Thread Amit Kapila
On Mon, Mar 7, 2016 at 9:13 AM, Tom Lane wrote: > > Amit Kapila writes: > >>>> Is there some reason why hash and nestloop are safe but merge isn't? > > > To make hash and nestloop work in parallel queries, we just push those > > nodes below gather node.

Re: [HACKERS] ExecGather() + nworkers

2016-03-07 Thread Amit Kapila
ter.sybase.com/archive/index.jsp?topic=/com.sybase.help.ase_15.0.sag1/html/sag1/sag1234.htm With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] WIP: Upper planner pathification

2016-03-07 Thread Amit Kapila
mtvrkkasy1xbshgzxkd6-hnxx3gq7x-p-dz0zt...@mail.gmail.com > > In summary, I think it's surprising that a max_parallel_degree of 1 > doesn't disable parallel workers entirely. > I have responded on the thread where you have raised that point with my thoughts, discussing it here on a separate point can dilute the purpose of this thread. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Relation extension scalability

2016-03-08 Thread Amit Kapila
On Mon, Mar 7, 2016 at 8:34 PM, Robert Haas wrote: > > On Fri, Mar 4, 2016 at 11:49 PM, Amit Kapila wrote: > > I think one thing which needs more thoughts about this approach is that we > > need to maintain some number of slots so that group extend for different > >

Re: [HACKERS] Relation extension scalability

2016-03-08 Thread Amit Kapila
On Tue, Mar 8, 2016 at 7:23 PM, Robert Haas wrote: > > On Tue, Mar 8, 2016 at 4:27 AM, Amit Kapila wrote: > >> Hmm. Can we drive this off of the heavyweight lock manager's idea of > >> how big the relation extension lock wait queue is, instead of adding > >

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-09 Thread Amit Kapila
th an error ERROR_ALREADY_EXISTS. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] WIP: Upper planner pathification

2016-03-09 Thread Amit Kapila
n speed-up of parallel queries especially for cases when target list contain costly expressions, so I am slightly inclined to follow that even though that looks more work. Thoughts? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com apply_tlist_partial_path_v1.patch De

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-09 Thread Amit Kapila
n't indicate any regression. [1] - http://www.postgresql.org/message-id/caa4ek1+zeb8pmwwktf+3brs0pt4ux6rs6aom0uip8c6shjw...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com extend_pg_stat_activity_v13.patch Description: Binary data -- Sent via pgsql-hac

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-09 Thread Amit Kapila
On Wed, Mar 9, 2016 at 7:17 PM, Robert Haas wrote: > > On Wed, Mar 9, 2016 at 8:31 AM, Amit Kapila wrote: > > Thanks for the suggestion. I have updated the patch to include wait_event_type information in the wait_event table. > > I think we should remove "a server proces

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-03-10 Thread Amit Kapila
LOG buffers (patch for same is posted in Speed up Clog thread [1]) as that also relieves contention on CLOG as per the tests I have done? [1] - http://www.postgresql.org/message-id/caa4ek1lmmgnq439bum0lcs3p0sb8s9kc-cugu_thnqmwa8_...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-10 Thread Amit Kapila
good to me and solves the problem mentioned. I have ran the regression tests with force_parallel_mode and doesn't see any problem. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com Analyze_select_into_disable_parallel_scan_v2.patch Description: Binary data -- Se

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-10 Thread Amit Kapila
On Wed, Mar 9, 2016 at 5:46 PM, Haribabu Kommi wrote: > On Wed, Mar 9, 2016 at 10:06 PM, Amit Kapila wrote: > > On Wed, Mar 9, 2016 at 11:46 AM, Haribabu Kommi < kommi.harib...@gmail.com> > > wrote: > >> > >> > >> I tried replacing the r

Re: [HACKERS] [COMMITTERS] pgsql: Provide much better wait information in pg_stat_activity.

2016-03-10 Thread Amit Kapila
tain some wait information and I think that will look odd to anybody new looking at the view. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-10 Thread Amit Kapila
leased buffer pins and heavyweight locks, so not sure if it makes sense to report wait end at that stage. I have noticed that in WaitOnLock(), on error the wait end is set, but now again thinking on it, it seems it will be better to set it in AbortTransaction/AbortSubTransaction at end. What

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-11 Thread Amit Kapila
On Fri, Mar 11, 2016 at 5:21 PM, Haribabu Kommi wrote: > > On Fri, Mar 11, 2016 at 12:00 AM, Amit Kapila wrote: > > > > Okay, so one probable theory for such an error could be that when there is > > already an object with same name exists, this API requests access to the

Re: [HACKERS] Relation extension scalability

2016-03-11 Thread Amit Kapila
ain when happen next time extend in multiple of load. > > How 20 comes ? > I tested with Multiple clients loads 1..64, with multiple load size 4 byte records to 1KB Records, COPY/ INSERT and found 20 works best. > Can you post the numbers for 1, 5, 10, 15, 25 or whatever other mul

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-11 Thread Amit Kapila
addition especially for testing one corner case. > > Explain Analyze produces planning time and execution time even with TIMING OFF > so not adding the same to regress tests. > Yeah, that makes the addition of test for this functionality difficult. Robert, do you have any idea wh

Re: [HACKERS] [COMMITTERS] pgsql: Provide much better wait information in pg_stat_activity.

2016-03-11 Thread Amit Kapila
micked the catalog for a specific Postgres version. > That makes sense to me if other people agree to it, but I think there will be some maintenance overhead for it, but I see that as worth the effort in terms of user convenience. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-12 Thread Amit Kapila
On Sat, Mar 12, 2016 at 2:02 PM, Mithun Cy wrote: > > > > On Sat, Mar 12, 2016 at 12:28 PM, Amit Kapila wrote > >I don't see how this test will fail with force_parallel_mode=regress and max_parallel_degree > 0 even without the patch proposed to fix the issue in >h

Re: [HACKERS] Background Processes and reporting

2016-03-12 Thread Amit Kapila
ave new view like pg_stat_background_activity with only relevant fields or try expose via individual views like pg_stat_bgwriter. Do you intend to get this done for 9.6 considering an add-on patch for wait event information displayed in pg_stat_activity? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Background Processes and reporting

2016-03-12 Thread Amit Kapila
for events like timing calculations where we suspect to have some performance penalty and during development if it is proven that none of the additional wait events cause any overhead, then we can keep them on by default. > 3) We don't want hook of wait events to be exposed. > > Can I concl

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-12 Thread Amit Kapila
On Sat, Mar 12, 2016 at 7:11 PM, Mithun Cy wrote: > > > > On Sat, Mar 12, 2016 at 2:32 PM, Amit Kapila wrote: > >With force_parallel_mode=on, I could see many other failures as well. I think it is better to have test, which tests this functionality with >force_parallel_mo

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-13 Thread Amit Kapila
On Fri, Mar 11, 2016 at 5:21 PM, Haribabu Kommi wrote: > > On Fri, Mar 11, 2016 at 12:00 AM, Amit Kapila wrote: > > > >> I am not able to find the reason for this error. This error is occurring > >> only > >> when the PostgreSQL is started as a servic

Re: [HACKERS] Prepared Statement support for Parallel query

2016-03-14 Thread Amit Kapila
ring execution that statements for non read-only operations should not enter into parallel mode similar to what we are doing for non-zero tuple count case. Attached patch fixes the problem. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com prepared_stmt_parallel_query_v2

Re: [HACKERS] Prepared Statement support for Parallel query

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 12:21 AM, Robert Haas wrote: > > On Mon, Mar 14, 2016 at 9:18 AM, Amit Kapila wrote: > > On Fri, Feb 26, 2016 at 4:37 PM, Robert Haas wrote: > > > > > > The failure cases fall into that category, basically wholePlanParallelSafe > > w

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 12:00 AM, David Steele wrote: > > On 2/26/16 11:37 PM, Amit Kapila wrote: > >> On Sat, Feb 27, 2016 at 10:03 AM, Amit Kapila > >> Here, we can see that there is a gain of ~15% to ~38% at higher >> client count. &

Re: [HACKERS] Explain [Analyze] produces parallel scan for select Into table statements.

2016-03-14 Thread Amit Kapila
On Tue, Mar 15, 2016 at 5:22 AM, Robert Haas wrote: > > On Sat, Mar 12, 2016 at 1:58 AM, Amit Kapila wrote: > > Yeah, that makes the addition of test for this functionality difficult. > > Robert, do you have any idea what kind of test would have caught this issue? > > Y

Re: [HACKERS] Background Processes and reporting

2016-03-15 Thread Amit Kapila
On Tue, Mar 15, 2016 at 1:32 AM, Andres Freund wrote: > > On 2016-03-12 16:29:11 +0530, Amit Kapila wrote: > > On Sat, Mar 12, 2016 at 3:10 AM, Andres Freund wrote: > > > > > > > > > > Similarly for the wait event stuff - checkpointer, wal writer, &

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-15 Thread Amit Kapila
On Tue, Mar 15, 2016 at 7:54 PM, David Steele wrote: > > On 3/15/16 1:17 AM, Amit Kapila wrote: > > > On Tue, Mar 15, 2016 at 12:00 AM, David Steele > > >> This patch no longer applies cleanly: > >> > >> $ git apply ../other/group_update_clog_v

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-15 Thread Amit Kapila
ared_mem' has no visible effect. > > wnum mem(MB) time(s) >0 16 247 >1 16 256 > It seems from you data that with 1 worker, you are always seeing slowdown, have you investigated the reason of same? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-16 Thread Amit Kapila
(cost=0.00..45063.68 rows=18 width=4) Output: (c1 + 2) Filter: (t1.c1 < 10) (6 rows) In the above plans, you can notice that target list expression (c1 + 2) is pushed beneath Gather node after patch. Thoughts? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.co

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-18 Thread Amit Kapila
idea sounds much better than current code in terms of understanding the free list concept as well. So, +1 for changing the code in this way. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-18 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund wrote: > > On 2016-03-18 20:14:07 +0530, Amit Kapila wrote: > > > I have done some > > tests on Windows with 0003 patch which includes running the regressions > > (vcregress check) and it passes. Will look into it tomorr

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-18 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:14 PM, Andres Freund wrote: > > > > On March 18, 2016 11:32:53 PM PDT, Amit Kapila wrote: > >On Sat, Mar 19, 2016 at 12:00 AM, Andres Freund > >wrote: > >> > >> On 2016-03-18 20:14:07 +0530, Amit Kapila wrote: > >> &

Re: Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 7:10 PM, Tom Lane wrote: > > Amit Kapila writes: > > While reading above code changes, it looks like it is assuming that subpath > > and subplan will always be same (as it is verifying projection capability > > of subpath and attaching the tlist

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 2:56 PM, Constantin S. Pan wrote: > > On Thu, 17 Mar 2016 13:21:32 +0530 > Amit Kapila wrote: > > > On Wed, Mar 16, 2016 at 7:50 PM, Constantin S. Pan > > wrote: > > > > > > On Wed, 16 Mar 2016 18:08:38 +0530 > > > Am

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
windows implementation. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 11:57 PM, Jesper Pedersen < jesper.peder...@redhat.com> wrote: > > On 03/15/2016 01:17 AM, Amit Kapila wrote: >> >> I have updated the comments and changed the name of one of a variable from >> "all_trans_same_page" to "

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 10:35 AM, David Rowley wrote: > > On 17 March 2016 at 01:19, Amit Kapila wrote: > > Few assorted comments: > > > > 2. > > AggPath * > > create_agg_path(PlannerInfo *root, > > @@ -2397,9 +2399,11 @@ create_agg_path(PlannerInfo *r

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund wrote: > > On March 18, 2016 11:52:08 PM PDT, Amit Kapila wrote: > >> >Won't the new code needs to ensure that ResetEvent(latchevent) > >should > >> >get > >> >called in case WaitForMultipleObj

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
Will look into it tomorrow once again and share if I find anything wrong with it, but feel to proceed if you want. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-19 Thread Amit Kapila
lly defining these functions under #ifndef WIN32, isn't it better to combine them all as they are at end of file. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: Pushdown target list below gather node (WAS Re: [HACKERS] WIP: Upper planner pathification)

2016-03-19 Thread Amit Kapila
and subplan correspond to different nodes when gating Result node is added on to top of scan plan by create_scan_plan(). I think it might be better to explain in comments, why it is safe to rely on projection capability of subpath to attach tlist to subplan. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 2:55 PM, Constantin S. Pan wrote: > > On Wed, 16 Mar 2016 12:14:51 +0530 > Amit Kapila wrote: > > > On Wed, Mar 16, 2016 at 5:41 AM, Constantin S. Pan > > wrote: > > > 3. Tested on some real data (GIN index on email message body > >

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
then, I think adding the reason for same in comments above function would be better. 7. tlist.c +} \ No newline at end of file There should be a new line at end of file. [1] - http://www.postgresql.org/message-id/CAA4eK1Jk8hm-2j-CKjvdd0CZTsdPX=edk_qhzc4689hq0xt...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Parallel Aggregate

2016-03-19 Thread Amit Kapila
On Thu, Mar 17, 2016 at 6:41 PM, David Rowley wrote: > > On 18 March 2016 at 01:22, Amit Kapila wrote: > > On Thu, Mar 17, 2016 at 10:35 AM, David Rowley > > wrote: > > Updated patch is attached. Thanks for the re-review. > Few more comments: 1. + if (parse-&

Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2016-03-19 Thread Amit Kapila
to have it in this release) if you don't see any patch for the same. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] POC: Cache data in GetSnapshotData()

2016-03-19 Thread Amit Kapila
04-111 NUMA node2 CPU(s): 80-87,112-119 NUMA node3 CPU(s): 88-95,120-127 NUMA node4 CPU(s): 1-8,33-40 NUMA node5 CPU(s): 9-16,41-48 NUMA node6 CPU(s): 17-24,49-56 NUMA node7 CPU(s): 25-32,57-64 With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] [WIP] speeding up GIN build with parallel workers

2016-03-19 Thread Amit Kapila
On Wed, Mar 16, 2016 at 7:50 PM, Constantin S. Pan wrote: > > On Wed, 16 Mar 2016 18:08:38 +0530 > Amit Kapila wrote: > > > > > Why backend just waits, why can't it does the same work as any worker > > does? In general, for other parallelism features the backe

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-20 Thread Amit Kapila
On Sat, Mar 19, 2016 at 7:02 PM, Robert Haas wrote: > > On Sat, Mar 19, 2016 at 12:28 AM, Amit Kapila wrote: > > Won't in theory, without patch as well nentries can overflow after running > > for very long time? I think with patch it is more prone to overflow because >

Re: [HACKERS] Patch: fix lock contention for HASHHDR.mutex

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 5:12 AM, Robert Haas wrote: > > On Sun, Mar 20, 2016 at 3:01 AM, Amit Kapila wrote: > > On Sat, Mar 19, 2016 at 7:02 PM, Robert Haas wrote: > >> On Sat, Mar 19, 2016 at 12:28 AM, Amit Kapila > >> wrote: > >> > Won't in theo

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Sun, Mar 20, 2016 at 7:13 AM, Andres Freund wrote: > > On 2016-03-19 15:43:27 +0530, Amit Kapila wrote: > > On Sat, Mar 19, 2016 at 12:40 PM, Andres Freund wrote: > > > > > > On March 18, 2016 11:52:08 PM PDT, Amit Kapila < amit.kapil...@gmail.com> >

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund wrote: > > > On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila < > amit.kapil...@gmail.com> wrote: > > >The article pointed by you justifies that the way ResetEvent is done by > >patch is correct. I am not sure,

Re: [HACKERS] Performance degradation in commit ac1d794

2016-03-20 Thread Amit Kapila
On Mon, Mar 21, 2016 at 10:26 AM, Amit Kapila wrote: > > On Mon, Mar 21, 2016 at 10:21 AM, Andres Freund wrote: >> >> >> >> On March 21, 2016 5:12:38 AM GMT+01:00, Amit Kapila < amit.kapil...@gmail.com> wrote: >> >> >The article pointed

Re: [HACKERS] Relation extension scalability

2016-03-21 Thread Amit Kapila
blocks, what is the need of same? 12. I think it is good to once test pgbench read-write tests to ensure that this doesn't introduce any new regression. [1] - http://www.postgresql.org/message-id/caa4ek1lonxz4qa_dquqbanspxsctjxrkexjii8h3gnd9z8u...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-21 Thread Amit Kapila
On Mon, Mar 21, 2016 at 6:16 PM, Haribabu Kommi wrote: > > On Mon, Mar 14, 2016 at 4:51 PM, Amit Kapila wrote: > >> Operating system - windows 7 > >> Binary - PostgreSQL 9.5 (This doesn't matter, 9.4+ can produce the > >> problem) > >> > >>

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-21 Thread Amit Kapila
p://www.postgresql.org/message-id/CAB7nPqT6gKj6iS9VTPth_h6Sz5Jo-177s6QJN_jrW66wyCjJ=w...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618”: Permission

2016-03-21 Thread Amit Kapila
On Tue, Mar 22, 2016 at 9:13 AM, Haribabu Kommi wrote: > > On Tue, Mar 22, 2016 at 2:19 PM, Amit Kapila wrote: > > On Mon, Mar 21, 2016 at 6:16 PM, Haribabu Kommi < kommi.harib...@gmail.com> > > wrote: > >> > >> On Mon, Mar 14, 2016 at 4:51 PM, Amit Ka

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-21 Thread Amit Kapila
On Tue, Mar 22, 2016 at 9:46 AM, Tom Lane wrote: > > Amit Kapila writes: > > On Mon, Mar 21, 2016 at 10:13 PM, Robert Haas wrote: > >> It is very difficult to believe that this is a good idea: > >> > >> --- a/src/backend/replication/libpqwalreceiver/li

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-22 Thread Amit Kapila
On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund wrote: > > Hi, > > On 2016-03-15 10:47:12 +0530, Amit Kapila wrote: > > @@ -248,12 +256,67 @@ set_status_by_pages(int nsubxids, TransactionId *subxids, > > * Record the final state of transaction entries in the commit log fo

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-22 Thread Amit Kapila
On Tue, Mar 22, 2016 at 6:29 PM, Andres Freund wrote: > > On 2016-03-22 18:19:48 +0530, Amit Kapila wrote: > > > I'm actually rather unconvinced that it's all that common that all > > > subtransactions are on one page. If you have concurrency - otherwise >

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-22 Thread Amit Kapila
On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund wrote: > > Hi, > > On 2016-03-15 10:47:12 +0530, Amit Kapila wrote: > > @@ -248,12 +256,67 @@ set_status_by_pages(int nsubxids, TransactionId *subxids, > > * Record the final state of transaction entries in the commit log fo

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Wed, Mar 23, 2016 at 12:26 PM, Amit Kapila wrote: > > On Tue, Mar 22, 2016 at 4:22 PM, Andres Freund wrote: > > > > > > I think it's worthwhile to create a benchmark that does something like > > BEGIN;SELECT ... FOR UPDATE; SELECT pg_sleep(random_time); &g

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Tue, Mar 22, 2016 at 12:33 PM, Dilip Kumar wrote: > > > On Thu, Mar 17, 2016 at 11:39 AM, Amit Kapila wrote: > > I have reviewed the patch.. here are some review comments, I will continue to review.. > > 1. > > + > + /* > + * Add the proc to list, if the

Re: [HACKERS] Relation extension scalability

2016-03-23 Thread Amit Kapila
ail that we should introduce a new API to do a more targeted search for such cases. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
mpleLruReadPage_ReadOnly) > > > My suspicion is that a better approach for now would be to take Simon's > patch, but add a (per-page?) 'ClogModificationLock'; to avoid the need > of doing something fancier in TransactionIdSetStatusBit(). > I think we can try that as well and if you see better results by that Approach, then we can use that instead of current patch. [1] - http://www.postgresql.org/message-id/cad__oujrdwqdjdovhahqldg-6ivu6ibci9ij1qpu6atuqpl...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-23 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund wrote: > > > > Even after changing to scale 500, the performance benefits on this, > > older 2 socket, machine were minor; even though contention on the > > ClogC

Re: [HACKERS] Relation extension scalability

2016-03-24 Thread Amit Kapila
On Thu, Mar 24, 2016 at 1:48 PM, Petr Jelinek wrote: > > On 24/03/16 07:04, Dilip Kumar wrote: >> >> >> On Thu, Mar 24, 2016 at 10:44 AM, Robert Haas > <mailto:robertmh...@gmail.com>> wrote: >> >> On Wed, Mar 23, 2016 at 9:43 PM, Amit Kapi

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-24 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund wrote: > > > > Have you, in your evaluation of the performance of this patch, done > > profiles over time? I.e. whether the performance benefits are the > >

Re: [HACKERS] Performance degradation in commit 6150a1b0

2016-03-24 Thread Amit Kapila
http://www.postgresql.org/message-id/capphfdsrot1jmsnrnccqpnzeu9vut7tx6b-n1wyouwwfhd6...@mail.gmail.com With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-25 Thread Amit Kapila
On Thu, Mar 24, 2016 at 8:08 AM, Amit Kapila wrote: > > On Thu, Mar 24, 2016 at 5:40 AM, Andres Freund wrote: > > > > Even after changing to scale 500, the performance benefits on this, > > older 2 socket, machine were minor; even though contention on the > > ClogC

Re: [HACKERS] Relation extension scalability

2016-03-27 Thread Amit Kapila
we reach a node with enough free space (as we must, since the root has enough space). So shouldn't it be able to find the new FSM page where the bulk extend rolls over? With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Speed up Clog Access by increasing CLOG buffers

2016-03-28 Thread Amit Kapila
On Fri, Sep 11, 2015 at 8:01 PM, Amit Kapila wrote: > > On Thu, Sep 3, 2015 at 5:11 PM, Andres Freund wrote: > > > > Updated comments and the patch (increate_clog_bufs_v2.patch) > containing the same is attached. > Andres mentioned to me in off-list discussion, that h

Re: [HACKERS] Relation extension scalability

2016-03-28 Thread Amit Kapila
gt; > Yes, that makes sense. One more point is that if the reason for v13 giving better performance is extra blocks (which we believe in certain cases can leak till the time Vacuum updates the FSM tree), do you think it makes sense to once test by increasing lockWaiters * 20 limit to may

Re: [HACKERS] OOM in libpq and infinite loop with getCopyStart()

2016-03-29 Thread Amit Kapila
I find that more or less we are at same situation as we were back in October. Let me know if you think anything more from myside can help in moving patch. [1] - http://www.postgresql.org/message-id/cab7npqtnrw8lr1hd7zlnojjc1bp1evw_emadgorv+s-sbcr...@mail.gmail.com [2] - http://www.postgresq

Re: [HACKERS] Updated backup APIs for non-exclusive backups

2016-03-30 Thread Amit Kapila
If yes, I think that will make this patch more invasive with respect to handling of failure modes. Also as David points out, I also feel that it will raise the bar for usage of this API. With Regards, Amit Kapila. EnterpriseDB: http://www.enterprisedb.com

  1   2   3   4   5   6   7   8   9   10   >