Re: RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-29 Thread Ali .
Ok, thanks -- Sent from Mail.ru app for Android Wednesday, 24 November 2021, 11:28pm +03:00 from Godfrin, Philippe E philippe.godf...@nov.com : >The notion of COPY blocks and asynchronously is very interesting >  >From: Gavin Roy < gav...@aweber.com> >Sent: Wednesday, November 24, 2021 1:50 PM

Re: [EXTERNAL] Re: Inserts and bad performance

2021-11-28 Thread Gavin Flower
On 28/11/21 17:17, Godfrin, Philippe E wrote: Right you are sir! I figured that out a few hours ago! pg *From:* Ron *Sent:* Wednesday, November 24, 2021 10:58 PM *To:* pgsql-general@lists.postgresql.org *Subject:* [EXTERNAL] Re: Inserts and bad performance On 11/24/21 1:15 PM, Godfrin, Phili

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-27 Thread Godfrin, Philippe E
Right you are sir! I figured that out a few hours ago! pg From: Ron Sent: Wednesday, November 24, 2021 10:58 PM To: pgsql-general@lists.postgresql.org Subject: [EXTERNAL] Re: Inserts and bad performance On 11/24/21 1:15 PM, Godfrin, Philippe E wrote: [snip] I dropped the unique index , rebuilt

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-27 Thread Godfrin, Philippe E
Hi Gavin – thanks I hadn’t realized that about psychopg. I’m on the earlier version, so I can’t use what you recommended at this point. But I did use copy_expert. Interestingly enough the performance of the copy statement is only slightly better than the insert, as I was running inserts with 50

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-26 Thread Godfrin, Philippe E
xplain >(analyze, buffers, verbose) and then rollback? Yes, I'm looking into that pg -Original Message- From: David Rowley Sent: Wednesday, November 24, 2021 7:13 PM To: Godfrin, Philippe E Cc: Tom Lane ; pgsql-general@lists.postgresql.org Subject: Re: [EXTERNAL] Re: Inserts a

Re: [EXTERNAL] Re: Inserts and bad performance

2021-11-24 Thread David Rowley
On Thu, 25 Nov 2021 at 08:59, Godfrin, Philippe E wrote: > Hi Tom. Good point about the index paging out of the buffer. I did that and > no change. I do have the shared buffers at 40GB, so there’s a good bit there, > but I also did all those things on the page you referred, except for using > c

Re: [EXTERNAL] Re: Inserts and bad performance

2021-11-24 Thread Michael Lewis
How many partitions? How many rows do they have when performance is slowing considerably? Does this table get many updates or is it insert only? What version of PostgreSQL? Are the inserts randomly distributed among the partitions or targeting one or a few partitions? Are you able to capture an exa

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-24 Thread Godfrin, Philippe E
The notion of COPY blocks and asynchronously is very interesting From: Gavin Roy Sent: Wednesday, November 24, 2021 1:50 PM To: Godfrin, Philippe E Cc: pgsql-general@lists.postgresql.org Subject: [EXTERNAL] Re: Inserts and bad performance On Wed, Nov 24, 2021 at 2:15 PM Godfrin, Philippe E mai

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-24 Thread Godfrin, Philippe E
Hi Tom. Good point about the index paging out of the buffer. I did that and no change. I do have the shared buffers at 40GB, so there's a good bit there, but I also did all those things on the page you referred, except for using copy. At this point the data has not been scrubbed, so I'm trapping

RE: [EXTERNAL] Re: Inserts and bad performance

2021-11-24 Thread Godfrin, Philippe E
My apologies for the dearth of details. No on both the cpu and errors. But I do believe it is IO related. I just can't find it. I thought maybe it was index splitting so I altered the unique index with filterfactor=40 and reindexed. No change. I then dropped the unique index. No change. I thoug