Re: [GENERAL] Improving performance of merging data between tables

2015-02-10 Thread Pawel Veselov
Sorry, it took me a while to respond, but I re-factored all of this process to suggestions. On Wed, Jan 7, 2015 at 7:49 PM, Maxim Boguk wrote: > > On Wed, Jan 7, 2015 at 8:49 PM, Pawel Veselov > wrote: >> >> PS: your setup look pretty complicated and hard to analyze without seeing >>> all involv

Re: [GENERAL] Improving performance of merging data between tables

2015-01-07 Thread Maxim Boguk
On Wed, Jan 7, 2015 at 8:49 PM, Pawel Veselov wrote: > > PPPS: and the last suggestion, after you finished with the "write all the >> data into its own tables", then application should perform analyze of these >> own tables (or you could have weird/inefficient plans during last stage). > > > Any

Re: [GENERAL] Improving performance of merging data between tables

2015-01-07 Thread Pawel Veselov
On Tue, Dec 30, 2014 at 7:25 PM, Maxim Boguk wrote: > > On Wed, Dec 31, 2014 at 11:10 AM, Pawel Veselov > wrote >> >> >> [skipped] >> >> 2) try pg_stat_statements, setting "pg_stat_statements.track = all". see: http://www.postgresql.org/docs/9.4/static/pgstatstatements.html I have

Re: [GENERAL] Improving performance of merging data between tables

2014-12-30 Thread Maxim Boguk
On Wed, Dec 31, 2014 at 11:10 AM, Pawel Veselov wrote > > > [skipped] > > 2) try pg_stat_statements, setting "pg_stat_statements.track = all". see: >>> http://www.postgresql.org/docs/9.4/static/pgstatstatements.html >>> >>> I have used this to profile some functions, and it worked pretty well. >>

Re: [GENERAL] Improving performance of merging data between tables

2014-12-30 Thread Pawel Veselov
On Mon, Dec 29, 2014 at 9:29 PM, Pawel Veselov wrote: [skipped] >>> 1) How do I find out what exactly is consuming the CPU in a PL/pgSQL >>> function? All I see is that the calls to merge_all() function take long >>> time, and the CPU is high while this is going on. >>> >>> [skipped] 2) try pg

Re: [GENERAL] Improving performance of merging data between tables

2014-12-30 Thread Andy Colson
On 12/29/2014 11:29 PM, Pawel Veselov wrote: Andy, thanks for looking into this. On Mon, Dec 29, 2014 at 9:00 AM, Andy Colson mailto:a...@squeakycode.net>> wrote: On 12/28/2014 3:49 PM, Pawel Veselov wrote: Hi. I was wondering if anybody would have any ideas on how to im

Re: [GENERAL] Improving performance of merging data between tables

2014-12-29 Thread Pawel Veselov
Andy, thanks for looking into this. On Mon, Dec 29, 2014 at 9:00 AM, Andy Colson wrote: > On 12/28/2014 3:49 PM, Pawel Veselov wrote: > >> Hi. >> >> I was wondering if anybody would have any ideas on how to improve >> certain operations that we are having. >> >> > >> >> Besides "can somebody

Re: [GENERAL] Improving performance of merging data between tables

2014-12-29 Thread Andy Colson
On 12/28/2014 3:49 PM, Pawel Veselov wrote: Hi. I was wondering if anybody would have any ideas on how to improve certain operations that we are having. Besides "can somebody please look at this and let me know if I'm doing something utterly stupid", here are my questions. 1) How do I find

[GENERAL] Improving performance of merging data between tables

2014-12-28 Thread Pawel Veselov
Hi. I was wondering if anybody would have any ideas on how to improve certain operations that we are having. PostgreSQL 9.3.3. Table layout: main_table: PK, N key columns, M data columns. The key columns are bound in a unique key together. PK is pure sequence number. There are few separate non-u