Re: Performance of INSERT into temporary tables using psqlODBC driver

2018-09-13 Thread Tim Cross
padusuma writes: > Hello Tim, > > I have tried the suggestions provided to the best of my knowledge, but I did > not see any improvement in the INSERT performance for temporary tables. The > Linux host on which PostgreSQL database is installed has 32 GB RAM. > Following are current settings I h

How Do You Associate a Query With its Invoking Procedure?

2018-09-13 Thread Fd Habash
In API function may invoke 10 queries. Ideally, I would like to know what queries are invoked by it and how long each took. I’m using pg_stat_statement. I can see the API function statement, but how do I deterministically identify all queries invoked by it? Thank you

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Andres Freund
Hi, On 2018-09-13 14:12:02 -0400, Tom Lane wrote: > > This is an Aurora cluster running on r4.2xlarge (8 vCPU, 61g). > > Don't know much about Aurora, but I wonder whether you paid for > guaranteed (provisioned) IOPS, and if so what service level. Given that aurora uses direct-io and has the sto

RE: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Fd Habash
Just checked metrics while the count was running … Read latency < 3.5 ms Write latency < 4 ms Read throughput ~ 40 MB/sec with sporadic peaks at 100 Read IOPS ~ 5000 QDepth < 3 Thank you From: Tom Lane Sent: Thursday, September 13, 2018 2:12 PM To: Fd Habash Cc: pgsql-performan

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Tom Lane
Fd Habash writes: > Based on my research in the forums and Google , it is described in multiple > places that ‘select count(*)’ is expected to be slow in Postgres because of > the MVCC controls imposed upon the query leading a table scan. Also, the > elapsed time increase linearly with table si

Re: Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Justin Pryzby
On Thu, Sep 13, 2018 at 01:33:54PM -0400, Fd Habash wrote: > Is this ET expected? If not, what could be slowing it down? I’m currently > running explain analyze and I’ll share the final output when done. explain(analyze,BUFFERS) is what's probably interesting You're getting an index-only-scan,

Select count(*) on a 2B Rows Tables Takes ~20 Hours

2018-09-13 Thread Fd Habash
Based on my research in the forums and Google , it is described in multiple places that ‘select count(*)’ is expected to be slow in Postgres because of the MVCC controls imposed upon the query leading a table scan. Also, the elapsed time increase linearly with table size. However, I do not kno

Re: Performance of INSERT into temporary tables using psqlODBC driver

2018-09-13 Thread padusuma
Hello Tim, I have tried the suggestions provided to the best of my knowledge, but I did not see any improvement in the INSERT performance for temporary tables. The Linux host on which PostgreSQL database is installed has 32 GB RAM. Following are current settings I have in postgresql.conf file: sha