Re: [PERFORM] PostgreSQL 9.6 wals management

2017-11-20 Thread Michael Paquier
On Mon, Nov 20, 2017 at 6:02 PM, Mariel Cherkassky wrote: > This morning , I set the wal_keep_segments to 100 and I set the > archive_timeout to 6 minutes. Now, after setting those settings and starting > the cluster wals switch is working fine and I didnt see that many wals were > However, doesnt

Re: [PERFORM] Low priority batch insert

2017-10-19 Thread Michael Paquier
On Fri, Oct 20, 2017 at 1:10 AM, Jean Baro wrote: > That's my first question in this mailing list! :) Welcome! > Is it possible (node.js connecting to PG 9.6 on RDS) to set a lower priority > to a connection so that that particular process (BATCH INSERT) would have a > low impact on other runnin

Re: [PERFORM] Unlogged tables

2017-08-09 Thread Michael Paquier
On Wed, Aug 9, 2017 at 5:20 AM, l...@laurent-hasson.com wrote: > We have a fairly large static dataset that we load into Postgres. We made > the tables UNLOGGED and saw a pretty significant performance improvement for > the loading. This was all fantastic until the server crashed and we were > sur

Re: [PERFORM] Unable to start the slave instance

2017-07-05 Thread Michael Paquier
On Thu, Jul 6, 2017 at 2:02 PM, Daulat Ram wrote: > We are using different releases of windows. Is this issue reported due to > different versions of windows releases. > Master server : Windows 7 Professional > Slave server : Windows 10 Professional Please do not top-post. That may be a problem

Re: [PERFORM] [BUGS] Invalid WAL segment size. Allowed values are 1,2,4,8,16,32,64

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 10:39 PM, Cocco Gianfranco wrote: > ./configure --with-wal-segsize=1024 > > checking for WAL segment size... configure: error: Invalid WAL segment size. > Allowed values are 1,2,4,8,16,32,64. > > Please, how can I do? When trying to compile Postgres 9.6, the maximum value

Re: [PERFORM] [BUGS] Invalid WAL segment size. Allowed values are 1,2,4,8,16,32,64

2017-06-13 Thread Michael Paquier
On Tue, Jun 13, 2017 at 6:10 PM, Cocco Gianfranco wrote: > If I understand it well, we can build a new postgreSQL server, by setting > this value into "configure" file? > > --with-wal-segsize=SEGSIZE becomes --with-wal-segsize=1024 ?? Yes, but as Jeff has pointed out upthread, this value can

Re: [PERFORM] [BUGS] Invalid WAL segment size. Allowed values are 1,2,4,8,16,32,64

2017-06-12 Thread Michael Paquier
On Tue, Jun 13, 2017 at 2:27 AM, Jeff Janes wrote: > But there is something preventing it. wal_segsize cannot exceed 64MB in > 9.2. v10 will be the first version which will allow sizes above 64MB. Yes, indeed. I have misread --with-segsize and --with-wal-segsize in the docs. Sorry for the confus

Re: [PERFORM] [BUGS] Invalid WAL segment size. Allowed values are 1,2,4,8,16,32,64

2017-06-09 Thread Michael Paquier
On Fri, Jun 9, 2017 at 10:55 PM, Cocco Gianfranco wrote: > Is there a way to fix “wal_segsize” to about 1 Gb in 9.2. version, and > “rebuild” postgreSQL server? As long as you are able to compile your own version of Postgres and your distribution does not allow that, there is nothing preventing

Re: [PERFORM] Slow query after 9.3 to 9.6 migration

2017-01-06 Thread Michael Paquier
On Fri, Jan 6, 2017 at 6:14 AM, Filipe Oliveira wrote: > Can you remove me from your mailing list? There is an unsubscribe action here: https://www.postgresql.org/community/lists/subscribe/ -- Michael -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make change

Re: [PERFORM] Invalid page header in block 25561983 of relation pg_tblspc

2016-12-27 Thread Michael Paquier
On Tue, Dec 27, 2016 at 6:24 PM, Dinesh Chandra 12108 wrote: > Can you please suggest me how to resolve it? I think its related to block > corruption. You may want to roll in a backup, and move to a different server: https://wiki.postgresql.org/wiki/Corruption > How can I find particular block i

Re: [PERFORM] Query hangs sometimes

2016-11-17 Thread Michael Paquier
On Thu, Nov 17, 2016 at 3:55 AM, Metatrader EA wrote: > How can I check this? Several options are listed in the docs: https://www.postgresql.org/docs/9.6/static/monitoring.html -- Michael -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your sub

Re: [PERFORM] pg_basebackup running slow

2016-10-14 Thread Michael Paquier
On Fri, Oct 14, 2016 at 10:21 PM, Samir Magar wrote: > What is the settings for max_wal_sender? > you can try increasing this parameter to improve backup performance. max_wal_senders has no influence on the performance of a base backup taken as a base backup is just sent through one single WAL se

Re: [PERFORM] array size exceeds the maximum allowed (1073741823) when building a json

2016-06-07 Thread Michael Paquier
On Tue, Jun 7, 2016 at 10:03 PM, Josh Berkus wrote: > On 06/07/2016 08:42 AM, Nicolas Paris wrote: >> You have to do something different. Using multiple columns and/or >> multiple rows might we workable. >> >> >> Certainly. Kind of disappointing, because I won't find any json builder >> a

Re: [PERFORM] PostgreSQL limitation

2015-11-02 Thread Michael Paquier
On Mon, Nov 2, 2015 at 7:52 PM, FattahRozzaq wrote: > If I install the PostgreSQL on Linux (Debian), > How much the limit of max_connections that PostgreSQL can take? > How much the limit of max_prepared_transactions that PostgreSQL can take? Per definition, those parameters have a max value of 2

Re: [PERFORM] dump restoration performance

2015-09-29 Thread Michael Paquier
On Fri, Sep 25, 2015 at 10:43 PM, rlemaroi wrote: > Please how long does it take approximately to restore a 300 Go database using > pg_restore ? Are there benchmarks for that ? That's not an exact science and this is really application-dependent. For example the more your schema has index entries

Re: [PERFORM] Yet another abort-early plan disaster on 9.3

2015-02-12 Thread Michael Paquier
On Wed, Dec 17, 2014 at 4:55 PM, Simon Riggs wrote: > On 12 December 2014 at 03:31, Simon Riggs wrote: > > > Also attached is a new parameter called enable_sortedpath which can be > > used to turn on/off the sorted path generated by the planner. > > Now with attachment. (Thanks Jeff!) > Moved t

Re: [PERFORM] trying to run pgbench-tools postgresql ubuntu ERROR: relation "branches" does not exist

2014-12-31 Thread Michael Paquier
On Wed, Dec 31, 2014 at 5:42 AM, cesar wrote: > After this it appears a message "Removing old pgbench tables" > First error message (seems not to be important) is: ERROR: table "accounts > does not exist" > ERROR: relation "branches" does not exist > LINE 1: select count(*) for branches > ERROR: A

Re: [PERFORM] Profiling PostgreSQL

2014-05-22 Thread Michael Paquier
On Thu, May 22, 2014 at 10:48 PM, Tom Lane wrote: > Call graph data usually isn't trustworthy unless you built the program > with -fno-omit-frame-pointer ... This page is full of ideas as well: https://wiki.postgresql.org/wiki/Profiling_with_perf -- Michael -- Sent via pgsql-performance mailin

Re: [PERFORM] pl/pgsql performance

2014-04-27 Thread Michael Paquier
On Fri, Apr 25, 2014 at 5:53 PM, Pavel Stehule wrote: > 2014-04-25 10:48 GMT+02:00 Mehdi Ravanbakhsh : >> 2- how i can chose max connection number for pgsql server based on cpu >> core and RAM capacity that have maximum Efficiency? > usually max performance is about 10 x CPU connections. But it hi

Re: [PERFORM] Sudden crazy high CPU usage

2014-04-02 Thread Michael Paquier
On Wed, Apr 2, 2014 at 7:16 AM, Andres Freund wrote: > On 2014-03-31 19:16:58 +0200, Niels Kristian Schjødt wrote: >> Yes, I could install "perf", though I'm not familiar with it. What would i >> do? :-) > > As root: > perf record -a sleep 5 > perf report > my-nice-perf-report.txt > > And then se

Re: [PERFORM] Slow Count-Distinct Query

2014-04-01 Thread Michael Paquier
On Wed, Apr 2, 2014 at 1:22 PM, Christopher Jackson wrote: > > Hi Bricklen, > > Thanks for the feedback. I'll play around with materialized views. My > understanding is they have to be manually triggered for refresh Yep. > and there's an exclusive lock on the view while the refresh is t

Re: [PERFORM] Can one Dump schema without index/constraints?

2014-02-14 Thread Michael Paquier
On Sat, Feb 15, 2014 at 6:06 AM, Tory M Blue wrote: > > > I'm working with slon and the index portion for at least 3 of my tables take > hours to complete and thus with this instance of slony being a wide area > replica, sessions time out and slon fails to complete. > > So I'm looking at dumping t

Re: [PERFORM] Postgresql in a Virtual Machine

2013-11-28 Thread Michael Paquier
On Fri, Nov 29, 2013 at 3:40 AM, Scott Marlowe wrote: > On Wed, Nov 27, 2013 at 7:58 PM, Dong Ye wrote: > >> As Heikki commented, VMware recently compared Postgres performance in >> an ESX (5.1) VM versus in a comparable native Linux. We saw 1. >> ESX-level locking causes no vertical scalability

Re: [PERFORM] Order By Clause, Slows Query Performance?

2013-11-12 Thread Michael Paquier
On Sun, Nov 10, 2013 at 4:40 PM, monalee_dba wrote: > Eg. SELECT col1, col2, col3,col10 FROM table1; > > For above query If I didn't mention ORDER BY clause, then I want to know > selected data will appear in which order by a query planner? The data will be selected in the order at which it is

Re: [PERFORM] BitMap Heap Scan & BitMap Index Scan

2013-11-12 Thread Michael Paquier
http://www.postgresql.org/docs/9.3/static/using-explain.html On Sun, Nov 10, 2013 at 4:32 PM, monalee_dba wrote: > I would like to know, What is BitMap Heap Scan & BitMap Index Scan? When I > use EXPLAIN for query, which has LEFT JOIN with 4 different table then some > time query planner uses Bit

Re: [PERFORM] postgresql recommendation memory

2013-11-05 Thread Michael Paquier
On Tue, Nov 5, 2013 at 8:37 AM, Евгений Селявка wrote: > I set work_mem to 1/4 from available RAM. I have 32Gb RAM so i set > shared_buffers to 8Gb. I am sure you are mentioning shared_buffers here and not work_mem. work_mem is a per-operation parameter. So if you are using an operation involving

Re: [PERFORM] limit connections pgpool

2013-10-10 Thread Michael Paquier
On Fri, Oct 11, 2013 at 12:00 AM, Jeison Bedoya Delgado wrote: > Hi, i want to know what is the pgpool parameter for close the connections > directly in the database, because the pgpool II close fine the the childs > with the life time, but the connection in the database continue open in idle > st

Re: [PERFORM] RESTORE multiple DBs concurrently

2013-09-07 Thread Michael Paquier
On Sat, Sep 7, 2013 at 12:52 AM, Roberto Grandi wrote: > Is this possible from your point of view to restore on the same server more > than 1 DB at time? Yes, it is possible: simply run multiple instances of pg_restore in parallel and just don't blow up your disk(s) I/O. Also, why not restoring

Re: [PERFORM] ORDER BY, LIMIT and indexes

2013-08-05 Thread Michael Paquier
On Tue, Aug 6, 2013 at 8:25 AM, Claudio Freire wrote: > On Mon, Aug 5, 2013 at 8:04 PM, Ivan Voras wrote: > > SELECT * FROM table ORDER BY id DESC LIMIT 10 OFFSET 10 > > > > SELECT * FROM table WHERE active ORDER BY id DESC LIMIT 10 OFFSET 10 > > Did you try explain? > And did you run ANALYZE on

Re: [PERFORM] to many locks held

2013-07-30 Thread Michael Paquier
On Tue, Jul 30, 2013 at 11:48 PM, bricklen wrote: > On Tue, Jul 30, 2013 at 3:52 AM, Jeison Bedoya > wrote: > memory ram: 128 GB >> cores: 32 >> >> max_connections: 900 >> > > I would say you might be better off using a connection pooler if you need > this many connections. > Yeah that's a lot.

Re: [PERFORM] Process in state BIND, authentication, PARSE

2013-07-08 Thread Michael Paquier
On Tue, Jul 9, 2013 at 2:01 AM, Jeison Bedoya wrote: > max_connections = 900 > work_mem = 1024MB > maintenance_work_mem = 1024MB Aren't work_mem and maintenance_work_mem too high? You need to keep in mind that those are per-operation settings, so for example if you have 100 clients performing quer

Re: [PERFORM] Performance autovaccum

2013-07-08 Thread Michael Paquier
On Tue, Jul 9, 2013 at 1:14 AM, Jeison Bedoya wrote: > Hi, i have a postgresql 9.2.2, You should update to 9.2.4. There are major security fixes in this subrelease. > but i don´t use autovaccum but i want to > begin to use it. some recommendation about the optimal configuration? or > some link t

Re: [PERFORM] 9.2.2 - semop hanging

2013-06-18 Thread Michael Paquier
On Tue, Jun 11, 2013 at 9:48 PM, Rafael Domiciano wrote: > postgres=# select version(); >version > -- > PostgreSQL 9.2.2 on x86_64-unknow