Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Jeff Janes
On Fri, Mar 6, 2015 at 5:38 PM, Tom Lane wrote: > Kevin Grittner writes: > > Tomas Vondra wrote: > >> How would fccebe421 explain the large amount of random writes (~4MB/s > >> for more than an hour), reported in the initial post? And why would that > >> only affect the EXPLAIN and not the bare

Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Tom Lane
Kevin Grittner writes: > Tomas Vondra wrote: >> How would fccebe421 explain the large amount of random writes (~4MB/s >> for more than an hour), reported in the initial post? And why would that >> only affect the EXPLAIN and not the bare query? >> But the random writes don't really match in this

Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Kevin Grittner
Tomas Vondra wrote: > On 6.3.2015 01:44, Tom Lane wrote: >> Tomas Vondra writes: >>> On 5.3.2015 16:01, Gunnlaugur Thor Briem wrote: - postgres version is 9.1.13 >>> >>> The only thing I can think of is some sort of memory exhaustion, >>> resulting in swapping out large amounts of memory. >>

Re: [PERFORM] PG 9.3 materialized view VS Views, indexes, shared memory

2015-03-06 Thread Nicolas Paris
According to this link http://postgresql.nabble.com/NULL-saves-disk-space-td4344106.html NULL values do not take place if only one other column are null for that row. Boolean takes 1 byte wheras smallint 2bytes. Then the space problem is not anymore a problem with boolean columns 95% empty One thi

Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Tom Lane
Tomas Vondra writes: > On 6.3.2015 01:44, Tom Lane wrote: >> I'm wondering about the issue addressed by commit fccebe421 ("Use >> SnapshotDirty rather than an active snapshot to probe index >> endpoints"). > How would fccebe421 explain the large amount of random writes (~4MB/s > for more than an

Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Jeff Janes
On Thu, Mar 5, 2015 at 4:44 PM, Tom Lane wrote: > Tomas Vondra writes: > > On 5.3.2015 16:01, Gunnlaugur Thor Briem wrote: > >> - postgres version is 9.1.13 > > > The only thing I can think of is some sort of memory exhaustion, > > resulting in swapping out large amounts of memory. > > I'm wonde

Re: [PERFORM] EXPLAIN (no ANALYZE) taking an hour for INSERT FROM SELECT

2015-03-06 Thread Tomas Vondra
On 6.3.2015 01:44, Tom Lane wrote: > Tomas Vondra writes: >> On 5.3.2015 16:01, Gunnlaugur Thor Briem wrote: >>> - postgres version is 9.1.13 > >> The only thing I can think of is some sort of memory exhaustion, >> resulting in swapping out large amounts of memory. > > I'm wondering about the is

Re: [PERFORM] slow server : s_lock and _bt_checkkeys on perf top

2015-03-06 Thread Soni M
Thanks All for the response, finally we figure it out. The slow is due to high number of dead rows on main table, repack these tables wipe out the issue. On Mar 6, 2015 9:31 PM, "Merlin Moncure" wrote: > On Wed, Mar 4, 2015 at 1:31 PM, Soni M wrote: > > Hello All, > > > > Master db size 1.5 TB >

Re: [PERFORM] slow server : s_lock and _bt_checkkeys on perf top

2015-03-06 Thread Merlin Moncure
On Wed, Mar 4, 2015 at 1:31 PM, Soni M wrote: > Hello All, > > Master db size 1.5 TB > All postgres 9.1.13 installed from RHEL package. > It has streaming replica and slony replica to another servers. > > Server performance is slower than usual, before that, there's a big query > got cancelled and

Re: [PERFORM] PG 9.3 materialized view VS Views, indexes, shared memory

2015-03-06 Thread Nicolas Paris
Thanks Jim, My suggestion is to test using nothing but plain views and plain indexes on > the base table Actualy the way I choose subset rows for views is complicated in terms of query. Then using simple views without partial indexes is terrible in terms of performance (I have tested that). You

Re: [PERFORM] PG 9.3 materialized view VS Views, indexes, shared memory

2015-03-06 Thread Jim Nasby
On 3/6/15 2:16 AM, Nicolas Paris wrote: If you have that many different views I doubt you want that many indexes anyway. ​It's a datawarehouse, then each view is used by many user for each query. Those views must be subset of the huge material table. All indexes are needed Yes, but the

Re: [PERFORM] PG 9.3 materialized view VS Views, indexes, shared memory

2015-03-06 Thread Nicolas Paris
> > If you have that many different views I doubt you want that many indexes > anyway. ​It's a datawarehouse, then each view is used by many user for each query. Those views must be subset of the huge material table. All indexes are needed ​ > ​Have you tried just hitting the base table and inde