Re: Benchmarking: How to identify bottleneck (limiting factor) and achieve "linear scalability"?

2019-01-24 Thread Saurabh Nanda
Is there any material on how to benchmark Postgres meaningfully? I'm getting very frustrated with the numbers that `pgbench` is reporting: -- allocating more resources to Postgres seems to be randomly dropping performance -- there seems to be no repeatability in the benchmarking numbers [1] -- the

Re: Zero throughput on a query on a very large table.

2019-01-24 Thread David Rowley
On Fri, 25 Jan 2019 at 19:24, Tom Lane wrote: > PS: On the third hand, you mention having created new indexes on this > table with apparently not a lot of pain, which is a tad surprising > if you don't have the patience to wait for a sort to finish. How > long did those index builds take? It wou

Re: Zero throughput on a query on a very large table.

2019-01-24 Thread Tom Lane
"l...@laurent-hasson.com" writes: > Query: > select * from tmp_outpatient_rev order by desy_sort_key, claim_no > Plan: > [ seqscan-and-sort ... parallelized, but still seqscan-and-sort ] > - I have tried to hack the planner to force an index scan (which would > avoid the sort/gather st

Re: Zero throughput on a query on a very large table.

2019-01-24 Thread Andreas Kretschmer
Am 25.01.19 um 06:20 schrieb l...@laurent-hasson.com: Hello, We have been stuck for the past week on a query that simply won’t “execute”. We have a table with 1.2B rows that took around 14h to load, but a simple select takes forever and after 10h, no records are coming through still. En

Zero throughput on a query on a very large table.

2019-01-24 Thread l...@laurent-hasson.com
Hello, We have been stuck for the past week on a query that simply won’t “execute”. We have a table with 1.2B rows that took around 14h to load, but a simple select takes forever and after 10h, no records are coming through still. Environment: - Table tmp_outpatient_rev with 41 VARCHA

Re: SELECT performance drop

2019-01-24 Thread Jan Nielsen
With Pascal's suggestion, I added a new index: CREATE INDEX fm_order_sid_type_idx ON fm_order (session_id, type); which improved the query to 2mS! https://explain.depesz.com/s/oxvs Thank you, Pascal! On Thu, Jan 24, 2019 at 9:52 AM Jan Nielsen wrote: > > > On Wed, Jan 23, 2019 at 12:37

Re: SELECT performance drop

2019-01-24 Thread Jan Nielsen
On Wed, Jan 23, 2019 at 12:37 PM legrand legrand < legrand_legr...@hotmail.com> wrote: > Hi, > is there an index on > fm_order(session_id,type)? > There isn't at the moment: table_name | index_name | column_name +--+

Re: ERROR: found xmin from before relfrozenxid

2019-01-24 Thread Mariel Cherkassky
I'm checking the full version. As you said I saw that in 9.6.9 there was a fix for the next bug : Avoid spuriously marking pages as all-visible (Dan Wood, Pavan Deolasee, Álvaro Herrera) This could happen if some tuples were locked (but not deleted). While queries would still function correctly,

Re: Parallel stats in execution plans

2019-01-24 Thread David Rowley
On Thu, 17 Jan 2019 at 00:31, David Conlin wrote: > How the time values combine with parallelism. For example, each execution of > the sort node takes an average of 48.5s, over three loops. This makes a total > running time of 145.5s. Even if this was perfectly distributed between the > two wor

Re: Parallel stats in execution plans

2019-01-24 Thread David Conlin
It seems like no-one has any ideas on this - does anyone know anywhere else I can try to look/ask to find out more? Is it possible that this is a bug? Thanks Dave On 16/01/2019 11:31, David Conlin wrote: Hi folks - I'm having trouble understanding what some of the stats mean in the execut