Re: Consecutive Query Executions with Increasing Execution Time

2019-12-19 Thread Shijia Wei
Hi everyone! Thanks a ton for this brilliant discussion here! It turned out that Nicolas was correct! I found that the CPU was broken and not spinning at all. With consecutive parallel query execution, the CPU temperature hits 100C almost immediately after 1 or 2 iterations. So the processor start

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-18 Thread Tom Lane
Laurenz Albe writes: > On Tue, 2019-12-17 at 11:11 -0500, Jeff Janes wrote: >> If it is doing a seq scan (I don't know if it is) they intentionally use a >> small ring buffer to, so they evict their own recently used blocks, rather >> than evicting other people's blocks. So these blocks won't bui

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-18 Thread Laurenz Albe
On Tue, 2019-12-17 at 11:11 -0500, Jeff Janes wrote: > On Tue, Dec 17, 2019 at 8:08 AM Laurenz Albe wrote: > > On Mon, 2019-12-16 at 15:50 -0500, Tom Lane wrote: > > > Peter Geoghegan writes: > > > > Why do the first and the twentieth executions of the query have almost > > > > identical "buffers

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-17 Thread Jeff Janes
On Tue, Dec 17, 2019 at 8:08 AM Laurenz Albe wrote: > On Mon, 2019-12-16 at 15:50 -0500, Tom Lane wrote: > > Peter Geoghegan writes: > > > Why do the first and the twentieth executions of the query have almost > > > identical "buffers shared/read" numbers? That seems odd. > > > > It's repeat exe

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-17 Thread Laurenz Albe
On Mon, 2019-12-16 at 15:50 -0500, Tom Lane wrote: > Peter Geoghegan writes: > > Why do the first and the twentieth executions of the query have almost > > identical "buffers shared/read" numbers? That seems odd. > > It's repeat execution of the same query, so that doesn't seem odd to me. Really

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Andres Freund
Hi, On 2019-12-16 17:48:16 -0500, Tom Lane wrote: > Hmm, that's an interesting thought. The OP did say the CPU type, > but according to Intel's spec page for it [1] the difference between > base and turbo frequency is only 4.0 vs 4.2 GHz, which doesn't seem > like enough to explain the results ..

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Sam Gendler
On Mon, Dec 16, 2019 at 2:48 PM Tom Lane wrote: > unless you suppose it actually > throttled to below base freq, which surely shouldn't happen that fast. > Might be worth watching the CPU frequency while doing the test though. > Wouldn't expect to see such linear progression if that were the cas

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Tom Lane
Nicolas Charles writes: > Could it be that your CPUs is warming and throttling? You didn't mention the > platform used, so I'm not sure whether it's a server or a laptop Hmm, that's an interesting thought. The OP did say the CPU type, but according to Intel's spec page for it [1] the difference

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Nicolas Charles
Could it be that your CPUs is warming and throttling? You didn't mention the platform used, so I'm not sure whether it's a server or a laptop Nicolas Le 16 décembre 2019 21:50:17 GMT+01:00, Tom Lane a écrit : >Peter Geoghegan writes: >> Why do the first and the twentieth executions of the q

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Tom Lane
Peter Geoghegan writes: > Why do the first and the twentieth executions of the query have almost > identical "buffers shared/read" numbers? That seems odd. It's repeat execution of the same query, so that doesn't seem odd to me. This last set of numbers suggests that there's some issue with the

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Peter Geoghegan
On Mon, Dec 16, 2019 at 9:28 AM Shijia Wei wrote: > 1st Query: > Buffers: shared hit=17074 read=16388 > 20th Query: > Buffers: shared hit=17037 read=16390 Why do the first and the twentieth executions of the query have almost identical "buffers shared/

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Shijia Wei
Hi Laurenz, Each time the identical query executes, the total number of rows selected stays the same. The table is actually not modified between/during runs. The query plan stays the same between fast and slow runs. Please find two copied here: The first one is the output of the first query in th

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Tom Lane
Shijia Wei writes: > I am running TPC-H on recent postgresql (12.0 and 12.1). > On some of the queries (that may involve parallel scans) I see this > interesting behavior: > When these queries are executed back-to-back (sent from psql interactive > terminal), the total execution time of them incre

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Laurenz Albe
On Sun, 2019-12-15 at 23:59 -0600, Shijia Wei wrote: > I am running TPC-H on recent postgresql (12.0 and 12.1). > On some of the queries (that may involve parallel scans) I see this > interesting behavior: > When these queries are executed back-to-back (sent from psql interactive > terminal), the

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Olivier Gautherot
Hi Shijia, If you're using fish, I suspect you're on a Mac - I don't have experience on this platform. Can you check with pgAdmin (3 or 4) what the server is busy doing after a few iterations? Check for locks, as it could be a cause. Also, do you have concurrent INSERTs? Olivier On Mon, Dec 16,

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Shijia Wei
Hi Olivier, I do not think that the queries are executed concurrently. The bash for loop ensures that the next command fires only after the first returns. Also for some 'complex' queries, even a wait-period that is longer than the total execution time does not completely avoid this effect. For exa

Re: Consecutive Query Executions with Increasing Execution Time

2019-12-16 Thread Olivier Gautherot
Hi Shijia, It sounds like concurrency on the queries: the second starts before the first ends, and so on. With a short wait in between you ensure sequential execution. Notice that you also have the overhead of concurrent psql... Sounds normal to me. Best regards Olivier On Mon, Dec 16, 2019, 0