Re: pb with big volumes

2023-08-13 Thread David Rowley
On Mon, 14 Aug 2023 at 11:14, Marc Millas wrote: > that's exactly my question. > does the analyze buffers data, generated when track_io_timing is on, keep > track of multiple reloads of the same data while executing one operation ? Yes, the timing for reads will include the time it took to fetc

Re: pb with big volumes

2023-08-13 Thread Marc Millas
Hi David, that's exactly my question. does the analyze buffers data, generated when track_io_timing is on, keep track of multiple reloads of the same data while executing one operation ? I ll do the test asap and report the results. Marc MILLAS Senior Architect +33607850334 www.mokadb.com On

Re: pb with big volumes

2023-08-11 Thread Ron
On 8/10/23 23:40, David Rowley wrote: On Fri, 11 Aug 2023 at 13:54, Ron wrote: Wouldn't IO contention make for additive timings instead of exponential? No, not necessarily. Imagine one query running that's doing a parameterised nested loop join resulting in the index on the inner side being de

Re: pb with big volumes

2023-08-11 Thread Marc Millas
No if there is a pumping effect ie. Loading data in the cache flushed by the other request and again and again. I have had this on an Oracle db years ago. Iostat 100% for days. Managed by partitionning and locking some data in the cache Le ven. 11 août 2023 à 03:54, Ron a écrit : > Wouldn't IO

Re: pb with big volumes

2023-08-10 Thread David Rowley
On Fri, 11 Aug 2023 at 13:54, Ron wrote: > Wouldn't IO contention make for additive timings instead of exponential? No, not necessarily. Imagine one query running that's doing a parameterised nested loop join resulting in the index on the inner side being descended several, say, million times. L

Re: pb with big volumes

2023-08-10 Thread Ron
Wouldn't IO contention make for additive timings instead of exponential? On 8/10/23 20:41, Adam Scott wrote: I think your concern is that 20 min + 30 min does not equal 3 hours. It might be natural to think the contention would, at max, be 50 min x 2 (1 hr 40 min). So what's going on? It se

Re: pb with big volumes

2023-08-10 Thread Adam Scott
I think your concern is that 20 min + 30 min does not equal 3 hours. It might be natural to think the contention would, at max, be 50 min x 2 (1 hr 40 min). So what's going on? It seems disk I/O is a primary suspect since you hint for an iostat replacement inside of Postgres. If it is due to d

Re: pb with big volumes

2023-08-10 Thread Ron
On 8/10/23 16:36, Marc Millas wrote: Hi, I have a 15 TB db on postgres 14 (soon 15). shared buffers is 32 GB. Does the system have 128GB AM? It's a db with max 15 users and often less, and currently 1 or 2. the biggest table have 133 partitions of 150M to 200M+ rows each. lots of request ac

pb with big volumes

2023-08-10 Thread Marc Millas
Hi, I have a 15 TB db on postgres 14 (soon 15). shared buffers is 32 GB. It's a db with max 15 users and often less, and currently 1 or 2. the biggest table have 133 partitions of 150M to 200M+ rows each. lots of request access explicitly one of those. When I, alone, run a query "reading" 15M bu