Re: [GENERAL] Query Using Massive Temp Space

2017-11-21 Thread Tom Lane
Thomas Munro writes: > On Wed, Nov 22, 2017 at 7:04 AM, Tom Lane wrote: >> Now, there's definitely something busted here; it should not have gone as >> far as 2 million batches before giving up on splitting. > I had been meaning to discuss this. We only give up when we reach the > point when a

Re: [GENERAL] Query Using Massive Temp Space

2017-11-21 Thread Thomas Munro
On Wed, Nov 22, 2017 at 7:04 AM, Tom Lane wrote: > Now, there's definitely something busted here; it should not have gone as > far as 2 million batches before giving up on splitting. I had been meaning to discuss this. We only give up when we reach the point when a batch is entirely entirely kep

Re: [GENERAL] Query Using Massive Temp Space

2017-11-21 Thread Tom Lane
Cory Tucker writes: > On Mon, Nov 20, 2017 at 9:36 AM Tom Lane wrote: >> The only thing I can think of offhand that could create temp files far in >> excess of the actual data volume is if a hash join repeatedly decides that >> it needs to increase the number of hash batches. We have seen that h

Re: [GENERAL] Query Using Massive Temp Space

2017-11-20 Thread Cory Tucker
On Mon, Nov 20, 2017 at 9:36 AM Tom Lane wrote: > 9.6.what exactly? > 9.6.5 > > The only thing I can think of offhand that could create temp files far in > excess of the actual data volume is if a hash join repeatedly decides that > it needs to increase the number of hash batches. We have see

Re: [GENERAL] Query Using Massive Temp Space

2017-11-20 Thread Tom Lane
Cory Tucker writes: > I have a query that is using a tremendous amount of temp disk space given > the overall size of the dataset. I'd love for someone to try to explain > what PG is doing and why its using so much space for the query. > First off, the system is PG 9.6 on Ubuntu with 4 cores and