SELECT creates millions of temp files in a single directory

2022-04-23 Thread Peter
In modern versions of postgres a simple SELECT writes a couple of millions of individual temp files into a single directory under pgsql_tmp. I know of no filesystem that would take such lightly, and even ZFS gets some problems with such extremely long directories. What is the rationale in this beh

PG14: "is of" vs pg_typeof

2022-04-23 Thread Karsten Hilbert
PostgreSQL 14.2-1 Dear readers, it used to be possible to say (inside plpgsql): if _value is of (text, char, varchar, name) then val_type := ''string''; elsif _value is of (smallint, integer, bigint, numeric, boolean) then val_type := ''numeric'';

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread David G. Johnston
On Sat, Apr 23, 2022 at 1:00 PM Peter wrote: > In modern versions of postgres a simple SELECT writes a couple > of millions of individual temp files into a single directory under > pgsql_tmp. > I know of no filesystem that would take such lightly, and even > ZFS gets some problems with such extre

Re: PG14: "is of" vs pg_typeof

2022-04-23 Thread Karsten Hilbert
Am Sat, Apr 23, 2022 at 10:14:03PM +0200 schrieb Karsten Hilbert: > I can't find anything in the changelog saying that "is of" > was removed. For what it's worth, nothing in the docs ever > said it existed either (though it did, as per real life). Oh, wait, https://www.postgresql.org/mes

Re: PG14: "is of" vs pg_typeof

2022-04-23 Thread Adrian Klaver
On 4/23/22 13:14, Karsten Hilbert wrote: PostgreSQL 14.2-1 Dear readers, I can't find anything in the changelog saying that "is of" was removed. For what it's worth, nothing in the docs ever said it existed either (though it did, as per real life). It works through version 13. I also am no

Re: PG14: "is of" vs pg_typeof

2022-04-23 Thread Adrian Klaver
On 4/23/22 13:36, Karsten Hilbert wrote: Am Sat, Apr 23, 2022 at 10:14:03PM +0200 schrieb Karsten Hilbert: I can't find anything in the changelog saying that "is of" was removed. For what it's worth, nothing in the docs ever said it existed either (though it did, as per real life). Oh, wait,

Re: PG14: "is of" vs pg_typeof

2022-04-23 Thread Adrian Klaver
On 4/23/22 13:14, Karsten Hilbert wrote: PostgreSQL 14.2-1 Dear readers, Now, pg_typeof is an alternative but that only goes so far: it requires laboriously constructing an array on the right hand side for the above use case, along the lines of: select pg_typeof('a'::text) = any(ARRA

Re: PG14: "is of" vs pg_typeof

2022-04-23 Thread Karsten Hilbert
Am Sat, Apr 23, 2022 at 01:43:52PM -0700 schrieb Adrian Klaver: > > select pg_typeof('a'::text) = any(ARRAY[pg_typeof(''::text), > > pg_typeof(''::name)]); > > > >Is there anything obvious I am missing for easily > >resurrecting the above "is of" use ? > > Actually it can be done as: > > sele

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Tom Lane
"David G. Johnston" writes: > v12 what? That ... > It would help if you can provide a self-contained demonstration > that others can then verify and debug (or explain). ... and that. As this message stands, it's undocumented whining. Please see https://wiki.postgresql.org/wiki/Slow_Query_Ques

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread David G. Johnston
On Sat, Apr 23, 2022 at 1:58 PM Tom Lane wrote: > "David G. Johnston" writes: > > v12 what? > > That ... > > > It would help if you can provide a self-contained demonstration > > that others can then verify and debug (or explain). > > ... and that. As this message stands, it's undocumented whin

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Adrian Klaver
On 4/23/22 12:50, Peter wrote: People seem to have been brainwashed by Web-Services and OLTP, and now think the working set must always fit in memory. But this is only one possible usecase, it is not the exclusive only one. This is no-win situation as most of the complaints in rec

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Tom Lane
"David G. Johnston" writes: > I'll add that given the nature of the problem that changing temp_file_limit > away from its default of unlimited may be useful. > https://www.postgresql.org/docs/current/runtime-config-resource.html#RUNTIME-CONFIG-RESOURCE-DISK Maybe ... that limits the total space i

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Peter
On Sat, Apr 23, 2022 at 02:11:00PM -0700, Adrian Klaver wrote: ! On 4/23/22 12:50, Peter wrote: ! ! ! > People seem to have been brainwashed by Web-Services and OLTP, ! > and now think the working set must always fit in memory. But this ! > is only one possible usecase, it is not the

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Adrian Klaver
On 4/23/22 14:58, Peter wrote: On Sat, Apr 23, 2022 at 02:11:00PM -0700, Adrian Klaver wrote: ! On 4/23/22 12:50, Peter wrote: ! ! ! > People seem to have been brainwashed by Web-Services and OLTP, ! > and now think the working set must always fit in memory. But this ! > is only one p

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Peter
On Sat, Apr 23, 2022 at 05:13:41PM -0400, Tom Lane wrote: ! "David G. Johnston" writes: ! > I'll add that given the nature of the problem that changing temp_file_limit ! > away from its default of unlimited may be useful. ! > https://www.postgresql.org/docs/current/runtime-config-resource.html#RU

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Adrian Klaver
On 4/23/22 15:40, Peter wrote: On Sat, Apr 23, 2022 at 05:13:41PM -0400, Tom Lane wrote: And btw, I probably lost-in-translation the relevant info about the running version: Name : postgresql12-server Version: 12.10 Installed on : Mon Apr 4 04:13:18 2022 CEST Origin

Re: SELECT creates millions of temp files in a single directory

2022-04-23 Thread Thomas Munro
On Sun, Apr 24, 2022 at 8:00 AM Peter wrote: > More than a million files in a single directory, this is > inacceptable. You didn't show EXPLAIN (ANALYZE) but if [Parallel] Hash is making insane numbers of temporary files then something is not working as intended... and I can take a guess at what