Re: Removing "long int"-related limit on hash table sizes

2021-07-27 Thread Andres Freund
Hi, On 2021-07-26 11:38:41 +0900, Michael Paquier wrote: > On Sun, Jul 25, 2021 at 12:28:04PM -0400, Tom Lane wrote: > > Andres Freund writes: > >> We really ought to just remove every single use of long. > > > > I have no objection to that as a long-term goal. But I'm not volunteering > > to d

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Alvaro Herrera
On 2021-Jul-26, Tom Lane wrote: > ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > > We also have the (U)INT64CONST() macros, which are about about two > > thirds as common as the U?LL? suffixes. > > Yeah. Ideally we'd forbid direct use of the suffixes and insist > you go

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Tom Lane
ilm...@ilmari.org (Dagfinn Ilmari =?utf-8?Q?Manns=C3=A5ker?=) writes: > We also have the (U)INT64CONST() macros, which are about about two > thirds as common as the U?LL? suffixes. Yeah. Ideally we'd forbid direct use of the suffixes and insist you go through those macros, but I don't know of any

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Dagfinn Ilmari Mannsåker
Alvaro Herrera writes: > On 2021-Jul-25, Ranier Vilela wrote: > >> > BTW, one aspect of this that I'm unsure how to tackle is the >> > common usage of "L" constants; in particular, "work_mem * 1024L" >> > is a really common idiom that we'll need to get rid of. Not sure >> > that grep will be a u

Re: Removing "long int"-related limit on hash table sizes

2021-07-26 Thread Alvaro Herrera
On 2021-Jul-25, Ranier Vilela wrote: > > BTW, one aspect of this that I'm unsure how to tackle is the > > common usage of "L" constants; in particular, "work_mem * 1024L" > > is a really common idiom that we'll need to get rid of. Not sure > > that grep will be a useful aid for finding those. > >

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Michael Paquier
On Sun, Jul 25, 2021 at 12:28:04PM -0400, Tom Lane wrote: > Andres Freund writes: >> We really ought to just remove every single use of long. > > I have no objection to that as a long-term goal. But I'm not volunteering > to do all the work, and in any case it wouldn't be a back-patchable fix. >

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Ranier Vilela
Em dom., 25 de jul. de 2021 às 15:53, Tom Lane escreveu: > Ranier Vilela writes: > > I think int64 is in most cases the counterpart of *long* on Windows. > > I'm not particularly on board with s/long/int64/g as a universal > solution. Sure, not a universal solution, I mean a start point. When I

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Tom Lane
Ranier Vilela writes: > I think int64 is in most cases the counterpart of *long* on Windows. I'm not particularly on board with s/long/int64/g as a universal solution. I think that most of these usages are concerned with memory sizes and would be better off as "size_t". We might need int64 in p

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Ranier Vilela
Em dom., 25 de jul. de 2021 às 13:28, Tom Lane escreveu: > Andres Freund writes: > > On 2021-07-23 17:15:24 -0400, Tom Lane wrote: > >> That's because they spill to disk where they did not before. The easy > >> answer of "raise hash_mem_multiplier" doesn't help, because on Windows > >> the prod

Re: Removing "long int"-related limit on hash table sizes

2021-07-25 Thread Tom Lane
Andres Freund writes: > On 2021-07-23 17:15:24 -0400, Tom Lane wrote: >> That's because they spill to disk where they did not before. The easy >> answer of "raise hash_mem_multiplier" doesn't help, because on Windows >> the product of work_mem and hash_mem_multiplier is clamped to 2GB, >> thanks

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Bruce Momjian
On Sat, Jul 24, 2021 at 06:25:53PM -0700, Andres Freund wrote: > > That's because they spill to disk where they did not before. The easy > > answer of "raise hash_mem_multiplier" doesn't help, because on Windows > > the product of work_mem and hash_mem_multiplier is clamped to 2GB, > > thanks to t

Re: Removing "long int"-related limit on hash table sizes

2021-07-24 Thread Andres Freund
Hi, On 2021-07-23 17:15:24 -0400, Tom Lane wrote: > Per the discussion at [1], users on Windows are seeing nasty performance > losses in v13/v14 (compared to prior releases) for hash aggregations that > required somewhat more than 2GB in the prior releases. Ugh :(. > That's because they spill t