Andrei Lepikhov писал(а) 2023-12-08 07:37:
On 28/11/2023 01:37, Alexander Korotkov wrote:
On Mon, Nov 27, 2023 at 8:07 PM Andres Freund <and...@anarazel.de>
wrote:
Sorry for the late answer, I missed this thread because of vacation.
On 2023-11-27 11:29:48 +0530, Ashutosh Bapat wrote:
How do we ensure that we are not making unnecessary copies of
Bitmapsets?
We don't - but that's not specific to this patch. Bitmapsets
typically aren't
very large, I doubt that it's a significant proportion of the memory
usage. Adding refcounts or such would likely add more overhead than
it'd save,
both in time and memory.
I'd already clashed with Tom on copying the required_relids field and
voluntarily made unnecessary copies in the project [1].
And ... stuck into huge memory consumption. The reason was in
Bitmapsets:
When we have 1E3-1E4 partitions and try to reparameterize a join, one
bitmapset field can have a size of about 1kB. Having bitmapset
referencing Relation with a large index value, we had a lot of (for
example, 1E4 * 1kB) copies on each reparametrization of such a field.
Alexander Pyhalov should remember that case.
Yes. If it matters, this happened during reparametrization when 2
partitioned tables with 1000 partitions each were joined. Then
asymmetric pw join managed to eat lots of memory for bitmapsets (by
lots of memory I mean all available on the test VM).
[1] Asymmetric partition-wise JOIN
https://www.postgresql.org/message-id/flat/CAOP8fzaVL_2SCJayLL9kj5pCA46PJOXXjuei6-3aFUV45j4LJQ%40mail.gmail.com
--
Best regards,
Alexander Pyhalov,
Postgres Professional
