Hello, Robert
Thanks for your review. I believe I fixed items 1, 2 and 3 (see
attachment). Also I would like to clarify item 4.
> 4. It mixes together multiple ideas in a single patch, not only
> introducing a hashing concept but also striping a brand-new layer of
> abstraction across the resource-owner mechanism. I am not sure that
> layer of abstraction is a very good idea, but if it needs to be done,
> I think it should be a separate patch.
Do I right understand that you suggest following?
Current patch should be split in two parts. In first patch we create
and use ResourceArray with array-based implementation (abstraction
layer). Then we apply second patch which change ResourceArray
implementation to hashing based (optimization).
Best regards,
Aleksander
On Tue, 8 Dec 2015 17:30:33 -0500
Robert Haas <robertmh...@gmail.com> wrote:
> On Fri, Dec 4, 2015 at 7:15 AM, Aleksander Alekseev
> <a.aleks...@postgrespro.ru> wrote:
> > Current implementation of ResourceOwner uses arrays to store
> > resources like TupleDescs, Snapshots, etc. When we want to release
> > one of these resources ResourceOwner finds it with linear scan.
> > Granted, resource array are usually small so its not a problem most
> > of the time. But it appears to be a bottleneck when we are working
> > with tables which have a lot of partitions.
> >
> > To reproduce this issue:
> > 1. run `./gen.pl 10000 | psql my_database postgres`
> > 2. run `pgbench -j 8 -c 8 -f q.sql -T 100 my_database`
> > 3. in second terminal run `sudo perf top -u postgres`
> >
> > Both gen.pl and q.sql are attached to this message.
> >
> > You will see that postgres spends a lot of time in
> > ResourceOwnerForget* procedures:
> >
> > 32.80% postgres [.] list_nth
> > 20.29% postgres [.] ResourceOwnerForgetRelationRef
> > 12.87% postgres [.] find_all_inheritors
> > 7.90% postgres [.] get_tabstat_entry
> > 6.68% postgres [.] ResourceOwnerForgetTupleDesc
> > 1.17% postgres [.] hash_search_with_hash_value
> > ... < 1% ...
> >
> > I would like to suggest a patch (see attachment) witch fixes this
> > bottleneck. Also I discovered that there is a lot of code
> > duplication in ResourceOwner. Patch fixes this too. The idea is
> > quite simple. I just replaced arrays with something that could be
> > considered hash tables, but with some specific optimizations.
> >
> > After applying this patch we can see that bottleneck is gone:
> >
> > 42.89% postgres [.] list_nth
> > 18.30% postgres [.] find_all_inheritors
> > 10.97% postgres [.] get_tabstat_entry
> > 1.82% postgres [.] hash_search_with_hash_value
> > 1.21% postgres [.] SearchCatCache
> > ... < 1% ...
> >
> > For tables with thousands partitions we gain in average 32.5% more
> > TPS.
> >
> > As far as I can see in the same time patch doesn't make anything
> > worse. `make check` passes with asserts enabled and disabled. There
> > is no performance degradation according to both standard pgbench
> > benchmark and benchmark described above for tables with 10 and 100
> > partitions.
>
> I do think it's interesting to try to speed up the ResourceOwner
> mechanism when there are many resources owned. We've had some forays
> in that direction in the past, and I think it's useful to continue
> that work. But I also think that this patch, while interesting, is
> not something we can seriously consider committing in its current
> form, because:
>
> 1. It lacks adequate comments and submission notes to explain the
> general theory of operation of the patch.
>
> 2. It seems to use uint8 * rather freely as a substitute for char * or
> void *, which doesn't seem like a great idea.
>
> 3. It doesn't follow PostgreSQL formatting conventions (uncuddled
> curly braces, space after if and before open paren, etc.).
>
> 4. It mixes together multiple ideas in a single patch, not only
> introducing a hashing concept but also striping a brand-new layer of
> abstraction across the resource-owner mechanism. I am not sure that
> layer of abstraction is a very good idea, but if it needs to be done,
> I think it should be a separate patch.
>
diff --git a/src/backend/utils/resowner/resowner.c b/src/backend/utils/resowner/resowner.c
index 0e7acbf..d2b37d5 100644
--- a/src/backend/utils/resowner/resowner.c
+++ b/src/backend/utils/resowner/resowner.c
@@ -29,6 +29,323 @@
#include "utils/snapmgr.h"
/*
+ * ResourceArray is a common structure for storing different types of resources.
+ *
+ * ResourceOwner can own `HeapTuple`s, `Relation`s, `Snapshot`s, etc. For
+ * each resource type there are procedures ResourceOwnerRemember* and
+ * ResourceOwnerForget*. There are also ResourceOwnerEnlarge* procedures
+ * which should be called before corresponding ResourceOwnerRemember* calls
+ * (see below). Internally each type of resource is stored in separate
+ * ResourceArray.
+ *
+ * There are two major reasons for using ResourceArray instead of, say,
+ * regular C arrays.
+ *
+ * Firstly we would like to prevent code duplication. For instance
+ * ResourceArray provides generic Remember/Forget/Enlarge procedures, so
+ * corresponding ResourceOwner* procedures are just a typesafe wrappers for
+ * these procedures. Note that different resources have different sizeof. Thus
+ * ResourceArray should know size of resource and store all data in char[]
+ * buffer.
+ *
+ * Secondly ResourceArray must be more efficient than regular C array.
+ * Previous implementation of ResourceOwner used arrays. It had O(1) complexity
+ * of Remember procedures and O(N) complexity of Forget procedures where N is a
+ * number of remembered resources. It turned out that such implementation
+ * creates a bottleneck in some cases, e.g. when working with partitioned
+ * tables which have a lot of (say, thousands) partitions. New implementation
+ * in general could be considered a hash table with some specific optimizations.
+ * It has O(1) complexity of both Remember and Forget procedures and
+ * apparently doesn't cause any performance degradation compared to previous
+ * implementation.
+ */
+typedef struct ResourceArray
+{
+ char *itemsarr; /* buffer for storing values */
+ uint32 itemsizelg:2; /* sizeof one item log 2 */
+ uint32 capacity:30; /* capacity of array */
+ uint32 nitems; /* how many items is stored in items array */
+ uint32 maxitems; /* precalculated RESARRAY_MAX_ITEMS(capacity) */
+} ResourceArray;
+
+/*
+ * This number is used as initial size of resource array. If given number of
+ * items is not enough, we double array size and reallocate memory.
+ *
+ * Should be power of two since we use (arrsize -1) as mask for hash value.
+ *
+ */
+#define RESARRAY_INIT_SIZE 16
+
+/*
+ * How many items could be stored in a resource array of given capacity. If
+ * this number is reached we need to resize an array to prevent hash collisions.
+ *
+ * This computation actually costs only two additions and one binary shift.
+ */
+#define RESARRAY_MAX_ITEMS(capacity) ((capacity)*3/4)
+
+/*
+ * Such type of callback function is called when resource stored in
+ * ResourceArray is released using ResourceArrayFree. Callback should
+ * _actually_ release a resource so nitems value will be decremented.
+ */
+typedef void (*ResourceArrayRemoveCallback) (const void *ref, bool isCommit);
+
+/* Used as argument to memcmp to determine if ResourceArray[i] is free. */
+static const char RESOURCE_ARRAY_ZERO_ELEMENT[sizeof(void *)] =
+{
+ 0
+};
+
+/*
+ * Calculate hash_any of given data. For uint32 values use faster hash_uint32.
+ */
+static Datum
+ResourceArrayHash(const void *data, int size)
+{
+ uint32 tmp;
+
+ Assert(size == sizeof(uint32) || size == sizeof(void *));
+
+ if (size == sizeof(uint32))
+ {
+ tmp = *((const uint32 *) data);
+ return hash_uint32(tmp);
+ }
+ else
+ {
+ return hash_any(data, size);
+ }
+}
+
+/*
+ * Initialize ResourceArray
+ */
+static void
+ResourceArrayInit(ResourceArray * resarr, uint32 itemsize)
+{
+ Assert(itemsize == sizeof(int) || itemsize == sizeof(void *));
+ Assert(resarr->itemsarr == NULL);
+ Assert(resarr->capacity == 0);
+ Assert(resarr->nitems == 0);
+ Assert(resarr->maxitems == 0);
+
+ resarr->itemsizelg = 0;
+ while (itemsize > 1)
+ {
+ resarr->itemsizelg++;
+ itemsize >>= 1;
+ }
+
+ Assert(resarr->itemsizelg == 2 || resarr->itemsizelg == 3);
+}
+
+/*
+ * Add a resource to ResourceArray
+ *
+ * Caller must have previously done ResourceArrayEnlarge()
+ */
+static void
+ResourceArrayAdd(ResourceArray * resarr, const void *dataref)
+{
+ char *itemptr;
+ Datum idx;
+ Datum mask = resarr->capacity - 1;
+ uint32 itemsize = 1 << resarr->itemsizelg;
+
+ Assert(memcmp(dataref, RESOURCE_ARRAY_ZERO_ELEMENT, itemsize) != 0);
+ Assert(resarr->maxitems > resarr->nitems);
+ Assert(resarr->capacity > 0);
+ Assert(resarr->itemsarr != NULL);
+
+ /*
+ * Hashing is quite expensive, so we use it only for large arrays. For
+ * small arrays we just use a linear scan.
+ */
+ if (resarr->capacity == RESARRAY_INIT_SIZE)
+ idx = resarr->nitems;
+ else
+ idx = ResourceArrayHash(dataref, itemsize);
+ idx &= mask;
+
+ while (true)
+ {
+ /*
+ * Read next line as:
+ *
+ * itemptr = &(itemsarr[idx])
+ *
+ * We use binary shift since compiler doesn't know that itemsize is
+ * always power of two. It would use multiplication instead of
+ * efficient binary shift in code `idx * itemsize`.
+ */
+ itemptr = resarr->itemsarr + (idx << resarr->itemsizelg);
+ if (memcmp(itemptr, RESOURCE_ARRAY_ZERO_ELEMENT, itemsize) == 0)
+ break;
+ idx = (idx + 1) & mask;
+ }
+
+ memcpy(itemptr, dataref, itemsize);
+ resarr->nitems++;
+}
+
+/*
+ * Remove a resource from ResourceArray
+ *
+ * Returns true on success, false if resource was not found
+ */
+static bool
+ResourceArrayRemove(ResourceArray * resarr, const void *dataref)
+{
+ char *itemptr;
+ Datum idx;
+ uint32 i;
+ Datum mask = resarr->capacity - 1;
+ uint32 itemsize = 1 << resarr->itemsizelg;
+
+ Assert(memcmp(dataref, RESOURCE_ARRAY_ZERO_ELEMENT, itemsize) != 0);
+ Assert(resarr->capacity > 0);
+ Assert(resarr->itemsarr != NULL);
+
+ /*
+ * Hashing is quite expensive, so we use it only for large arrays. For
+ * small arrays we use a linear scan.
+ */
+ if (resarr->capacity == RESARRAY_INIT_SIZE)
+ idx = 0;
+ else
+ idx = ResourceArrayHash(dataref, itemsize);
+ idx &= mask;
+
+ for (i = 0; i < resarr->capacity; i++)
+ {
+ /*
+ * Read next line as:
+ *
+ * itemptr = &(itemsarr[idx])
+ *
+ * We use binary shift since compiler doesn't know that itemsize is
+ * always power of two. It would use multiplication instead of
+ * efficient binary shift in code `idx * itemsize`.
+ */
+ itemptr = resarr->itemsarr + (idx << resarr->itemsizelg);
+ if (memcmp(itemptr, dataref, itemsize) == 0)
+ {
+ memset(itemptr, 0, itemsize);
+ resarr->nitems--;
+ return true;
+ }
+ idx = (idx + 1) & mask;
+ }
+
+ return false;
+}
+
+/*
+ * Make sure there is a room for at least one more resource in an array.
+ *
+ * This is separate from actually inserting a resource because if we run out
+ * of memory, it's critical to do so *before* acquiring the resource.
+ */
+static void
+ResourceArrayEnlarge(ResourceArray * resarr)
+{
+ uint32 oldcap;
+ char *olditemsarr;
+ char *olditemptr;
+ uint32 itemsize = 1 << resarr->itemsizelg;
+
+ Assert(resarr->itemsizelg != 0);
+
+ if (resarr->nitems < resarr->maxitems)
+ return; /* nothing to do */
+
+ olditemsarr = resarr->itemsarr;
+ oldcap = resarr->capacity;
+
+ resarr->capacity = oldcap > 0 ? oldcap * 2 : RESARRAY_INIT_SIZE;
+ resarr->itemsarr = (char *)
+ MemoryContextAllocZero(TopMemoryContext,
+ resarr->capacity * itemsize);
+ resarr->maxitems = RESARRAY_MAX_ITEMS(resarr->capacity);
+ resarr->nitems = 0;
+
+ if (olditemsarr != NULL)
+ {
+ while (oldcap > 0)
+ {
+ oldcap--;
+
+ /*
+ * Read next line as:
+ *
+ * olditemptr = &(olditemsarr[oldcap])
+ *
+ * We use binary shift since compiler doesn't know that itemsize
+ * is always power of two. It would use multiplication instead of
+ * efficient binary shift in code `oldcap * itemsize`.
+ */
+ olditemptr = olditemsarr + (oldcap << resarr->itemsizelg);
+ if (memcmp(olditemptr, RESOURCE_ARRAY_ZERO_ELEMENT, itemsize) != 0)
+ ResourceArrayAdd(resarr, olditemptr);
+ }
+ pfree(olditemsarr);
+ }
+}
+
+/*
+ * Remove all resources in array and call a callback function for each release.
+ */
+static void
+ResourceArrayRemoveAll(ResourceArray * resarr,
+ ResourceArrayRemoveCallback releasecb,
+ bool isCommit)
+{
+ uint32 idx = 0;
+ char *itemptr;
+ uint32 itemsize = 1 << resarr->itemsizelg;
+
+ while (resarr->nitems > 0)
+ {
+ /*
+ * Read next line as:
+ *
+ * itemptr = &(itemsarr[idx])
+ *
+ * We use binary shift since compiler doesn't know that itemsize is
+ * always power of two. It would use multiplication instead of
+ * efficient binary shift in code `idx * itemsize`.
+ */
+ itemptr = resarr->itemsarr + (idx << resarr->itemsizelg);
+ if (memcmp(itemptr, RESOURCE_ARRAY_ZERO_ELEMENT, itemsize) == 0)
+ {
+ idx++;
+ continue;
+ }
+ releasecb(itemptr, isCommit);
+ }
+}
+
+/*
+ * Return ResourceArray to initial state
+ */
+static void
+ResourceArrayFree(ResourceArray * resarr)
+{
+ Assert(resarr->nitems == 0);
+
+ resarr->capacity = 0;
+ resarr->maxitems = 0;
+
+ if (!resarr->itemsarr)
+ return;
+
+ pfree(resarr->itemsarr);
+ resarr->itemsarr = NULL;
+}
+
+/*
* To speed up bulk releasing or reassigning locks from a resource owner to
* its parent, each resource owner has a small cache of locks it owns. The
* lock manager has the same information in its local lock hash table, and
@@ -56,53 +373,22 @@ typedef struct ResourceOwnerData
ResourceOwner nextchild; /* next child of same parent */
const char *name; /* name (just for debugging) */
- /* We have built-in support for remembering owned buffers */
- int nbuffers; /* number of owned buffer pins */
- Buffer *buffers; /* dynamically allocated array */
- int maxbuffers; /* currently allocated array size */
-
/* We can remember up to MAX_RESOWNER_LOCKS references to local locks. */
int nlocks; /* number of owned locks */
LOCALLOCK *locks[MAX_RESOWNER_LOCKS]; /* list of owned locks */
- /* We have built-in support for remembering catcache references */
- int ncatrefs; /* number of owned catcache pins */
- HeapTuple *catrefs; /* dynamically allocated array */
- int maxcatrefs; /* currently allocated array size */
-
- int ncatlistrefs; /* number of owned catcache-list pins */
- CatCList **catlistrefs; /* dynamically allocated array */
- int maxcatlistrefs; /* currently allocated array size */
-
- /* We have built-in support for remembering relcache references */
- int nrelrefs; /* number of owned relcache pins */
- Relation *relrefs; /* dynamically allocated array */
- int maxrelrefs; /* currently allocated array size */
-
- /* We have built-in support for remembering plancache references */
- int nplanrefs; /* number of owned plancache pins */
- CachedPlan **planrefs; /* dynamically allocated array */
- int maxplanrefs; /* currently allocated array size */
-
- /* We have built-in support for remembering tupdesc references */
- int ntupdescs; /* number of owned tupdesc references */
- TupleDesc *tupdescs; /* dynamically allocated array */
- int maxtupdescs; /* currently allocated array size */
-
- /* We have built-in support for remembering snapshot references */
- int nsnapshots; /* number of owned snapshot references */
- Snapshot *snapshots; /* dynamically allocated array */
- int maxsnapshots; /* currently allocated array size */
-
- /* We have built-in support for remembering open temporary files */
- int nfiles; /* number of owned temporary files */
- File *files; /* dynamically allocated array */
- int maxfiles; /* currently allocated array size */
-
- /* We have built-in support for remembering dynamic shmem segments */
- int ndsms; /* number of owned shmem segments */
- dsm_segment **dsms; /* dynamically allocated array */
- int maxdsms; /* currently allocated array size */
+ /* We have built-in support for remembering: */
+
+ ResourceArray catrefarr; /* `HeapTuple`s */
+ ResourceArray catlistrefarr; /* `ResourceOwner`s */
+ ResourceArray relrefarr; /* `Relation`s */
+ ResourceArray planrefarr; /* `CachedPlan*`s */
+ ResourceArray tupdescarr; /* `TupleDesc`s */
+ ResourceArray snapshotarr; /* `Snapshot`s */
+ ResourceArray dsmarr; /* `dsm_segment*`s */
+ ResourceArray bufferarr; /* `Buffer`s */
+ ResourceArray filearr; /* `File`s */
+
} ResourceOwnerData;
@@ -168,6 +454,16 @@ ResourceOwnerCreate(ResourceOwner parent, const char *name)
parent->firstchild = owner;
}
+ ResourceArrayInit(&(owner->catrefarr), sizeof(HeapTuple));
+ ResourceArrayInit(&(owner->catlistrefarr), sizeof(ResourceOwner));
+ ResourceArrayInit(&(owner->relrefarr), sizeof(Relation));
+ ResourceArrayInit(&(owner->planrefarr), sizeof(CachedPlan *));
+ ResourceArrayInit(&(owner->tupdescarr), sizeof(TupleDesc));
+ ResourceArrayInit(&(owner->snapshotarr), sizeof(Snapshot));
+ ResourceArrayInit(&(owner->dsmarr), sizeof(dsm_segment *));
+ ResourceArrayInit(&(owner->bufferarr), sizeof(Buffer));
+ ResourceArrayInit(&(owner->filearr), sizeof(File));
+
return owner;
}
@@ -221,6 +517,96 @@ ResourceOwnerRelease(ResourceOwner owner,
}
static void
+ReleaseCachedPlanCallback(const void *dataref, bool isCommit)
+{
+ CachedPlan *res = *((CachedPlan **) dataref);
+
+ if (isCommit)
+ PrintPlanCacheLeakWarning(res);
+ ReleaseCachedPlan(res, true);
+}
+
+static void
+ReleaseDSMCallback(const void *dataref, bool isCommit)
+{
+ dsm_segment *res = *((dsm_segment **) dataref);
+
+ if (isCommit)
+ PrintDSMLeakWarning(res);
+ dsm_detach(res);
+}
+
+static void
+ReleaseTupleDescCallback(const void *dataref, bool isCommit)
+{
+ TupleDesc res = *((TupleDesc *) dataref);
+
+ if (isCommit)
+ PrintTupleDescLeakWarning(res);
+ DecrTupleDescRefCount(res);
+}
+
+static void
+ReleaseSnapshotCallback(const void *dataref, bool isCommit)
+{
+ Snapshot res = *((Snapshot *) dataref);
+
+ if (isCommit)
+ PrintSnapshotLeakWarning(res);
+ UnregisterSnapshot(res);
+}
+
+static void
+ReleaseRelationCallback(const void *dataref, bool isCommit)
+{
+ Relation res = *((Relation *) dataref);
+
+ if (isCommit)
+ PrintRelCacheLeakWarning(res);
+ RelationClose(res);
+}
+
+static void
+ReleaseCatCacheListCallback(const void *dataref, bool isCommit)
+{
+ CatCList *res = *((CatCList **) dataref);
+
+ if (isCommit)
+ PrintCatCacheListLeakWarning(res);
+ ReleaseCatCacheList(res);
+}
+
+static void
+ReleaseCatCacheCallback(const void *dataref, bool isCommit)
+{
+ HeapTuple res = *((HeapTuple *) dataref);
+
+ if (isCommit)
+ PrintCatCacheLeakWarning(res);
+ ReleaseCatCache(res);
+}
+
+static void
+ReleaseBufferCallback(const void *dataref, bool isCommit)
+{
+ Buffer res = *((Buffer *) dataref);
+
+ if (isCommit)
+ PrintBufferLeakWarning(res);
+ ReleaseBuffer(res);
+}
+
+static void
+ReleaseFileCallback(const void *dataref, bool isCommit)
+{
+ File res = *((File *) dataref);
+
+ if (isCommit)
+ PrintFileLeakWarning(res);
+ FileClose(res);
+}
+
+static void
ResourceOwnerReleaseInternal(ResourceOwner owner,
ResourceReleasePhase phase,
bool isCommit,
@@ -252,46 +638,17 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
* During a commit, there shouldn't be any remaining pins --- that
* would indicate failure to clean up the executor correctly --- so
* issue warnings. In the abort case, just clean up quietly.
- *
- * We are careful to do the releasing back-to-front, so as to avoid
- * O(N^2) behavior in ResourceOwnerForgetBuffer().
*/
- while (owner->nbuffers > 0)
- {
- if (isCommit)
- PrintBufferLeakWarning(owner->buffers[owner->nbuffers - 1]);
- ReleaseBuffer(owner->buffers[owner->nbuffers - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->bufferarr),
+ ReleaseBufferCallback, isCommit);
- /*
- * Release relcache references. Note that RelationClose will remove
- * the relref entry from my list, so I just have to iterate till there
- * are none.
- *
- * As with buffer pins, warn if any are left at commit time, and
- * release back-to-front for speed.
- */
- while (owner->nrelrefs > 0)
- {
- if (isCommit)
- PrintRelCacheLeakWarning(owner->relrefs[owner->nrelrefs - 1]);
- RelationClose(owner->relrefs[owner->nrelrefs - 1]);
- }
+ /* Ditto for relcache references. */
+ ResourceArrayRemoveAll(&(owner->relrefarr),
+ ReleaseRelationCallback, isCommit);
- /*
- * Release dynamic shared memory segments. Note that dsm_detach()
- * will remove the segment from my list, so I just have to iterate
- * until there are none.
- *
- * As in the preceding cases, warn if there are leftover at commit
- * time.
- */
- while (owner->ndsms > 0)
- {
- if (isCommit)
- PrintDSMLeakWarning(owner->dsms[owner->ndsms - 1]);
- dsm_detach(owner->dsms[owner->ndsms - 1]);
- }
+ /* Ditto for dynamic shared memory segments */
+ ResourceArrayRemoveAll(&(owner->dsmarr),
+ ReleaseDSMCallback, isCommit);
}
else if (phase == RESOURCE_RELEASE_LOCKS)
{
@@ -351,48 +708,28 @@ ResourceOwnerReleaseInternal(ResourceOwner owner,
* As with buffer pins, warn if any are left at commit time, and
* release back-to-front for speed.
*/
- while (owner->ncatrefs > 0)
- {
- if (isCommit)
- PrintCatCacheLeakWarning(owner->catrefs[owner->ncatrefs - 1]);
- ReleaseCatCache(owner->catrefs[owner->ncatrefs - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->catrefarr),
+ ReleaseCatCacheCallback, isCommit);
+
/* Ditto for catcache lists */
- while (owner->ncatlistrefs > 0)
- {
- if (isCommit)
- PrintCatCacheListLeakWarning(owner->catlistrefs[owner->ncatlistrefs - 1]);
- ReleaseCatCacheList(owner->catlistrefs[owner->ncatlistrefs - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->catlistrefarr),
+ ReleaseCatCacheListCallback, isCommit);
+
/* Ditto for plancache references */
- while (owner->nplanrefs > 0)
- {
- if (isCommit)
- PrintPlanCacheLeakWarning(owner->planrefs[owner->nplanrefs - 1]);
- ReleaseCachedPlan(owner->planrefs[owner->nplanrefs - 1], true);
- }
+ ResourceArrayRemoveAll(&(owner->planrefarr),
+ ReleaseCachedPlanCallback, isCommit);
+
/* Ditto for tupdesc references */
- while (owner->ntupdescs > 0)
- {
- if (isCommit)
- PrintTupleDescLeakWarning(owner->tupdescs[owner->ntupdescs - 1]);
- DecrTupleDescRefCount(owner->tupdescs[owner->ntupdescs - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->tupdescarr),
+ ReleaseTupleDescCallback, isCommit);
+
/* Ditto for snapshot references */
- while (owner->nsnapshots > 0)
- {
- if (isCommit)
- PrintSnapshotLeakWarning(owner->snapshots[owner->nsnapshots - 1]);
- UnregisterSnapshot(owner->snapshots[owner->nsnapshots - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->snapshotarr),
+ ReleaseSnapshotCallback, isCommit);
/* Ditto for temporary files */
- while (owner->nfiles > 0)
- {
- if (isCommit)
- PrintFileLeakWarning(owner->files[owner->nfiles - 1]);
- FileClose(owner->files[owner->nfiles - 1]);
- }
+ ResourceArrayRemoveAll(&(owner->filearr),
+ ReleaseFileCallback, isCommit);
/* Clean up index scans too */
ReleaseResources_hash();
@@ -418,16 +755,7 @@ ResourceOwnerDelete(ResourceOwner owner)
Assert(owner != CurrentResourceOwner);
/* And it better not own any resources, either */
- Assert(owner->nbuffers == 0);
Assert(owner->nlocks == 0 || owner->nlocks == MAX_RESOWNER_LOCKS + 1);
- Assert(owner->ncatrefs == 0);
- Assert(owner->ncatlistrefs == 0);
- Assert(owner->nrelrefs == 0);
- Assert(owner->ndsms == 0);
- Assert(owner->nplanrefs == 0);
- Assert(owner->ntupdescs == 0);
- Assert(owner->nsnapshots == 0);
- Assert(owner->nfiles == 0);
/*
* Delete children. The recursive call will delink the child from me, so
@@ -444,25 +772,15 @@ ResourceOwnerDelete(ResourceOwner owner)
ResourceOwnerNewParent(owner, NULL);
/* And free the object. */
- if (owner->buffers)
- pfree(owner->buffers);
- if (owner->catrefs)
- pfree(owner->catrefs);
- if (owner->catlistrefs)
- pfree(owner->catlistrefs);
- if (owner->relrefs)
- pfree(owner->relrefs);
- if (owner->planrefs)
- pfree(owner->planrefs);
- if (owner->tupdescs)
- pfree(owner->tupdescs);
- if (owner->snapshots)
- pfree(owner->snapshots);
- if (owner->files)
- pfree(owner->files);
- if (owner->dsms)
- pfree(owner->dsms);
-
+ ResourceArrayFree(&(owner->catrefarr));
+ ResourceArrayFree(&(owner->catlistrefarr));
+ ResourceArrayFree(&(owner->relrefarr));
+ ResourceArrayFree(&(owner->planrefarr));
+ ResourceArrayFree(&(owner->tupdescarr));
+ ResourceArrayFree(&(owner->snapshotarr));
+ ResourceArrayFree(&(owner->dsmarr));
+ ResourceArrayFree(&(owner->bufferarr));
+ ResourceArrayFree(&(owner->filearr));
pfree(owner);
}
@@ -575,26 +893,9 @@ UnregisterResourceReleaseCallback(ResourceReleaseCallback callback, void *arg)
void
ResourceOwnerEnlargeBuffers(ResourceOwner owner)
{
- int newmax;
-
- if (owner == NULL ||
- owner->nbuffers < owner->maxbuffers)
- return; /* nothing to do */
-
- if (owner->buffers == NULL)
- {
- newmax = 16;
- owner->buffers = (Buffer *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(Buffer));
- owner->maxbuffers = newmax;
- }
- else
- {
- newmax = owner->maxbuffers * 2;
- owner->buffers = (Buffer *)
- repalloc(owner->buffers, newmax * sizeof(Buffer));
- owner->maxbuffers = newmax;
- }
+ if (owner == NULL)
+ return;
+ ResourceArrayEnlarge(&(owner->bufferarr));
}
/*
@@ -608,12 +909,9 @@ ResourceOwnerEnlargeBuffers(ResourceOwner owner)
void
ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer)
{
- if (owner != NULL)
- {
- Assert(owner->nbuffers < owner->maxbuffers);
- owner->buffers[owner->nbuffers] = buffer;
- owner->nbuffers++;
- }
+ if (owner == NULL)
+ return;
+ ResourceArrayAdd(&(owner->bufferarr), (const void *) &buffer);
}
/*
@@ -625,33 +923,15 @@ ResourceOwnerRememberBuffer(ResourceOwner owner, Buffer buffer)
void
ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer)
{
- if (owner != NULL)
- {
- Buffer *buffers = owner->buffers;
- int nb1 = owner->nbuffers - 1;
- int i;
+ bool res;
- /*
- * Scan back-to-front because it's more likely we are releasing a
- * recently pinned buffer. This isn't always the case of course, but
- * it's the way to bet.
- */
- for (i = nb1; i >= 0; i--)
- {
- if (buffers[i] == buffer)
- {
- while (i < nb1)
- {
- buffers[i] = buffers[i + 1];
- i++;
- }
- owner->nbuffers = nb1;
- return;
- }
- }
+ if (owner == NULL)
+ return;
+
+ res = ResourceArrayRemove(&(owner->bufferarr), (const void *) &buffer);
+ if (!res)
elog(ERROR, "buffer %d is not owned by resource owner %s",
buffer, owner->name);
- }
}
/*
@@ -667,6 +947,8 @@ ResourceOwnerForgetBuffer(ResourceOwner owner, Buffer buffer)
void
ResourceOwnerRememberLock(ResourceOwner owner, LOCALLOCK *locallock)
{
+ Assert(locallock != NULL);
+
if (owner->nlocks > MAX_RESOWNER_LOCKS)
return; /* we have already overflowed */
@@ -714,25 +996,7 @@ ResourceOwnerForgetLock(ResourceOwner owner, LOCALLOCK *locallock)
void
ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->ncatrefs < owner->maxcatrefs)
- return; /* nothing to do */
-
- if (owner->catrefs == NULL)
- {
- newmax = 16;
- owner->catrefs = (HeapTuple *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(HeapTuple));
- owner->maxcatrefs = newmax;
- }
- else
- {
- newmax = owner->maxcatrefs * 2;
- owner->catrefs = (HeapTuple *)
- repalloc(owner->catrefs, newmax * sizeof(HeapTuple));
- owner->maxcatrefs = newmax;
- }
+ ResourceArrayEnlarge(&(owner->catrefarr));
}
/*
@@ -743,9 +1007,7 @@ ResourceOwnerEnlargeCatCacheRefs(ResourceOwner owner)
void
ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple)
{
- Assert(owner->ncatrefs < owner->maxcatrefs);
- owner->catrefs[owner->ncatrefs] = tuple;
- owner->ncatrefs++;
+ ResourceArrayAdd(&(owner->catrefarr), (const void *) &tuple);
}
/*
@@ -754,25 +1016,12 @@ ResourceOwnerRememberCatCacheRef(ResourceOwner owner, HeapTuple tuple)
void
ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple)
{
- HeapTuple *catrefs = owner->catrefs;
- int nc1 = owner->ncatrefs - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->catrefarr),
+ (const void *) &tuple);
- for (i = nc1; i >= 0; i--)
- {
- if (catrefs[i] == tuple)
- {
- while (i < nc1)
- {
- catrefs[i] = catrefs[i + 1];
- i++;
- }
- owner->ncatrefs = nc1;
- return;
- }
- }
- elog(ERROR, "catcache reference %p is not owned by resource owner %s",
- tuple, owner->name);
+ if (!res)
+ elog(ERROR, "catcache reference %p is not owned by resource owner %s",
+ tuple, owner->name);
}
/*
@@ -785,25 +1034,7 @@ ResourceOwnerForgetCatCacheRef(ResourceOwner owner, HeapTuple tuple)
void
ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->ncatlistrefs < owner->maxcatlistrefs)
- return; /* nothing to do */
-
- if (owner->catlistrefs == NULL)
- {
- newmax = 16;
- owner->catlistrefs = (CatCList **)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(CatCList *));
- owner->maxcatlistrefs = newmax;
- }
- else
- {
- newmax = owner->maxcatlistrefs * 2;
- owner->catlistrefs = (CatCList **)
- repalloc(owner->catlistrefs, newmax * sizeof(CatCList *));
- owner->maxcatlistrefs = newmax;
- }
+ ResourceArrayEnlarge(&(owner->catlistrefarr));
}
/*
@@ -814,9 +1045,7 @@ ResourceOwnerEnlargeCatCacheListRefs(ResourceOwner owner)
void
ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list)
{
- Assert(owner->ncatlistrefs < owner->maxcatlistrefs);
- owner->catlistrefs[owner->ncatlistrefs] = list;
- owner->ncatlistrefs++;
+ ResourceArrayAdd(&(owner->catlistrefarr), (const void *) &list);
}
/*
@@ -825,25 +1054,12 @@ ResourceOwnerRememberCatCacheListRef(ResourceOwner owner, CatCList *list)
void
ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list)
{
- CatCList **catlistrefs = owner->catlistrefs;
- int nc1 = owner->ncatlistrefs - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->catlistrefarr),
+ (const void *) &list);
- for (i = nc1; i >= 0; i--)
- {
- if (catlistrefs[i] == list)
- {
- while (i < nc1)
- {
- catlistrefs[i] = catlistrefs[i + 1];
- i++;
- }
- owner->ncatlistrefs = nc1;
- return;
- }
- }
- elog(ERROR, "catcache list reference %p is not owned by resource owner %s",
- list, owner->name);
+ if (!res)
+ elog(ERROR, "catcache list reference %p is not owned by resource owner %s",
+ list, owner->name);
}
/*
@@ -856,25 +1072,7 @@ ResourceOwnerForgetCatCacheListRef(ResourceOwner owner, CatCList *list)
void
ResourceOwnerEnlargeRelationRefs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->nrelrefs < owner->maxrelrefs)
- return; /* nothing to do */
-
- if (owner->relrefs == NULL)
- {
- newmax = 16;
- owner->relrefs = (Relation *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(Relation));
- owner->maxrelrefs = newmax;
- }
- else
- {
- newmax = owner->maxrelrefs * 2;
- owner->relrefs = (Relation *)
- repalloc(owner->relrefs, newmax * sizeof(Relation));
- owner->maxrelrefs = newmax;
- }
+ ResourceArrayEnlarge(&(owner->relrefarr));
}
/*
@@ -885,9 +1083,7 @@ ResourceOwnerEnlargeRelationRefs(ResourceOwner owner)
void
ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel)
{
- Assert(owner->nrelrefs < owner->maxrelrefs);
- owner->relrefs[owner->nrelrefs] = rel;
- owner->nrelrefs++;
+ ResourceArrayAdd(&(owner->relrefarr), (const void *) &rel);
}
/*
@@ -896,25 +1092,12 @@ ResourceOwnerRememberRelationRef(ResourceOwner owner, Relation rel)
void
ResourceOwnerForgetRelationRef(ResourceOwner owner, Relation rel)
{
- Relation *relrefs = owner->relrefs;
- int nr1 = owner->nrelrefs - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->relrefarr),
+ (const void *) &rel);
- for (i = nr1; i >= 0; i--)
- {
- if (relrefs[i] == rel)
- {
- while (i < nr1)
- {
- relrefs[i] = relrefs[i + 1];
- i++;
- }
- owner->nrelrefs = nr1;
- return;
- }
- }
- elog(ERROR, "relcache reference %s is not owned by resource owner %s",
- RelationGetRelationName(rel), owner->name);
+ if (!res)
+ elog(ERROR, "relcache reference %s is not owned by resource owner %s",
+ RelationGetRelationName(rel), owner->name);
}
/*
@@ -937,25 +1120,7 @@ PrintRelCacheLeakWarning(Relation rel)
void
ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->nplanrefs < owner->maxplanrefs)
- return; /* nothing to do */
-
- if (owner->planrefs == NULL)
- {
- newmax = 16;
- owner->planrefs = (CachedPlan **)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(CachedPlan *));
- owner->maxplanrefs = newmax;
- }
- else
- {
- newmax = owner->maxplanrefs * 2;
- owner->planrefs = (CachedPlan **)
- repalloc(owner->planrefs, newmax * sizeof(CachedPlan *));
- owner->maxplanrefs = newmax;
- }
+ ResourceArrayEnlarge(&(owner->planrefarr));
}
/*
@@ -966,9 +1131,7 @@ ResourceOwnerEnlargePlanCacheRefs(ResourceOwner owner)
void
ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan)
{
- Assert(owner->nplanrefs < owner->maxplanrefs);
- owner->planrefs[owner->nplanrefs] = plan;
- owner->nplanrefs++;
+ ResourceArrayAdd(&(owner->planrefarr), (const void *) &plan);
}
/*
@@ -977,25 +1140,12 @@ ResourceOwnerRememberPlanCacheRef(ResourceOwner owner, CachedPlan *plan)
void
ResourceOwnerForgetPlanCacheRef(ResourceOwner owner, CachedPlan *plan)
{
- CachedPlan **planrefs = owner->planrefs;
- int np1 = owner->nplanrefs - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->planrefarr),
+ (const void *) &plan);
- for (i = np1; i >= 0; i--)
- {
- if (planrefs[i] == plan)
- {
- while (i < np1)
- {
- planrefs[i] = planrefs[i + 1];
- i++;
- }
- owner->nplanrefs = np1;
- return;
- }
- }
- elog(ERROR, "plancache reference %p is not owned by resource owner %s",
- plan, owner->name);
+ if (!res)
+ elog(ERROR, "plancache reference %p is not owned by resource owner %s",
+ plan, owner->name);
}
/*
@@ -1017,25 +1167,7 @@ PrintPlanCacheLeakWarning(CachedPlan *plan)
void
ResourceOwnerEnlargeTupleDescs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->ntupdescs < owner->maxtupdescs)
- return; /* nothing to do */
-
- if (owner->tupdescs == NULL)
- {
- newmax = 16;
- owner->tupdescs = (TupleDesc *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(TupleDesc));
- owner->maxtupdescs = newmax;
- }
- else
- {
- newmax = owner->maxtupdescs * 2;
- owner->tupdescs = (TupleDesc *)
- repalloc(owner->tupdescs, newmax * sizeof(TupleDesc));
- owner->maxtupdescs = newmax;
- }
+ ResourceArrayEnlarge(&(owner->tupdescarr));
}
/*
@@ -1046,9 +1178,7 @@ ResourceOwnerEnlargeTupleDescs(ResourceOwner owner)
void
ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
{
- Assert(owner->ntupdescs < owner->maxtupdescs);
- owner->tupdescs[owner->ntupdescs] = tupdesc;
- owner->ntupdescs++;
+ ResourceArrayAdd(&(owner->tupdescarr), (const void *) &tupdesc);
}
/*
@@ -1057,25 +1187,12 @@ ResourceOwnerRememberTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
void
ResourceOwnerForgetTupleDesc(ResourceOwner owner, TupleDesc tupdesc)
{
- TupleDesc *tupdescs = owner->tupdescs;
- int nt1 = owner->ntupdescs - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->tupdescarr),
+ (const void *) &tupdesc);
- for (i = nt1; i >= 0; i--)
- {
- if (tupdescs[i] == tupdesc)
- {
- while (i < nt1)
- {
- tupdescs[i] = tupdescs[i + 1];
- i++;
- }
- owner->ntupdescs = nt1;
- return;
- }
- }
- elog(ERROR, "tupdesc reference %p is not owned by resource owner %s",
- tupdesc, owner->name);
+ if (!res)
+ elog(ERROR, "tupdesc reference %p is not owned by resource owner %s",
+ tupdesc, owner->name);
}
/*
@@ -1099,25 +1216,7 @@ PrintTupleDescLeakWarning(TupleDesc tupdesc)
void
ResourceOwnerEnlargeSnapshots(ResourceOwner owner)
{
- int newmax;
-
- if (owner->nsnapshots < owner->maxsnapshots)
- return; /* nothing to do */
-
- if (owner->snapshots == NULL)
- {
- newmax = 16;
- owner->snapshots = (Snapshot *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(Snapshot));
- owner->maxsnapshots = newmax;
- }
- else
- {
- newmax = owner->maxsnapshots * 2;
- owner->snapshots = (Snapshot *)
- repalloc(owner->snapshots, newmax * sizeof(Snapshot));
- owner->maxsnapshots = newmax;
- }
+ ResourceArrayEnlarge(&(owner->snapshotarr));
}
/*
@@ -1128,9 +1227,7 @@ ResourceOwnerEnlargeSnapshots(ResourceOwner owner)
void
ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot)
{
- Assert(owner->nsnapshots < owner->maxsnapshots);
- owner->snapshots[owner->nsnapshots] = snapshot;
- owner->nsnapshots++;
+ ResourceArrayAdd(&(owner->snapshotarr), (const void *) &snapshot);
}
/*
@@ -1139,25 +1236,12 @@ ResourceOwnerRememberSnapshot(ResourceOwner owner, Snapshot snapshot)
void
ResourceOwnerForgetSnapshot(ResourceOwner owner, Snapshot snapshot)
{
- Snapshot *snapshots = owner->snapshots;
- int ns1 = owner->nsnapshots - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->snapshotarr),
+ (const void *) &snapshot);
- for (i = ns1; i >= 0; i--)
- {
- if (snapshots[i] == snapshot)
- {
- while (i < ns1)
- {
- snapshots[i] = snapshots[i + 1];
- i++;
- }
- owner->nsnapshots = ns1;
- return;
- }
- }
- elog(ERROR, "snapshot reference %p is not owned by resource owner %s",
- snapshot, owner->name);
+ if (!res)
+ elog(ERROR, "snapshot reference %p is not owned by resource owner %s",
+ snapshot, owner->name);
}
/*
@@ -1182,25 +1266,7 @@ PrintSnapshotLeakWarning(Snapshot snapshot)
void
ResourceOwnerEnlargeFiles(ResourceOwner owner)
{
- int newmax;
-
- if (owner->nfiles < owner->maxfiles)
- return; /* nothing to do */
-
- if (owner->files == NULL)
- {
- newmax = 16;
- owner->files = (File *)
- MemoryContextAlloc(TopMemoryContext, newmax * sizeof(File));
- owner->maxfiles = newmax;
- }
- else
- {
- newmax = owner->maxfiles * 2;
- owner->files = (File *)
- repalloc(owner->files, newmax * sizeof(File));
- owner->maxfiles = newmax;
- }
+ ResourceArrayEnlarge(&(owner->filearr));
}
/*
@@ -1211,9 +1277,7 @@ ResourceOwnerEnlargeFiles(ResourceOwner owner)
void
ResourceOwnerRememberFile(ResourceOwner owner, File file)
{
- Assert(owner->nfiles < owner->maxfiles);
- owner->files[owner->nfiles] = file;
- owner->nfiles++;
+ ResourceArrayAdd(&(owner->filearr), (const void *) &file);
}
/*
@@ -1222,25 +1286,12 @@ ResourceOwnerRememberFile(ResourceOwner owner, File file)
void
ResourceOwnerForgetFile(ResourceOwner owner, File file)
{
- File *files = owner->files;
- int ns1 = owner->nfiles - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->filearr),
+ (const void *) &file);
- for (i = ns1; i >= 0; i--)
- {
- if (files[i] == file)
- {
- while (i < ns1)
- {
- files[i] = files[i + 1];
- i++;
- }
- owner->nfiles = ns1;
- return;
- }
- }
- elog(ERROR, "temporery file %d is not owned by resource owner %s",
- file, owner->name);
+ if (!res)
+ elog(ERROR, "temporery file %d is not owned by resource owner %s",
+ file, owner->name);
}
@@ -1265,26 +1316,7 @@ PrintFileLeakWarning(File file)
void
ResourceOwnerEnlargeDSMs(ResourceOwner owner)
{
- int newmax;
-
- if (owner->ndsms < owner->maxdsms)
- return; /* nothing to do */
-
- if (owner->dsms == NULL)
- {
- newmax = 16;
- owner->dsms = (dsm_segment **)
- MemoryContextAlloc(TopMemoryContext,
- newmax * sizeof(dsm_segment *));
- owner->maxdsms = newmax;
- }
- else
- {
- newmax = owner->maxdsms * 2;
- owner->dsms = (dsm_segment **)
- repalloc(owner->dsms, newmax * sizeof(dsm_segment *));
- owner->maxdsms = newmax;
- }
+ ResourceArrayEnlarge(&(owner->dsmarr));
}
/*
@@ -1295,9 +1327,7 @@ ResourceOwnerEnlargeDSMs(ResourceOwner owner)
void
ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg)
{
- Assert(owner->ndsms < owner->maxdsms);
- owner->dsms[owner->ndsms] = seg;
- owner->ndsms++;
+ ResourceArrayAdd(&(owner->dsmarr), (const void *) &seg);
}
/*
@@ -1306,26 +1336,12 @@ ResourceOwnerRememberDSM(ResourceOwner owner, dsm_segment *seg)
void
ResourceOwnerForgetDSM(ResourceOwner owner, dsm_segment *seg)
{
- dsm_segment **dsms = owner->dsms;
- int ns1 = owner->ndsms - 1;
- int i;
+ bool res = ResourceArrayRemove(&(owner->dsmarr),
+ (const void *) &seg);
- for (i = ns1; i >= 0; i--)
- {
- if (dsms[i] == seg)
- {
- while (i < ns1)
- {
- dsms[i] = dsms[i + 1];
- i++;
- }
- owner->ndsms = ns1;
- return;
- }
- }
- elog(ERROR,
- "dynamic shared memory segment %u is not owned by resource owner %s",
- dsm_segment_handle(seg), owner->name);
+ if (!res)
+ elog(ERROR, "dynamic shared memory segment %u is not owned by resource"
+ " owner %s", dsm_segment_handle(seg), owner->name);
}
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
