On Thu, 19 Aug 2021 at 13:44, Andres Freund <and...@anarazel.de> wrote:
>
> > Another fun thing --- and, I fear, another good argument against just
> > raising NAMEDATALEN --- is what about TupleDescs, which last I checked
> > used an array of fixed-width pg_attribute images. But maybe we could
> > replace that with an array of pointers. Andres already did a lot of
> > the heavy code churn required to hide that data structure behind
> > TupleDescAttr() macros, so changing the representation should be much
> > less painful than it would once have been.
>
> I was recently wondering if we shouldn't go to a completely bespoke
> datastructure for TupleDesc->attrs, rather than reusing FormData_pg_attribute.
>
> Right now every attribute uses nearly two cachelines (112 bytes). Given how
> frequent a task tuple [de]forming is, and how often it's a bottleneck,
> increasing the cache efficiency of tupledescs would worth quite a bit of
> effort - I do see tupledesc attr cache misses in profiles. A secondary benefit
> would be that we do create a lot of short-lived descs in the executor,
> slimming those down obviously would be good on its own. A third benefit would
> be that we could get rid of attcacheoff in pg_attribute, that always smelled
> funny to me.
>
> One possible way to structure such future tupledescs would be to have multiple
> arrays in struct TupleDescData. With an array of just the data necessary for
> [de]forming at the place ->attrs is, and other stuff in one or more separate
> arrays. The other option could perhaps be omitted for some tupledescs or
> computed lazily.
>
> For deforming we just need attlen (2byte), attbyval (1 byte), attalign (1byte)
> and optionally attcacheoff (4 byte), for forming we also need attstorage (1
> byte). Naively that ends up being 12 bytes - 5 attrs / cacheline is a heck of
> a lot better than ~0.5.
I tried to implement this 'compact attribute access descriptor' a few
months ago in my effort to improve btree index performance.
I abandoned the idea at the time as I didn't find any measurable
difference for the (limited!) tests I ran, where the workload was
mainly re-indexing, select * into, and similar items while
benchmarking reindexing in the 'pp-complete' dataset. But, seeing that
there might be interest outside this effort on a basis seperate from
just plain performance, I'll share the results.
Attached is the latest version of my patch that I could find; it might
be incorrect or fail, as this is something I sent to myself between 2
of my systems during development of the patch. Also, attached as .txt,
as I don't want any CFBot coverage on this (this is not proposed for
inclusion, it is just a show of work, and might be basis for future
work).
The patch allocates an array of 'TupleAttrAlignData'-structs at the
end of the attrs-array, fills it with the correct data upon
TupleDesc-creation, and uses this TupleAttrAlign-data for constructing
and destructing tuples.
One main difference from what you described was that I used a union
for storing attbyval and attstorage, as the latter is only applicable
to attlen < 0, and the first only for attlen >= 0. This keeps the
whole structure in 8 bytes, whilst also being useable in both tuple
forming and deforming.
I hope this can is useful, otherwise sorry for the noise.
Kind regards,
Matthias van de Meent
From 3dd3f470ab78b8811015c9f374d0bdc44ffef531 Mon Sep 17 00:00:00 2001
From: Matthias van de Meent <boekew...@gmail.com>
Date: Wed, 23 Jun 2021 20:38:47 +0200
Subject: [PATCH] Some work on storing attribute access fields more compactly
This would increase the amount of useful data in each cache line, potentially
increasing performance. It _does_ use more memory (8 bytes / attribute extra
overhead)
---
src/backend/access/common/heaptuple.c | 95 +++++++++++++-------------
src/backend/access/common/indextuple.c | 52 ++++++++------
src/backend/access/common/relation.c | 2 +
src/backend/access/common/tupdesc.c | 74 +++++++++++++++++++-
src/backend/access/spgist/spgutils.c | 3 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 4 ++
src/backend/utils/cache/relcache.c | 18 ++++-
src/include/access/htup_details.h | 6 +-
src/include/access/itup.h | 9 +--
src/include/access/tupdesc.h | 35 +++++++++-
src/include/access/tupmacs.h | 2 +-
src/include/catalog/pg_attribute.h | 5 ++
13 files changed, 226 insertions(+), 81 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c
b/src/backend/access/common/heaptuple.c
index 0b56b0fa5a..3600f88b25 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -123,19 +123,18 @@ heap_compute_data_size(TupleDesc tupleDesc,
Size data_length = 0;
int i;
int numberOfAttributes = tupleDesc->natts;
+ TupleAttrAlign att;
- for (i = 0; i < numberOfAttributes; i++)
+ for (i = 0, att = TupleDescAttrAlign(tupleDesc, i); i <
numberOfAttributes; i++, att++)
{
Datum val;
- Form_pg_attribute atti;
if (isnull[i])
continue;
val = values[i];
- atti = TupleDescAttr(tupleDesc, i);
- if (ATT_IS_PACKABLE(atti) &&
+ if (ATT_IS_PACKABLE(att) &&
VARATT_CAN_MAKE_SHORT(DatumGetPointer(val)))
{
/*
@@ -144,21 +143,21 @@ heap_compute_data_size(TupleDesc tupleDesc,
*/
data_length +=
VARATT_CONVERTED_SHORT_SIZE(DatumGetPointer(val));
}
- else if (atti->attlen == -1 &&
+ else if (att->attlen == -1 &&
VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(val)))
{
/*
* we want to flatten the expanded value so that the
constructed
* tuple doesn't depend on it
*/
- data_length = att_align_nominal(data_length,
atti->attalign);
+ data_length = att_align_nominal(data_length,
att->attalign);
data_length += EOH_get_flat_size(DatumGetEOHP(val));
}
else
{
- data_length = att_align_datum(data_length,
atti->attalign,
-
atti->attlen, val);
- data_length = att_addlength_datum(data_length,
atti->attlen,
+ data_length = att_align_datum(data_length,
att->attalign,
+
att->attlen, val);
+ data_length = att_addlength_datum(data_length,
att->attlen,
val);
}
}
@@ -172,7 +171,7 @@ heap_compute_data_size(TupleDesc tupleDesc,
* Fill in either a data value or a bit in the null bitmask
*/
static inline void
-fill_val(Form_pg_attribute att,
+fill_val(TupleAttrAlign att,
bits8 **bit,
int *bitmask,
char **dataP,
@@ -211,7 +210,7 @@ fill_val(Form_pg_attribute att,
* XXX we use the att_align macros on the pointer value itself, not on
an
* offset. This is a bit of a hack.
*/
- if (att->attbyval)
+ if (att->attbyval && att->attlen >= 0)
{
/* pass-by-value */
data = (char *) att_align_nominal(data, att->attalign);
@@ -310,6 +309,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
int bitmask;
int i;
int numberOfAttributes = tupleDesc->natts;
+ TupleAttrAlign attr;
#ifdef USE_ASSERT_CHECKING
char *start = data;
@@ -329,10 +329,8 @@ heap_fill_tuple(TupleDesc tupleDesc,
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
- for (i = 0; i < numberOfAttributes; i++)
+ for (i = 0, attr = TupleDescAttrAlign(tupleDesc, i); i <
numberOfAttributes; i++, attr++)
{
- Form_pg_attribute attr = TupleDescAttr(tupleDesc, i);
-
fill_val(attr,
bitP ? &bitP : NULL,
&bitmask,
@@ -429,6 +427,7 @@ nocachegetattr(HeapTuple tuple,
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
bool slow = false; /* do we have to walk attrs? */
int off; /* current offset
within data */
+ TupleAttrAlign thisAttrAlign;
/* ----------------
* Three cases:
@@ -474,13 +473,13 @@ nocachegetattr(HeapTuple tuple,
if (!slow)
{
- Form_pg_attribute att;
+ TupleAttrAlign att;
/*
* If we get here, there are no nulls up to and including the
target
* attribute. If we have a cached offset, we can use it.
*/
- att = TupleDescAttr(tupleDesc, attnum);
+ att = TupleDescAttrAlign(tupleDesc, attnum);
if (att->attcacheoff >= 0)
return fetchatt(att, tp + att->attcacheoff);
@@ -492,10 +491,11 @@ nocachegetattr(HeapTuple tuple,
if (HeapTupleHasVarWidth(tuple))
{
int j;
+ att = TupleDescAttrAlign(tupleDesc, 0);
- for (j = 0; j <= attnum; j++)
+ for (j = 0; j <= attnum; j++, att++)
{
- if (TupleDescAttr(tupleDesc, j)->attlen <= 0)
+ if (att->attlen <= 0)
{
slow = true;
break;
@@ -508,6 +508,7 @@ nocachegetattr(HeapTuple tuple,
{
int natts = tupleDesc->natts;
int j = 1;
+ TupleAttrAlign att;
/*
* If we get here, we have a tuple with no nulls or var-widths
up to
@@ -518,19 +519,18 @@ nocachegetattr(HeapTuple tuple,
* fixed-width columns, in hope of avoiding future visits to
this
* routine.
*/
- TupleDescAttr(tupleDesc, 0)->attcacheoff = 0;
+ att = TupleDescAttrAlign(tupleDesc, 0);
+ att->attcacheoff = 0;
+ att++;
/* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescAttr(tupleDesc, j)->attcacheoff >
0)
- j++;
+ while (j < natts && att->attcacheoff > 0)
+ j++, att++;
- off = TupleDescAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescAttr(tupleDesc, j - 1)->attlen;
+ off = (att - 1)->attcacheoff + (att - 1)->attlen;
- for (; j < natts; j++)
+ for (; j < natts; j++, att++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, j);
-
if (att->attlen <= 0)
break;
@@ -543,12 +543,14 @@ nocachegetattr(HeapTuple tuple,
Assert(j > attnum);
- off = TupleDescAttr(tupleDesc, attnum)->attcacheoff;
+ thisAttrAlign = TupleDescAttrAlign(tupleDesc, attnum);
+ off = thisAttrAlign->attcacheoff;
}
else
{
bool usecache = true;
int i;
+ TupleAttrAlign att = TupleDescAttrAlign(tupleDesc, 0);
/*
* Now we know that we have to walk the tuple CAREFULLY. But
we still
@@ -561,10 +563,8 @@ nocachegetattr(HeapTuple tuple,
* attcacheoff until we reach either a null or a var-width
attribute.
*/
off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
+ for (i = 0;; i++, att++) /* loop exit is at
"break" */
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
-
if (HeapTupleHasNulls(tuple) && att_isnull(i, bp))
{
usecache = false;
@@ -602,7 +602,10 @@ nocachegetattr(HeapTuple tuple,
}
if (i == attnum)
+ {
+ thisAttrAlign = att;
break;
+ }
off = att_addlength_pointer(off, att->attlen, tp + off);
@@ -611,7 +614,7 @@ nocachegetattr(HeapTuple tuple,
}
}
- return fetchatt(TupleDescAttr(tupleDesc, attnum), tp + off);
+ return fetchatt(thisAttrAlign, tp + off);
}
/* ----------------
@@ -925,7 +928,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
for (attnum = sourceNatts; attnum < natts; attnum++)
{
- Form_pg_attribute attr = TupleDescAttr(tupleDesc, attnum);
+ TupleAttrAlign attr = TupleDescAttrAlign(tupleDesc, attnum);
if (attrmiss && attrmiss[attnum].am_present)
{
@@ -1258,6 +1261,7 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
uint32 off; /* offset in tuple data */
bits8 *bp = tup->t_bits; /* ptr to null bitmap in tuple */
bool slow = false; /* can we use/set attcacheoff? */
+ TupleAttrAlign att;
natts = HeapTupleHeaderGetNatts(tup);
@@ -1272,10 +1276,9 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
off = 0;
- for (attnum = 0; attnum < natts; attnum++)
+ for (attnum = 0, att = TupleDescAttrAlign(tupleDesc, attnum);
+ attnum < natts; attnum++, att++)
{
- Form_pg_attribute thisatt = TupleDescAttr(tupleDesc, attnum);
-
if (hasnulls && att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
@@ -1286,9 +1289,9 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
isnull[attnum] = false;
- if (!slow && thisatt->attcacheoff >= 0)
- off = thisatt->attcacheoff;
- else if (thisatt->attlen == -1)
+ if (!slow && att->attcacheoff >= 0)
+ off = att->attcacheoff;
+ else if (att->attlen == -1)
{
/*
* We can only cache the offset for a varlena attribute
if the
@@ -1297,11 +1300,11 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
* an aligned or unaligned value.
*/
if (!slow &&
- off == att_align_nominal(off,
thisatt->attalign))
- thisatt->attcacheoff = off;
+ off == att_align_nominal(off, att->attalign))
+ att->attcacheoff = off;
else
{
- off = att_align_pointer(off, thisatt->attalign,
-1,
+ off = att_align_pointer(off, att->attalign, -1,
tp + off);
slow = true;
}
@@ -1309,17 +1312,17 @@ heap_deform_tuple(HeapTuple tuple, TupleDesc tupleDesc,
else
{
/* not varlena, so safe to use att_align_nominal */
- off = att_align_nominal(off, thisatt->attalign);
+ off = att_align_nominal(off, att->attalign);
if (!slow)
- thisatt->attcacheoff = off;
+ att->attcacheoff = off;
}
- values[attnum] = fetchatt(thisatt, tp + off);
+ values[attnum] = fetchatt(att, tp + off);
- off = att_addlength_pointer(off, thisatt->attlen, tp + off);
+ off = att_addlength_pointer(off, att->attlen, tp + off);
- if (thisatt->attlen <= 0)
+ if (att->attlen <= 0)
slow = true; /* can't use attcacheoff
anymore */
}
diff --git a/src/backend/access/common/indextuple.c
b/src/backend/access/common/indextuple.c
index 8df882da7a..b2c08216db 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -228,7 +228,7 @@ nocache_index_getattr(IndexTuple tup,
bool slow = false; /* do we have to walk attrs? */
int data_off; /* tuple data offset */
int off; /* current offset
within data */
-
+ TupleAttrAlign thisAttrAlign;
/* ----------------
* Three cases:
*
@@ -242,6 +242,8 @@ nocache_index_getattr(IndexTuple tup,
attnum--;
+ Assert(TupleDescAttrAlignsIsConsistent(tupleDesc));
+
if (IndexTupleHasNulls(tup))
{
/*
@@ -284,13 +286,13 @@ nocache_index_getattr(IndexTuple tup,
if (!slow)
{
- Form_pg_attribute att;
+ TupleAttrAlign att;
/*
* If we get here, there are no nulls up to and including the
target
* attribute. If we have a cached offset, we can use it.
*/
- att = TupleDescAttr(tupleDesc, attnum);
+ att = TupleDescAttrAlign(tupleDesc, attnum);
if (att->attcacheoff >= 0)
return fetchatt(att, tp + att->attcacheoff);
@@ -302,10 +304,11 @@ nocache_index_getattr(IndexTuple tup,
if (IndexTupleHasVarwidths(tup))
{
int j;
+ att = TupleDescAttrAlign(tupleDesc, 0);
- for (j = 0; j <= attnum; j++)
+ for (j = 0; j <= attnum; j++, att++)
{
- if (TupleDescAttr(tupleDesc, j)->attlen <= 0)
+ if (att->attlen <= 0)
{
slow = true;
break;
@@ -318,6 +321,7 @@ nocache_index_getattr(IndexTuple tup,
{
int natts = tupleDesc->natts;
int j = 1;
+ TupleAttrAlign att;
/*
* If we get here, we have a tuple with no nulls or var-widths
up to
@@ -328,19 +332,18 @@ nocache_index_getattr(IndexTuple tup,
* fixed-width columns, in hope of avoiding future visits to
this
* routine.
*/
- TupleDescAttr(tupleDesc, 0)->attcacheoff = 0;
+ att = TupleDescAttrAlign(tupleDesc, 0);
+ att->attcacheoff = 0;
+ att++;
/* we might have set some offsets in the slow path previously */
- while (j < natts && TupleDescAttr(tupleDesc, j)->attcacheoff >
0)
- j++;
+ while (j < natts && att->attcacheoff > 0)
+ j++, att++;
- off = TupleDescAttr(tupleDesc, j - 1)->attcacheoff +
- TupleDescAttr(tupleDesc, j - 1)->attlen;
+ off = (att - 1)->attcacheoff + (att - 1)->attlen;
- for (; j < natts; j++)
+ for (; j < natts; j++, att++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, j);
-
if (att->attlen <= 0)
break;
@@ -353,13 +356,14 @@ nocache_index_getattr(IndexTuple tup,
Assert(j > attnum);
- off = TupleDescAttr(tupleDesc, attnum)->attcacheoff;
+ thisAttrAlign = TupleDescAttrAlign(tupleDesc, attnum);
+ off = thisAttrAlign->attcacheoff;
}
else
{
bool usecache = true;
int i;
-
+ TupleAttrAlign att;
/*
* Now we know that we have to walk the tuple CAREFULLY. But
we still
* might be able to cache some offsets for next time.
@@ -371,10 +375,10 @@ nocache_index_getattr(IndexTuple tup,
* attcacheoff until we reach either a null or a var-width
attribute.
*/
off = 0;
- for (i = 0;; i++) /* loop exit is at "break" */
- {
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ att = TupleDescAttrAlign(tupleDesc, 0);
+ for (i = 0;; i++, att++) /* loop exit is at
"break" */
+ {
if (IndexTupleHasNulls(tup) && att_isnull(i, bp))
{
usecache = false;
@@ -412,7 +416,10 @@ nocache_index_getattr(IndexTuple tup,
}
if (i == attnum)
+ {
+ thisAttrAlign = att;
break;
+ }
off = att_addlength_pointer(off, att->attlen, tp + off);
@@ -421,7 +428,7 @@ nocache_index_getattr(IndexTuple tup,
}
}
- return fetchatt(TupleDescAttr(tupleDesc, attnum), tp + off);
+ return fetchatt(thisAttrAlign, tp + off);
}
/*
@@ -465,14 +472,14 @@ index_deform_tuple_internal(TupleDesc tupleDescriptor,
int attnum;
int off = 0; /* offset in tuple data
*/
bool slow = false; /* can we use/set attcacheoff? */
+ TupleAttrAlign thisatt;
/* Assert to protect callers who allocate fixed-size arrays */
Assert(natts <= INDEX_MAX_KEYS);
- for (attnum = 0; attnum < natts; attnum++)
+ for (attnum = 0, thisatt = TupleDescAttrAlign(tupleDescriptor, attnum);
+ attnum < natts; attnum++, thisatt++)
{
- Form_pg_attribute thisatt = TupleDescAttr(tupleDescriptor,
attnum);
-
if (hasnulls && att_isnull(attnum, bp))
{
values[attnum] = (Datum) 0;
@@ -572,6 +579,7 @@ index_truncate_tuple(TupleDesc sourceDescriptor, IndexTuple
source,
truncdesc = palloc(TupleDescSize(sourceDescriptor));
TupleDescCopy(truncdesc, sourceDescriptor);
truncdesc->natts = leavenatts;
+ TupleDescInitAttrAligns(truncdesc);
/* Deform, form copy of tuple with fewer attributes */
index_deform_tuple(source, truncdesc, values, isnull);
diff --git a/src/backend/access/common/relation.c
b/src/backend/access/common/relation.c
index 632d13c1ea..629d60b947 100644
--- a/src/backend/access/common/relation.c
+++ b/src/backend/access/common/relation.c
@@ -61,6 +61,8 @@ relation_open(Oid relationId, LOCKMODE lockmode)
if (!RelationIsValid(r))
elog(ERROR, "could not open relation with OID %u", relationId);
+ Assert(TupleDescAttrAlignsIsConsistent(r->rd_att));
+
/*
* If we didn't get the lock ourselves, assert that caller holds one,
* except in bootstrap mode where no locks are used.
diff --git a/src/backend/access/common/tupdesc.c
b/src/backend/access/common/tupdesc.c
index 4c63bd4dc6..3ee74ce9fc 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -63,8 +63,9 @@ CreateTemplateTupleDesc(int natts)
* could be less due to trailing padding, although with the current
* definition of pg_attribute there probably isn't any padding.
*/
- desc = (TupleDesc) palloc(offsetof(struct TupleDescData, attrs) +
- natts *
sizeof(FormData_pg_attribute));
+ desc = (TupleDesc) palloc(MAXALIGN((offsetof(struct TupleDescData,
attrs)) +
+ natts *
sizeof(FormData_pg_attribute)) +
+ natts *
sizeof(TupleAttrAlignData));
/*
* Initialize other fields of the tupdesc.
@@ -75,6 +76,9 @@ CreateTemplateTupleDesc(int natts)
desc->tdtypmod = -1;
desc->tdrefcount = -1; /* assume not reference-counted */
+ desc->attralignoff = MAXALIGN((offsetof(struct TupleDescData, attrs)) +
+ natts *
sizeof(FormData_pg_attribute));
+
return desc;
}
@@ -97,6 +101,8 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
for (i = 0; i < natts; ++i)
memcpy(TupleDescAttr(desc, i), attrs[i],
ATTRIBUTE_FIXED_PART_SIZE);
+ TupleDescInitAttrAligns(desc);
+
return desc;
}
@@ -135,6 +141,8 @@ CreateTupleDescCopy(TupleDesc tupdesc)
att->attgenerated = '\0';
}
+ TupleDescInitAttrAligns(desc);
+
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
@@ -210,6 +218,10 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->constr = cpy;
}
+ memcpy(TupleDescAttrAlign(desc, 0),
+ TupleDescAttrAlign(tupdesc, 0),
+ tupdesc->natts * sizeof(TupleAttrAlignData));
+
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
@@ -300,6 +312,10 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
dstAtt->attgenerated = '\0';
+
+ memcpy(TupleDescAttrAlign(dst, dstAttno - 1),
+ TupleDescAttrAlign(src, srcAttno - 1),
+ sizeof(TupleAttrAlignData));
}
/*
@@ -645,6 +661,8 @@ TupleDescInitEntry(TupleDesc desc,
att->attcompression = InvalidCompressionMethod;
att->attcollation = typeForm->typcollation;
+ CopyAttrToAlign(att, TupleDescAttrAlign(desc, attributeNumber - 1));
+
ReleaseSysCache(tuple);
}
@@ -742,6 +760,10 @@ TupleDescInitBuiltinEntry(TupleDesc desc,
default:
elog(ERROR, "unsupported type %u", oidtypeid);
}
+
+ TupleAttrAlign attrAlign = TupleDescAttrAlign(desc, attributeNumber -
1);
+
+ CopyAttrToAlign(att, attrAlign);
}
/*
@@ -765,6 +787,49 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
+/*
+ * TupleDescInitAttrAligns
+ *
+ * Initialize the attraligns field of the TupleDesc with data from
+ * tupledesc->attrs.
+ */
+void
+TupleDescInitAttrAligns(TupleDesc desc)
+{
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+ TupleAttrAlign attrAlign = TupleDescAttrAlign(desc, i);
+ CopyAttrToAlign(attr, attrAlign);
+ }
+}
+
+bool
+TupleDescAttrAlignsIsConsistent(TupleDesc desc)
+{
+ for (int i = 0; i < desc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(desc, i);
+ TupleAttrAlign attrAlign = TupleDescAttrAlign(desc, i);
+ if (!(attrAlign->attcacheoff == attr->attcacheoff ||
attrAlign->attcacheoff == -1 || attr->attcacheoff == -1))
+ return false;
+ if (attrAlign->attlen != attr->attlen)
+ return false;
+ if (attrAlign->attalign != attr->attalign)
+ return false;
+ if (attrAlign->attlen < 0)
+ {
+ if (attrAlign->attstorage != attr->attstorage)
+ return false;
+ }
+ else
+ {
+ if (attrAlign->attbyval != attr->attbyval)
+ return false;
+ }
+ }
+ return true;
+}
/*
* BuildDescForRelation
@@ -801,6 +866,7 @@ BuildDescForRelation(List *schema)
ColumnDef *entry = lfirst(l);
AclResult aclresult;
Form_pg_attribute att;
+ TupleAttrAlign align;
/*
* for each entry in the list, get the name and type
information from
@@ -828,11 +894,15 @@ BuildDescForRelation(List *schema)
TupleDescInitEntry(desc, attnum, attname,
atttypid, atttypmod, attdim);
att = TupleDescAttr(desc, attnum - 1);
+ align = TupleDescAttrAlign(desc, attnum - 1);
/* Override TupleDescInitEntry's settings as requested */
TupleDescInitEntryCollation(desc, attnum, attcollation);
if (entry->storage)
+ {
att->attstorage = entry->storage;
+ align->attstorage = entry->storage;
+ }
/* Fill in additional stuff not handled by TupleDescInitEntry */
att->attnotnull = entry->is_not_null;
diff --git a/src/backend/access/spgist/spgutils.c
b/src/backend/access/spgist/spgutils.c
index 9ff280a252..bc46dd1b02 100644
--- a/src/backend/access/spgist/spgutils.c
+++ b/src/backend/access/spgist/spgutils.c
@@ -306,9 +306,12 @@ getSpGistTupleDesc(Relation index, SpGistTypeDesc *keyType)
/* We shouldn't need to bother with making these valid: */
att->attcompression = InvalidCompressionMethod;
att->attcollation = InvalidOid;
+
/* In case we changed typlen, we'd better reset following
offsets */
for (int i = spgFirstIncludeColumn; i < outTupDesc->natts; i++)
TupleDescAttr(outTupDesc, i)->attcacheoff = -1;
+
+ TupleDescInitAttrAligns(outTupDesc);
}
return outTupDesc;
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 50b7a16bce..28d1a7a032 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -471,6 +471,8 @@ ConstructTupleDescriptor(Relation heapRelation,
}
}
+ TupleDescInitAttrAligns(indexTupDesc);
+
pfree(amroutine);
return indexTupDesc;
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index bf81f6ccc5..a872ce50c7 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -221,6 +221,10 @@ create_toast_table(Relation rel, Oid toastOid, Oid
toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ CopyAttrToAlign(TupleDescAttr(tupdesc, 0), TupleDescAttrAlign(tupdesc,
0));
+ CopyAttrToAlign(TupleDescAttr(tupdesc, 1), TupleDescAttrAlign(tupdesc,
1));
+ CopyAttrToAlign(TupleDescAttr(tupdesc, 2), TupleDescAttrAlign(tupdesc,
2));
+
/* Toast field should not be compressed */
TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/utils/cache/relcache.c
b/src/backend/utils/cache/relcache.c
index d55ae016d0..b1aeab6e06 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -633,6 +633,8 @@ RelationBuildTupleDesc(Relation relation)
systable_endscan(pg_attribute_scan);
table_close(pg_attribute_desc, AccessShareLock);
+ TupleDescInitAttrAligns(relation->rd_att);
+
if (need != 0)
elog(ERROR, "pg_attribute catalog is missing %d attribute(s)
for relation OID %u",
need, RelationGetRelid(relation));
@@ -647,7 +649,8 @@ RelationBuildTupleDesc(Relation relation)
int i;
for (i = 0; i < RelationGetNumberOfAttributes(relation); i++)
- Assert(TupleDescAttr(relation->rd_att, i)->attcacheoff
== -1);
+ Assert(TupleDescAttr(relation->rd_att, i)->attcacheoff
== -1 &&
+ TupleDescAttrAlign(relation->rd_att,
i)->attcacheoff == -1);
}
#endif
@@ -657,7 +660,10 @@ RelationBuildTupleDesc(Relation relation)
* for attnum=1 that used to exist in fastgetattr() and index_getattr().
*/
if (RelationGetNumberOfAttributes(relation) > 0)
+ {
TupleDescAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescAttrAlign(relation->rd_att, 0)->attcacheoff = 0;
+ }
/*
* Set up constraint/default info
@@ -1901,6 +1907,8 @@ formrdesc(const char *relationName, Oid relationReltype,
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc
*/
TupleDescAttr(relation->rd_att, 0)->attcacheoff = 0;
+ TupleDescInitAttrAligns(relation->rd_att);
+
/* mark not-null status */
if (has_not_null)
{
@@ -2035,6 +2043,7 @@ RelationIdGetRelation(Oid relationId)
Assert(rd->rd_isvalid ||
(rd->rd_isnailed &&
!criticalRelcachesBuilt));
}
+ Assert(TupleDescAttrAlignsIsConsistent(rd->rd_att));
return rd;
}
@@ -2044,7 +2053,10 @@ RelationIdGetRelation(Oid relationId)
*/
rd = RelationBuildDesc(relationId, true);
if (RelationIsValid(rd))
+ {
RelationIncrementReferenceCount(rd);
+ Assert(TupleDescAttrAlignsIsConsistent(rd->rd_att));
+ }
return rd;
}
@@ -4208,6 +4220,8 @@ BuildHardcodedDescriptor(int natts, const
FormData_pg_attribute *attrs)
/* initialize first attribute's attcacheoff, cf RelationBuildTupleDesc
*/
TupleDescAttr(result, 0)->attcacheoff = 0;
+ TupleDescInitAttrAligns(result);
+
/* Note: we don't bother to set up a TupleConstr entry */
MemoryContextSwitchTo(oldcxt);
@@ -6088,6 +6102,8 @@ load_relcache_init_file(bool shared)
rel->rd_amcache = NULL;
MemSet(&rel->pgstat_info, 0, sizeof(rel->pgstat_info));
+ TupleDescInitAttrAligns(rel->rd_att);
+
/*
* Recompute lock and physical addressing info. This is needed
in
* case the pg_internal.init file was copied from some other
database
diff --git a/src/include/access/htup_details.h
b/src/include/access/htup_details.h
index 960772f76b..bc7892be9b 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -714,11 +714,11 @@ struct MinimalTupleData
(*(isnull) = false),
\
HeapTupleNoNulls(tup) ?
\
(
\
- TupleDescAttr((tupleDesc), (attnum)-1)->attcacheoff >= 0 ?
\
+ TupleDescAttrAlign((tupleDesc), (attnum)-1)->attcacheoff >= 0 ?
\
(
\
- fetchatt(TupleDescAttr((tupleDesc), (attnum)-1),
\
+ fetchatt(TupleDescAttrAlign((tupleDesc), (attnum)-1),
\
(char *) (tup)->t_data + (tup)->t_data->t_hoff
+ \
- TupleDescAttr((tupleDesc),
(attnum)-1)->attcacheoff)\
+ TupleDescAttrAlign((tupleDesc),
(attnum)-1)->attcacheoff)\
)
\
:
\
nocachegetattr((tup), (attnum), (tupleDesc))
\
diff --git a/src/include/access/itup.h b/src/include/access/itup.h
index 1917375cde..7b739dcd4f 100644
--- a/src/include/access/itup.h
+++ b/src/include/access/itup.h
@@ -99,15 +99,16 @@ typedef IndexAttributeBitMapData * IndexAttributeBitMap;
*/
#define index_getattr(tup, attnum, tupleDesc, isnull) \
( \
- AssertMacro(PointerIsValid(isnull) && (attnum) > 0), \
+ AssertMacro(PointerIsValid(isnull) && (attnum) > 0 && \
+ TupleDescAttrAlignsIsConsistent(tupleDesc)), \
*(isnull) = false, \
!IndexTupleHasNulls(tup) ? \
( \
- TupleDescAttr((tupleDesc), (attnum)-1)->attcacheoff >= 0 ? \
+ TupleDescAttrAlign((tupleDesc), (attnum)-1)->attcacheoff >= 0 ?
\
( \
- fetchatt(TupleDescAttr((tupleDesc), (attnum)-1), \
+ fetchatt(TupleDescAttrAlign((tupleDesc), (attnum)-1), \
(char *) (tup) + IndexInfoFindDataOffset((tup)->t_info)
\
- + TupleDescAttr((tupleDesc), (attnum)-1)->attcacheoff) \
+ + TupleDescAttrAlign((tupleDesc),
(attnum)-1)->attcacheoff) \
) \
: \
nocache_index_getattr((tup), (attnum), (tupleDesc)) \
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index f45d47aab7..9e8ce52cf3 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -45,6 +45,19 @@ typedef struct TupleConstr
bool has_generated_stored;
} TupleConstr;
+typedef struct TupleAttrAlignData
+{
+ int32 attcacheoff;
+ int16 attlen;
+ char attalign;
+ union {
+ char attstorage;
+ bool attbyval;
+ };
+} TupleAttrAlignData;
+
+typedef TupleAttrAlignData *TupleAttrAlign;
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -82,6 +95,7 @@ typedef struct TupleDescData
Oid tdtypeid; /* composite type ID
for tuple type */
int32 tdtypmod; /* typmod for tuple type */
int tdrefcount; /* reference count, or
-1 if not counting */
+ int attralignoff; /* offset to a co-allocated
array of TupleAttrAlignData */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
FormData_pg_attribute attrs[FLEXIBLE_ARRAY_MEMBER];
@@ -90,6 +104,8 @@ typedef struct TupleDescData *TupleDesc;
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+/* Accessor for the i'th attribute of attralign. */
+#define TupleDescAttrAlign(tupdesc, i) ((TupleAttrAlign) ((char *)(tupdesc) +
(tupdesc)->attralignoff + (i) * sizeof(TupleAttrAlignData)))
extern TupleDesc CreateTemplateTupleDesc(int natts);
@@ -100,8 +116,9 @@ extern TupleDesc CreateTupleDescCopy(TupleDesc tupdesc);
extern TupleDesc CreateTupleDescCopyConstr(TupleDesc tupdesc);
#define TupleDescSize(src) \
- (offsetof(struct TupleDescData, attrs) + \
- (src)->natts * sizeof(FormData_pg_attribute))
+ (MAXALIGN(offsetof(struct TupleDescData, attrs) + \
+ (src)->natts * sizeof(FormData_pg_attribute)) + \
+ (src)->natts * sizeof(TupleAttrAlignData))
extern void TupleDescCopy(TupleDesc dst, TupleDesc src);
@@ -125,6 +142,17 @@ extern void DecrTupleDescRefCount(TupleDesc tupdesc);
DecrTupleDescRefCount(tupdesc); \
} while (0)
+#define CopyAttrToAlign(att, align) \
+ do { \
+ (align)->attcacheoff = (att)->attcacheoff; \
+ (align)->attlen = (att)->attlen; \
+ (align)->attalign = (att)->attalign; \
+ if ((att)->attlen < 0) \
+ (align)->attstorage = (att)->attstorage; \
+ else \
+ (align)->attbyval = (att)->attbyval; \
+ } while (0)
+
extern bool equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2);
extern uint32 hashTupleDesc(TupleDesc tupdesc);
@@ -147,6 +175,9 @@ extern void TupleDescInitEntryCollation(TupleDesc desc,
AttrNumber attributeNumber,
Oid collationid);
+extern void TupleDescInitAttrAligns(TupleDesc desc);
+extern bool TupleDescAttrAlignsIsConsistent(TupleDesc desc);
+
extern TupleDesc BuildDescForRelation(List *schema);
extern TupleDesc BuildDescFromLists(List *names, List *types, List *typmods,
List *collations);
diff --git a/src/include/access/tupmacs.h b/src/include/access/tupmacs.h
index 65ac1ef3fc..0451dd6bb8 100644
--- a/src/include/access/tupmacs.h
+++ b/src/include/access/tupmacs.h
@@ -47,7 +47,7 @@
#define fetch_att(T,attbyval,attlen) \
( \
- (attbyval) ? \
+ (attbyval && attlen >= 0) ? \
( \
(attlen) == (int) sizeof(Datum) ? \
*((Datum *)(T)) \
diff --git a/src/include/catalog/pg_attribute.h
b/src/include/catalog/pg_attribute.h
index 603392fe81..d3194041ba 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -94,6 +94,11 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP
BKI_ROWTYPE_OID(75,
* no cached value. But when we copy these tuples into a tuple
* descriptor, we may then update attcacheoff in the copies. This speeds
* up the attribute walking process.
+ *
+ * Note: Although the maximum offset encountered in stored tuples is
+ * limited to the max BLCKSZ (2**15), FormData_pg_attribute is used for
+ * all internal tuples as well, so attcacheoff may be larger for those
+ * tuples, and it is therefore not safe to use int16.
*/
int32 attcacheoff BKI_DEFAULT(-1);
--
2.20.1
