Hey, this is my first patch for Postgres, so please let me know if I got something about the contribution policies wrong.
Our main workload does a lot of index-only scans, and in perf profiles of it a good part of the time showed up under ExecScan(). Digging into the code I noticed that IndexScan, IndexOnlyScan and BitmapHeapScan all still go through the generic out-of-line ExecScan(). So each tuple pays for an indirect call to the access method and for the runtime EPQ / qual / projection checks. SeqScan got rid of that with fb9f95502 (and 83a53572a6f made sure SeqNext actually gets inlined). The attached patch does the same for index-only scans. There are now ExecIndexOnlyScan, ...WithQuals, ...WithProject, ...WithQualProject and ...EPQ, the variant is picked in ExecInitIndexOnlyScan(), and IndexOnlyNext() is pg_always_inline. The EPQ variant just calls ExecScan() like ExecSeqScanEPQ does. One thing that cost me bit more time: the runtime key setup (which may call ExecReScan()) has to happen before the pg_assume() calls. If it comes after, gcc can't know that pstate->qual is still non-NULL after the call, keeps the !qual && !projInfo branch of ExecScanExtended() and you end up with a second inlined copy of IndexOnlyNext() in each variant. I added a comment for that. Test setup: table with id int primary key and two more int columns, 5M rows, vacuumed so it's all-visible. enable_seqscan and enable_bitmapscan off, jit off, no parallel. gcc 12, -O2, i7-13700H. instructions latency SELECT count(*) FROM t WHERE id BETWEEN 1 AND 1000000 -5.2% -5 to -11% same, plus AND id % 3 = 0 -4.8% -3 to -5% SELECT id + 1 FROM t WHERE id BETWEEN 1 AND 1000000 OFFSET 2000000 -6.3% -4 to -10% Instructions are from callgrind (single user mode), so those are kinda deterministic. Latency is median pgbench latency over 5 interleaved rounds, server pinned to one P-core. It's a range because I ran everything a second time with -falign-functions=64 and the wall clock results moved by several percent just from the the layout change. So I wouldn't put too much weight on the exact latency numbers, but it was faster in all cases. tbh the code gets bigger: the per-tuple function is 576-858 bytes depending on the variant, and the IOS functions together go from ~0.6kB to ~3.5kB. Tests: regress and isolation pass with cassert, and CI on my github fork is green on all platforms. There is no existing test that runs an index-only scan under EPQ though. I checked that path manually with a correlated subquery in an UPDATE that has to wait for a concurrent update. I can turn that into an isolation test if people think it's worth it. any thoughts? Best, Matthias
From 8ce54e0e21c3c669c0a4834e8cdb7fb42506f93a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Matthias=20D=C3=B6tsch?= <[email protected]> Date: Tue, 13 Oct 2026 13:12:12 +0200 Subject: [PATCH v2] Specialize ExecIndexOnlyScan() like ExecSeqScan() ExecIndexOnlyScan() went through the out-of-line ExecScan(), which calls IndexOnlyNext() through a function pointer for every tuple and checks for EPQ, qual and projection at runtime. As fb9f95502 and 83a53572a6f did for SeqScan, add ExecIndexOnlyScan*() variants that pass const-NULLs to the inline ExecScanExtended() for whatever the node doesn't need, and choose one in ExecInitIndexOnlyScan(). The EPQ case keeps using ExecScan(). Mark IndexOnlyNext() as pg_always_inline so that it gets inlined into each variant. The runtime-key setup has to come before the pg_assume() calls, because the compiler can't rely on them across the ExecReScan() call. It would then keep the code for the no-qual, no-projection case too, with a second copy of IndexOnlyNext(). In a 1M-row index-only range scan (gcc 12, -O2), executed instructions drop by 5.2% without qual or projection, by 4.8% with a qual and by 6.3% with a projection (callgrind). Wall-clock time for the same queries drops by 3-11%, but that varies by several percent with code layout alone. --- src/backend/executor/nodeIndexonlyscan.c | 145 ++++++++++++++++++++++- 1 file changed, 140 insertions(+), 5 deletions(-) diff --git a/src/backend/executor/nodeIndexonlyscan.c b/src/backend/executor/nodeIndexonlyscan.c index b1009027ac8..20d798e3e02 100644 --- a/src/backend/executor/nodeIndexonlyscan.c +++ b/src/backend/executor/nodeIndexonlyscan.c @@ -33,6 +33,7 @@ #include "access/genam.h" #include "access/relscan.h" #include "access/tableam.h" +#include "executor/execScan.h" #include "executor/executor.h" #include "executor/instrument.h" #include "executor/nodeIndexonlyscan.h" @@ -50,7 +51,7 @@ static TupleTableSlot *IndexOnlyNext(IndexOnlyScanState *node); * Retrieve a tuple from the IndexOnlyScan node's index. * ---------------------------------------------------------------- */ -static TupleTableSlot * +static pg_always_inline TupleTableSlot * IndexOnlyNext(IndexOnlyScanState *node) { EState *estate; @@ -164,8 +165,28 @@ IndexOnlyRecheck(IndexOnlyScanState *node, TupleTableSlot *slot) return false; /* keep compiler quiet */ } +/* + * IndexOnlyScanSetupRuntimeKeys -- compute runtime keys if not done yet + */ +static pg_always_inline void +IndexOnlyScanSetupRuntimeKeys(IndexOnlyScanState *node) +{ + /* + * If we have runtime keys and they've not already been set up, do it now. + */ + if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady) + ExecReScan((PlanState *) node); +} + /* ---------------------------------------------------------------- * ExecIndexOnlyScan(node) + * + * Returns the next qualifying tuple from the index-only scan. This + * variant is used when there is no es_epq_active, no qual and no + * projection. As with the ExecSeqScan() variants, passing const-NULLs + * for these to ExecScanExtended() allows the compiler to eliminate the + * additional code that would ordinarily be required for the evaluation + * of these, and to inline IndexOnlyNext(). * ---------------------------------------------------------------- */ static TupleTableSlot * @@ -173,11 +194,104 @@ ExecIndexOnlyScan(PlanState *pstate) { IndexOnlyScanState *node = castNode(IndexOnlyScanState, pstate); + IndexOnlyScanSetupRuntimeKeys(node); + + Assert(pstate->state->es_epq_active == NULL); + Assert(pstate->qual == NULL); + Assert(pstate->ps_ProjInfo == NULL); + + return ExecScanExtended(&node->ss, + (ExecScanAccessMtd) IndexOnlyNext, + (ExecScanRecheckMtd) IndexOnlyRecheck, + NULL, + NULL, + NULL); +} + +/* + * Variant of ExecIndexOnlyScan() but when qual evaluation is required. + */ +static TupleTableSlot * +ExecIndexOnlyScanWithQual(PlanState *pstate) +{ + IndexOnlyScanState *node = castNode(IndexOnlyScanState, pstate); + + IndexOnlyScanSetupRuntimeKeys(node); + /* - * If we have runtime keys and they've not already been set up, do it now. + * Use pg_assume() for != NULL tests to make the compiler realize no + * runtime check for the field is needed in ExecScanExtended(). This has + * to come after IndexOnlyScanSetupRuntimeKeys(), as the compiler can't + * know that ExecReScan() leaves the field alone. */ - if (node->ioss_NumRuntimeKeys != 0 && !node->ioss_RuntimeKeysReady) - ExecReScan((PlanState *) node); + Assert(pstate->state->es_epq_active == NULL); + pg_assume(pstate->qual != NULL); + Assert(pstate->ps_ProjInfo == NULL); + + return ExecScanExtended(&node->ss, + (ExecScanAccessMtd) IndexOnlyNext, + (ExecScanRecheckMtd) IndexOnlyRecheck, + NULL, + pstate->qual, + NULL); +} + +/* + * Variant of ExecIndexOnlyScan() but when projection is required. + */ +static TupleTableSlot * +ExecIndexOnlyScanWithProject(PlanState *pstate) +{ + IndexOnlyScanState *node = castNode(IndexOnlyScanState, pstate); + + IndexOnlyScanSetupRuntimeKeys(node); + + Assert(pstate->state->es_epq_active == NULL); + Assert(pstate->qual == NULL); + pg_assume(pstate->ps_ProjInfo != NULL); + + return ExecScanExtended(&node->ss, + (ExecScanAccessMtd) IndexOnlyNext, + (ExecScanRecheckMtd) IndexOnlyRecheck, + NULL, + NULL, + pstate->ps_ProjInfo); +} + +/* + * Variant of ExecIndexOnlyScan() but when qual evaluation and projection are + * required. + */ +static TupleTableSlot * +ExecIndexOnlyScanWithQualProject(PlanState *pstate) +{ + IndexOnlyScanState *node = castNode(IndexOnlyScanState, pstate); + + IndexOnlyScanSetupRuntimeKeys(node); + + Assert(pstate->state->es_epq_active == NULL); + pg_assume(pstate->qual != NULL); + pg_assume(pstate->ps_ProjInfo != NULL); + + return ExecScanExtended(&node->ss, + (ExecScanAccessMtd) IndexOnlyNext, + (ExecScanRecheckMtd) IndexOnlyRecheck, + NULL, + pstate->qual, + pstate->ps_ProjInfo); +} + +/* + * Variant of ExecIndexOnlyScan() for when EPQ evaluation is required. As + * with ExecSeqScanEPQ(), we don't bother adding variants of this for + * with/without qual and projection, and just use the out-of-line ExecScan(). + */ +static TupleTableSlot * +ExecIndexOnlyScanEPQ(PlanState *pstate) +{ + IndexOnlyScanState *node = castNode(IndexOnlyScanState, pstate); + + IndexOnlyScanSetupRuntimeKeys(node); return ExecScan(&node->ss, (ExecScanAccessMtd) IndexOnlyNext, @@ -368,7 +482,6 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) indexstate = makeNode(IndexOnlyScanState); indexstate->ss.ps.plan = (Plan *) node; indexstate->ss.ps.state = estate; - indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScan; /* * Miscellaneous initialization @@ -416,6 +529,28 @@ ExecInitIndexOnlyScan(IndexOnlyScan *node, EState *estate, int eflags) indexstate->recheckqual = ExecInitQual(node->recheckqual, (PlanState *) indexstate); + /* + * Assign ExecProcNode based on whether EvalPlanQual is in use, and on the + * presence of qual and projection. Each ExecIndexOnlyScan*() variant is + * optimized for the specific combination of these conditions. + */ + if (estate->es_epq_active != NULL) + indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScanEPQ; + else if (indexstate->ss.ps.qual == NULL) + { + if (indexstate->ss.ps.ps_ProjInfo == NULL) + indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScan; + else + indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScanWithProject; + } + else + { + if (indexstate->ss.ps.ps_ProjInfo == NULL) + indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScanWithQual; + else + indexstate->ss.ps.ExecProcNode = ExecIndexOnlyScanWithQualProject; + } + /* * If we are just doing EXPLAIN (ie, aren't going to run the plan), stop * here. This allows an index-advisor plugin to EXPLAIN a plan containing -- 2.53.0
