Hi all,
I am very glad that we have finally completed the merge of Postgres 15 and
16, and everyone is more
than welcome to review it.Here are some of the new features:
Key features and changes, sorted by importance:
1.
MERGE command
-
SQL-standard MERGE for INSERT/UPDATE/DELETE in a single statement.
-
ruleutils decompilation, PlaceHolderVar support in actions
-
Fixes for partitioned tables, cross-partition triggers, RLS,
-
EvalPlanQual, self-modified tuples, pg_stat_statements
2.
Logical replication enhancements
-
Built-in support for prepared transactions (2PC)
-
Enable two-phase via replication protocol
-
Streaming changes after speculative aborts
-
New pg_create_subscription predefined role
-
Parallel apply workers (PG 16)
-
Row/column filtering, schema-level publication (PG 15)
-
Critical fix for data loss in logical replication
-
Slot invalidation, snapshot, and memory leak fixes
3.
SQL/JSON
-
IS JSON predicate
-
Standard JSON constructor functions (json_array, json_object,
json_scalar, json_serialize)
-
Auto-generated query jumbling via gen_node_support.pl
-
Fix json_array() subquery double transformation
4.
WAL and storage subsystem
-
LZ4 compression for full-page writes (FPW)
-
Custom WAL resource managers
-
Skip WAL recycling/preallocation during archive recovery
-
XLOG_FPI_FOR_HINT honors full_page_writes setting
-
Direct I/O support on macOS
-
Buffer manager infrastructure refactor for faster relation extension
-
New smgrzeroextend(), FileZero(), FileFallocate()
-
Remove HeapBitmapScan skip_fetch optimization (correctness issue)
-
WAL/Recovery separation: xlogrecovery.c, xlogprefetcher.c (PG 15)
-
Backup sink architecture (PG 15)
5.
Query optimizer / executor
Performance:
-
Hash table acceleration for NOT IN(values)
-
Datum sorts for single-column sorts
-
Result Cache renamed to Memoize with ongoing fixes
-
Parallel Hash Full Join
-
Reduced planning cost for deeply-nested views
-
Fewer pallocs when building partition bounds
-
Outer join optimization enhancement (PG 16)
-
Symmetric hash join optimization (PG 16)
Critical fixes:
-
Planner failure to identify multiple hashable ScalarArrayOpExprs
-
Consistent whole-row Var construction in parsing and planning
-
WindowAgg evaluation crash
-
WITH RECURSIVE UNION assert failure
-
ORDER BY / DISTINCT aggregates with FILTER
-
setrefs.c missing expression processing on prune steps
-
Parallel Hash Join extreme skew detection
6.
pgstat monitoring enhancements
pgstat shared memory rewrite (PG 15):
-
Stats collector process removed; stats now in shared memory
-
Monolithic pgstat.c split into 14 files under utils/activity/
pg_stat_io (PG 16 new view):
-
Detailed I/O statistics: shared buffer hits, I/O times, writeback
pg_stat_statements:
-
JIT counters, temp file block I/O timing
-
Utility query string normalization, MERGE support
-
32-bit integer overflow fix
Other:
-
pg_stat_wal time accumulation as instr_time
-
Macro-generated pg_stat_get*() functions for tables and databases
-
Enhanced pg_stat_reset_single_table_counters
-
SP-GiST index scans counted in pg_stat
-
New test helpers: pg_stat_force_next_flush(), pg_stat_have_stats()
7.
libpq and client protocol
-
New PQsendFlushRequest
-
Pipeline mode state machine fix
-
SASL code refactored to generic interface
-
Escape function fixes for invalid encoding data (security)
-
PQescapeLiteral()/PQescapeIdentifier() length handling fix
-
New pg_encoding_set_invalid()
-
Build-time check that libpq doesn't call exit()/abort()
8.
Security and privilege model
-
CREATEROLE privilege restrictions (major security improvement)
-
New GUCs: createrole_self_grant, reserved_connections
-
New predefined roles: pg_maintain, pg_create_subscription
-
Non-superuser predefined roles for vacuum/analyze
-
Revoke PUBLIC CREATE from public schema (now owned by
pg_database_owner)
-
Security invoker views (SECURITY INVOKER)
-
session_authorization and role interaction fixes
-
scram_SaltedPassword() integer overflow fix
9.
pg_dump / pg_upgrade / pg_basebackup toolchain
pg_dump:
-
Generic compression API, zstd support, LZ4 frame-only format
pg_basebackup:
-
Extended compression options, server-side compression with -Fp
-
Client-side LZ4 decompression, parallel zstd compression
pg_upgrade:
-
Fix unintentional 'NULL' string literal
-
Fix XMLSERIALIZE(NO INDENT) cross-version upgrade
-
pg_dumpall handling of dangling OIDs in pg_auth_members
10.
Partitioned table improvements
-
Self-referencing FKs in partitioned tables
-
Detach partition with top-level FK fix
-
Reset relhassubclass on ATTACH
-
Trigger rename consistency, preserve firing state on clone
-
Disallow partitionwise join/grouping on collation mismatch
11.
ICU and collation
-
Build ICU support by default
-
initdb uses uloc_getDefault()
-
CREATE DATABASE LOCALE applies to all collation providers
12.
Node support function auto-generation (FIXME)
-
gen_node_support.pl auto-generates
copyfuncs/equalfuncs/outfuncs/readfuncs
-
Query jumbling code auto-generation
-
NodeTag ABI stability check
13.
Global renames and structural changes
-
RelFileNode -> RelFileLocator (pervasive rename across hundreds of
files)
-
GUC system split: guc.c -> guc.c + guc_tables.c + guc_funcs.c
-
varatt.h separated from postgres.h
-
Test framework: PostgresNode.pm -> PostgreSQL::Test::Cluster
14.
SQL types and functions
-
Numeric scale allows negative or greater than precision
-
New ANY_VALUE aggregate function
-
unnest(multirange), range_agg with multirange inputs
-
pg_size_pretty/pg_size_bytes support petabytes
-
ALTER TABLE SET ACCESS METHOD
-
SYSTEM_USER function
-
numeric_mul() overflow fix
15.
psql improvements
-
PSQL_WATCH_PAGER for \watch command
-
New \drg command for role grants
-
\copy from sends data in larger chunks
-
pg_waldump --rmgr multiple specification
16.
Critical data corruption and crash fixes
-
Fix data corruption when relation truncation fails
-
Fail instead of corrupting page header on non-LP_NORMAL TID update
-
Fix unsafe BufferDescriptors access
-
GB18030 SIGSEGV from out-of-bounds read
-
Snowball stemmer null pointer dereference after OOM
-
Rare standby assertion failure on primary restart
-
catcache invalidation during list entry construction
17.
Vacuum / Autovacuum
-
Parallel VACUUM (PG 16, vacuumparallel.c new file)
-
ON COMMIT DELETE ROWS avoids ERROR after relhassubclass=f
-
Use WaitLatch() instead of pg_usleep() at end-of-vacuum truncation
-
Prevent numeric overflow in parallel numeric aggregates
-
ANALYZE preserves relhasindex for partitioned tables
Currently, there are still some features that are not fully functional yet.
We have left some "FIXME" tags,
which will be fixed one after another in the coming weeks. We will create a
feature for each "FIXME" in
the project board, and anyone is welcome to help fix them:
https://github.com/orgs/apache/projects/497
Since the amount of code in this merge is massive, handling conflicts will
be very troublesome if new code
is merged into the main branch. Therefore, please temporarily pause merging
any code for the next few days.
Below is the list of contributors to this merge.
Jinbao Chen <[email protected]>
liushengsong <[email protected]>
reshke <[email protected]>
Hao Wu <[email protected]>
Jianghua.yjh <[email protected]>
Dianjin Wang <[email protected]>
