On Fri Oct 31, 2025 at 11:52 PM CET, Jacob Champion wrote:
On Fri, Oct 31, 2025 at 2:56 PM Jelte Fennema-Nio <[email protected]> wrote: Yeah, I feel better signing on to a plan like this if we think the breakage is likely to lead to a comprehensive fix for NegotiateProtocolVersion (as opposed to "we greased this thing in 2025 and now this thing in 2026 and now...").
Attached is a new version of the patch that reserves version 3.9999 and _pq_.test_protocol_negotiation and starts requesting those by default (which we should revert before at least PG19rc1 and possibly earlier).
From 51fefaabed1b56aae891d37d91e9891382a9d659 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <[email protected]> Date: Thu, 23 Oct 2025 15:16:05 +0200 Subject: [PATCH v3 1/2] Add test for libpq its default protocol version We did not test libpq its default protocol version. Defaults are important so we should test them. --- .../modules/libpq_pipeline/libpq_pipeline.c | 33 ++++++++++++++++--- 1 file changed, 28 insertions(+), 5 deletions(-) diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c index b3af70fa09b..3af79e03d24 100644 --- a/src/test/modules/libpq_pipeline/libpq_pipeline.c +++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c @@ -1328,7 +1328,7 @@ test_protocol_version(PGconn *conn) int nopts; PQconninfoOption *opts = PQconninfo(conn); int protocol_version; - int max_protocol_version_index; + int max_protocol_version_index = -1; int i; /* @@ -1351,14 +1351,37 @@ test_protocol_version(PGconn *conn) { keywords[i] = opt->keyword; vals[i] = opt->val; + if (strcmp(opt->keyword, "max_protocol_version") == 0) + { + max_protocol_version_index = i; + } + i++; } } - max_protocol_version_index = i; - keywords[i] = "max_protocol_version"; /* value is filled in below */ - i++; - keywords[i] = vals[i] = NULL; + if (max_protocol_version_index == -1) + { + max_protocol_version_index = i; + keywords[i] = "max_protocol_version"; /* value is filled in below */ + i++; + } + + /* + * Test default protocol_version + */ + vals[max_protocol_version_index] = ""; + conn = PQconnectdbParams(keywords, vals, false); + + if (PQstatus(conn) != CONNECTION_OK) + pg_fatal("Connection to database failed: %s", + PQerrorMessage(conn)); + + protocol_version = PQfullProtocolVersion(conn); + if (protocol_version != 30000) + pg_fatal("expected 30000, got %d", protocol_version); + + PQfinish(conn); /* * Test max_protocol_version=3.0 base-commit: ad25744f436ed7809fc754e1a44630b087812fbc -- 2.51.1
From 31851ddff8cb40f732aac0bfac364da61ed9fa30 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <[email protected]> Date: Thu, 23 Oct 2025 15:08:52 +0200 Subject: [PATCH v3 2/2] libpq: Request protocol version 3.9999 to GREASE the ecosystem The main reason that libpq does not request protocol version 3.2 by default is because other proxy/server implementations don't implement the negotiation. This is a bit of a chicken and egg problem: We don't bump the default version that libpq asks, but proxies will only implement version negotiation when their users run into issues. This patch defines 3.999 as an explicitly unsupported protocol version number and _pq_.test_protocol_negotiation as an explicitly unsupported protocol extension. It also starts requesting that version and protocol extension by default. This change to the default will be reverted before we release PG19 release candidates (when exactly to revert before that time is TBD). The intent is to stress test the ecosystem for servers/middleware that don't support protocol version negotiation, so that those servers/middleware can implement the negotiation. This is similar to the GREASE[1] mechanism that TLS has. It's still possible for users to connect to servers that don't support protocol negotiation by using max_protocol_version=3.0 in their connection string. Only the default connection behaviour is impacted. [1]: https://www.rfc-editor.org/rfc/rfc8701.html Author: Jelte Fennema-Nio <[email protected]> --- doc/src/sgml/libpq.sgml | 24 ++++++---- doc/src/sgml/protocol.sgml | 43 +++++++++++++++-- src/include/libpq/pqcomm.h | 7 +++ src/interfaces/libpq/fe-connect.c | 20 ++++---- src/interfaces/libpq/fe-protocol3.c | 46 +++++++++++++++++-- .../modules/libpq_pipeline/libpq_pipeline.c | 6 +-- 6 files changed, 119 insertions(+), 27 deletions(-) diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index e76da383714..6b1e4625d17 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -2213,15 +2213,21 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <listitem> <para> Specifies the protocol version to request from the server. - The default is to use version <literal>3.0</literal> of the - <productname>PostgreSQL</productname> protocol, unless the connection - string specifies a feature that relies on a higher protocol version, - in which case the latest version supported by libpq is used. If the - server does not support the protocol version requested by the client, - the connection is automatically downgraded to a lower minor protocol - version that the server supports. After the connection attempt has - completed you can use <xref linkend="libpq-PQfullProtocolVersion"/> to - find out which exact protocol version was negotiated. + During the PostgreSQL 19 beta period, the default is to use + <literal>3.9999</literal>, a GREASE (Generate Random Extensions And + Sustain Extensibility) value that tests proper protocol negotiation + implementation. If the server does not support the protocol version + requested by the client, the connection is automatically downgraded to + a lower minor protocol version that the server supports. After the + connection attempt has completed you can use + <xref linkend="libpq-PQfullProtocolVersion"/> to find out which exact + protocol version was negotiated. + </para> + + <para> + For servers that don't properly implement protocol version negotiation, + you can set <literal>max_protocol_version=3.0</literal> to connect + successfully. </para> <para> diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 9d755232873..66c540c6029 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -192,10 +192,13 @@ <title>Protocol Versions</title> <para> - The current, latest version of the protocol is version 3.2. However, for - backwards compatibility with old server versions and middleware that don't - support the version negotiation yet, libpq still uses protocol version 3.0 - by default. + The current, latest version of the protocol is version 3.2. During the + PostgreSQL 19 beta period, libpq defaults to requesting protocol version + 3.9999 to test that servers and middleware properly implement protocol + version negotiation. Servers that support negotiation will automatically + downgrade to version 3.2 or 3.0. For servers that don't support + negotiation, users can connect by explicitly setting + <literal>max_protocol_version=3.0</literal> in their connection string. </para> <para> @@ -238,6 +241,20 @@ </thead> <tbody> + <row> + <entry>3.9999</entry> + <entry>-</entry> + <entry>GREASE (Generate Random Extensions And Sustain Extensibility) + version. This version number is intentionally reserved and will never + be implemented. During the PostgreSQL 19 beta period, libpq requests + this version by default to test that servers and middleware properly + implement protocol version negotiation via + <literal>NegotiateProtocolVersion</literal>. Servers should respond + by downgrading to a supported version. This mechanism helps ensure + the ecosystem is ready for future protocol versions. libpq will revert + to defaulting to version 3.2 before the PostgreSQL 19 final release. + </entry> + </row> <row> <entry>3.2</entry> <entry>PostgreSQL 18 and later</entry> @@ -6148,6 +6165,24 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;" </para> </listitem> </varlistentry> + <varlistentry> + <term><literal>_pq_.test_protocol_negotiation</literal></term> + <listitem> + <para> + A reserved protocol extension requested by libpq during the + PostgreSQL 19 beta period to test that servers properly implement + protocol version negotiation. When the client requests the GREASE + protocol version (3.9999), this parameter is automatically + included in the startup packet too. Servers should report it as + unsupported in their <literal>NegotiateProtocolVersion</literal> + response. In GREASE mode the connection will fail if the server + doesn't report this parameter as unsupported, ensuring + comprehensive implementation of protocol negotiation. This + parameter is reserved and will never actually be implemented by a + server. + </para> + </listitem> + </varlistentry> </variablelist> In addition to the above, other parameters may be listed. diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h index f04ca135653..4c20105e998 100644 --- a/src/include/libpq/pqcomm.h +++ b/src/include/libpq/pqcomm.h @@ -91,10 +91,17 @@ is_unixsock_path(const char *path) /* * The earliest and latest frontend/backend protocol version supported. + * + * PG_PROTOCOL_GREASE is an intentionally unsupported protocol version used + * for GREASE (Generate Random Extensions And Sustain Extensibility). This + * helps ensure that servers properly implement protocol version negotiation + * via NegotiateProtocolVersion. Version 3.9999 was chosen to be safely within + * the valid range but unlikely to ever be implemented. */ #define PG_PROTOCOL_EARLIEST PG_PROTOCOL(3,0) #define PG_PROTOCOL_LATEST PG_PROTOCOL(3,2) +#define PG_PROTOCOL_GREASE PG_PROTOCOL(3,9999) typedef uint32 ProtocolVersion; /* FE/BE protocol version number */ diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index a3d12931fff..e8306d567c1 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -2131,15 +2131,12 @@ pqConnectOptions2(PGconn *conn) else { /* - * To not break connecting to older servers/poolers that do not yet - * support NegotiateProtocolVersion, default to the 3.0 protocol at - * least for a while longer. Except when min_protocol_version is set - * to something larger, then we might as well default to the latest. + * Default to the GREASE protocol version to test that servers + * properly implement NegotiateProtocolVersion. The server will + * automatically downgrade to a supported version. This will be + * changed to a supported version before the PG19 release. */ - if (conn->min_pversion > PG_PROTOCOL(3, 0)) - conn->max_pversion = PG_PROTOCOL_LATEST; - else - conn->max_pversion = PG_PROTOCOL(3, 0); + conn->max_pversion = PG_PROTOCOL_GREASE; } if (conn->min_pversion > conn->max_pversion) @@ -4375,6 +4372,13 @@ keep_going: /* We will come back to here until there is goto error_return; } + if (conn->max_pversion == PG_PROTOCOL_GREASE && + conn->pversion == PG_PROTOCOL_GREASE) + { + libpq_append_conn_error(conn, "server incorrectly accepted reserved GREASE protocol version 3.9999 without negotiation"); + goto error_return; + } + /* Almost there now ... */ conn->status = CONNECTION_CHECK_TARGET; goto keep_going; diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index da7a8db68c8..720bc27bb5d 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -1429,6 +1429,8 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) { int their_version; int num; + bool found_test_protocol_negotiation; + bool expect_test_protocol_negotiation; if (pqGetInt(&their_version, 4, conn) != 0) goto eof; @@ -1456,6 +1458,13 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) goto failure; } + /* The GREASE protocol version is intentionally unsupported and reserved */ + if (their_version == PG_PROTOCOL_GREASE) + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server claimed to support reserved GREASE protocol version 3.9999"); + goto failure; + } + if (num < 0) { libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported negative number of unsupported parameters"); @@ -1484,9 +1493,12 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) conn->pversion = their_version; /* - * We don't currently request any protocol extensions, so we don't expect - * the server to reply with any either. + * Check that all expected unsupported parameters are reported by the + * server. */ + found_test_protocol_negotiation = false; + expect_test_protocol_negotiation = (conn->max_pversion == PG_PROTOCOL_GREASE); + for (int i = 0; i < num; i++) { if (pqGets(&conn->workBuffer, conn)) @@ -1498,7 +1510,27 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported unsupported parameter name without a \"%s\" prefix (\"%s\")", "_pq_.", conn->workBuffer.data); goto failure; } - libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported an unsupported parameter that was not requested (\"%s\")", conn->workBuffer.data); + + /* Check if this is the expected test parameter */ + if (expect_test_protocol_negotiation && strcmp(conn->workBuffer.data, "_pq_.test_protocol_negotiation") == 0) + { + found_test_protocol_negotiation = true; + } + else + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported an unsupported parameter that was not requested (\"%s\")", conn->workBuffer.data); + goto failure; + } + } + + /* + * If we requested the GREASE protocol version, the server must report + * _pq_.test_protocol_negotiation as unsupported. This ensures + * comprehensive NegotiateProtocolVersion implementation. + */ + if (expect_test_protocol_negotiation && !found_test_protocol_negotiation) + { + libpq_append_conn_error(conn, "server did not report the unsupported `_pq_.test_protocol_negotiation` parameter in its protocol negotiation message"); goto failure; } @@ -2439,6 +2471,14 @@ build_startup_packet(const PGconn *conn, char *packet, if (conn->client_encoding_initial && conn->client_encoding_initial[0]) ADD_STARTUP_OPTION("client_encoding", conn->client_encoding_initial); + /* + * Add the test protocol negotiation option if we're using the GREASE + * protocol version. This tests that servers properly report unsupported + * protocol options in their NegotiateProtocolVersion response. + */ + if (conn->pversion == PG_PROTOCOL_GREASE) + ADD_STARTUP_OPTION("_pq_.test_protocol_negotiation", ""); + /* Add any environment-driven GUC settings needed */ for (next_eo = options; next_eo->envName; next_eo++) { diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c index 3af79e03d24..2372a77908b 100644 --- a/src/test/modules/libpq_pipeline/libpq_pipeline.c +++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c @@ -1368,7 +1368,7 @@ test_protocol_version(PGconn *conn) } /* - * Test default protocol_version + * Test default protocol_version (GREASE - should negotiate down to 3.2) */ vals[max_protocol_version_index] = ""; conn = PQconnectdbParams(keywords, vals, false); @@ -1378,8 +1378,8 @@ test_protocol_version(PGconn *conn) PQerrorMessage(conn)); protocol_version = PQfullProtocolVersion(conn); - if (protocol_version != 30000) - pg_fatal("expected 30000, got %d", protocol_version); + if (protocol_version != 30002) + pg_fatal("expected 30002, got %d", protocol_version); PQfinish(conn); -- 2.51.1
