On Mon, 9 Sept 2024 at 17:58, Robert Haas <robertmh...@gmail.com> wrote: > > On Fri, Aug 16, 2024 at 11:29 AM Robert Haas <robertmh...@gmail.com> wrote: > > > I'll split this patch like that, to make it easier to compare and merge > > > with Jelte's corresponding patches. > > > > That sounds great. IMHO, comparing and merging the patches is the next > > step here and would be great to see. > > Heikki, do you have any update on this work?
My patchset in the other protocol thread needed a rebase. So I took that as an opportunity to rebase this patchset on top of it, because this seems to be the protocol change that we can most easily push over the finish line. 1. I changed the last patch from Heikki to only contain the changes for the cancel lengthening. The general protocol change related things I merged with mine (I only kept some error handling and docs). 2. I also removed the length field in the new BackendKey definition, eventhough I asked for that addition previously. I agree with Heikki that it's actually easier to parse and create without, because you can use the same code for both versions. 3. I made our timingsafe_bcmp implementation call into OpenSSL's CRYPTO_memcmp. One open question on the last patch is: Document what the maximum size of the cancel key is that the client can expect? I think Jacob might have some ideas on that.
From 0168dd6d463eb989d2e944c8acccf7cc620f5db1 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <postg...@jeltef.nl> Date: Mon, 23 Dec 2024 16:49:10 +0100 Subject: [PATCH v4 1/7] libpq: Add PQfullProtocolVersion to exports.txt This is necessary to be able to actually use the function on Windows. --- src/interfaces/libpq/exports.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt index 9b789cbec0b..d5143766858 100644 --- a/src/interfaces/libpq/exports.txt +++ b/src/interfaces/libpq/exports.txt @@ -209,3 +209,4 @@ PQservice 206 PQsetAuthDataHook 207 PQgetAuthDataHook 208 PQdefaultAuthDataHook 209 +PQfullProtocolVersion 210 base-commit: bba2fbc6238b2a0a7f348fbbb5c31ffa7623bc39 -- 2.43.0
From 804da9044f08c62db598b42296b973284ca5b32b Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <jelte.fenn...@microsoft.com> Date: Wed, 5 Jun 2024 11:40:04 +0200 Subject: [PATCH v4 2/7] libpq: Trace NegotiateProtocolVersion correctly This changes the libpq tracing code to correctly trace the NegotiateProtocolVersion message. Previously it wasn't important that tracing of the NegotiateProtocolVersion message worked correctly, because in practice libpq never received it. Now that we are planning to introduce protocol changes in future commits it starts to become more useful for testing/debugging. --- src/interfaces/libpq/fe-trace.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/interfaces/libpq/fe-trace.c b/src/interfaces/libpq/fe-trace.c index 641e70f321c..a45f0d85587 100644 --- a/src/interfaces/libpq/fe-trace.c +++ b/src/interfaces/libpq/fe-trace.c @@ -578,9 +578,15 @@ pqTraceOutput_RowDescription(FILE *f, const char *message, int *cursor, bool reg static void pqTraceOutput_NegotiateProtocolVersion(FILE *f, const char *message, int *cursor) { + int nparams; + fprintf(f, "NegotiateProtocolVersion\t"); pqTraceOutputInt32(f, message, cursor, false); - pqTraceOutputInt32(f, message, cursor, false); + nparams = pqTraceOutputInt32(f, message, cursor, false); + for (int i = 0; i < nparams; i++) + { + pqTraceOutputString(f, message, cursor, false); + } } static void -- 2.43.0
From d4264c2487cb7c84efeaee09cbe52685487c0b88 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <postg...@jeltef.nl> Date: Wed, 14 Aug 2024 18:25:16 +0200 Subject: [PATCH v4 3/7] libpq: Handle NegotiateProtocolVersion message differently Previously libpq would always error when the server returns a NegotiateProtocolVersion message. This was fine because libpq only supported a single protocol version and did not support any protocol parameters. But we now that we're discussing protocol changes we need to change this behaviour, and introduce a fallback mechanism when connecting to an older server. This patch modifies the client side checks to allow a range of supported protocol versions, instead of only allowing the exact version that was requested. Currently this "range" only contains the 3.0 version, but in a future commit we'll change this. In addition it changes the errors for protocol to say that they error because we did not request any parameters, not because the server did not support some of them. In a future commit more infrastructure for protocol parameters will be added, so that these checks can also succeed when receiving unsupported parameters back. Note that at the moment this change does not have any behavioural effect, because libpq will only request version 3.0 and will never send protocol parameters. Which means that the client never receives a NegotiateProtocolVersion message from the server. --- src/interfaces/libpq/fe-connect.c | 3 +- src/interfaces/libpq/fe-protocol3.c | 80 +++++++++++++++++------------ 2 files changed, 49 insertions(+), 34 deletions(-) diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index d5051f5e820..281710b0616 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -4091,9 +4091,10 @@ keep_going: /* We will come back to here until there is libpq_append_conn_error(conn, "received invalid protocol negotiation message"); goto error_return; } + /* OK, we read the message; mark data consumed */ pqParseDone(conn, conn->inCursor); - goto error_return; + goto keep_going; } /* It is an authentication request. */ diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index 95dd456f076..68a94281eb4 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -1399,58 +1399,72 @@ reportErrorPosition(PQExpBuffer msg, const char *query, int loc, int encoding) /* - * Attempt to read a NegotiateProtocolVersion message. + * Attempt to read a NegotiateProtocolVersion message. Sets conn->pversion + * to the version that's negotiated by the server. * Entry: 'v' message type and length have already been consumed. * Exit: returns 0 if successfully consumed message. - * returns EOF if not enough data. + * returns 1 on failure. The error message is filled in. */ int pqGetNegotiateProtocolVersion3(PGconn *conn) { - int tmp; - ProtocolVersion their_version; + int their_version; int num; - PQExpBufferData buf; - if (pqGetInt(&tmp, 4, conn) != 0) - return EOF; - their_version = tmp; + if (pqGetInt(&their_version, 4, conn) != 0) + goto eof; if (pqGetInt(&num, 4, conn) != 0) - return EOF; + goto eof; - initPQExpBuffer(&buf); - for (int i = 0; i < num; i++) + if (their_version > conn->pversion) { - if (pqGets(&conn->workBuffer, conn)) - { - termPQExpBuffer(&buf); - return EOF; - } - if (buf.len > 0) - appendPQExpBufferChar(&buf, ' '); - appendPQExpBufferStr(&buf, conn->workBuffer.data); + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server requests downgrade to higher-numbered version"); + goto failure; } - if (their_version < conn->pversion) - libpq_append_conn_error(conn, "protocol version not supported by server: client uses %u.%u, server supports up to %u.%u", - PG_PROTOCOL_MAJOR(conn->pversion), PG_PROTOCOL_MINOR(conn->pversion), - PG_PROTOCOL_MAJOR(their_version), PG_PROTOCOL_MINOR(their_version)); - if (num > 0) + if (their_version < PG_PROTOCOL(3, 0)) + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server requests downgrade to pre-3.0 protocol version"); + goto failure; + } + + if (num < 0) { - appendPQExpBuffer(&conn->errorMessage, - libpq_ngettext("protocol extension not supported by server: %s", - "protocol extensions not supported by server: %s", num), - buf.data); - appendPQExpBufferChar(&conn->errorMessage, '\n'); + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported negative number of unsupported parameters"); + goto failure; } - /* neither -- server shouldn't have sent it */ - if (!(their_version < conn->pversion) && !(num > 0)) - libpq_append_conn_error(conn, "invalid %s message", "NegotiateProtocolVersion"); + if (their_version == conn->pversion && num == 0) + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server negotiated but asks for no changes"); + goto failure; + } + + conn->pversion = their_version; + for (int i = 0; i < num; i++) + { + if (pqGets(&conn->workBuffer, conn)) + { + goto eof; + } + if (strncmp(conn->workBuffer.data, "_pq_.", 5) != 0) + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported unsupported parameter name without a _pq_. prefix (\"%s\")", conn->workBuffer.data); + goto failure; + } + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported an unsupported parameter that was not requested (\"%s\")", conn->workBuffer.data); + goto failure; + } - termPQExpBuffer(&buf); return 0; + +eof: + libpq_append_conn_error(conn, "received invalid protocol negotation message: message too short"); +failure: + conn->asyncStatus = PGASYNC_READY; + pqSaveErrorResult(conn); + return 1; } -- 2.43.0
From 007e89278b3a5ab4c407c86363210205e0eb2b40 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <postg...@jeltef.nl> Date: Wed, 14 Aug 2024 18:25:16 +0200 Subject: [PATCH v4 4/7] libpq: Add min/max_protocol_version connection options All officially supported version of the PostgreSQL server send the NegotiateProtocolVersion message when an unsupported minor protocol version is requested by a client. But many other applications that implement the PostgreSQL protocol (connection poolers, or other databases) do not, and the same is true for many unspported PostgreSQL server versions. Connecting to such other applications thus fails if a client requests a protocol version different than 3.0. This patch adds a max_protocol_version connection option to libpq that specifies the protocol version that libpq should request from the server. Currently all allowed values result in the use of 3.0, but that will be changed in a future commit that bumps the protocol version. Even after that version bump the default will likely stay 3.0 for the time being. Once more of the ecosystem supports the NegotiateProtocolVersion message we might want to change the default to the latest minor version. We also add the similar min_protocol_version connection option, to allow a client to specify that connecting should fail if a lower protocol version is attempted by the server. This can be used to ensure certain protocol features are in used, which can be particularly useful if those features impact security. --- doc/src/sgml/libpq.sgml | 71 ++++++++++++++++++- src/interfaces/libpq/fe-connect.c | 104 +++++++++++++++++++++++++++- src/interfaces/libpq/fe-protocol3.c | 12 ++++ src/interfaces/libpq/libpq-int.h | 4 ++ 4 files changed, 189 insertions(+), 2 deletions(-) diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index ddb3596df83..f249bbd9263 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -2144,6 +2144,56 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname </listitem> </varlistentry> + <varlistentry id="libpq-connect-min-protocol-version" xreflabel="min_protocol_version"> + <term><literal>min_protocol_version</literal></term> + <listitem> + <para> + Specifies the minimum protocol version to allow for the connection. + The default is to allow any version of the + <productname>PostgreSQL</productname> protocol supported by libpq, + which currently means <literal>3.0</literal>. If the server + does not support at least this protocol version the connection will be + closed. + </para> + + <para> + The current supported values are + <literal>3.0</literal> + and <literal>latest</literal>. The <literal>latest</literal> value is + equivalent to the latest protocol version that is supported by the used + libpq version, which currently is <literal>3.2</literal>, but this will + change in future libpq releases. + </para> + </listitem> + </varlistentry> + + <varlistentry id="libpq-connect-max-protocol-version" xreflabel="max_protocol_version"> + <term><literal>max_protocol_version</literal></term> + <listitem> + <para> + Specifies the protocol version to request from the server. + The default is to use version <literal>3.0</literal> of the + <productname>PostgreSQL</productname> protocol, unless the connection + string specifies a feature that relies on a higher protocol version, in + that case the latest version supported by libpq is used. If the server + does not support the requested protocol version of the client the + connection will be automatically downgraded to a lower minor protocol + version, which the server does support. After the connection attempt has + completed you can use <xref linkend="libpq-PQprotocolVersion"/> to find + out which exact protocol version was negotiated. + </para> + + <para> + The current supported values are + <literal>3.0</literal> + and <literal>latest</literal>. The <literal>latest</literal> value is + equivalent to the latest protocol version that is supported by the used + libpq version, which currently is <literal>3.0</literal>, but this will + change in future libpq releases. + </para> + </listitem> + </varlistentry> + <varlistentry id="libpq-connect-ssl-max-protocol-version" xreflabel="ssl_max_protocol_version"> <term><literal>ssl_max_protocol_version</literal></term> <listitem> @@ -2482,7 +2532,6 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname </para> </listitem> </varlistentry> - </variablelist> </para> </sect2> @@ -9329,6 +9378,26 @@ myEventProc(PGEventId evtId, void *evtInfo, void *passThrough) linkend="libpq-connect-load-balance-hosts"/> connection parameter. </para> </listitem> + + <listitem> + <para> + <indexterm> + <primary><envar>PGMINPROTOCOLVERSION</envar></primary> + </indexterm> + <envar>PGMINPROTOCOLVERSION</envar> behaves the same as the <xref + linkend="libpq-connect-min-protocol-version"/> connection parameter. + </para> + </listitem> + + <listitem> + <para> + <indexterm> + <primary><envar>PGMAXPROTOCOLVERSION</envar></primary> + </indexterm> + <envar>PGMAXPROTOCOLVERSION</envar> behaves the same as the <xref + linkend="libpq-connect-max-protocol-version"/> connection parameter. + </para> + </listitem> </itemizedlist> </para> diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 281710b0616..7f08b143fa7 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -325,6 +325,16 @@ static const internalPQconninfoOption PQconninfoOptions[] = { "Require-Auth", "", 14, /* sizeof("scram-sha-256") == 14 */ offsetof(struct pg_conn, require_auth)}, + {"min_protocol_version", "PGMINPROTOCOLVERSION", + NULL, NULL, + "Min-Protocol-Version", "", 6, /* sizeof("latest") = 6 */ + offsetof(struct pg_conn, min_protocol_version)}, + + {"max_protocol_version", "PGMAXPROTOCOLVERSION", + NULL, NULL, + "Max-Protocol-Version", "", 6, /* sizeof("latest") = 6 */ + offsetof(struct pg_conn, max_protocol_version)}, + {"ssl_min_protocol_version", "PGSSLMINPROTOCOLVERSION", "TLSv1.2", NULL, "SSL-Minimum-Protocol-Version", "", 8, /* sizeof("TLSv1.x") == 8 */ offsetof(struct pg_conn, ssl_min_protocol_version)}, @@ -483,6 +493,7 @@ static void pgpassfileWarning(PGconn *conn); static void default_threadlock(int acquire); static bool sslVerifyProtocolVersion(const char *version); static bool sslVerifyProtocolRange(const char *min, const char *max); +static bool pqParseProtocolVersion(const char *value, ProtocolVersion *result, PGconn *conn, const char *context); /* global variable because fe-auth.c needs to access it */ @@ -2080,6 +2091,42 @@ pqConnectOptions2(PGconn *conn) } } + if (conn->min_protocol_version) + { + if (!pqParseProtocolVersion(conn->min_protocol_version, &conn->min_pversion, conn, "min_protocol_version")) + return false; + } + else + { + conn->min_pversion = PG_PROTOCOL_EARLIEST; + } + + if (conn->max_protocol_version) + { + if (!pqParseProtocolVersion(conn->max_protocol_version, &conn->max_pversion, conn, "max_protocol_version")) + return false; + } + else + { + /* + * To not break connecting to older servers/poolers that do not yet + * support NegotiateProtocolVersion, default to the 3.0 protocol at + * least for a while longer. Except when min_protocol_version is set + * to something larger, then we might as well default to the latest. + */ + if (conn->min_pversion > PG_PROTOCOL(3, 0)) + conn->max_pversion = PG_PROTOCOL_LATEST; + else + conn->max_pversion = PG_PROTOCOL(3, 0); + } + + if (conn->min_pversion > conn->max_pversion) + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "min_protocol_version is greater than max_protocol_version"); + return false; + } + /* * Resolve special "auto" client_encoding from the locale */ @@ -3083,7 +3130,7 @@ keep_going: /* We will come back to here until there is * must persist across individual connection attempts, but we must * reset them when we start to consider a new server. */ - conn->pversion = PG_PROTOCOL(3, 0); + conn->pversion = conn->max_pversion; conn->send_appname = true; conn->failed_enc_methods = 0; conn->current_enc_method = 0; @@ -4094,6 +4141,7 @@ keep_going: /* We will come back to here until there is /* OK, we read the message; mark data consumed */ pqParseDone(conn, conn->inCursor); + goto keep_going; } @@ -8149,6 +8197,60 @@ error: return false; } +/* + * Parse and try to interpret "value" as a ProtocolVersion value, and if successful, + * store it in *result. + */ +static bool +pqParseProtocolVersion(const char *value, ProtocolVersion *result, PGconn *conn, + const char *context) +{ + char *end; + int major; + int minor; + ProtocolVersion version; + + if (strcmp(value, "latest") == 0) + { + *result = PG_PROTOCOL_LATEST; + return true; + } + + major = strtol(value, &end, 10); + if (*end != '.') + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + context, + value); + return false; + } + + minor = strtol(&end[1], &end, 10); + if (*end != '\0') + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + context, + value); + return false; + } + + version = PG_PROTOCOL(major, minor); + if (version > PG_PROTOCOL_LATEST || + version < PG_PROTOCOL_EARLIEST) + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + context, + value); + return false; + } + + *result = version; + return true; +} + /* * To keep the API consistent, the locking stubs are always provided, even * if they are not required. diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index 68a94281eb4..221627bdea6 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -1441,6 +1441,18 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) goto failure; } + if (their_version < conn->min_pversion) + { + libpq_append_conn_error(conn, "server only supports protocol version %d.%d, but min_protocol_version was set to %d.%d", + PG_PROTOCOL_MAJOR(conn->pversion), + PG_PROTOCOL_MINOR(conn->pversion), + PG_PROTOCOL_MAJOR(conn->min_pversion), + PG_PROTOCOL_MINOR(conn->min_pversion)); + + goto failure; + } + + conn->pversion = their_version; for (int i = 0; i < num; i++) { diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h index f36f7f19d58..c016408338a 100644 --- a/src/interfaces/libpq/libpq-int.h +++ b/src/interfaces/libpq/libpq-int.h @@ -425,6 +425,8 @@ struct pg_conn char *gsslib; /* What GSS library to use ("gssapi" or * "sspi") */ char *gssdelegation; /* Try to delegate GSS credentials? (0 or 1) */ + char *min_protocol_version; /* minimum used protocol version */ + char *max_protocol_version; /* maximum used protocol version */ char *ssl_min_protocol_version; /* minimum TLS protocol version */ char *ssl_max_protocol_version; /* maximum TLS protocol version */ char *target_session_attrs; /* desired session properties */ @@ -545,6 +547,8 @@ struct pg_conn void *scram_client_key_binary; /* binary SCRAM client key */ size_t scram_server_key_len; void *scram_server_key_binary; /* binary SCRAM server key */ + ProtocolVersion min_pversion; /* protocol version to request */ + ProtocolVersion max_pversion; /* protocol version to request */ /* Miscellaneous stuff */ int be_pid; /* PID of backend --- needed for cancels */ -- 2.43.0
From fd49b5ec9bad0e50ebea84b48a6d7eadacabe450 Mon Sep 17 00:00:00 2001 From: Jelte Fennema-Nio <postg...@jeltef.nl> Date: Wed, 14 Aug 2024 18:25:16 +0200 Subject: [PATCH v4 5/7] Bump protocol version to 3.2 In preparation of new additions to the protocol in a follow up commit this bumps the minor protocol version number. Instead of bumping the version number to 3.1, which would be the next minor version, we skip that one and bump straight to 3.2. The reason for this is that many PgBouncer releases have, due to an off-by-one bug, reported 3.1 as supported[1]. These versions would interpret 3.1 as equivalent to 3.0. So if we would now add extra messages to the 3.1 protocol, clients would succeed to connect to PgBouncer, but would then cause connection failures when sending such new messages. So instead of bumping to 3.1, we bump the protocol version to 3.2, for which these buggy PgBouncer releases will correctly close the connection at the startup packet. It's a bit strange to skip a version number due to a bug in a third-party application, but PgBouncer is used widely enough that it seems worth it to not cause user confusion when connecting to recent versions of it. Especially since skipping a single minor version number in the protocol versioning doesn't really cost us anything. So, while this is not the most theoretically sound decission, it is the most pragmatic one. [1]: https://github.com/pgbouncer/pgbouncer/pull/1007 --- doc/src/sgml/libpq.sgml | 8 +- doc/src/sgml/protocol.sgml | 31 ++++- src/include/libpq/pqcomm.h | 3 +- src/interfaces/libpq/fe-connect.c | 9 ++ src/interfaces/libpq/fe-protocol3.c | 8 +- .../modules/libpq_pipeline/libpq_pipeline.c | 110 +++++++++++++++--- 6 files changed, 142 insertions(+), 27 deletions(-) diff --git a/doc/src/sgml/libpq.sgml b/doc/src/sgml/libpq.sgml index f249bbd9263..ed099598c38 100644 --- a/doc/src/sgml/libpq.sgml +++ b/doc/src/sgml/libpq.sgml @@ -2158,7 +2158,8 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <para> The current supported values are - <literal>3.0</literal> + <literal>3.0</literal>, + <literal>3.2</literal>, and <literal>latest</literal>. The <literal>latest</literal> value is equivalent to the latest protocol version that is supported by the used libpq version, which currently is <literal>3.2</literal>, but this will @@ -2185,10 +2186,11 @@ postgresql://%2Fvar%2Flib%2Fpostgresql/dbname <para> The current supported values are - <literal>3.0</literal> + <literal>3.0</literal>, + <literal>3.2</literal>, and <literal>latest</literal>. The <literal>latest</literal> value is equivalent to the latest protocol version that is supported by the used - libpq version, which currently is <literal>3.0</literal>, but this will + libpq version, which currently is <literal>3.2</literal>, but this will change in future libpq releases. </para> </listitem> diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 3bd9e68e6ce..6bb97dbba55 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -18,17 +18,23 @@ </para> <para> - This document describes version 3.0 of the protocol, implemented in + This document describes version 3.2 of the protocol, introduced in + <productname>PostgreSQL</productname> version 17. The server is compatible with + protocol version 3.0, implemented in <productname>PostgreSQL</productname> 7.4 and later. For descriptions - of the earlier protocol versions, see previous releases of the - <productname>PostgreSQL</productname> documentation. A single server + of earlier protocol versions, see previous releases of the + <productname>PostgreSQL</productname> documentation. + </para> + + <para> + A single server can support multiple protocol versions. The initial startup-request message tells the server which protocol version the client is attempting to use. If the major version requested by the client is not supported by the server, the connection will be rejected (for example, this would occur if the client requested protocol version 4.0, which does not exist as of this writing). If the minor version requested by the client is not - supported by the server (e.g., the client requests version 3.1, but the + supported by the server (e.g., the client requests version 3.2, but the server supports only 3.0), the server may either reject the connection or may respond with a NegotiateProtocolVersion message containing the highest minor protocol version which it supports. The client may then choose either @@ -36,6 +42,13 @@ to abort the connection. </para> + <para> + The protocol negotiation was introduced in + <productname>PostgreSQL</productname> version 9.3.21. Earlier versions would + reject the connection if the client requested a minor version that was not + supported by the server. + </para> + <para> In order to serve multiple clients efficiently, the server launches a new <quote>backend</quote> process for each client. @@ -413,8 +426,14 @@ this message indicates the highest supported minor version. This message will also be sent if the client requested unsupported protocol options (i.e., beginning with <literal>_pq_.</literal>) in the - startup packet. This message will be followed by an ErrorResponse or - a message indicating the success or failure of authentication. + startup packet. + </para> + <para> + After this message, the authentication will continue using the version + indicated by the server. If the client does not support the older + version, it should immediately close the connection. If the server + does not send this message, it supports the client's requested + protocol version and all the protocol options. </para> </listitem> </varlistentry> diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h index 46b37e0e4eb..0aceb7147c7 100644 --- a/src/include/libpq/pqcomm.h +++ b/src/include/libpq/pqcomm.h @@ -91,11 +91,10 @@ is_unixsock_path(const char *path) /* * The earliest and latest frontend/backend protocol version supported. - * (Only protocol version 3 is currently supported) */ #define PG_PROTOCOL_EARLIEST PG_PROTOCOL(3,0) -#define PG_PROTOCOL_LATEST PG_PROTOCOL(3,0) +#define PG_PROTOCOL_LATEST PG_PROTOCOL(3,2) typedef uint32 ProtocolVersion; /* FE/BE protocol version number */ diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 7f08b143fa7..559964a6a25 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -8216,6 +8216,15 @@ pqParseProtocolVersion(const char *value, ProtocolVersion *result, PGconn *conn, return true; } + if (strcmp(value, "3.1") == 0) + { + conn->status = CONNECTION_BAD; + libpq_append_conn_error(conn, "invalid %s value: \"%s\"", + context, + value); + return false; + } + major = strtol(value, &end, 10); if (*end != '.') { diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index 221627bdea6..f33766c7df4 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -1428,7 +1428,13 @@ pqGetNegotiateProtocolVersion3(PGconn *conn) libpq_append_conn_error(conn, "received invalid protocol negotiation message: server requests downgrade to pre-3.0 protocol version"); goto failure; } - + + if (their_version == PG_PROTOCOL(3, 1)) + { + libpq_append_conn_error(conn, "received invalid protocol negotiation message: server requests downgrade to 3.1 protocol version"); + goto failure; + } + if (num < 0) { libpq_append_conn_error(conn, "received invalid protocol negotiation message: server reported negative number of unsupported parameters"); diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/test/modules/libpq_pipeline/libpq_pipeline.c index 7ff18e91e66..656f437826e 100644 --- a/src/test/modules/libpq_pipeline/libpq_pipeline.c +++ b/src/test/modules/libpq_pipeline/libpq_pipeline.c @@ -198,24 +198,15 @@ send_cancellable_query_impl(int line, PGconn *conn, PGconn *monitorConn) } /* - * Create a new connection with the same conninfo as the given one. + * Fills keywords and vals, with the same options as the ones in the opts + * linked-list. Returns the length of the filled in list. */ -static PGconn * -copy_connection(PGconn *conn) +static +int +copy_connection_options(PQconninfoOption *opts, const char **keywords, const char **vals) { - PGconn *copyConn; - PQconninfoOption *opts = PQconninfo(conn); - const char **keywords; - const char **vals; - int nopts = 1; int i = 0; - for (PQconninfoOption *opt = opts; opt->keyword != NULL; ++opt) - nopts++; - - keywords = pg_malloc(sizeof(char *) * nopts); - vals = pg_malloc(sizeof(char *) * nopts); - for (PQconninfoOption *opt = opts; opt->keyword != NULL; ++opt) { if (opt->val) @@ -225,8 +216,28 @@ copy_connection(PGconn *conn) i++; } } - keywords[i] = vals[i] = NULL; + return i; +} + +/* + * Create a new connection with the same conninfo as the given one. + */ +static PGconn * +copy_connection(PGconn *conn) +{ + const char **keywords; + const char **vals; + PGconn *copyConn; + PQconninfoOption *opts = PQconninfo(conn); + int nopts = 1; /* 1 for the NULL terminator */ + for (PQconninfoOption *opt = opts; opt->keyword != NULL; ++opt) + nopts++; + + keywords = pg_malloc0(sizeof(char *) * nopts); + vals = pg_malloc0(sizeof(char *) * nopts); + + copy_connection_options(opts, keywords, vals); copyConn = PQconnectdbParams(keywords, vals, false); if (PQstatus(copyConn) != CONNECTION_OK) @@ -1406,6 +1417,72 @@ test_prepared(PGconn *conn) fprintf(stderr, "ok\n"); } +static void +test_protocol_version(PGconn *conn) +{ + const char **keywords; + const char **vals; + int nopts = 2; /* NULL terminator + max_protocol_version */ + PQconninfoOption *opts = PQconninfo(conn); + int protocol_version; + int max_protocol_version_index; + + for (PQconninfoOption *opt = opts; opt->keyword != NULL; ++opt) + nopts++; + + keywords = pg_malloc0(sizeof(char *) * nopts); + vals = pg_malloc0(sizeof(char *) * nopts); + + max_protocol_version_index = copy_connection_options(opts, keywords, vals); + + keywords[max_protocol_version_index] = "max_protocol_version"; + vals[max_protocol_version_index] = "3.0"; + + conn = PQconnectdbParams(keywords, vals, false); + + if (PQstatus(conn) != CONNECTION_OK) + pg_fatal("Connection to database failed: %s", + PQerrorMessage(conn)); + + protocol_version = PQfullProtocolVersion(conn); + if (protocol_version != 30000) + pg_fatal("expected 30000, got %d", protocol_version); + + PQfinish(conn); + + vals[max_protocol_version_index] = "3.1"; + conn = PQconnectdbParams(keywords, vals, false); + + if (PQstatus(conn) != CONNECTION_BAD) + pg_fatal("Connecting with max_protocol_version 3.1 should have failed."); + + PQfinish(conn); + + vals[max_protocol_version_index] = "3.2"; + conn = PQconnectdbParams(keywords, vals, false); + + if (PQstatus(conn) != CONNECTION_OK) + pg_fatal("Connection to database failed: %s", + PQerrorMessage(conn)); + + protocol_version = PQfullProtocolVersion(conn); + if (protocol_version != 30002) + pg_fatal("expected 30002, got %d", protocol_version); + PQfinish(conn); + + vals[max_protocol_version_index] = "latest"; + conn = PQconnectdbParams(keywords, vals, false); + + if (PQstatus(conn) != CONNECTION_OK) + pg_fatal("Connection to database failed: %s", + PQerrorMessage(conn)); + + protocol_version = PQfullProtocolVersion(conn); + if (protocol_version != 30002) + pg_fatal("expected 30002, got %d", protocol_version); + PQfinish(conn); +} + /* Notice processor: print notices, and count how many we got */ static void notice_processor(void *arg, const char *message) @@ -2154,6 +2231,7 @@ print_test_list(void) printf("pipeline_idle\n"); printf("pipelined_insert\n"); printf("prepared\n"); + printf("protocol_version\n"); printf("simple_pipeline\n"); printf("singlerow\n"); printf("transaction\n"); @@ -2264,6 +2342,8 @@ main(int argc, char **argv) test_pipelined_insert(conn, numrows); else if (strcmp(testname, "prepared") == 0) test_prepared(conn); + else if (strcmp(testname, "protocol_version") == 0) + test_protocol_version(conn); else if (strcmp(testname, "simple_pipeline") == 0) test_simple_pipeline(conn); else if (strcmp(testname, "singlerow") == 0) -- 2.43.0
From 86e307c7534dfdd2b25afac273211e2c63de737c Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <heikki.linnakan...@iki.fi> Date: Thu, 15 Aug 2024 15:49:46 +0300 Subject: [PATCH v4 6/7] Add timingsafe_bcmp(), for constant-time memory comparison timingsafe_bcmp() should be used instead of memcmp() or a naive for-loop, when comparing passwords or secret tokens, to avoid leaking information about the secret token by timing. This commit just introduces the function but does not change any existing code to use it yet. --- configure | 23 ++++++++++++++++++++ configure.ac | 3 ++- meson.build | 2 ++ src/include/port.h | 4 ++++ src/port/meson.build | 1 + src/port/timingsafe_bcmp.c | 43 ++++++++++++++++++++++++++++++++++++++ 6 files changed, 75 insertions(+), 1 deletion(-) create mode 100644 src/port/timingsafe_bcmp.c diff --git a/configure b/configure index 93fddd69981..73648354a65 100755 --- a/configure +++ b/configure @@ -15792,6 +15792,16 @@ fi cat >>confdefs.h <<_ACEOF #define HAVE_DECL_STRSEP $ac_have_decl _ACEOF +ac_fn_c_check_decl "$LINENO" "timingsafe_bcmp" "ac_cv_have_decl_timingsafe_bcmp" "$ac_includes_default" +if test "x$ac_cv_have_decl_timingsafe_bcmp" = xyes; then : + ac_have_decl=1 +else + ac_have_decl=0 +fi + +cat >>confdefs.h <<_ACEOF +#define HAVE_DECL_TIMINGSAFE_BCMP $ac_have_decl +_ACEOF # We can't use AC_CHECK_FUNCS to detect these functions, because it @@ -15952,6 +15962,19 @@ esac fi +ac_fn_c_check_func "$LINENO" "timingsafe_bcmp" "ac_cv_func_timingsafe_bcmp" +if test "x$ac_cv_func_timingsafe_bcmp" = xyes; then : + $as_echo "#define HAVE_TIMINGSAFE_BCMP 1" >>confdefs.h + +else + case " $LIBOBJS " in + *" timingsafe_bcmp.$ac_objext "* ) ;; + *) LIBOBJS="$LIBOBJS timingsafe_bcmp.$ac_objext" + ;; +esac + +fi + ac_fn_c_check_func "$LINENO" "pthread_barrier_wait" "ac_cv_func_pthread_barrier_wait" diff --git a/configure.ac b/configure.ac index b6d02f5ecc7..af6e237b26d 100644 --- a/configure.ac +++ b/configure.ac @@ -1793,7 +1793,7 @@ AC_CHECK_DECLS(posix_fadvise, [], [], [#include <fcntl.h>]) ]) # fi AC_CHECK_DECLS(fdatasync, [], [], [#include <unistd.h>]) -AC_CHECK_DECLS([strlcat, strlcpy, strnlen, strsep]) +AC_CHECK_DECLS([strlcat, strlcpy, strnlen, strsep, timingsafe_bcmp]) # We can't use AC_CHECK_FUNCS to detect these functions, because it # won't handle deployment target restrictions on macOS @@ -1813,6 +1813,7 @@ AC_REPLACE_FUNCS(m4_normalize([ strlcpy strnlen strsep + timingsafe_bcmp ])) AC_REPLACE_FUNCS(pthread_barrier_wait) diff --git a/meson.build b/meson.build index 13c13748e5d..5a7e77ddbe6 100644 --- a/meson.build +++ b/meson.build @@ -2513,6 +2513,7 @@ decl_checks = [ ['strlcpy', 'string.h'], ['strnlen', 'string.h'], ['strsep', 'string.h'], + ['timingsafe_bcmp', 'string.h'], ] # Need to check for function declarations for these functions, because @@ -2754,6 +2755,7 @@ func_checks = [ ['strsignal'], ['sync_file_range'], ['syncfs'], + ['timingsafe_bcmp'], ['uselocale'], ['wcstombs_l'], ] diff --git a/src/include/port.h b/src/include/port.h index 703cad868ba..8db8b9f2a6f 100644 --- a/src/include/port.h +++ b/src/include/port.h @@ -464,6 +464,10 @@ extern size_t strnlen(const char *str, size_t maxlen); extern char *strsep(char **stringp, const char *delim); #endif +#if !HAVE_DECL_TIMINGSAFE_BCMP +extern int timingsafe_bcmp(const void *b1, const void *b2, size_t len); +#endif + /* * Callers should use the qsort() macro defined below instead of calling * pg_qsort() directly. diff --git a/src/port/meson.build b/src/port/meson.build index 7fcfa728d43..16134bb6065 100644 --- a/src/port/meson.build +++ b/src/port/meson.build @@ -71,6 +71,7 @@ replace_funcs_neg = [ ['strlcpy'], ['strnlen'], ['strsep'], + ['timingsafe_bcmp'], ] if host_system != 'windows' diff --git a/src/port/timingsafe_bcmp.c b/src/port/timingsafe_bcmp.c new file mode 100644 index 00000000000..288865f50d1 --- /dev/null +++ b/src/port/timingsafe_bcmp.c @@ -0,0 +1,43 @@ +/* + * src/port/timingsafe_bcmp.c + * + * $OpenBSD: timingsafe_bcmp.c,v 1.3 2015/08/31 02:53:57 guenther Exp $ + */ + +/* + * Copyright (c) 2010 Damien Miller. All rights reserved. + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include "c.h" + +#ifdef USE_SSL +#include <openssl/crypto.h> +#endif + +int +timingsafe_bcmp(const void *b1, const void *b2, size_t n) +{ +#ifdef USE_SSL + return CRYPTO_memcmp(b1, b2, n); +#else + const unsigned char *p1 = b1, + *p2 = b2; + int ret = 0; + + for (; n > 0; n--) + ret |= *p1++ ^ *p2++; + return (ret != 0); +#endif +} -- 2.43.0
From 6d5332914bfd0562708151fc0b4033dfd6bb356a Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <heikki.linnakan...@iki.fi> Date: Thu, 15 Aug 2024 19:49:29 +0300 Subject: [PATCH v4 7/7] Make cancel request keys longer Currently, cancel request key is 32-bit token, which isn't very much entropy. If you want to cancel another session's query, you can brute-force it. In most environments, an unauthorized cancellation of a query isn't very serious, but it nevertheless would be nice to have more protection from it. Hence make the key longer, to make it harder to guess. The new longer key length is not hardcoded in the protocol anymore, the client is expected to deal with variable length keys, up to some reasonable upper limit (TODO: document the maximum). This flexibility allows e.g. a connection pooler to add more information to the cancel key, which might be useful for finding the connection. Reviewed-by: Jelte Fennema-Nio Discussion: https://www.postgresql.org/message-id/508d0505-8b7a-4864-a681-e7e5edfe3...@iki.fi --- doc/src/sgml/protocol.sgml | 19 +++- src/backend/storage/ipc/procsignal.c | 23 ++-- src/backend/tcop/backend_startup.c | 55 ++++++---- src/backend/tcop/postgres.c | 15 ++- src/backend/utils/init/globals.c | 5 +- src/backend/utils/init/postinit.c | 2 +- src/include/libpq/pqcomm.h | 7 +- src/include/miscadmin.h | 4 +- src/include/storage/procsignal.h | 14 ++- src/interfaces/libpq/fe-cancel.c | 102 ++++++++++++++---- src/interfaces/libpq/fe-connect.c | 20 ++-- src/interfaces/libpq/fe-protocol3.c | 45 +++++++- src/interfaces/libpq/libpq-int.h | 7 +- .../libpq_pipeline/t/001_libpq_pipeline.pl | 12 ++- src/tools/pgindent/typedefs.list | 1 + 15 files changed, 247 insertions(+), 84 deletions(-) diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml index 6bb97dbba55..a5dc8721bc3 100644 --- a/doc/src/sgml/protocol.sgml +++ b/doc/src/sgml/protocol.sgml @@ -49,6 +49,13 @@ supported by the server. </para> + <para> + The difference between protocol version 3.0 and 3.2 is that the secret key + used in query cancellation was enlarged from 4 bytes to a variable length + field. The BackendKeyData message was changed to accomodate that, and the + CancelRequest message was redefined to have a variable length payload. + </para> + <para> In order to serve multiple clients efficiently, the server launches a new <quote>backend</quote> process for each client. @@ -4064,7 +4071,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;" </varlistentry> <varlistentry> - <term>Int32(12)</term> + <term>Int32</term> <listitem> <para> Length of message contents in bytes, including self. @@ -4082,7 +4089,7 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;" </varlistentry> <varlistentry> - <term>Int32</term> + <term>Byten</term> <listitem> <para> The secret key of this backend. @@ -4295,14 +4302,18 @@ psql "dbname=postgres replication=database" -c "IDENTIFY_SYSTEM;" </varlistentry> <varlistentry> - <term>Int32</term> + <term>Byte4</term> <listitem> <para> - The secret key for the target backend. + The secret key for the target backend. This field extends to the end of the + message, indicated by the length field. The maximum key length is 256 bytes. </para> </listitem> </varlistentry> </variablelist> + <para> + Before protocol version 3.2, the secret key was always 4 bytes long. + </para> </listitem> </varlistentry> diff --git a/src/backend/storage/ipc/procsignal.c b/src/backend/storage/ipc/procsignal.c index 7401b6e625e..6ad800c717e 100644 --- a/src/backend/storage/ipc/procsignal.c +++ b/src/backend/storage/ipc/procsignal.c @@ -63,8 +63,8 @@ typedef struct { pg_atomic_uint32 pss_pid; - bool pss_cancel_key_valid; - int32 pss_cancel_key; + int pss_cancel_key_len; /* 0 means no cancellation is possible */ + char pss_cancel_key[MAX_CANCEL_KEY_LENGTH]; volatile sig_atomic_t pss_signalFlags[NUM_PROCSIGNALS]; slock_t pss_mutex; /* protects the above fields */ @@ -148,8 +148,7 @@ ProcSignalShmemInit(void) SpinLockInit(&slot->pss_mutex); pg_atomic_init_u32(&slot->pss_pid, 0); - slot->pss_cancel_key_valid = false; - slot->pss_cancel_key = 0; + slot->pss_cancel_key_len = 0; MemSet(slot->pss_signalFlags, 0, sizeof(slot->pss_signalFlags)); pg_atomic_init_u64(&slot->pss_barrierGeneration, PG_UINT64_MAX); pg_atomic_init_u32(&slot->pss_barrierCheckMask, 0); @@ -163,11 +162,12 @@ ProcSignalShmemInit(void) * Register the current process in the ProcSignal array */ void -ProcSignalInit(bool cancel_key_valid, int32 cancel_key) +ProcSignalInit(char *cancel_key, int cancel_key_len) { ProcSignalSlot *slot; uint64 barrier_generation; + Assert(cancel_key_len >= 0 && cancel_key_len <= MAX_CANCEL_KEY_LENGTH); if (MyProcNumber < 0) elog(ERROR, "MyProcNumber not set"); if (MyProcNumber >= NumProcSignalSlots) @@ -202,8 +202,9 @@ ProcSignalInit(bool cancel_key_valid, int32 cancel_key) pg_atomic_read_u64(&ProcSignal->psh_barrierGeneration); pg_atomic_write_u64(&slot->pss_barrierGeneration, barrier_generation); - slot->pss_cancel_key_valid = cancel_key_valid; - slot->pss_cancel_key = cancel_key; + if (cancel_key_len > 0) + memcpy(slot->pss_cancel_key, cancel_key, cancel_key_len); + slot->pss_cancel_key_len = cancel_key_len; pg_atomic_write_u32(&slot->pss_pid, MyProcPid); SpinLockRelease(&slot->pss_mutex); @@ -252,8 +253,7 @@ CleanupProcSignalState(int status, Datum arg) /* Mark the slot as unused */ pg_atomic_write_u32(&slot->pss_pid, 0); - slot->pss_cancel_key_valid = false; - slot->pss_cancel_key = 0; + slot->pss_cancel_key_len = 0; /* * Make this slot look like it's absorbed all possible barriers, so that @@ -723,7 +723,7 @@ procsignal_sigusr1_handler(SIGNAL_ARGS) * fields in the ProcSignal slots. */ void -SendCancelRequest(int backendPID, int32 cancelAuthCode) +SendCancelRequest(int backendPID, char *cancel_key, int cancel_key_len) { Assert(backendPID != 0); @@ -752,7 +752,8 @@ SendCancelRequest(int backendPID, int32 cancelAuthCode) } else { - match = slot->pss_cancel_key_valid && slot->pss_cancel_key == cancelAuthCode; + match = slot->pss_cancel_key_len == cancel_key_len && + timingsafe_bcmp(slot->pss_cancel_key, cancel_key, cancel_key_len) == 0; SpinLockRelease(&slot->pss_mutex); diff --git a/src/backend/tcop/backend_startup.c b/src/backend/tcop/backend_startup.c index 9ee738eb1e0..b73cec88721 100644 --- a/src/backend/tcop/backend_startup.c +++ b/src/backend/tcop/backend_startup.c @@ -45,6 +45,7 @@ bool Trace_connection_negotiation = false; static void BackendInitialize(ClientSocket *client_sock, CAC_state cac); static int ProcessSSLStartup(Port *port); static int ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done); +static void ProcessCancelRequestPacket(Port *port, void *pkt, int pktlen); static void SendNegotiateProtocolVersion(List *unrecognized_protocol_options); static void process_startup_packet_die(SIGNAL_ARGS); static void StartupPacketTimeoutHandler(void); @@ -542,28 +543,7 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done) if (proto == CANCEL_REQUEST_CODE) { - /* - * The client has sent a cancel request packet, not a normal - * start-a-new-connection packet. Perform the necessary processing. - * Nothing is sent back to the client. - */ - CancelRequestPacket *canc; - int backendPID; - int32 cancelAuthCode; - - if (len != sizeof(CancelRequestPacket)) - { - ereport(COMMERROR, - (errcode(ERRCODE_PROTOCOL_VIOLATION), - errmsg("invalid length of startup packet"))); - return STATUS_ERROR; - } - canc = (CancelRequestPacket *) buf; - backendPID = (int) pg_ntoh32(canc->backendPID); - cancelAuthCode = (int32) pg_ntoh32(canc->cancelAuthCode); - - if (backendPID != 0) - SendCancelRequest(backendPID, cancelAuthCode); + ProcessCancelRequestPacket(port, buf, len); /* Not really an error, but we don't want to proceed further */ return STATUS_ERROR; } @@ -863,6 +843,37 @@ ProcessStartupPacket(Port *port, bool ssl_done, bool gss_done) return STATUS_OK; } +/* + * The client has sent a cancel request packet, not a normal + * start-a-new-connection packet. Perform the necessary processing. Nothing + * is sent back to the client. + */ +static void +ProcessCancelRequestPacket(Port *port, void *pkt, int pktlen) +{ + CancelRequestPacket *canc; + int len; + + if (pktlen < offsetof(CancelRequestPacket, cancelAuthCode)) + { + ereport(COMMERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("invalid length of query cancel packet"))); + return; + } + len = pktlen - offsetof(CancelRequestPacket, cancelAuthCode); + if (len == 0 || len > 256) + { + ereport(COMMERROR, + (errcode(ERRCODE_PROTOCOL_VIOLATION), + errmsg("invalid length of query cancel key"))); + return; + } + + canc = (CancelRequestPacket *) pkt; + SendCancelRequest(pg_ntoh32(canc->backendPID), canc->cancelAuthCode, len); +} + /* * Send a NegotiateProtocolVersion to the client. This lets the client know * that they have either requested a newer minor protocol version than we are diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c index f2f75aa0f88..ef413b33227 100644 --- a/src/backend/tcop/postgres.c +++ b/src/backend/tcop/postgres.c @@ -4225,16 +4225,20 @@ PostgresMain(const char *dbname, const char *username) * Generate a random cancel key, if this is a backend serving a * connection. InitPostgres() will advertise it in shared memory. */ - Assert(!MyCancelKeyValid); + Assert(MyCancelKeyLength == 0); if (whereToSendOutput == DestRemote) { - if (!pg_strong_random(&MyCancelKey, sizeof(int32))) + int len; + + len = (MyProcPort == NULL || MyProcPort->proto >= PG_PROTOCOL(3, 1)) + ? MAX_CANCEL_KEY_LENGTH : 4; + if (!pg_strong_random(&MyCancelKey, len)) { ereport(ERROR, (errcode(ERRCODE_INTERNAL_ERROR), errmsg("could not generate random cancel key"))); } - MyCancelKeyValid = true; + MyCancelKeyLength = len; } /* @@ -4289,10 +4293,11 @@ PostgresMain(const char *dbname, const char *username) { StringInfoData buf; - Assert(MyCancelKeyValid); + Assert(MyCancelKeyLength > 0); pq_beginmessage(&buf, PqMsg_BackendKeyData); pq_sendint32(&buf, (int32) MyProcPid); - pq_sendint32(&buf, (int32) MyCancelKey); + + pq_sendbytes(&buf, MyCancelKey, MyCancelKeyLength); pq_endmessage(&buf); /* Need not flush since ReadyForQuery will do it. */ } diff --git a/src/backend/utils/init/globals.c b/src/backend/utils/init/globals.c index b844f9fdaef..2152aad97d9 100644 --- a/src/backend/utils/init/globals.c +++ b/src/backend/utils/init/globals.c @@ -24,6 +24,7 @@ #include "miscadmin.h" #include "postmaster/postmaster.h" #include "storage/procnumber.h" +#include "storage/procsignal.h" ProtocolVersion FrontendProtocol; @@ -48,8 +49,8 @@ pg_time_t MyStartTime; TimestampTz MyStartTimestamp; struct ClientSocket *MyClientSocket; struct Port *MyProcPort; -bool MyCancelKeyValid = false; -int32 MyCancelKey = 0; +char MyCancelKey[MAX_CANCEL_KEY_LENGTH]; +uint8 MyCancelKeyLength = 0; int MyPMChildSlot; /* diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c index 01bb6a410cb..a59871422cb 100644 --- a/src/backend/utils/init/postinit.c +++ b/src/backend/utils/init/postinit.c @@ -723,7 +723,7 @@ InitPostgres(const char *in_dbname, Oid dboid, */ SharedInvalBackendInit(false); - ProcSignalInit(MyCancelKeyValid, MyCancelKey); + ProcSignalInit(MyCancelKey, MyCancelKeyLength); /* * Also set up timeout handlers needed for backend operation. We need diff --git a/src/include/libpq/pqcomm.h b/src/include/libpq/pqcomm.h index 0aceb7147c7..d0f2c1fb03a 100644 --- a/src/include/libpq/pqcomm.h +++ b/src/include/libpq/pqcomm.h @@ -128,7 +128,11 @@ typedef uint32 AuthRequest; * * The cancel request code must not match any protocol version number * we're ever likely to use. This random choice should do. + * + * Before PostgreSQL v17 and the protocol version bump from 3.0 to 3.1, the + * cancel key was always 4 bytes. Starting with v17, it's variable length. */ + #define CANCEL_REQUEST_CODE PG_PROTOCOL(1234,5678) typedef struct CancelRequestPacket @@ -136,7 +140,8 @@ typedef struct CancelRequestPacket /* Note that each field is stored in network byte order! */ MsgType cancelRequestCode; /* code to identify a cancel request */ uint32 backendPID; /* PID of client's backend */ - uint32 cancelAuthCode; /* secret key to authorize cancel */ + char cancelAuthCode[FLEXIBLE_ARRAY_MEMBER]; /* secret key to + * authorize cancel */ } CancelRequestPacket; /* Application-Layer Protocol Negotiation is required for direct connections diff --git a/src/include/miscadmin.h b/src/include/miscadmin.h index a2b63495eec..2eda5b0bd4d 100644 --- a/src/include/miscadmin.h +++ b/src/include/miscadmin.h @@ -191,8 +191,8 @@ extern PGDLLIMPORT pg_time_t MyStartTime; extern PGDLLIMPORT TimestampTz MyStartTimestamp; extern PGDLLIMPORT struct Port *MyProcPort; extern PGDLLIMPORT struct Latch *MyLatch; -extern PGDLLIMPORT bool MyCancelKeyValid; -extern PGDLLIMPORT int32 MyCancelKey; +extern PGDLLIMPORT char MyCancelKey[]; +extern PGDLLIMPORT uint8 MyCancelKeyLength; extern PGDLLIMPORT int MyPMChildSlot; extern PGDLLIMPORT char OutputFileName[]; diff --git a/src/include/storage/procsignal.h b/src/include/storage/procsignal.h index 022fd8ed933..b14c9252582 100644 --- a/src/include/storage/procsignal.h +++ b/src/include/storage/procsignal.h @@ -56,16 +56,26 @@ typedef enum PROCSIGNAL_BARRIER_SMGRRELEASE, /* ask smgr to close files */ } ProcSignalBarrierType; +/* + * Length of query cancel keys generated. + * + * Note that the protocol allows for longer keys, or shorter, but this is the + * length we actually generate. Client code, and the server code that handles + * incoming cancellation packets from clients, cannot use this hardcoded + * length. + */ +#define MAX_CANCEL_KEY_LENGTH 32 + /* * prototypes for functions in procsignal.c */ extern Size ProcSignalShmemSize(void); extern void ProcSignalShmemInit(void); -extern void ProcSignalInit(bool cancel_key_valid, int32 cancel_key); +extern void ProcSignalInit(char *cancel_key, int cancel_key_len); extern int SendProcSignal(pid_t pid, ProcSignalReason reason, ProcNumber procNumber); -extern void SendCancelRequest(int backendPID, int32 cancelAuthCode); +extern void SendCancelRequest(int backendPID, char *cancel_key, int cancel_key_len); extern uint64 EmitProcSignalBarrier(ProcSignalBarrierType type); extern void WaitForProcSignalBarrier(uint64 generation); diff --git a/src/interfaces/libpq/fe-cancel.c b/src/interfaces/libpq/fe-cancel.c index 7ebaa335bba..e84e64bf2a7 100644 --- a/src/interfaces/libpq/fe-cancel.c +++ b/src/interfaces/libpq/fe-cancel.c @@ -1,7 +1,7 @@ /*------------------------------------------------------------------------- * * fe-cancel.c - * functions related to setting up a connection to the backend + * functions related to query cancellation * * Portions Copyright (c) 1996-2025, PostgreSQL Global Development Group * Portions Copyright (c) 1994, Regents of the University of California @@ -41,7 +41,6 @@ struct pg_cancel { SockAddr raddr; /* Remote address */ int be_pid; /* PID of to-be-canceled backend */ - int be_key; /* cancel key of to-be-canceled backend */ int pgtcp_user_timeout; /* tcp user timeout */ int keepalives; /* use TCP keepalives? */ int keepalives_idle; /* time between TCP keepalives */ @@ -49,6 +48,10 @@ struct pg_cancel * retransmits */ int keepalives_count; /* maximum number of TCP keepalive * retransmits */ + + /* Pre-constructed cancel request packet starts here */ + int32 cancel_pkt_len; /* in network-byte-order */ + char cancel_req[FLEXIBLE_ARRAY_MEMBER]; /* CancelRequestPacket */ }; @@ -83,6 +86,13 @@ PQcancelCreate(PGconn *conn) return (PGcancelConn *) cancelConn; } + /* Check that we have received a cancellation key */ + if (conn->be_cancel_key_len == 0) + { + libpq_append_conn_error(cancelConn, "no cancellation key received"); + return (PGcancelConn *) cancelConn; + } + /* * Indicate that this connection is used to send a cancellation */ @@ -101,7 +111,15 @@ PQcancelCreate(PGconn *conn) * Copy cancellation token data from the original connection */ cancelConn->be_pid = conn->be_pid; - cancelConn->be_key = conn->be_key; + if (conn->be_cancel_key != NULL) + { + cancelConn->be_cancel_key = malloc(conn->be_cancel_key_len); + if (!conn->be_cancel_key) + goto oom_error; + memcpy(cancelConn->be_cancel_key, conn->be_cancel_key, conn->be_cancel_key_len); + } + cancelConn->be_cancel_key_len = conn->be_cancel_key_len; + cancelConn->pversion = conn->pversion; /* * Cancel requests should not iterate over all possible hosts. The request @@ -349,6 +367,8 @@ PGcancel * PQgetCancel(PGconn *conn) { PGcancel *cancel; + int cancel_req_len; + CancelRequestPacket *req; if (!conn) return NULL; @@ -356,13 +376,17 @@ PQgetCancel(PGconn *conn) if (conn->sock == PGINVALID_SOCKET) return NULL; - cancel = malloc(sizeof(PGcancel)); + /* Check that we have received a cancellation key */ + if (conn->be_cancel_key_len == 0) + return NULL; + + cancel_req_len = offsetof(CancelRequestPacket, cancelAuthCode) + conn->be_cancel_key_len; + cancel = malloc(offsetof(PGcancel, cancel_req) + cancel_req_len); if (cancel == NULL) return NULL; memcpy(&cancel->raddr, &conn->raddr, sizeof(SockAddr)); - cancel->be_pid = conn->be_pid; - cancel->be_key = conn->be_key; + /* We use -1 to indicate an unset connection option */ cancel->pgtcp_user_timeout = -1; cancel->keepalives = -1; @@ -405,6 +429,13 @@ PQgetCancel(PGconn *conn) goto fail; } + req = (CancelRequestPacket *) &cancel->cancel_req; + req->cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE); + req->backendPID = pg_hton32(conn->be_pid); + memcpy(req->cancelAuthCode, conn->be_cancel_key, conn->be_cancel_key_len); + /* include the length field itself in the length */ + cancel->cancel_pkt_len = pg_hton32(cancel_req_len + 4); + return cancel; fail: @@ -412,6 +443,42 @@ fail: return NULL; } +/* + * PQsendCancelRequest + * Submit a CancelRequest message, but don't wait for it to finish + * + * Returns: 1 if successfully submitted + * 0 if error (conn->errorMessage is set) + */ +int +PQsendCancelRequest(PGconn *cancelConn) +{ + CancelRequestPacket req; + + /* Start the message. */ + if (pqPutMsgStart(0, cancelConn)) + return STATUS_ERROR; + + /* Send the message body. */ + memset(&req, 0, offsetof(CancelRequestPacket, cancelAuthCode)); + req.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE); + req.backendPID = pg_hton32(cancelConn->be_pid); + if (pqPutnchar((char *) &req, offsetof(CancelRequestPacket, cancelAuthCode), cancelConn)) + return STATUS_ERROR; + if (pqPutnchar(cancelConn->be_cancel_key, cancelConn->be_cancel_key_len, cancelConn)) + return STATUS_ERROR; + + /* Finish the message. */ + if (pqPutMsgEnd(cancelConn)) + return STATUS_ERROR; + + /* Flush to ensure backend gets it. */ + if (pqFlush(cancelConn)) + return STATUS_ERROR; + + return STATUS_OK; +} + /* PQfreeCancel: free a cancel structure */ void PQfreeCancel(PGcancel *cancel) @@ -465,11 +532,8 @@ PQcancel(PGcancel *cancel, char *errbuf, int errbufsize) int save_errno = SOCK_ERRNO; pgsocket tmpsock = PGINVALID_SOCKET; int maxlen; - struct - { - uint32 packetlen; - CancelRequestPacket cp; - } crp; + char recvbuf; + int cancel_pkt_len; if (!cancel) { @@ -571,15 +635,15 @@ retry3: goto cancel_errReturn; } - /* Create and send the cancel request packet. */ - - crp.packetlen = pg_hton32((uint32) sizeof(crp)); - crp.cp.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE); - crp.cp.backendPID = pg_hton32(cancel->be_pid); - crp.cp.cancelAuthCode = pg_hton32(cancel->be_key); + cancel_pkt_len = pg_ntoh32(cancel->cancel_pkt_len); retry4: - if (send(tmpsock, (char *) &crp, sizeof(crp), 0) != (int) sizeof(crp)) + + /* + * Send the cancel request packet. It starts with the message length at + * cancel_pkt_len, followed by the actual packet. + */ + if (send(tmpsock, (char *) &cancel->cancel_pkt_len, cancel_pkt_len, 0) != cancel_pkt_len) { if (SOCK_ERRNO == EINTR) /* Interrupted system call - we'll just try again */ @@ -596,7 +660,7 @@ retry4: * read to obtain any data, we are just waiting for EOF to be signaled. */ retry5: - if (recv(tmpsock, (char *) &crp, 1, 0) < 0) + if (recv(tmpsock, &recvbuf, 1, 0) < 0) { if (SOCK_ERRNO == EINTR) /* Interrupted system call - we'll just try again */ diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c index 559964a6a25..1423311b1ee 100644 --- a/src/interfaces/libpq/fe-connect.c +++ b/src/interfaces/libpq/fe-connect.c @@ -695,7 +695,12 @@ pqDropServerData(PGconn *conn) if (!conn->cancelRequest) { conn->be_pid = 0; - conn->be_key = 0; + if (conn->be_cancel_key != NULL) + { + free(conn->be_cancel_key); + conn->be_cancel_key = NULL; + } + conn->be_cancel_key_len = 0; } } @@ -3685,13 +3690,7 @@ keep_going: /* We will come back to here until there is */ if (conn->cancelRequest) { - CancelRequestPacket cancelpacket; - - packetlen = sizeof(cancelpacket); - cancelpacket.cancelRequestCode = (MsgType) pg_hton32(CANCEL_REQUEST_CODE); - cancelpacket.backendPID = pg_hton32(conn->be_pid); - cancelpacket.cancelAuthCode = pg_hton32(conn->be_key); - if (pqPacketSend(conn, 0, &cancelpacket, packetlen) != STATUS_OK) + if (PQsendCancelRequest(conn) != STATUS_OK) { libpq_append_conn_error(conn, "could not send cancel packet: %s", SOCK_STRERROR(SOCK_ERRNO, sebuf, sizeof(sebuf))); @@ -4135,7 +4134,10 @@ keep_going: /* We will come back to here until there is { if (pqGetNegotiateProtocolVersion3(conn)) { - libpq_append_conn_error(conn, "received invalid protocol negotiation message"); + /* + * Negotiation failed. The error message was filled + * in already. + */ goto error_return; } diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c index f33766c7df4..d7570705bc4 100644 --- a/src/interfaces/libpq/fe-protocol3.c +++ b/src/interfaces/libpq/fe-protocol3.c @@ -48,6 +48,7 @@ static int getRowDescriptions(PGconn *conn, int msgLength); static int getParamDescriptions(PGconn *conn, int msgLength); static int getAnotherTuple(PGconn *conn, int msgLength); static int getParameterStatus(PGconn *conn); +static int getBackendKeyData(PGconn *conn, int msgLength); static int getNotify(PGconn *conn); static int getCopyStart(PGconn *conn, ExecStatusType copytype); static int getReadyForQuery(PGconn *conn); @@ -308,9 +309,7 @@ pqParseInput3(PGconn *conn) * just as easy to handle it as part of the main loop. * Save the data and continue processing. */ - if (pqGetInt(&(conn->be_pid), 4, conn)) - return; - if (pqGetInt(&(conn->be_key), 4, conn)) + if (getBackendKeyData(conn, msgLength)) return; break; case PqMsg_RowDescription: @@ -1514,6 +1513,46 @@ getParameterStatus(PGconn *conn) return 0; } +/* + * parseInput subroutine to read a BackendKeyData message. + * Entry: 'v' message type and length have already been consumed. + * Exit: returns 0 if successfully consumed message. + * returns EOF if not enough data. + */ +static int +getBackendKeyData(PGconn *conn, int msgLength) +{ + uint8 cancel_key_len; + + if (conn->be_cancel_key) + { + free(conn->be_cancel_key); + conn->be_cancel_key = NULL; + conn->be_cancel_key_len = 0; + } + + if (pqGetInt(&(conn->be_pid), 4, conn)) + return EOF; + + cancel_key_len = 5 + msgLength - (conn->inCursor - conn->inStart); + + conn->be_cancel_key = malloc(cancel_key_len); + if (conn->be_cancel_key == NULL) + { + libpq_append_conn_error(conn, "out of memory"); + /* discard the message */ + return EOF; + } + if (pqGetnchar(conn->be_cancel_key, cancel_key_len, conn)) + { + free(conn->be_cancel_key); + conn->be_cancel_key = NULL; + return EOF; + } + conn->be_cancel_key_len = cancel_key_len; + return 0; +} + /* * Attempt to read a Notify response message. diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h index c016408338a..587315ad890 100644 --- a/src/interfaces/libpq/libpq-int.h +++ b/src/interfaces/libpq/libpq-int.h @@ -552,7 +552,8 @@ struct pg_conn /* Miscellaneous stuff */ int be_pid; /* PID of backend --- needed for cancels */ - int be_key; /* key of backend --- needed for cancels */ + char *be_cancel_key; /* query cancellation key and its length */ + uint16 be_cancel_key_len; pgParameterStatus *pstatus; /* ParameterStatus data */ int client_encoding; /* encoding id */ bool std_strings; /* standard_conforming_strings */ @@ -772,6 +773,10 @@ extern PGresult *pqFunctionCall3(PGconn *conn, Oid fnid, int result_is_int, const PQArgBlock *args, int nargs); +/* === in fe-cancel.c === */ + +extern int PQsendCancelRequest(PGconn *cancelConn); + /* === in fe-misc.c === */ /* diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl index e301c378264..d900355c5cf 100644 --- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl +++ b/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl @@ -49,11 +49,11 @@ for my $testname (@tests) push @extraargs, "-t", $traceout; } - # Execute the test + # Execute the test using the latest protocol version. $node->command_ok( [ 'libpq_pipeline', @extraargs, - $testname, $node->connstr('postgres') + $testname, $node->connstr('postgres') . " max_protocol_version=latest" ], "libpq_pipeline $testname"); @@ -72,6 +72,14 @@ for my $testname (@tests) } } +# There were changes to query cancellation in protocol version 3.2, so +# test separately that it still works the old protocol version too. +$node->command_ok( + [ + 'libpq_pipeline', 'cancel', $node->connstr('postgres') . " max_protocol_version=3.0" + ], + "libpq_pipeline cancel with protocol 3.0"); + $node->stop('fast'); done_testing(); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index b09d8af7183..cdf0d5dff2b 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -382,6 +382,7 @@ CachedPlanSource CallContext CallStmt CancelRequestPacket +CancelRequestExtendedPacket Cardinality CaseExpr CaseKind -- 2.43.0