On Mon, 2021-06-14 at 10:51 -0400, Robert Haas wrote:
> but if
> > > you use the extended query protocol, then the result is a
> > > hopeless
> > > mess, because the protocol is badly designed:
> > >
After looking in more detail, I think I understand a bit better.
Clients don't differentiate between:
* A normal command, where you know that you've sent everything that you
will send. In this case, the client needs to send the Sync message in
order to get the ReadyForQuery message.
* A command that initiates CopyIn/CopyBoth, where you are going to send
more data after the command. In this case, sending the Sync eagerly is
wrong, and you can't pipeline more queries in the middle of
CopyIn/CopyBoth mode. Instead, the client should send Sync after
receiving an ErrorResponse, or after sending a CopyDone/CopyFail
(right?).
One thing I don't fully understand is what would happen if the client
issued the Sync as the *first* message in an extended-protocol series.
> > > But I think you're correct in saying that the discard-until-Sync
> > > behavior only happens if the extended query protocol is used, so
> > > I
> > > agree that the current text is wrong.
> >
> > Should we just document how CopyBoth works with the simple query
> > protocol, or should we make it match the CopyIn docs?
>
> I think it would make sense to make it match the CopyIn docs.
> Possibly
> the CopyOut docs should be made more similar as well.
I attached a doc patch that hopefully clarifies this point as well as
the weirdness around CopyIn/CopyBoth and the extended protocol. I
reorganized the sections, as well.
Regards,
Jeff Davis
diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index bc2a2feb0b9..6d0819785cf 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -1132,59 +1132,60 @@ SELCT 1/0;<!-- this typo is intentional -->
<title>COPY Operations</title>
<para>
- The <command>COPY</command> command allows high-speed bulk data transfer
- to or from the server. Copy-in and copy-out operations each switch
- the connection into a distinct sub-protocol, which lasts until the
- operation is completed.
+ There are three special modes to handle bulk data transfer to or from the
+ server: copy-in, copy-out, and copy-both. Executing a command that
+ initiates one of these special modes switches the connection into a
+ distinct sub-protocol, which lasts until the operation is completed.
</para>
<para>
- Copy-in mode (data transfer to the server) is initiated when the
- backend executes a <command>COPY FROM STDIN</command> SQL statement. The backend
- sends a CopyInResponse message to the frontend. The frontend should
- then send zero or more CopyData messages, forming a stream of input
- data. (The message boundaries are not required to have anything to do
- with row boundaries, although that is often a reasonable choice.)
- The frontend can terminate the copy-in mode by sending either a CopyDone
- message (allowing successful termination) or a CopyFail message (which
- will cause the <command>COPY</command> SQL statement to fail with an
- error). The backend then reverts to the command-processing mode it was
- in before the <command>COPY</command> started, which will be either simple or
- extended query protocol. It will next send either CommandComplete
- (if successful) or ErrorResponse (if not).
+ Copy-in mode (data transfer to the server) is initiated when the backend
+ executes a <command>COPY FROM STDIN</command> SQL statement. The backend
+ sends a CopyInResponse message to the frontend. The frontend should then
+ send zero or more CopyData messages, forming a stream of input data. (The
+ message boundaries are not required to have anything to do with row
+ boundaries, although that is often a reasonable choice.) The frontend can
+ terminate the copy-in mode by sending either a CopyDone message (allowing
+ successful termination) or a CopyFail message (which will cause the
+ <command>COPY</command> SQL statement to fail with an error). The backend
+ then reverts to the command-processing mode it was in before the
+ <command>COPY</command> started, which will be either simple or extended
+ query protocol. It will next send either CommandComplete (if successful)
+ or ErrorResponse (if not).
</para>
<para>
- In the event of a backend-detected error during copy-in mode (including
- receipt of a CopyFail message), the backend will issue an ErrorResponse
- message. If the <command>COPY</command> command was issued via an extended-query
- message, the backend will now discard frontend messages until a Sync
- message is received, then it will issue ReadyForQuery and return to normal
- processing. If the <command>COPY</command> command was issued in a simple
- Query message, the rest of that message is discarded and ReadyForQuery
- is issued. In either case, any subsequent CopyData, CopyDone, or CopyFail
- messages issued by the frontend will simply be dropped.
- </para>
-
- <para>
- The backend will ignore Flush and Sync messages received during copy-in
- mode. Receipt of any other non-copy message type constitutes an error
- that will abort the copy-in state as described above. (The exception for
- Flush and Sync is for the convenience of client libraries that always
- send Flush or Sync after an Execute message, without checking whether
- the command to be executed is a <command>COPY FROM STDIN</command>.)
+ Copy-out mode (data transfer from the server) is initiated when the
+ backend executes a <command>COPY TO STDOUT</command> SQL statement. The
+ backend sends a CopyOutResponse message to the frontend, followed by zero
+ or more CopyData messages (always one per row), followed by CopyDone. The
+ backend then reverts to the command-processing mode it was in before the
+ <command>COPY</command> started, and sends CommandComplete. The frontend
+ cannot abort the transfer (except by closing the connection or issuing a
+ Cancel request), but it can discard unwanted CopyData and CopyDone
+ messages. Copy-out mode will also be initiated after a backend in
+ walsender mode executes a <command>BASE_BACKUP</command> statement; but
+ that statement must be sent as a simple Query message, and the server will
+ return multiple result sets before initiating copy-out mode (see <xref
+ linkend="protocol-replication-base-backup"/> for details).
</para>
<para>
- Copy-out mode (data transfer from the server) is initiated when the
- backend executes a <command>COPY TO STDOUT</command> SQL statement. The backend
- sends a CopyOutResponse message to the frontend, followed by
- zero or more CopyData messages (always one per row), followed by CopyDone.
- The backend then reverts to the command-processing mode it was
- in before the <command>COPY</command> started, and sends CommandComplete.
- The frontend cannot abort the transfer (except by closing the connection
- or issuing a Cancel request),
- but it can discard unwanted CopyData and CopyDone messages.
+ Copy-both mode, which allows high-speed bulk data transfer to
+ <emphasis>and</emphasis> from the server, is initiated when a backend in
+ walsender mode executes a <command>START_REPLICATION</command> statement.
+ The backend sends a CopyBothResponse message to the frontend. Both the
+ backend and the frontend may then send CopyData messages until either end
+ sends a CopyDone message. After the client sends a CopyDone message, the
+ connection goes from copy-both mode to copy-out mode, and the client may
+ not send any more CopyData messages. Similarly, when the server sends a
+ CopyDone message, the connection goes into copy-in mode, and the server
+ may not send any more CopyData messages. After both sides have sent a
+ CopyDone message, the copy mode is terminated, and the backend reverts to
+ the command-processing mode. The <command>START_REPLICATION</command> can
+ only be sent as a simple Query message, and after copy-both mode is
+ finished, a result set may also be sent (see <xref
+ linkend="protocol-replication"/> for details).
</para>
<para>
@@ -1194,6 +1195,18 @@ SELCT 1/0;<!-- this typo is intentional -->
terminating the copy-out mode.
</para>
+ <para>
+ In the event of a backend-detected error during copy-in mode or copy-both
+ mode (including receipt of a CopyFail message), the backend will issue an
+ ErrorResponse message. If the command was issued via an extended-query
+ message, the backend will now discard frontend messages until a Sync
+ message is received, then it will issue ReadyForQuery and return to normal
+ processing. If the command was issued in a simple Query message, the rest
+ of that message is discarded and ReadyForQuery is issued. In either case,
+ any subsequent CopyData, CopyDone, or CopyFail messages issued by the
+ frontend will simply be dropped.
+ </para>
+
<para>
It is possible for NoticeResponse and ParameterStatus messages to be
interspersed between CopyData messages; frontends must handle these cases,
@@ -1203,26 +1216,13 @@ SELCT 1/0;<!-- this typo is intentional -->
</para>
<para>
- There is another Copy-related mode called copy-both, which allows
- high-speed bulk data transfer to <emphasis>and</emphasis> from the server.
- Copy-both mode is initiated when a backend in walsender mode
- executes a <command>START_REPLICATION</command> statement. The
- backend sends a CopyBothResponse message to the frontend. Both
- the backend and the frontend may then send CopyData messages
- until either end sends a CopyDone message. After the client
- sends a CopyDone message, the connection goes from copy-both mode to
- copy-out mode, and the client may not send any more CopyData messages.
- Similarly, when the server sends a CopyDone message, the connection
- goes into copy-in mode, and the server may not send any more CopyData
- messages. After both sides have sent a CopyDone message, the copy mode
- is terminated, and the backend reverts to the command-processing mode.
- In the event of a backend-detected error during copy-both mode,
- the backend will issue an ErrorResponse message, discard frontend messages
- until a Sync message is received, and then issue ReadyForQuery and return
- to normal processing. The frontend should treat receipt of ErrorResponse
- as terminating the copy in both directions; no CopyDone should be sent
- in this case. See <xref linkend="protocol-replication"/> for more
- information on the subprotocol transmitted over copy-both mode.
+ When executing a command that initiates copy-in or copy-both mode, the
+ frontend should not send a Sync message until it has either received an
+ ErrorResponse message, or it has sent a CopyDone or CopyFail message. The
+ backend will ignore Flush and Sync messages received during copy-in or
+ copy-both mode for the convenience of client libraries that don't follow
+ this rule; but such client libraries will have difficulty handling errors
+ properly.
</para>
<para>
