Hi, hackers.
psql \copy command can stream data from client host just like the normal
copy command can do on server host. Let's assume we want to stream a
local data file from psql:
pgsql =# \copy tbl from '/tmp/datafile' (format 'text');
If there's error inside the data file, \copy will still stream the
whole data file
before it reports the error. This is undesirable If the data file is very large,
or it's an infinite pipe. The normal copy command which reads file on the
server host can report error immediately as expected.
The problem seems to be pqParseInput3(). When error occurs in server
backend, it will send 'E' packet back to client. During \copy command, the
connection's asyncStatus is PGASYNC_COPY_IN, any 'E' packet will
get ignored by this path:
else if (conn->asyncStatus != PGASYNC_BUSY)
{
/* If not IDLE state, just wait ... */
if (conn->asyncStatus != PGASYNC_IDLE)
return;
So the client can't detect the error sent back by server.
I've attached a patch to demonstrate one way to workaround this. Save
the error via pqGetErrorNotice3() if the conn is PGASYNC_COPY_IN
status. The client code(psql) can detect the error via PQerrorMessage().
Probably still lots of details need to be considered but should be good
enough to start this discussion. Any thoughts on this issue?
Best regards
Peifeng Qiu
From a155adf6c1e6a6c31cf4146ce53827180247f384 Mon Sep 17 00:00:00 2001
From: Peifeng Qiu <pg...@qiupf.dev>
Date: Thu, 29 Dec 2022 11:37:19 +0900
Subject: [PATCH] psql: stop at error immediately during \copy
---
src/bin/psql/copy.c | 7 +++++++
src/interfaces/libpq/fe-protocol3.c | 13 +++++++++++++
2 files changed, 20 insertions(+)
diff --git a/src/bin/psql/copy.c b/src/bin/psql/copy.c
index 0f66ebc2ed..1d285312c2 100644
--- a/src/bin/psql/copy.c
+++ b/src/bin/psql/copy.c
@@ -580,6 +580,7 @@ handleCopyIn(PGconn *conn, FILE *copystream, bool isbinary, PGresult **res)
bool copydone = false;
int buflen;
bool at_line_begin = true;
+ char *err;
/*
* In text mode, we have to read the input one line at a time, so that
@@ -660,6 +661,12 @@ handleCopyIn(PGconn *conn, FILE *copystream, bool isbinary, PGresult **res)
OK = false;
break;
}
+ err = PQerrorMessage(conn);
+ if (err && *err)
+ {
+ /* We got error from server backend. Stop processing. */
+ break;
+ }
buflen = 0;
}
diff --git a/src/interfaces/libpq/fe-protocol3.c b/src/interfaces/libpq/fe-protocol3.c
index 364bad2b88..b50ce7da2e 100644
--- a/src/interfaces/libpq/fe-protocol3.c
+++ b/src/interfaces/libpq/fe-protocol3.c
@@ -150,6 +150,19 @@ pqParseInput3(PGconn *conn)
if (pqGetErrorNotice3(conn, false))
return;
}
+ else if (conn->asyncStatus == PGASYNC_COPY_IN)
+ {
+ /*
+ * Process and save error message during COPY. Client can check
+ * this error via PQerrorMessage.
+ */
+ if (id == 'E')
+ {
+ if (pqGetErrorNotice3(conn, true))
+ return;
+ break;
+ }
+ }
else if (conn->asyncStatus != PGASYNC_BUSY)
{
/* If not IDLE state, just wait ... */
--
2.39.0
