On 3/16/23 01:20, Justin Pryzby wrote:
> On Mon, Mar 13, 2023 at 10:47:12PM +0100, Tomas Vondra wrote:
>>> Rearrange functions to their original order allowing a cleaner diff to the
>>> prior code;
>>
>> OK. I wasn't very enthusiastic about this initially, but after thinking
>> about it a bit I think it's meaningful to make diffs clearer. But I
>> don't see much difference with/without the patch. The
>>
>> git diff --diff-algorithm=minimal -w
>> e9960732a~:src/bin/pg_dump/compress_io.c src/bin/pg_dump/compress_gzip.c
>>
>> Produces ~25k diff with/without the patch. What am I doing wrong?
>
> Do you mean 25 kB of diff ?
Yes, if you redirect the git-diff to a file, it's a 25kB file.
> I agree that the statistics of the diff output don't change a lot:
>
> 1 file changed, 201 insertions(+), 570 deletions(-)
> 1 file changed, 198 insertions(+), 548 deletions(-)
>
> But try reading the diff while looking for the cause of a bug. It's the
> difference between reading 50, two-line changes, and reading a hunk that
> replaces 100 lines with a different 100 lines, with empty/unrelated
> lines randomly thrown in as context.
>
> When the diff is readable, the pg_fatal() also stands out.
>
I don't know, maybe I'm doing something wrong or maybe I just am bad at
looking at diffs, but if I apply the patch you submitted on 8/3 and do
the git-diff above (output attached), it seems pretty incomprehensible
to me :-( I don't see 50 two-line changes (I certainly wouldn't be able
to identify the root cause of the bug based on that).
>>> Change pg_fatal() to an assertion+comment;
>>
>> Yeah, that's reasonable. I'd even ditch the assert/comment, TBH. We
>> could add such protections against "impossible" stuff to a zillion other
>> places and the confusion likely outweighs the benefits.
>>
>>> Update the commit message and fix a few typos;
>>
>> Thanks. I don't want to annoy you too much, but could you split the
>> patch into the "empty-data" fix and all the other changes (rearranging
>> functions etc.)? I'd rather not mix those in the same commit.
>
> I don't know if that makes sense? The "empty-data" fix creates a new
> function called DeflateCompressorInit(). My proposal was to add the new
> function in the same place in the file as it used to be.
>
Got it. In that case I agree it's fine to do that in a single commit.
> The patch also moves the pg_fatal() that's being removed. I don't think
> it's going to look any cleaner to read a history involving the
> pg_fatal() first being added, then moved, then removed. Anyway, I'll
> wait while the community continues discussion about the pg_fatal().
>
I think the agreement was to replace the pg_fatal with and assert, and I
see your patch already does that.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
diff --git a/src/bin/pg_dump/compress_io.c b/src/bin/pg_dump/compress_gzip.c
index 5ac21f091f0..3c9ac55c266 100644
--- a/src/bin/pg_dump/compress_io.c
+++ b/src/bin/pg_dump/compress_gzip.c
@@ -1,285 +1,118 @@
/*-------------------------------------------------------------------------
*
- * compress_io.c
- * Routines for archivers to write an uncompressed or compressed data
- * stream.
+ * compress_gzip.c
+ * Routines for archivers to read or write a gzip compressed data stream.
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * This file includes two APIs for dealing with compressed data. The first
- * provides more flexibility, using callbacks to read/write data from the
- * underlying stream. The second API is a wrapper around fopen/gzopen and
- * friends, providing an interface similar to those, but abstracts away
- * the possible compression. Both APIs use libz for the compression, but
- * the second API uses gzip headers, so the resulting files can be easily
- * manipulated with the gzip utility.
- *
- * Compressor API
- * --------------
- *
- * The interface for writing to an archive consists of three functions:
- * AllocateCompressor, WriteDataToArchive and EndCompressor. First you call
- * AllocateCompressor, then write all the data by calling
WriteDataToArchive
- * as many times as needed, and finally EndCompressor. WriteDataToArchive
- * and EndCompressor will call the WriteFunc that was provided to
- * AllocateCompressor for each chunk of compressed data.
- *
- * The interface for reading an archive consists of just one function:
- * ReadDataFromArchive. ReadDataFromArchive reads the whole compressed
input
- * stream, by repeatedly calling the given ReadFunc. ReadFunc returns the
- * compressed data chunk at a time, and ReadDataFromArchive decompresses it
- * and passes the decompressed data to ahwrite(), until ReadFunc returns 0
- * to signal EOF.
- *
- * The interface is the same for compressed and uncompressed streams.
- *
- * Compressed stream API
- * ----------------------
- *
- * The compressed stream API is a wrapper around the C standard fopen() and
- * libz's gzopen() APIs. It allows you to use the same functions for
- * compressed and uncompressed streams. cfopen_read() first tries to open
- * the file with given name, and if it fails, it tries to open the same
- * file with the .gz suffix. cfopen_write() opens a file for writing, an
- * extra argument specifies if the file should be compressed, and adds the
- * .gz suffix to the filename if so. This allows you to easily handle both
- * compressed and uncompressed files.
- *
* IDENTIFICATION
- * src/bin/pg_dump/compress_io.c
+ * src/bin/pg_dump/compress_gzip.c
*
*-------------------------------------------------------------------------
*/
#include "postgres_fe.h"
+#include <unistd.h>
-#include "compress_io.h"
+#include "compress_gzip.h"
#include "pg_backup_utils.h"
#ifdef HAVE_LIBZ
-#include <zlib.h>
-#endif
-
-/*----------------------
- * Generic functions
- *----------------------
- */
-
-/*
- * Checks whether a compression algorithm is supported.
- *
- * On success returns NULL, otherwise returns a malloc'ed string which can be
- * used by the caller in an error message.
- */
-char *
-supports_compression(const pg_compress_specification compression_spec)
-{
- const pg_compress_algorithm algorithm = compression_spec.algorithm;
- bool supported = false;
-
- if (algorithm == PG_COMPRESSION_NONE)
- supported = true;
-#ifdef HAVE_LIBZ
- if (algorithm == PG_COMPRESSION_GZIP)
- supported = true;
-#endif
-
- if (!supported)
- return psprintf("this build does not support compression with
%s",
-
get_compress_algorithm_name(algorithm));
-
- return NULL;
-}
+#include "zlib.h"
/*----------------------
* Compressor API
*----------------------
*/
-
-/* typedef appears in compress_io.h */
-struct CompressorState
+typedef struct GzipCompressorState
{
- pg_compress_specification compression_spec;
- WriteFunc writeF;
-
-#ifdef HAVE_LIBZ
z_streamp zp;
- char *zlibOut;
- size_t zlibOutSize;
-#endif
-};
-/* Routines that support zlib compressed data I/O */
-#ifdef HAVE_LIBZ
-static void InitCompressorZlib(CompressorState *cs, int level);
-static void DeflateCompressorZlib(ArchiveHandle *AH, CompressorState *cs,
- bool flush);
-static void ReadDataFromArchiveZlib(ArchiveHandle *AH, ReadFunc readF);
-static void WriteDataToArchiveZlib(ArchiveHandle *AH, CompressorState *cs,
- const char
*data, size_t dLen);
-static void EndCompressorZlib(ArchiveHandle *AH, CompressorState *cs);
-#endif
-
-/* Routines that support uncompressed data I/O */
-static void ReadDataFromArchiveNone(ArchiveHandle *AH, ReadFunc readF);
-static void WriteDataToArchiveNone(ArchiveHandle *AH, CompressorState *cs,
- const char
*data, size_t dLen);
-
-/* Public interface routines */
-
-/* Allocate a new compressor */
-CompressorState *
-AllocateCompressor(const pg_compress_specification compression_spec,
- WriteFunc writeF)
-{
- CompressorState *cs;
-
-#ifndef HAVE_LIBZ
- if (compression_spec.algorithm == PG_COMPRESSION_GZIP)
- pg_fatal("this build does not support compression with %s",
"gzip");
-#endif
-
- cs = (CompressorState *) pg_malloc0(sizeof(CompressorState));
- cs->writeF = writeF;
- cs->compression_spec = compression_spec;
-
- /*
- * Perform compression algorithm specific initialization.
- */
-#ifdef HAVE_LIBZ
- if (cs->compression_spec.algorithm == PG_COMPRESSION_GZIP)
- InitCompressorZlib(cs, cs->compression_spec.level);
-#endif
-
- return cs;
-}
-
-/*
- * Read all compressed data from the input stream (via readF) and print it
- * out with ahwrite().
- */
-void
-ReadDataFromArchive(ArchiveHandle *AH,
- const pg_compress_specification
compression_spec,
- ReadFunc readF)
-{
- if (compression_spec.algorithm == PG_COMPRESSION_NONE)
- ReadDataFromArchiveNone(AH, readF);
- if (compression_spec.algorithm == PG_COMPRESSION_GZIP)
- {
-#ifdef HAVE_LIBZ
- ReadDataFromArchiveZlib(AH, readF);
-#else
- pg_fatal("this build does not support compression with %s",
"gzip");
-#endif
- }
-}
-
-/*
- * Compress and write data to the output stream (via writeF).
- */
-void
-WriteDataToArchive(ArchiveHandle *AH, CompressorState *cs,
- const void *data, size_t dLen)
-{
- switch (cs->compression_spec.algorithm)
- {
- case PG_COMPRESSION_GZIP:
-#ifdef HAVE_LIBZ
- WriteDataToArchiveZlib(AH, cs, data, dLen);
-#else
- pg_fatal("this build does not support compression with
%s", "gzip");
-#endif
- break;
- case PG_COMPRESSION_NONE:
- WriteDataToArchiveNone(AH, cs, data, dLen);
- break;
- case PG_COMPRESSION_LZ4:
- /* fallthrough */
- case PG_COMPRESSION_ZSTD:
- pg_fatal("invalid compression method");
- break;
- }
-}
-
-/*
- * Terminate compression library context and flush its buffers.
- */
-void
-EndCompressor(ArchiveHandle *AH, CompressorState *cs)
-{
-#ifdef HAVE_LIBZ
- if (cs->compression_spec.algorithm == PG_COMPRESSION_GZIP)
- EndCompressorZlib(AH, cs);
-#endif
- free(cs);
-}
+ void *outbuf;
+ size_t outsize;
+} GzipCompressorState;
-/* Private routines, specific to each compression method. */
-#ifdef HAVE_LIBZ
-/*
- * Functions for zlib compressed output.
- */
+/* Private routines that support gzip compressed data I/O */
+static void DeflateCompressorInit(CompressorState *cs);
+static void DeflateCompressorEnd(ArchiveHandle *AH, CompressorState *cs);
+static void DeflateCompressorCommon(ArchiveHandle *AH, CompressorState *cs,
+ bool
flush);
+static void EndCompressorGzip(ArchiveHandle *AH, CompressorState *cs);
+static void WriteDataToArchiveGzip(ArchiveHandle *AH, CompressorState *cs,
+ const void
*data, size_t dLen);
+static void ReadDataFromArchiveGzip(ArchiveHandle *AH, CompressorState *cs);
static void
-InitCompressorZlib(CompressorState *cs, int level)
+DeflateCompressorInit(CompressorState *cs)
{
+ GzipCompressorState *gzipcs;
z_streamp zp;
- zp = cs->zp = (z_streamp) pg_malloc(sizeof(z_stream));
+ gzipcs = (GzipCompressorState *)
pg_malloc0(sizeof(GzipCompressorState));
+ zp = gzipcs->zp = (z_streamp) pg_malloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
/*
- * zlibOutSize is the buffer size we tell zlib it can output to. We
- * actually allocate one extra byte because some routines want to
append a
- * trailing zero byte to the zlib output.
+ * outsize is the buffer size we tell zlib it can output to. We
actually
+ * allocate one extra byte because some routines want to append a
trailing
+ * zero byte to the zlib output.
*/
- cs->zlibOut = (char *) pg_malloc(ZLIB_OUT_SIZE + 1);
- cs->zlibOutSize = ZLIB_OUT_SIZE;
+ gzipcs->outbuf = pg_malloc(ZLIB_OUT_SIZE + 1);
+ gzipcs->outsize = ZLIB_OUT_SIZE;
- if (deflateInit(zp, level) != Z_OK)
- pg_fatal("could not initialize compression library: %s",
- zp->msg);
+ /* -Z 0 uses the "None" compressor -- not zlib with no compression */
+ Assert(cs->compression_spec.level != 0);
+
+ if (deflateInit(zp, cs->compression_spec.level) != Z_OK)
+ pg_fatal("could not initialize compression library: %s",
zp->msg);
/* Just be paranoid - maybe End is called after Start, with no Write */
- zp->next_out = (void *) cs->zlibOut;
- zp->avail_out = cs->zlibOutSize;
+ zp->next_out = gzipcs->outbuf;
+ zp->avail_out = gzipcs->outsize;
+
+ /* Keep track of gzipcs */
+ cs->private_data = gzipcs;
}
static void
-EndCompressorZlib(ArchiveHandle *AH, CompressorState *cs)
+DeflateCompressorEnd(ArchiveHandle *AH, CompressorState *cs)
{
- z_streamp zp = cs->zp;
+ GzipCompressorState *gzipcs = (GzipCompressorState *) cs->private_data;
+ z_streamp zp;
+ zp = gzipcs->zp;
zp->next_in = NULL;
zp->avail_in = 0;
/* Flush any remaining data from zlib buffer */
- DeflateCompressorZlib(AH, cs, true);
+ DeflateCompressorCommon(AH, cs, true);
if (deflateEnd(zp) != Z_OK)
pg_fatal("could not close compression stream: %s", zp->msg);
- free(cs->zlibOut);
- free(cs->zp);
+ pg_free(gzipcs->outbuf);
+ pg_free(gzipcs->zp);
+ pg_free(gzipcs);
+ cs->private_data = NULL;
}
static void
-DeflateCompressorZlib(ArchiveHandle *AH, CompressorState *cs, bool flush)
+DeflateCompressorCommon(ArchiveHandle *AH, CompressorState *cs, bool flush)
{
- z_streamp zp = cs->zp;
- char *out = cs->zlibOut;
+ GzipCompressorState *gzipcs = (GzipCompressorState *) cs->private_data;
+ z_streamp zp = gzipcs->zp;
+ void *out = gzipcs->outbuf;
int res = Z_OK;
- while (cs->zp->avail_in != 0 || flush)
+ while (gzipcs->zp->avail_in != 0 || flush)
{
res = deflate(zp, flush ? Z_FINISH : Z_NO_FLUSH);
if (res == Z_STREAM_ERROR)
pg_fatal("could not compress data: %s", zp->msg);
- if ((flush && (zp->avail_out < cs->zlibOutSize))
+ if ((flush && (zp->avail_out < gzipcs->outsize))
|| (zp->avail_out == 0)
|| (zp->avail_in != 0)
)
@@ -289,18 +122,18 @@ DeflateCompressorZlib(ArchiveHandle *AH, CompressorState
*cs, bool flush)
* chunk is the EOF marker in the custom format. This
should never
* happen but ...
*/
- if (zp->avail_out < cs->zlibOutSize)
+ if (zp->avail_out < gzipcs->outsize)
{
/*
* Any write function should do its own error
checking but to
* make sure we do a check here as well ...
*/
- size_t len = cs->zlibOutSize -
zp->avail_out;
+ size_t len = gzipcs->outsize -
zp->avail_out;
- cs->writeF(AH, out, len);
+ cs->writeF(AH, (char *) out, len);
}
- zp->next_out = (void *) out;
- zp->avail_out = cs->zlibOutSize;
+ zp->next_out = out;
+ zp->avail_out = gzipcs->outsize;
}
if (res == Z_STREAM_END)
@@ -309,16 +142,26 @@ DeflateCompressorZlib(ArchiveHandle *AH, CompressorState
*cs, bool flush)
}
static void
-WriteDataToArchiveZlib(ArchiveHandle *AH, CompressorState *cs,
- const char *data, size_t dLen)
+EndCompressorGzip(ArchiveHandle *AH, CompressorState *cs)
{
- cs->zp->next_in = (void *) unconstify(char *, data);
- cs->zp->avail_in = dLen;
- DeflateCompressorZlib(AH, cs, false);
+ /* If deflation was initialized, finalize it */
+ if (cs->private_data)
+ DeflateCompressorEnd(AH, cs);
}
static void
-ReadDataFromArchiveZlib(ArchiveHandle *AH, ReadFunc readF)
+WriteDataToArchiveGzip(ArchiveHandle *AH, CompressorState *cs,
+ const void *data, size_t dLen)
+{
+ GzipCompressorState *gzipcs = (GzipCompressorState *) cs->private_data;
+
+ gzipcs->zp->next_in = (void *) unconstify(void *, data);
+ gzipcs->zp->avail_in = dLen;
+ DeflateCompressorCommon(AH, cs, false);
+}
+
+static void
+ReadDataFromArchiveGzip(ArchiveHandle *AH, CompressorState *cs)
{
z_streamp zp;
char *out;
@@ -342,7 +185,7 @@ ReadDataFromArchiveZlib(ArchiveHandle *AH, ReadFunc readF)
zp->msg);
/* no minimal chunk size for zlib */
- while ((cnt = readF(AH, &buf, &buflen)))
+ while ((cnt = cs->readF(AH, &buf, &buflen)))
{
zp->next_in = (void *) buf;
zp->avail_in = cnt;
@@ -382,389 +225,196 @@ ReadDataFromArchiveZlib(ArchiveHandle *AH, ReadFunc
readF)
free(out);
free(zp);
}
-#endif /* HAVE_LIBZ */
-
-/*
- * Functions for uncompressed output.
- */
-
-static void
-ReadDataFromArchiveNone(ArchiveHandle *AH, ReadFunc readF)
-{
- size_t cnt;
- char *buf;
- size_t buflen;
-
- buf = pg_malloc(ZLIB_OUT_SIZE);
- buflen = ZLIB_OUT_SIZE;
-
- while ((cnt = readF(AH, &buf, &buflen)))
+/* Public routines that support gzip compressed data I/O */
+void
+InitCompressorGzip(CompressorState *cs,
+ const pg_compress_specification
compression_spec)
{
- ahwrite(buf, 1, cnt, AH);
- }
+ cs->readData = ReadDataFromArchiveGzip;
+ cs->writeData = WriteDataToArchiveGzip;
+ cs->end = EndCompressorGzip;
- free(buf);
-}
+ cs->compression_spec = compression_spec;
-static void
-WriteDataToArchiveNone(ArchiveHandle *AH, CompressorState *cs,
- const char *data, size_t dLen)
-{
- cs->writeF(AH, data, dLen);
+ /*
+ * If the caller has defined a write function, prepare the necessary
+ * state. Note that if the data is empty, End may be called immediately
+ * after Init, without ever calling Write.
+ */
+ if (cs->writeF)
+ DeflateCompressorInit(cs);
}
/*----------------------
- * Compressed stream API
+ * Compress File API
*----------------------
*/
-/*
- * cfp represents an open stream, wrapping the underlying FILE or gzFile
- * pointer. This is opaque to the callers.
- */
-struct cfp
+static size_t
+Gzip_read(void *ptr, size_t size, CompressFileHandle *CFH)
{
- FILE *uncompressedfp;
-#ifdef HAVE_LIBZ
- gzFile compressedfp;
-#endif
-};
-
-#ifdef HAVE_LIBZ
-static int hasSuffix(const char *filename, const char *suffix);
-#endif
+ gzFile gzfp = (gzFile) CFH->private_data;
+ size_t ret;
-/* free() without changing errno; useful in several places below */
-static void
-free_keep_errno(void *p)
-{
- int save_errno = errno;
-
- free(p);
- errno = save_errno;
-}
-
-/*
- * Open a file for reading. 'path' is the file to open, and 'mode' should
- * be either "r" or "rb".
- *
- * If the file at 'path' does not exist, we append the ".gz" suffix (if 'path'
- * doesn't already have it) and try again. So if you pass "foo" as 'path',
- * this will open either "foo" or "foo.gz".
- *
- * On failure, return NULL with an error code in errno.
- */
-cfp *
-cfopen_read(const char *path, const char *mode)
+ ret = gzread(gzfp, ptr, size);
+ if (ret != size && !gzeof(gzfp))
{
- cfp *fp;
-
- pg_compress_specification compression_spec = {0};
+ int errnum;
+ const char *errmsg = gzerror(gzfp, &errnum);
-#ifdef HAVE_LIBZ
- if (hasSuffix(path, ".gz"))
- {
- compression_spec.algorithm = PG_COMPRESSION_GZIP;
- fp = cfopen(path, mode, compression_spec);
+ pg_fatal("could not read from input file: %s",
+ errnum == Z_ERRNO ? strerror(errno) : errmsg);
}
- else
-#endif
- {
- compression_spec.algorithm = PG_COMPRESSION_NONE;
- fp = cfopen(path, mode, compression_spec);
-#ifdef HAVE_LIBZ
- if (fp == NULL)
- {
- char *fname;
- fname = psprintf("%s.gz", path);
- compression_spec.algorithm = PG_COMPRESSION_GZIP;
- fp = cfopen(fname, mode, compression_spec);
- free_keep_errno(fname);
- }
-#endif
- }
- return fp;
+ return ret;
}
-/*
- * Open a file for writing. 'path' indicates the path name, and 'mode' must
- * be a filemode as accepted by fopen() and gzopen() that indicates writing
- * ("w", "wb", "a", or "ab").
- *
- * If 'compression_spec.algorithm' is GZIP, a gzip compressed stream is opened,
- * and 'compression_spec.level' used. The ".gz" suffix is automatically added
to
- * 'path' in that case.
- *
- * On failure, return NULL with an error code in errno.
- */
-cfp *
-cfopen_write(const char *path, const char *mode,
- const pg_compress_specification compression_spec)
+static size_t
+Gzip_write(const void *ptr, size_t size, CompressFileHandle *CFH)
{
- cfp *fp;
-
- if (compression_spec.algorithm == PG_COMPRESSION_NONE)
- fp = cfopen(path, mode, compression_spec);
- else
- {
-#ifdef HAVE_LIBZ
- char *fname;
+ gzFile gzfp = (gzFile) CFH->private_data;
- fname = psprintf("%s.gz", path);
- fp = cfopen(fname, mode, compression_spec);
- free_keep_errno(fname);
-#else
- pg_fatal("this build does not support compression with %s",
"gzip");
- fp = NULL; /* keep compiler quiet
*/
-#endif
+ return gzwrite(gzfp, ptr, size);
}
- return fp;
-}
-
-/*
- * This is the workhorse for cfopen() or cfdopen(). It opens file 'path' or
- * associates a stream 'fd', if 'fd' is a valid descriptor, in 'mode'. The
- * descriptor is not dup'ed and it is the caller's responsibility to do so.
- * The caller must verify that the 'compress_algorithm' is supported by the
- * current build.
- *
- * On failure, return NULL with an error code in errno.
- */
-static cfp *
-cfopen_internal(const char *path, int fd, const char *mode,
- pg_compress_specification compression_spec)
-{
- cfp *fp = pg_malloc0(sizeof(cfp));
- if (compression_spec.algorithm == PG_COMPRESSION_GZIP)
- {
-#ifdef HAVE_LIBZ
- if (compression_spec.level != Z_DEFAULT_COMPRESSION)
+static int
+Gzip_getc(CompressFileHandle *CFH)
{
- /* user has specified a compression level, so tell zlib
to use it */
- char mode_compression[32];
+ gzFile gzfp = (gzFile) CFH->private_data;
+ int ret;
- snprintf(mode_compression, sizeof(mode_compression),
"%s%d",
- mode, compression_spec.level);
- if (fd >= 0)
- fp->compressedfp = gzdopen(fd,
mode_compression);
- else
- fp->compressedfp = gzopen(path,
mode_compression);
- }
- else
+ errno = 0;
+ ret = gzgetc(gzfp);
+ if (ret == EOF)
{
- /* don't specify a level, just use the zlib default */
- if (fd >= 0)
- fp->compressedfp = gzdopen(fd, mode);
+ if (!gzeof(gzfp))
+ pg_fatal("could not read from input file: %s",
strerror(errno));
else
- fp->compressedfp = gzopen(path, mode);
+ pg_fatal("could not read from input file: end of file");
}
- if (fp->compressedfp == NULL)
- {
- free_keep_errno(fp);
- fp = NULL;
- }
-#else
- pg_fatal("this build does not support compression with %s",
"gzip");
-#endif
+ return ret;
}
- else
- {
- if (fd >= 0)
- fp->uncompressedfp = fdopen(fd, mode);
- else
- fp->uncompressedfp = fopen(path, mode);
- if (fp->uncompressedfp == NULL)
+static char *
+Gzip_gets(char *ptr, int size, CompressFileHandle *CFH)
{
- free_keep_errno(fp);
- fp = NULL;
- }
- }
+ gzFile gzfp = (gzFile) CFH->private_data;
- return fp;
+ return gzgets(gzfp, ptr, size);
}
-/*
- * Opens file 'path' in 'mode' and compression as defined in
- * compression_spec. The caller must verify that the compression
- * is supported by the current build.
- *
- * On failure, return NULL with an error code in errno.
- */
-cfp *
-cfopen(const char *path, const char *mode,
- const pg_compress_specification compression_spec)
+static int
+Gzip_close(CompressFileHandle *CFH)
{
- return cfopen_internal(path, -1, mode, compression_spec);
-}
+ gzFile gzfp = (gzFile) CFH->private_data;
-/*
- * Associates a stream 'fd', if 'fd' is a valid descriptor, in 'mode'
- * and compression as defined in compression_spec. The caller must
- * verify that the compression is supported by the current build.
- *
- * On failure, return NULL with an error code in errno.
- */
-cfp *
-cfdopen(int fd, const char *mode,
- const pg_compress_specification compression_spec)
-{
- return cfopen_internal(NULL, fd, mode, compression_spec);
+ CFH->private_data = NULL;
+
+ return gzclose(gzfp);
}
-int
-cfread(void *ptr, int size, cfp *fp)
+static int
+Gzip_eof(CompressFileHandle *CFH)
{
- int ret;
+ gzFile gzfp = (gzFile) CFH->private_data;
- if (size == 0)
- return 0;
+ return gzeof(gzfp);
+}
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- {
- ret = gzread(fp->compressedfp, ptr, size);
- if (ret != size && !gzeof(fp->compressedfp))
+static const char *
+Gzip_get_error(CompressFileHandle *CFH)
{
+ gzFile gzfp = (gzFile) CFH->private_data;
+ const char *errmsg;
int errnum;
- const char *errmsg = gzerror(fp->compressedfp, &errnum);
- pg_fatal("could not read from input file: %s",
- errnum == Z_ERRNO ? strerror(errno) :
errmsg);
- }
- }
- else
-#endif
- {
- ret = fread(ptr, 1, size, fp->uncompressedfp);
- if (ret != size && !feof(fp->uncompressedfp))
- READ_ERROR_EXIT(fp->uncompressedfp);
- }
- return ret;
-}
+ errmsg = gzerror(gzfp, &errnum);
+ if (errnum == Z_ERRNO)
+ errmsg = strerror(errno);
-int
-cfwrite(const void *ptr, int size, cfp *fp)
-{
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- return gzwrite(fp->compressedfp, ptr, size);
- else
-#endif
- return fwrite(ptr, 1, size, fp->uncompressedfp);
+ return errmsg;
}
-int
-cfgetc(cfp *fp)
+static int
+Gzip_open(const char *path, int fd, const char *mode, CompressFileHandle *CFH)
{
- int ret;
+ gzFile gzfp;
+ char mode_compression[32];
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- {
- ret = gzgetc(fp->compressedfp);
- if (ret == EOF)
+ if (CFH->compression_spec.level != Z_DEFAULT_COMPRESSION)
{
- if (!gzeof(fp->compressedfp))
- pg_fatal("could not read from input file: %s",
strerror(errno));
- else
- pg_fatal("could not read from input file: end
of file");
- }
+ /*
+ * user has specified a compression level, so tell zlib to use
it
+ */
+ snprintf(mode_compression, sizeof(mode_compression), "%s%d",
+ mode, CFH->compression_spec.level);
}
else
-#endif
- {
- ret = fgetc(fp->uncompressedfp);
- if (ret == EOF)
- READ_ERROR_EXIT(fp->uncompressedfp);
- }
-
- return ret;
-}
+ strcpy(mode_compression, mode);
-char *
-cfgets(cfp *fp, char *buf, int len)
-{
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- return gzgets(fp->compressedfp, buf, len);
+ if (fd >= 0)
+ gzfp = gzdopen(dup(fd), mode_compression);
else
-#endif
- return fgets(buf, len, fp->uncompressedfp);
-}
+ gzfp = gzopen(path, mode_compression);
-int
-cfclose(cfp *fp)
-{
- int result;
+ if (gzfp == NULL)
+ return 1;
- if (fp == NULL)
- {
- errno = EBADF;
- return EOF;
- }
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- {
- result = gzclose(fp->compressedfp);
- fp->compressedfp = NULL;
- }
- else
-#endif
- {
- result = fclose(fp->uncompressedfp);
- fp->uncompressedfp = NULL;
- }
- free_keep_errno(fp);
+ CFH->private_data = gzfp;
- return result;
+ return 0;
}
-int
-cfeof(cfp *fp)
+static int
+Gzip_open_write(const char *path, const char *mode, CompressFileHandle *CFH)
{
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- return gzeof(fp->compressedfp);
- else
-#endif
- return feof(fp->uncompressedfp);
-}
+ char *fname;
+ int ret;
+ int save_errno;
-const char *
-get_cfp_error(cfp *fp)
-{
-#ifdef HAVE_LIBZ
- if (fp->compressedfp)
- {
- int errnum;
- const char *errmsg = gzerror(fp->compressedfp, &errnum);
+ fname = psprintf("%s.gz", path);
+ ret = CFH->open_func(fname, -1, mode, CFH);
- if (errnum != Z_ERRNO)
- return errmsg;
- }
-#endif
- return strerror(errno);
+ save_errno = errno;
+ pg_free(fname);
+ errno = save_errno;
+
+ return ret;
}
-#ifdef HAVE_LIBZ
-static int
-hasSuffix(const char *filename, const char *suffix)
+void
+InitCompressFileHandleGzip(CompressFileHandle *CFH,
+ const
pg_compress_specification compression_spec)
{
- int filenamelen = strlen(filename);
- int suffixlen = strlen(suffix);
+ CFH->open_func = Gzip_open;
+ CFH->open_write_func = Gzip_open_write;
+ CFH->read_func = Gzip_read;
+ CFH->write_func = Gzip_write;
+ CFH->gets_func = Gzip_gets;
+ CFH->getc_func = Gzip_getc;
+ CFH->close_func = Gzip_close;
+ CFH->eof_func = Gzip_eof;
+ CFH->get_error_func = Gzip_get_error;
- if (filenamelen < suffixlen)
- return 0;
+ CFH->compression_spec = compression_spec;
- return memcmp(&filename[filenamelen - suffixlen],
- suffix,
- suffixlen) == 0;
+ CFH->private_data = NULL;
+}
+#else /* HAVE_LIBZ */
+void
+InitCompressorGzip(CompressorState *cs,
+ const pg_compress_specification
compression_spec)
+{
+ pg_fatal("this build does not support compression with %s", "gzip");
}
-#endif
+void
+InitCompressFileHandleGzip(CompressFileHandle *CFH,
+ const
pg_compress_specification compression_spec)
+{
+ pg_fatal("this build does not support compression with %s", "gzip");
+}
+#endif /* HAVE_LIBZ */
