Hello all,
I'm not sure if this is the correct list for extensions, if not please let
me know where this should be sent too.
I have a mid sized append only table that is read fairly infrequently and
contains 2 columns that contain on average about 0.5Kb and 2Kb of xml per
row. In an attempt to same some space I am planning to deflate them with a
predefined dictionary (we have a set schema for the xml) as unfortunately
changing the table design to a more traditional structure is off the table.
In general the compression and decompression will be done in the
application layer but because we sometimes need to look at the content I am
writing some PG functions to inflate and deflate them with this dictionary.
What I have found however is that when I go to load my extensions I get
this error:
test=# create extension pg_zlib;
ERROR: incompatible library "/usr/local/lib/postgresql/pg_zlib.so":
missing magic block
HINT: Extension libraries are required to use the PG_MODULE_MAGIC macro.
And I can't for the life of me work out why. As far as I understand them,
the docs indicate that this should only really happen if I'm compiling
against an 8.* headers. I'm on OSX el capitan, PG 9.5.3 (from brew), clang
is Apple LLVM version 7.3.0 (clang-703.0.31). I have manually checked all
the lib and include dirs and other than some suspicious looking ones in
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/
(which I removed) everything looks like it is correctly linked to the 9.5.3
version.
I don't regularly work in C and I've never worked on a PG extension before
so most of what I have is cobbled together by looking at what the mysql_fdw
extension does. If I'm missing something fundamental to either C or
extensions or if you can think of anything else I should check please let
me know.
I have inlined the output from `make install installcheck` and the relevant
parts of the files below. Thanks for you time and sorry about the
formatting.
Brent
make install installcheck
[~/work/pg_zlib (master)]$ make install installcheck || cat regression.diffs
clang -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-Wno-unused-command-line-argument -O2 -std=c11 -I. -I./
-I/usr/local/Cellar/postgresql/9.5.3/include/server
-I/usr/local/Cellar/postgresql/9.5.3/include/internal
-I/usr/local/opt/openssl/include -I/usr/local/opt/readline/include
-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2
-c -o pg_zlib.o pg_zlib.c
pg_zlib.c:24:1: warning: type specifier missing, defaults to 'int'
[-Wimplicit-int]
PG_MODULE_MAGIC;
^
1 warning generated.
clang -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute
-Wformat-security -fno-strict-aliasing -fwrapv
-Wno-unused-command-line-argument -O2 -bundle -multiply_defined suppress
-Wl,-undefined,dynamic_lookup -o pg_zlib.so pg_zlib.o -L/usr/local/lib
-L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib
-L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/lib
-Wl,-dead_strip_dylibs -bundle_loader
/usr/local/Cellar/postgresql/9.5.3/bin/postgres
/bin/sh
/usr/local/lib/postgresql/pgxs/src/makefiles/../../config/install-sh -c -d
'/usr/local/lib/postgresql'
/bin/sh
/usr/local/lib/postgresql/pgxs/src/makefiles/../../config/install-sh -c -d
'/usr/local/share/postgresql/extension'
/bin/sh
/usr/local/lib/postgresql/pgxs/src/makefiles/../../config/install-sh -c -d
'/usr/local/share/postgresql/extension'
/usr/bin/install -c -m 755 pg_zlib.so
'/usr/local/lib/postgresql/pg_zlib.so'
/usr/bin/install -c -m 644 .//pg_zlib.control
'/usr/local/share/postgresql/extension/'
/usr/bin/install -c -m 644 .//pg_zlib--1.0.sql
'/usr/local/share/postgresql/extension/'
/usr/local/lib/postgresql/pgxs/src/makefiles/../../src/test/regress/pg_regress
--inputdir=./ --bindir='/usr/local/Cellar/postgresql/9.5.3/bin'
--dbname=contrib_regression pg_zlib
(using postmaster on Unix socket, default port)
============== dropping database "contrib_regression" ==============
DROP DATABASE
============== creating database "contrib_regression" ==============
CREATE DATABASE
ALTER DATABASE
============== running regression test queries ==============
test pg_zlib ... FAILED
======================
1 of 1 tests failed.
======================
The differences that caused some tests to fail can be viewed in the
file "/Users/bdouglas/work/pg_zlib/regression.diffs". A copy of the test
summary that you see
above is saved in the file "/Users/bdouglas/work/pg_zlib/regression.out".
make: *** [installcheck] Error 1
*** /Users/bdouglas/work/pg_zlib/expected/pg_zlib.out Thu Sep 1 23:59:01
2016
--- /Users/bdouglas/work/pg_zlib/results/pg_zlib.out Fri Sep 2 10:26:14
2016
***************
*** 0 ****
--- 1,3 ----
+ create extension pg_zlib;
+ ERROR: incompatible library "/usr/local/lib/postgresql/pg_zlib.so":
missing magic block
+ HINT: Extension libraries are required to use the PG_MODULE_MAGIC macro.
======================================================================
Makefile
MODULE_big = pg_zlib
OBJS = pg_zlib.o
EXTENSION = pg_zlib
DATA = pg_zlib--1.0.sql
REGRESS = pg_zlib
pg_zlib.o: override CFLAGS += -std=c11
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
include $(PGXS)
pg_zlib--1.0.sql
create function inf(bytea)
returns text
as 'MODULE_PATHNAME'
language C strict;
create function def(text)
returns bytea
as 'MODULE_PATHNAME'
language C strict;
pg_zlib.control
comment = 'zlib extension for compressing and decompressing text'
default_version = '1.0'
module_pathname = '$libdir/pg_zlib'
relocatable = true
pg_zlib.c
#include "postgres.h"
#include "zlib.h"
#define CHUNK 16384
#define DICT_SIZE 32768
PG_MODULE_MAGIC;
text *
def(const bytea *in);
bytea *
inf(const text *in);
extern PGDLLEXPORT void _PG_init(void);
char *DICT;
size_t DICT_LEN;
void
_PG_init(void)
{
//Load dict
}
bytea *
inf(const text *in)
{
//inflate
}
text *
def(const bytea *in)
{
//deflate
}
sql/pg_zlib.sql
create extension pg_zlib;
