Re: [HACKERS] [ADMIN] postgres 9.0 crash when bringing up hot standby
Thanks. Yes, the LOAD command does work, on another database cluster on the same AIX machine. -Original Message- From: Fujii Masao [mailto:masao.fu...@gmail.com] Sent: Friday, August 06, 2010 10:31 AM To: Alanoly Andrews Cc: pgsql-ad...@postgresql.org; PostgreSQL-development Subject: Re: [ADMIN] postgres 9.0 crash when bringing up hot standby On Fri, Aug 6, 2010 at 10:10 PM, Alanoly Andrews wrote: > I'm testing "hot standby" using "streaming WAL records". On trying to bring > up the hot standby, I see the following error in the log: Thanks for the report! > LOG: database system was interrupted; last known up at 2010-08-05 14:46:36 > LOG: entering standby mode > LOG: restored log file "00010007" from archive > LOG: redo starts at 0/720 > LOG: consistent recovery state reached at 0/800 > LOG: database system is ready to accept read only connections > cp: /pgarclog/pg1/00010008: A file or directory in the path > name does not exist. > LOG: WAL receiver process (PID 1073206) was terminated by signal 11 > LOG: terminating any other active server processes > > There is a core dump. The debugger indicates the crash sequence as follows: > > (dbx) where > _alloc_initial_pthread(??) at 0x949567c > __pth_init(??) at 0x9493ba4 > uload(??, ??, ??, ??, ??, ??, ??, ??) at 0x9fff0001954 > load_64.load(??, ??, ??) at 0x904686c > loadAndInit() at 0x947bd7c > dlopen(??, ??) at 0x911cc4c > internal_load_library(libname = > "/apps/pg_9.0_b4/lib/postgresql/libpqwalreceiver.so"), line 234 in "dfmgr.c" > load_file(filename = "libpqwalreceiver", restricted = '\0'), line 156 in > "dfmgr.c" > WalReceiverMain(), line 248 in "walreceiver.c" > AuxiliaryProcessMain(argc = 2, argv = 0x0fffa8b8), line 428 in > "bootstrap.c" > StartChildProcess(type = WalReceiverProcess), line 4405 in "postmaster.c" > sigusr1_handler(postgres_signal_arg = 30), line 4227 in "postmaster.c" > __fd_select(??, ??, ??, ??, ??) at 0x911805c > postmaster.select(__fds = 5, __readlist = 0x0fffd0a8, __writelist = > (nil), __exceptlist = (nil), __timeout = 0x00c0), line 229 in > "time.h" > unnamed block in ServerLoop(), line 1391 in "postmaster.c" > unnamed block in ServerLoop(), line 1391 in "postmaster.c" > ServerLoop(), line 1391 in "postmaster.c" > PostmasterMain(argc = 1, argv = 0x0001102aa4b0), line 1092 in > "postmaster.c" > main(argc = 1, argv = 0x0001102aa4b0), line 188 in "main.c" > > Any pointers on how to resolve the issue will be much appreciated. Sorry, I have no idea what's wrong :( Is the simple LOAD command successful on your AIX? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately. Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [ADMIN] postgres 9.0 crash when bringing up hot standby
Ok..in response to the questions from Heikki, 1. Yes, "contrib/dblink" does work. Here's the output from the command used to "make" dblink: postgres:thimar> /usr/bin/gmake -C contrib/dblink install gmake: Entering directory `/dinabkp/faouzis/postgresql-9.0beta1/contrib/dblink' /bin/sh ../../config/install-sh -c -d '/dinabkp/faouzis/local2/pgsql/lib' /bin/sh ../../config/install-sh -c -d '/dinabkp/faouzis/local2/pgsql/share/contrib' /bin/sh ../../config/install-sh -c -m 755 dblink.so '/dinabkp/faouzis/local2/pgsql/lib/dblink.so' /bin/sh ../../config/install-sh -c -m 644 ./uninstall_dblink.sql '/dinabkp/faouzis/local2/pgsql/share/contrib' /bin/sh ../../config/install-sh -c -m 644 dblink.sql '/dinabkp/faouzis/local2/pgsql/share/contrib' gmake: Leaving directory `/dinabkp/faouzis/postgresql-9.0beta1/contrib/dblink' 2. I don't have records of the build logs for the regular postgres executables (which contains the libpqwalreceiver) but can do a new compile/make if that is required. But they were compiled and installed using the regular make files supplied along with the postgres source code. The following flags were added during the compilation: --without-readline --without-zlib--enable-debug --enable-cassert --enable-thread-safety Thanks. Alanoly. -Original Message- From: Robert Haas [mailto:robertmh...@gmail.com] Sent: Wednesday, August 11, 2010 10:13 AM To: Heikki Linnakangas Cc: Alanoly Andrews; pgsql-ad...@postgresql.org; PostgreSQL-development Subject: Re: [HACKERS] [ADMIN] postgres 9.0 crash when bringing up hot standby On Fri, Aug 6, 2010 at 3:53 PM, Heikki Linnakangas wrote: > So, loading libpqwalreceiver library crashes. It looks like it might be > pthread-related. Perhaps something wrong with our makefiles, causing > libpqwalreceiver to be built with wrong flags? Does contrib/dblink work? If > you look at the build log, what is the command line used to compile > libpqwalreceiver, and what is the command line used to build other > libraries, like contrib/dblink? I haven't seen any response to this from the OP, but it seems worrisome. Has anyone else tested a Hot Standby configuraration - successfully or otherwise - on AIX? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise Postgres Company This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately. Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS/SR on AIX
Fujii, All my tests so far were done on "clean" installs. Every version I tested on, beta2 through beta4, was compiled and installed in different and unique directories. Regards. Alanoly. -Original Message- From: Fujii Masao [mailto:masao.fu...@gmail.com] Sent: Thursday, August 26, 2010 1:44 AM To: Steve Singer Cc: Tom Lane; PostgreSQL-development Hackers; Alanoly Andrews Subject: Re: [HACKERS] HS/SR on AIX On Thu, Aug 26, 2010 at 12:45 AM, Steve Singer wrote: > A clean build from the beta4 source tarball where I'm careful to install > into a clean (ie no old beta2 artifacts laying around waiting to be > overwritten) isn't reproducing the issue. > > I'm happy to try other things if people suggest them (or if the original > reporter is still getting this after making sure he cleans up old files > first) but I'm thinking that was the issue. Thanks for the report! Alanoly, could you do a clean install and try the test again? Regards, -- Fujii Masao NIPPON TELEGRAPH AND TELEPHONE CORPORATION NTT Open Source Software Center This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately. Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] HS/SR on AIX
Steve, I have tried all the LIBPATH's that you suggested. Besides, I don't think the problem is that postgres cannot find the "libpqwalreceiver" library. It does find it, but crashes on loading it. See below a repeat of the copy and paste from my first post, showing the sequence just before the crash: (dbx) where _alloc_initial_pthread(??) at 0x949567c __pth_init(??) at 0x9493ba4 uload(??, ??, ??, ??, ??, ??, ??, ??) at 0x9fff0001954 load_64.load(??, ??, ??) at 0x904686c loadAndInit() at 0x947bd7c dlopen(??, ??) at 0x911cc4c internal_load_library(libname = "/apps/pg_9.0_b4/lib/postgresql/libpqwalreceiver.so"), line 234 in "dfmgr.c" load_file(filename = "libpqwalreceiver", restricted = '\0'), line 156 in "dfmgr.c" WalReceiverMain(), line 248 in "walreceiver.c" AuxiliaryProcessMain(argc = 2, argv = 0x0fffa8b8), line 428 in "bootstrap.c" StartChildProcess(type = WalReceiverProcess), line 4405 in "postmaster.c" sigusr1_handler(postgres_signal_arg = 30), line 4227 in "postmaster.c" __fd_select(??, ??, ??, ??, ??) at 0x911805c postmaster.select(__fds = 5, __readlist = 0x0fffd0a8, __writelist = (nil), __exceptlist = (nil), __timeout = 0x00c0), line 229 in "time.h" unnamed block in ServerLoop(), line 1391 in "postmaster.c" unnamed block in ServerLoop(), line 1391 in "postmaster.c" ServerLoop(), line 1391 in "postmaster.c" PostmasterMain(argc = 1, argv = 0x0001102aa4b0), line 1092 in "postmaster.c" main(argc = 1, argv = 0x0001102aa4b0), line 188 in "main.c" Alanoly. -Original Message- From: Steve Singer [mailto:ssin...@ca.afilias.info] Sent: Friday, August 27, 2010 3:45 PM To: Alanoly Andrews Cc: 'Fujii Masao'; Tom Lane; PostgreSQL-development Hackers; pgsql-ad...@postgresql.org Subject: Re: [HACKERS] HS/SR on AIX Alanoly Andrews wrote: > Fujii, > > All my tests so far were done on "clean" installs. Every version I tested on, > beta2 through beta4, was compiled and installed in different and unique > directories. > > Regards. Alanoly, If you do an export LIBPATH=/apps/pg_9.0_b4/lib before starting postgres on the replica does it make a difference? How about with a LIBPATH=/apps/pg_9.0_b4/lib/postgresql (I'm not exactly sure where libpq.a is on your install) > > Alanoly. -- Steve Singer Afilias Canada Data Services Developer 416-673-1142 This e-mail may be privileged and/or confidential, and the sender does not waive any related rights and obligations. Any distribution, use or copying of this e-mail or the information it contains by other than an intended recipient is unauthorized. If you received this e-mail in error, please advise me (by return e-mail or otherwise) immediately. Ce courriel est confidentiel et protégé. L'expéditeur ne renonce pas aux droits et obligations qui s'y rapportent. Toute diffusion, utilisation ou copie de ce message ou des renseignements qu'il contient par une personne autre que le (les) destinataire(s) désigné(s) est interdite. Si vous recevez ce courriel par erreur, veuillez m'en aviser immédiatement, par retour de courriel ou par un autre moyen. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
