Re: [BUGS] BUG #6425: Bus error in slot_deform_tuple
I just wanted to say thanks to everyone who has been working so hard on
this issue. I realize it's not certain that this would fix the issues
we're seeing, but we'd be willing to try it out and report back. The only
caveat is we would need to deploy it to production, so if someone could let
us know what the risk factor is here (e.g. the potential to make things
worse), that would help us plan out how and when we would want to try it.
Thanks again, I'm really hopeful that this will fix the issues we're seeing
- and, if not, at least there seems to be good momentum towards getting to
the root of the problem.
-B
On Thu, Feb 2, 2012 at 10:45 PM, Tom Lane wrote:
> I wrote:
> > I have not gotten very far with the coredump, except to observe that
> > gdb says the Assert ought to have passed: ...
> > This suggests very strongly that indeed the buffer was changing under
> > us.
>
> I probably ought to let the test case run overnight before concluding
> anything, but at this point it's run for two-plus hours with no errors
> after applying this patch:
>
> diff --git a/src/backend/access/transam/xlog.c
> b/src/backend/access/transam/xlog.c
> index cce87a3..b128bfd 100644
> *** a/src/backend/access/transam/xlog.c
> --- b/src/backend/access/transam/xlog.c
> *** RestoreBkpBlocks(XLogRecPtr lsn, XLogRec
> *** 3716,3724
>}
>else
>{
> - /* must zero-fill the hole */
> - MemSet((char *) page, 0, BLCKSZ);
>memcpy((char *) page, blk, bkpb.hole_offset);
>memcpy((char *) page + (bkpb.hole_offset +
> bkpb.hole_length),
> blk + bkpb.hole_offset,
> BLCKSZ - (bkpb.hole_offset +
> bkpb.hole_length));
> --- 3716,3724
>}
>else
>{
>memcpy((char *) page, blk, bkpb.hole_offset);
> + /* must zero-fill the hole */
> + MemSet((char *) page + bkpb.hole_offset, 0,
> bkpb.hole_length);
>memcpy((char *) page + (bkpb.hole_offset +
> bkpb.hole_length),
> blk + bkpb.hole_offset,
> BLCKSZ - (bkpb.hole_offset +
> bkpb.hole_length));
>
>
> The existing code makes the page state transiently invalid (all zeroes)
> for no particularly good reason, and consumes useless cycles to do so,
> so this would be a good change in any case. The reason it is relevant
> to our current problem is that even though RestoreBkpBlocks faithfully
> takes exclusive lock on the buffer, *that is not enough to guarantee
> that no one else is touching that buffer*. Another backend that has
> already located a visible tuple on a page is entitled to keep accessing
> that tuple with only a buffer pin. So the existing code transiently
> wipes the data from underneath the other backend's pin.
>
> It's clear how this explains the symptoms I saw (Assert reporting wrong
> value of lp_flags even though the backend must previously have seen the
> right value, and the eventual coredump captured the right value too).
> It's less clear though whether this explains the symptoms seen by Duncan
> and Bridget. They presumably are running without asserts enabled, so
> it's unsurprising that they don't see the Assert failure, but what
> happens if control gets past that? There seem to be several possible
> failure modes:
>
> * Reader picks up zero lp_off/lp_len from the line pointer, and then
> tries to interpret the page header as a tuple. The results would be
> predictable only until RestoreBkpBlocks puts back nonzero data there,
> and then it's a bit of a mess. (In particular, t_hoff would be read out
> of the pd_prune_xid field if I counted right, and so would have a rather
> unpredictable value.)
>
> * Reader finds correct location of tuple, but sees t_hoff and/or
> t_infomask as zeroes (the latter possibly causing it to not check for
> nulls, if it doesn't think HEAP_HASNULL is set). Until RestoreBkpBlocks
> puts back the data, this would devolve to the next case, but after that
> it's a bit unpredictable again.
>
> * Reader finds correct location of data, but sees zeroes there.
>
> I believe that the reported failures involving palloc(-3) in
> text_to_cstring can be explained as instances of seeing zeroes where a
> text or varchar value is expected. Zeroes would look like a long-format
> varlena header with value zero, and the code would subtract 4 to get the
> data length, then add 1 for the trailing NUL byte that's needed in
> cstring representation, and thus ask for -3 bytes for the cstring
> equivalent. Furthermore, all three of the above cases end up doing that
> as long as the page stays all-zero for long enough. If some nonzero
> data gets restored before we're done examining the page, you get some
> other behavior of doubtful predictab
[BUGS] BUG #6430: Wal Segment Size is 128GB
The following bug has been logged on the website: Bug reference: 6430 Logged by: Girish Email address: giri.anami...@gmail.com PostgreSQL version: 9.0.0 Operating system: Linux Description: For Postgres Advanced Server 9.0 Version - default sever configuration has WAL SEGMENT SIZE set to 128GB. But the Wal Segment Files are of 16MB only. This behavior is not there in 9.1AS or any other non advanced Postgres version. -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6430: Wal Segment Size is 128GB
On 03.02.2012 11:25, giri.anami...@gmail.com wrote: The following bug has been logged on the website: Bug reference: 6430 Logged by: Girish Email address: giri.anami...@gmail.com PostgreSQL version: 9.0.0 Operating system: Linux Description: For Postgres Advanced Server 9.0 Version - default sever configuration has WAL SEGMENT SIZE set to 128GB. But the Wal Segment Files are of 16MB only. This behavior is not there in 9.1AS or any other non advanced Postgres version. This is a community mailing list, please send queries related to proprietary products directly to the vendor. supp...@enterprisedb.com would be the right address in this case. -- Heikki Linnakangas EnterpriseDB http://www.enterprisedb.com -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6347: Reopening bug #6085
On Mon, Dec 19, 2011 at 03:06:31PM +, alexander.for...@gmail.com wrote: > The following bug has been logged on the website: > > Bug reference: 6347 > Logged by: Alexander Fortin > Email address: alexander.for...@gmail.com > PostgreSQL version: 9.1.2 > Operating system: Ubuntu 10.04.3 > Description: > > Hi folks. I'm testing 9.1.2 (source compiled) pg_upgrade (upgrading from > 8.4.9) and it seems that the problem exposed in bug #6085 is still there. In > my case, the only way to make pg_upgrade work is to actually force > unix_socket_directory = '/tmp/' for the 8.4.9 cluster. > > Running in verbose mode > Performing Consistency Checks on Old Live Server > > Checking current, bin, and data directories ok > Checking cluster versions ok > connection to database failed: could not connect to server: No such file or > directory > Is the server running locally and accepting > connections on Unix domain socket "/tmp/.s.PGSQL.5432"? Yes. I wasn't clear in my email reply: http://archives.postgresql.org/pgsql-bugs/2011-07/msg00092.php When I said this will be fixed in 9.1, I meant pg_ctl will work in 9.1 for non-default socket directories, but when the 9.1 pg_upgrade accesses the 8.4 server, it has to use the 8.4 pg_ctl to do it, and that can't be fixed in a back-branch. I think we can only call this fixed when the old and new server is >= PG 9.1. Yeah, this isn't good, but it is the best we can do. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6425: Bus error in slot_deform_tuple
On 3 Feb 2012, at 06:45, Tom Lane wrote: > > I probably ought to let the test case run overnight before concluding > anything, but at this point it's run for two-plus hours with no errors > after applying this patch: > > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c Thank Tom! I've had this running for a few hours now without problems. Previously, on Sparc, the problem would occur in less than a minute. I did try a build with --enable-cassert and it didn't actually cause the problem. I think I left it for about an hour. Although a a relatively modern machine, this Sparc box I am using is painfully slow. My guess is that the extra time taken to perform the Assert code is hiding the problem. Now it's time to persuade the customer to use a patched version of pg ;) Cheers, Duncan P.S. I've been looking for an OS project to contribute to, and I think I'll see if I can help with pg. Time to look a the TODO list :)
Re: [BUGS] BUG #6425: Bus error in slot_deform_tuple
Duncan Rance writes: > On 3 Feb 2012, at 06:45, Tom Lane wrote: >> I probably ought to let the test case run overnight before concluding >> anything, but at this point it's run for two-plus hours with no errors >> after applying this patch: > Thank Tom! I've had this running for a few hours now without problems. > Previously, on Sparc, the problem would occur in less than a minute. > I did try a build with --enable-cassert and it didn't actually cause the > problem. I think I left it for about an hour. Although a a relatively modern > machine, this Sparc box I am using is painfully slow. My guess is that the > extra time taken to perform the Assert code is hiding the problem. My machine has been running the test case for twelve hours now with no errors, whereas with the bug the MTTF seemed to be half an hour or so. (Hm, I wonder whether turning off asserts would reduce the time to failure? Probably not worth the trouble to experiment now.) So I think we've got it, or at least we've found the problems this test case can expose. I'm still going to go read all the other WAL replay code... > Now it's time to persuade the customer to use a patched version of pg ;) FWIW, this bug might persuade us to do a set of releases pretty soon. regards, tom lane -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6404: postgres account not created during unattended install
Thank you for the reply. I have spent the better part of this week on this and
related issues.
My test procedure is as follows: create VM using Fusion 4.x with a fresh
install of OS X 10.7. The VM is hosted on a Mac also running 10.7. I create a
snapshot of this clean image. The first step of each test run is to restore the
snapshot image.
With this in mind, the comment about the previous installation does not seem to
apply to my report.
After considering your remarks and modifying the app install script, the app
installation completes normally with a functioning PostgreSQL installation.
However, the errors reported previous still appear in the logs.
The unattended mode results in the appearance of a user "PostgreSQL" in the
User Accounts preferences panel as well as the login window. It is interesting
to note that a user named "postgres" in any variant does not appear in /Users/
directory, nor in the file /etc/passwd/, with the exception of the os x
standard postgres account.
$ cat /etc/passwd | grep "Postgre"
_postgres:*:216:216:PostgreSQL Server:/var/empty:/usr/bin/false
The "one click" installer does not result in the appearance of the "PostgreSQL"
user account in the os x GUI.
To recap the questions:
1. are the errors reported in the postgres install log of a type to cause a
malfunction for end users?
2. how can I suppress the appearance of the user account "PostgreSQL" in the os
x login window?
Many thanks,
- Mark
On Jan 30, 2012, at 11:34 PM, Dharmendra Goyal wrote:
> Hi Mark,
>
> Install log shows that your db installation is successful. The error which
> you had sent is coming because your earlier installation failed to create
> 'postgres' user and when you ran the installer again, installer read
> /etc/postgres-reg.ini file to check any previous installation and found that
> previous installation was done. Hence installer tries to stop the db server
> in case it is running using postgres user. This command fails as postgres
> user was not created in your first installation because of some reason. But
> in latest installation (for which you have sent the log), postgres user has
> been created successfully and installation went successfully. You need to
> check why postgres user was not created in first go by checking logs of that
> installation.
>
> You are checking return status of the installbuilder.sh command run which is
> not 0 hence you are getting error ("install of DBMS failed".) in your script.
> Can yu please check what is the return value in your script of
> installbuilder.sh.
>
> Regards,
>
> On Tue, Jan 31, 2012 at 5:37 AM, Mark Phillips
> wrote:
>
> On Jan 29, 2012, at 11:07 PM, Dharmendra Goyal wrote:
>
>> On Sat, Jan 21, 2012 at 3:01 AM, wrote:
>> The following bug has been logged on the website:
>>
>> Bug reference: 6404
>> Logged by: Mark Phillips
>> Email address: mark.phill...@mophilly.com
>> PostgreSQL version: 9.1.2
>> Operating system: Mac OS X 10.7
>> Description:
>>
>> for a stand alone app that uses postgres, the app installer invokes the
>> shell script installbuilder.sh for the "unattended" install of postgres.
>>
>> The pg installer creates a user "PostgreSQL" that is not hidden, but does
>> not create a hidden account "postgres". As a result, the invocation of
>> pg_ctl fails. Error is "unknown login: postgres".
>> pg installer creates "postgres" user which somehow seems to be failing at
>> your machine. Can you please send installation log which can be found under
>> /tmp/install-postgresql.log. Also if you can run
>> installation_path/installer/server/createuser.sh script manually and check
>> the output, it can be helpful.
>>
>> Thanks,
>> --
>> Dharmendra Goyal
>
> Thank you for the reply. I apologize for the delay in responding.
>
> File install-postgresql.log attached.
>
> Please review the link below to a post in the postgresql forum for a bit more
> detail about this issue:
>
> http://forums.enterprisedb.com/posts/list/3042.page
>
> I looked into running the createuser.sh script. However, I encountered a bit
> of new information before I could get that far so I include it here in case
> it is relevant.
>
> Beginning anew with a clean os install, I ran the app installer that invokes
> the postgresql installer.
>
> Here is a snippet of the installer script that relates to installing postgres:
>
> # adjust current settings to allow the install to execute properly
> log "adjust current shared memory settings"
> log "..refer to
> http://www.postgresql.org/docs/9.1/interactive/kernel-resources.html#SYSVIPC";
> sysctl -w kern.sysv.shmmax=1610612736
> sysctl -w kern.sysv.shmall=393216
> sysctl -w kern.sysv.shmmin=1
> sysctl -w kern.sysv.shmmni=32
> sysctl -w kern.sysv.shmseg=8
> sysctl -w kern.maxprocperuid=512
> sysctl -w kern.maxproc=2048
>
> # install postgres
>
> cd "${BASE}"
> log "change directory to ${BASE}"
>
> log "==
Re: [BUGS] BUG #6404: postgres account not created during unattended install
On Fri, Feb 3, 2012 at 10:37 PM, Mark Phillips
wrote:
> Thank you for the reply. I have spent the better part of this week on this
> and related issues.
>
> My test procedure is as follows: create VM using Fusion 4.x with a fresh
> install of OS X 10.7. The VM is hosted on a Mac also running 10.7. I create
> a snapshot of this clean image. The first step of each test run is
> to restore the snapshot image.
>
> With this in mind, the comment about the previous installation does not
> seem to apply to my report.
>
> After considering your remarks and modifying the app install script, the
> app installation completes normally with a functioning PostgreSQL
> installation. However, the errors reported previous still appear in the
> logs.
>
If erros which you sent last tim appear even when you run the installation
first time, it means that there was an existing installation of postgres.
You can confirm same by checking /etc/postgres-reg.ini file.
>
> The unattended mode results in the appearance of a user "PostgreSQL" in
> the User Accounts preferences panel as well as the login window. It is
> interesting to note that a user named "postgres" in any variant does not
> appear in /Users/ directory, nor in the file /etc/passwd/, with the
> exception of the os x standard postgres account.
> $ cat /etc/passwd | grep "Postgre"
> _postgres:*:216:216:PostgreSQL Server:/var/empty:/usr/bin/false
>
I agree that _postgres exists because of the inbuilt pg which is bundled
with OS. But not sure why PostgreSQL user is there.
>
> The "one click" installer does not result in the appearance of the
> "PostgreSQL" user account in the os x GUI.
>
> To recap the questions:
> 1. are the errors reported in the postgres install log of a type to cause
> a malfunction for end users?
>
Even if the error (postgres user not found) is there in the log, it will
not cause any problem to the end user. Error is coming because installer
finds a previous installation of PG hence tries to stop the existing pg
server if it is running. So even if it is not able to stop any, it is not
going to cause any issue as subsequent installation of pg goes fine.
> 2. how can I suppress the appearance of the user account "PostgreSQL" in
> the os x login window?
>
Please check /etc/postgres-reg.ini file for bindled pg installation.
>
> Many thanks,
>
> - Mark
>
> On Jan 30, 2012, at 11:34 PM, Dharmendra Goyal wrote:
>
> Hi Mark,
>
> Install log shows that your db installation is successful. The error which
> you had sent is coming because your earlier installation failed to create
> 'postgres' user and when you ran the installer again, installer read
> /etc/postgres-reg.ini file to check any previous installation and found
> that previous installation was done. Hence installer tries to stop the db
> server in case it is running using postgres user. This command fails as
> postgres user was not created in your first installation because of some
> reason. But in latest installation (for which you have sent the log),
> postgres user has been created successfully and installation went
> successfully. You need to check why postgres user was not created in first
> go by checking logs of that installation.
>
> You are checking return status of the installbuilder.sh command run which
> is not 0 hence you are getting error ("install of DBMS failed".) in your
> script. Can yu please check what is the return value in your script of
> installbuilder.sh.
>
> Regards,
>
> On Tue, Jan 31, 2012 at 5:37 AM, Mark Phillips > wrote:
>
>>
>> On Jan 29, 2012, at 11:07 PM, Dharmendra Goyal wrote:
>>
>> On Sat, Jan 21, 2012 at 3:01 AM, wrote:
>>
>>> The following bug has been logged on the website:
>>>
>>> Bug reference: 6404
>>> Logged by: Mark Phillips
>>> Email address: mark.phill...@mophilly.com
>>> PostgreSQL version: 9.1.2
>>> Operating system: Mac OS X 10.7
>>> Description:
>>>
>>> for a stand alone app that uses postgres, the app installer invokes the
>>> shell script installbuilder.sh for the "unattended" install of postgres.
>>>
>>> The pg installer creates a user "PostgreSQL" that is not hidden, but does
>>> not create a hidden account "postgres". As a result, the invocation of
>>> pg_ctl fails. Error is "unknown login: postgres".
>>>
>> pg installer creates "postgres" user which somehow seems to be failing at
>> your machine. Can you please send installation log which can be found under
>> /tmp/install-postgresql.log. Also if you can run
>> installation_path/installer/server/createuser.sh script manually and check
>> the output, it can be helpful.
>>
>> Thanks,
>> --
>> Dharmendra Goyal
>>
>>
>> Thank you for the reply. I apologize for the delay in responding.
>>
>> File install-postgresql.log attached.
>>
>> Please review the link below to a post in the postgresql forum for a bit
>> more detail about this issue:
>>
>> http://forums.enterprisedb.com/posts/list/3042.page
>>
>> I looked into running the createuser.sh script. However, I encountere
Re: [BUGS] BUG #6347: Reopening bug #6085
On Fri, Feb 03, 2012 at 09:59:07AM -0500, Bruce Momjian wrote: > On Mon, Dec 19, 2011 at 03:06:31PM +, alexander.for...@gmail.com wrote: > > The following bug has been logged on the website: > > > > Bug reference: 6347 > > Logged by: Alexander Fortin > > Email address: alexander.for...@gmail.com > > PostgreSQL version: 9.1.2 > > Operating system: Ubuntu 10.04.3 > > Description: > > > > Hi folks. I'm testing 9.1.2 (source compiled) pg_upgrade (upgrading from > > 8.4.9) and it seems that the problem exposed in bug #6085 is still there. In > > my case, the only way to make pg_upgrade work is to actually force > > unix_socket_directory = '/tmp/' for the 8.4.9 cluster. > > > > Running in verbose mode > > Performing Consistency Checks on Old Live Server > > > > Checking current, bin, and data directories ok > > Checking cluster versions ok > > connection to database failed: could not connect to server: No such file or > > directory > > Is the server running locally and accepting > > connections on Unix domain socket "/tmp/.s.PGSQL.5432"? > > Yes. I wasn't clear in my email reply: > > http://archives.postgresql.org/pgsql-bugs/2011-07/msg00092.php > > When I said this will be fixed in 9.1, I meant pg_ctl will work in 9.1 > for non-default socket directories, but when the 9.1 pg_upgrade accesses > the 8.4 server, it has to use the 8.4 pg_ctl to do it, and that can't be > fixed in a back-branch. > > I think we can only call this fixed when the old and new server is >= PG > 9.1. Yeah, this isn't good, but it is the best we can do. Actually, thinking more about this, the old pg_upgrade didn't use pg_ctl wait/-w mode, but rather kept trying to connect until the server was up. Once pg_ctl -w worked in more cases in PG 9.1, the new pg_upgrade started using pg_ctl -w, but I didn't consider that we were unable to fix pg_ctl -w for non-standard settings in back branches. This can be seen as a regression in pg_upgrade functionality. Not sure what we can do about this, but perhaps there should be a mention in the pg_upgrad docs. I am going to wait to see if anyone else reports this problem --- the last report was against Postgres 9.0 in July, 2011. FYI, here is the 9.1 relesase not mention of the fix: Improve pg_ctl start's "wait" (-w) option (Bruce Momjian, Tom Lane) The wait mode is now significantly more robust. It will not get confused by non-default postmaster port numbers, non-default Unix-domain socket locations, permission problems, or stale postmaster lock files. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6347: Reopening bug #6085
Excerpts from Bruce Momjian's message of vie feb 03 15:52:29 -0300 2012: > Actually, thinking more about this, the old pg_upgrade didn't use pg_ctl > wait/-w mode, but rather kept trying to connect until the server was up. > Once pg_ctl -w worked in more cases in PG 9.1, the new pg_upgrade > started using pg_ctl -w, but I didn't consider that we were unable to > fix pg_ctl -w for non-standard settings in back branches. Hm, so what was wrong with just keep trying to connect? Surely it's not optimal, but if it's more robust than the alternative, maybe it's preferrable. -- Álvaro Herrera The PostgreSQL Company - Command Prompt, Inc. PostgreSQL Replication, Consulting, Custom Development, 24x7 support -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
Re: [BUGS] BUG #6347: Reopening bug #6085
On Fri, Feb 03, 2012 at 04:17:50PM -0300, Alvaro Herrera wrote: > > Excerpts from Bruce Momjian's message of vie feb 03 15:52:29 -0300 2012: > > > Actually, thinking more about this, the old pg_upgrade didn't use pg_ctl > > wait/-w mode, but rather kept trying to connect until the server was up. > > Once pg_ctl -w worked in more cases in PG 9.1, the new pg_upgrade > > started using pg_ctl -w, but I didn't consider that we were unable to > > fix pg_ctl -w for non-standard settings in back branches. > > Hm, so what was wrong with just keep trying to connect? Surely it's not > optimal, but if it's more robust than the alternative, maybe it's > preferrable. Well, it didn't always work. What we used to do, and still do, is to pass the port number in via -o '-p ', but that didn't handle the socket location, which is the case for the bug reporter. Now that I think of it, we might not have a regression from 9.0 --- my big point is that the socket location, while fixed in 9.1, didn't fix it in back branches, and therefore pg_upgrade doesn't handle them for old pre-9.1 clusters. I was unclear why the original pg_upgrade code used a separate connection loop instead of pg_ctl -w, but when I found how broken pg_ctl -w was, I fixed pg_ctl so at least going forward, it works for all use-cases. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-bugs
