Re: [BUGS] BUG #7517: terminated by exception 0xC0000409

2012-09-04 Thread Magnus Hagander
On Tue, Sep 4, 2012 at 3:00 AM,   wrote:
> The following bug has been logged on the website:
>
> Bug reference:  7517
> Logged by:  Katsuhito Habaguchi
> Email address:  katsuhito.habagu...@fujixerox.co.jp
> PostgreSQL version: 8.3.20
> Operating system:   Windows Server 2008 R2
> Description:
>
> I'm Katsuhito Habaguchi. I'm working for Fuji Xerox Co,Ltd.
> In our server, PostgreSQL service stopped by the following error message.
>
> -
> LOG:  server process (PID 1124) was terminated by exception 0xC409
> HINT:  See C include file "ntstatus.h" for a description of the hexadecimal
> value.
> LOG:  terminating any other active server processes
> WARNING:  terminating connection because of crash of another server process
> DETAIL:  The postmaster has commanded this server process to roll back the
> current transaction and exit, because another server process exited
> abnormally and possibly corrupted shared memory.
> HINT:  In a moment you should be able to reconnect to the database and
> repeat your command.
> LOG:  all server processes terminated; reinitializing
> -

Interesting - 0xC409 means STACK_BUFFER_OVERRUN... The only case
I've seen that happen on connection startup is, as usual, with broken
antivirus/antispyware on the machine. If you have any such product
installed, try uninstalling it (not just disabling it) and see if the
problem goes away.


> OS: Windows Server 2008 R2
> PostgreSQL: 8.3.15
> Client: libpq C++ Application
>
> In client application log, we confirm that the above error occured in
> PQconnectDB or PQsetClientEncoding function.

The error occurred on the server, not the client if you got that
message. The question is what you did on the server when it happened.
If it happened connection, then I would look into things like anything
you have configured in shared_preload_libraries for example.


> The above error occured only one times.
> Is this problem fixed by latest package(8.3.20)?

Not that I know of, but it's always a possible side-effect from
something else - so it's always worth applying the latest patches.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


[BUGS] BUG #7518: "FORMAT" in the "COPY" command

2012-09-04 Thread matsusaka
The following bug has been logged on the website:

Bug reference:  7518
Logged by:  Daichi Matsusaka
Email address:  matsus...@sraoss.co.jp
PostgreSQL version: 9.1.5
Operating system:   (Centos5)
Description:

Hi,

I think manual "COPY" command to be in the wrong.

 postgres=# COPY table_name TO '/tmp/out.csv' CSV;
 COPY 1

 postgres=# COPY table_name TO '/tmp/out.csv' FORMAT 'CSV';
 ERROR:  syntax error at or near "FORMAT"
 LINE 1: COPY table_name TO '/tmp/out.csv' FORMAT CSV;

But, help is...

 postgres=# \h COPY
   :
   :
 COPY { table_name [ ( column [, ...] ) ] | ( query ) }
 TO { 'filename' | STDOUT }
 [ [ WITH ] ( option [, ...] ) ]
 
 where option can be one of:
 
 FORMAT format_name-- Is here.
 OIDS [ boolean ]
 DELIMITER 'delimiter_character'
   :

Is the same manual.
http://www.postgresql.org/docs/9.2/static/sql-copy.html



-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #7518: "FORMAT" in the "COPY" command

2012-09-04 Thread TAKATSUKA Haruka
Hi,

It's not bug.

>  postgres=# COPY table_name TO '/tmp/out.csv' FORMAT 'CSV';

This should be the following.

  COPY table_name TO '/tmp/out.csv' (FORMAT 'CSV')


On Tue, 04 Sep 2012 09:02:12 +
matsus...@sraoss.co.jp wrote:

> The following bug has been logged on the website:
> 
> Bug reference:  7518
> Logged by:  Daichi Matsusaka
> Email address:  matsus...@sraoss.co.jp
> PostgreSQL version: 9.1.5
> Operating system:   (Centos5)
> Description:
> 
> Hi,
> 
> I think manual "COPY" command to be in the wrong.
> 
>  postgres=# COPY table_name TO '/tmp/out.csv' CSV;
>  COPY 1
> 
>  postgres=# COPY table_name TO '/tmp/out.csv' FORMAT 'CSV';
>  ERROR:  syntax error at or near "FORMAT"
>  LINE 1: COPY table_name TO '/tmp/out.csv' FORMAT CSV;
> 
> But, help is...
> 
>  postgres=# \h COPY
>:
>:
>  COPY { table_name [ ( column [, ...] ) ] | ( query ) }
>  TO { 'filename' | STDOUT }
>  [ [ WITH ] ( option [, ...] ) ]
>  
>  where option can be one of:
>  
>  FORMAT format_name-- Is here.
>  OIDS [ boolean ]
>  DELIMITER 'delimiter_character'
>:
> 
> Is the same manual.
> http://www.postgresql.org/docs/9.2/static/sql-copy.html
> 
> 
> 
> -- 
> Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-bugs

---
TAKATSUKA Haruka / haru...@sraoss.co.jp



-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #7514: postgres -k no longer works with spaces in the path

2012-09-04 Thread Tom Lane
murr...@murrayc.com writes:
> At some point, probably in 9.1.5, the -k option to Postgres ("Unix-domain
> socket location" in --help), stopped accepting paths that contain spaces.

> For instance,
> -k '/tmp/testglom2FPDKW/path with spaces/some_postgres_data'

> It now fails with this error:
> FATAL:  invalid list syntax for "unix_socket_directories"

Ah, you're using the latest Fedora packaging of 9.1.x, which includes a
back-ported version of the unix_socket_directories change that's in
HEAD.  -k now effectively takes a list of directory names, not just
one, and it's pickier about whitespace.

IIRC, you can make it work if you put double quotes around the
space-containing name, so it'd look like this:

-k '"/blah blah blah"'


regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6528: pglesslog still referenced in docs, but no 9.1 support

2012-09-04 Thread Kevin Grittner
Bruce Momjian  wrote:
> On Mon, Apr  9, 2012 at 02:07:43PM -0500, Kevin Grittner wrote:
>> Bruce Momjian  wrote:
>>> On Mon, Apr 09, 2012 at 03:37:09PM -0300, Alvaro Herrera wrote:
>>> 
 (Another related tool is clearxlogtail which zeroes areas from
 WAL files when they are empty because of an early switch due to
 archive timeout).
>>> 
>>> Should we document that?
>>  
>> Our shop has been using that since before pglesslog existed, and
>> it has continued to work across many major releases with no
>> change to source code because it doesn't get down to the level of
>> looking at the xlog records themselves, just the segment and
>> page-level structures.  I should probably put packaging that up
>> as a proper extension and posting it to PGXN as another thing on
>> the list of things I'll do when I stumble over that pot of round
>> tuits at the end of the rainbow.  Right now the source is on
>> pgfoundry.  If anybody wants to do anything with it before I find
>> time, feel free.
> 
> I was going to add a mention of this to our docs, but it seems
> there are no released files for the project:
 
There is a single 175 line C source file to generate a stand-alone
executable filter.  The source has not needed any changes since it
was finished over 4 years ago.  Besides that C file, there is only a
make file.  I wasn't sure what form of packaging would be easier
than just checking out the source from the SCM.
 
What do you think would be the right thing to do with it at this
point?
 
-Kevin


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6528: pglesslog still referenced in docs, but no 9.1 support

2012-09-04 Thread Bruce Momjian
On Tue, Sep  4, 2012 at 12:11:53PM -0500, Kevin Grittner wrote:
> Bruce Momjian  wrote:
> > On Mon, Apr  9, 2012 at 02:07:43PM -0500, Kevin Grittner wrote:
> >> Bruce Momjian  wrote:
> >>> On Mon, Apr 09, 2012 at 03:37:09PM -0300, Alvaro Herrera wrote:
> >>> 
>  (Another related tool is clearxlogtail which zeroes areas from
>  WAL files when they are empty because of an early switch due to
>  archive timeout).
> >>> 
> >>> Should we document that?
> >>  
> >> Our shop has been using that since before pglesslog existed, and
> >> it has continued to work across many major releases with no
> >> change to source code because it doesn't get down to the level of
> >> looking at the xlog records themselves, just the segment and
> >> page-level structures.  I should probably put packaging that up
> >> as a proper extension and posting it to PGXN as another thing on
> >> the list of things I'll do when I stumble over that pot of round
> >> tuits at the end of the rainbow.  Right now the source is on
> >> pgfoundry.  If anybody wants to do anything with it before I find
> >> time, feel free.
> > 
> > I was going to add a mention of this to our docs, but it seems
> > there are no released files for the project:
>  
> There is a single 175 line C source file to generate a stand-alone
> executable filter.  The source has not needed any changes since it
> was finished over 4 years ago.  Besides that C file, there is only a
> make file.  I wasn't sure what form of packaging would be easier
> than just checking out the source from the SCM.
>  
> What do you think would be the right thing to do with it at this
> point?

Well, there should probably be a tar files with a README.  What about
adding this to /contrib?

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6528: pglesslog still referenced in docs, but no 9.1 support

2012-09-04 Thread Kevin Grittner
Bruce Momjian  wrote:
> On Tue, Sep  4, 2012 at 12:11:53PM -0500, Kevin Grittner wrote:
 
>> What do you think would be the right thing to do with it at this
>> point?
> 
> Well, there should probably be a tar files with a README.  What
> about adding this to /contrib?
 
I would be happy to add it to contrib if people want it there. 
There was some discussion of that when I wrote it, but then there
was a feeling that it would be uninteresting once people had
streaming replication.  I'm skeptical that people will ever lose
interest in archiving WAL files, because of the not-uncommon need to
keep archival backups or restore to a "point in time" to recover
from a mangled table.  As long as those needs are there, I suspect
that people will want to compress WAL files, including making
partially-filled files smaller.
 
-Kevin


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6528: pglesslog still referenced in docs, but no 9.1 support

2012-09-04 Thread Bruce Momjian
On Tue, Sep  4, 2012 at 12:49:40PM -0500, Kevin Grittner wrote:
> Bruce Momjian  wrote:
> > On Tue, Sep  4, 2012 at 12:11:53PM -0500, Kevin Grittner wrote:
>  
> >> What do you think would be the right thing to do with it at this
> >> point?
> > 
> > Well, there should probably be a tar files with a README.  What
> > about adding this to /contrib?
>  
> I would be happy to add it to contrib if people want it there. 
> There was some discussion of that when I wrote it, but then there
> was a feeling that it would be uninteresting once people had
> streaming replication.  I'm skeptical that people will ever lose
> interest in archiving WAL files, because of the not-uncommon need to
> keep archival backups or restore to a "point in time" to recover
> from a mangled table.  As long as those needs are there, I suspect
> that people will want to compress WAL files, including making
> partially-filled files smaller.

Agreed.  Please move forward on the contrib idea.

-- 
  Bruce Momjian  http://momjian.us
  EnterpriseDB http://enterprisedb.com

  + It's impossible for everything to be true. +


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6704: ALTER EXTENSION postgis SET SCHEMA leaves dangling relations

2012-09-04 Thread Alvaro Herrera
Excerpts from Tom Lane's message of vie ago 31 17:50:41 -0400 2012:
> Alvaro Herrera  writes:

> > 2. During ALTER EXTENSION execution, skip moving objects that have
> > already been moved.  Not really sure how this would be implemented;
> 
> +1 for this approach.  I'm a bit surprised we didn't hit this before,
> because in general there can be multiple dependency chains leading from
> object A to object B.  Most code that is doing more than trivial
> dependency-walking has to be prepared to cope with reaching the same
> object multiple times.
> 
> Implementation like this seems reasonable:
> 
> > 4. Maybe we could have AlterRelationNamespaceInternal check what the
> > current namespace is for the object, and do nothing if it's already the
> > target namespace.
> 
> We already have some such shortcut for ALTER OWNER, IIRC, so why not
> for SET SCHEMA as well?  I suspect that AlterRelationNamespaceInternal
> is not the only function that needs it, too.

It doesn't work :-(  The problem is that the outer sysscan in
extension.c gets to the table first, recurses there and updates the
sequence pg_depend tuple; then it gets out and the outer scan gets to
the sequence directly.  But this fails to notice that it has already
been updated, because we haven't done a CommandCounterIncrement.
However, if I add one, we get into Halloween problem because the
sequence is updated, command counter incremented, and the outer scan
sees the updated tuple (because it's using SnapshotNow) for the table so
it recurses again, and instead of "tuple updated by self" we get this:

alvherre=# alter extension isn set schema baz;
ERROR:  relation "test_b_seq" already exists in schema "baz"

So I think my other proposal is the way to fix the problem: each
AlterFooNamespace routine must update an ObjectAddresses array of
relocated objects, and the outer scan (extension.c) must skip objects
that are already in that list.

I have tested this theory (attached patch) and it solves the problem at
hand.  The patch is not complete: I haven't updated all the
AlterFooNamespace routines, only those necessary to fix this problem.
If we agree that this is the way to go I can complete and push.

Putting this kind of change this late in the 9.2 release process makes
me a bit nervous, but I don't see a simpler way to solve the problem.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services


alter-extension-schema-2.patch
Description: Binary data

-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #6704: ALTER EXTENSION postgis SET SCHEMA leaves dangling relations

2012-09-04 Thread Tom Lane
Alvaro Herrera  writes:
> Putting this kind of change this late in the 9.2 release process makes
> me a bit nervous, but I don't see a simpler way to solve the problem.

This is a pre-existing bug, not something new in 9.2, and quite honestly
I don't think we should try to fix it under time pressure if there is
any doubt about how to do so.  I don't like the proposed patch very
much either, so let's think about it some more.

regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


[BUGS] BUG #7519: incresed data base size and query performance lost

2012-09-04 Thread lokendra . dixit
The following bug has been logged on the website:

Bug reference:  7519
Logged by:  Lokendra Dixit
Email address:  lokendra.di...@rmsi.com
PostgreSQL version: 8.4.11
Operating system:   Windows 7
Description:

Hi,
1.When I am taking bacup and restore two three times when databse size
automatically incresed please descrive the solution. 

2. I have created non cluster index on tables to increase query performance
at same time performance incresed but after one month on same query on same
database again query lost performance two three time what is reson behind
this.
any suggestion ?

With regards
Lokendra




-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #7514: postgres -k no longer works with spaces in the path

2012-09-04 Thread Murray Cumming
On Tue, 2012-09-04 at 10:31 -0400, Tom Lane wrote:
> murr...@murrayc.com writes:
> > At some point, probably in 9.1.5, the -k option to Postgres ("Unix-domain
> > socket location" in --help), stopped accepting paths that contain spaces.
> 
> > For instance,
> > -k '/tmp/testglom2FPDKW/path with spaces/some_postgres_data'
> 
> > It now fails with this error:
> > FATAL:  invalid list syntax for "unix_socket_directories"
> 
> Ah, you're using the latest Fedora packaging of 9.1.x, which includes a
> back-ported version of the unix_socket_directories change that's in
> HEAD.  -k now effectively takes a list of directory names, not just
> one, and it's pickier about whitespace.
> 
> IIRC, you can make it work if you put double quotes around the
> space-containing name, so it'd look like this:
> 
>   -k '"/blah blah blah"'

Yes, thanks, that works for spaces, though it's rather odd.

However, how should I now specify a path that has a " or a /, or \,
which now cause similar errors with -k? Previously I could just use
g_shell_quote() and forget about it:
http://developer.gnome.org/glib/stable/glib-Shell-related-Utilities.html#g-shell-quote

To be honest, I don't have much interest in, or understanding of, this
-k option anyway. I just use it because, when starting my temporary
postgresql instance, it otherwise defaults to /var/run/postgresql/ (at
least in this distro build) which is not available to normal users. I
currently just specify the same directory that the postgresql data is
in. Maybe I should just use /tmp instead.

-- 
murr...@murrayc.com
www.murrayc.com
www.openismus.com



-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #7519: incresed data base size and query performance lost

2012-09-04 Thread Kevin Grittner
 wrote:
 
> PostgreSQL version: 8.4.11
 
> 1.When I am taking bacup and restore two three times when databse
> size automatically incresed please descrive the solution. 
> 
> 2. I have created non cluster index on tables to increase query
> performance at same time performance incresed but after one month
> on same query on same database again query lost performance two
> three time what is reson behind this.
 
Everything you are describing sounds like the kind of bloat you get
if you don't have autovacuum configured aggressively enough.  To
give people a little more to work with, please describe you hardware
and show the results of running the query on this page:
 
http://wiki.postgresql.org/wiki/Server_Configuration
 
-Kevin


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


Re: [BUGS] BUG #7517: terminated by exception 0xC0000409

2012-09-04 Thread fx HABAGUCHI KATSUHITO
Dear Mr.Hagander.
Thank you for your answer to my report.

> The error occurred on the server, not the client if you got that message.
> The question is what you did on the server when it happened.

I describe the situation when the error occurred a little in detail.

The client application connect with the server by "localhost(127.0.0.1)" 
connection.
It output error-code to application log when an error will occurred in libpq 
function.
When this error happened, I found the error-code means PQconnectDB or 
PQsetClientEncoding function in application log.

So I suppose that it occurred when this error connect with server.

> If it happened connection, then I would look into things like anything you
> have configured in shared_preload_libraries for example.

I specified following definition in postgresql.conf.
shared_preload_libraries = '$libdir/plugins/plugin_debugger.dll'

Sincerely yours.

//Katsuhito Habaguchi

> -Original Message-
> From: Magnus Hagander [mailto:mag...@hagander.net]
> Sent: Tuesday, September 04, 2012 4:04 PM
> To: fx HABAGUCHI KATSUHITO
> Cc: pgsql-bugs@postgresql.org
> Subject: Re: [BUGS] BUG #7517: terminated by exception 0xC409
> 
> On Tue, Sep 4, 2012 at 3:00 AM,  
> wrote:
> > The following bug has been logged on the website:
> >
> > Bug reference:  7517
> > Logged by:  Katsuhito Habaguchi
> > Email address:  katsuhito.habagu...@fujixerox.co.jp
> > PostgreSQL version: 8.3.20
> > Operating system:   Windows Server 2008 R2
> > Description:
> >
> > I'm Katsuhito Habaguchi. I'm working for Fuji Xerox Co,Ltd.
> > In our server, PostgreSQL service stopped by the following error message.
> >
> > -
> > LOG:  server process (PID 1124) was terminated by exception 0xC409
> > HINT:  See C include file "ntstatus.h" for a description of the
> > hexadecimal value.
> > LOG:  terminating any other active server processes
> > WARNING:  terminating connection because of crash of another server
> > process
> > DETAIL:  The postmaster has commanded this server process to roll back
> > the current transaction and exit, because another server process
> > exited abnormally and possibly corrupted shared memory.
> > HINT:  In a moment you should be able to reconnect to the database and
> > repeat your command.
> > LOG:  all server processes terminated; reinitializing
> > -
> 
> Interesting - 0xC409 means STACK_BUFFER_OVERRUN... The only case I've
> seen that happen on connection startup is, as usual, with broken
> antivirus/antispyware on the machine. If you have any such product installed,
> try uninstalling it (not just disabling it) and see if the problem goes
> away.
> 
> 
> > OS: Windows Server 2008 R2
> > PostgreSQL: 8.3.15
> > Client: libpq C++ Application
> >
> > In client application log, we confirm that the above error occured in
> > PQconnectDB or PQsetClientEncoding function.
> 
> The error occurred on the server, not the client if you got that message.
> The question is what you did on the server when it happened.
> If it happened connection, then I would look into things like anything you
> have configured in shared_preload_libraries for example.
> 
> 
> > The above error occured only one times.
> > Is this problem fixed by latest package(8.3.20)?
> 
> Not that I know of, but it's always a possible side-effect from something
> else - so it's always worth applying the latest patches.
> 
> --
>  Magnus Hagander
>  Me: http://www.hagander.net/
>  Work: http://www.redpill-linpro.com/


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


[BUGS] BUG #7520: regexp_matches does not work as expected

2012-09-04 Thread sbasurto
The following bug has been logged on the website:

Bug reference:  7520
Logged by:  Sergio Basurto
Email address:  sbasu...@soft-gator.com
PostgreSQL version: 9.1.4
Operating system:   Gentoo
Description:

I am using regexp_matches in a function like this
create or replace function test (v_string   in text)
returns varchar as $$
declare
i_strings   text[];
i_stringtext[];

i_strings :=
regexp_matches(v_string,E'[a-zA-Z0-9:\\s\\-\\.#%]*:[A-Za-z0-9\\s\\-\\.#%]+','g');

The I use the results 
foreach i_string slice 1 in array i_strings
loop
raise notice 'row = %',i_string;
end loop;

when I run the function like this
select test('1:Warehouse1;2:Warehouse2;');

postgresql complains:
ERROR:  query "SELECT
regexp_matches(v_string,E'[a-zA-Z0-9:\\s\\-\\.#%]*:[A-Za-z0-9\\s\\-\\.#%]+','g')"
returned more than one row

Off course I am expecting more than one row!

If I run:
select
regexp_matches('1:Warehouse1;2:Warehouse2;',E'[a-zA-Z0-9:\\s\\-\\.#%]*:[A-Za-z0-9\\s\\-\\.#%]+','g');
 regexp_matches 

 {1:Warehouse1}
 {2:Warehouse2}
(2 rows)




-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs


[BUGS] windows 8 RTM compatibility issue (could not reserve shared memory region for child)

2012-09-04 Thread Dave Vitek

Hello pgsql-bugs list,

I have attached a patch file that I believe resolves a compatibility 
issue with Windows 8 RTM and postgresql.  The impatient might want to 
just read the patch, this email is longer than it probably should be.  I 
have CC'd Seiko Ishida who expressed an interest in Windows 8 
compatibility on this list about a year ago.


We test postgres pretty heavily at my place of work (probably thousands 
of DBs created and exercised each day) on a number of platforms.  We've 
been doing compatibility testing with the Windows 8 previews and 
everything has been working well.  We are using the latest postgres release.


However, last week we upgraded from a preview version to the RTM version 
of Windows 8 x64, and it is clear that something changed. Since 
upgrading, we have been getting this error message a few times a day.  
Still very rare, but it never happened before the upgrade.


LOG:  could not reserve shared memory region (addr=0141) for child
0F8C: 487
LOG:  could not fork new process for connection: A blocking operation was
interrupted by a call to WSACancelBlockingCall.


This corresponds to VirtualAllocEx failing with ERROR_INVALID_ADDRESS 
inside win32_shmem.c (search for the error message).


Postgres uses a shared memory block to do much of its IPC.  This shared 
memory block presumably stores pointers to itself, and so must be 
allocated at the same address inside every postgres process.  In order 
to maximize the probability that this address will be available in child 
processes, the address should be reserved as early as possible in the 
lifetime of the child process (before the address space gets polluted).  
In order to achieve this goal, the postmaster starts its children in a 
suspended state and reserves the address before any code has executed in 
the child process.


However, there are a bunch of chunks of the virtual address space 
already reserved even when the child process is in this suspended 
state.  At least some of them are memory mapped images of binaries 
(duh).  I believe VirtualAllocEx is failing because something is already 
mapped (in the child) to the address the postmaster wants the shared 
memory segment to live at.


I wrote a small program that repeatedly starts postgres.exe in suspended 
mode and then tries to VirtualAllocEx 0x141.  The address is never 
blocked on Windows 7, but is blocked 2% of the time on Windows 8.  I 
attached windbg to the troublesome postgres process and used "!vadump 
-v" to see that there is a file mapped to the contentious address while 
postgres is in the suspended state.  I don't know if the failure rate is 
this bad for all addresses or just this one, but the possibility of 
conflict exists, since the postmaster was willing to use this address in 
at least one run.


So why hasn't this ever happened before?  I'm guessing that ASLR got 
better in the latest windows 8 patch, or maybe there's just more stuff 
in the virtual address space of a newborn process.


The postmaster originally decides where to place the shared memory 
segment by letting Windows (MapViewOfFileEx) choose where to put it.  So 
if the postmaster ends up using address 0x141, and then the 
postgres.exe image (for example) gets mapped to that same address in the 
child, you'll end up with the error message above.


I assume Windows changed so that the addresses in use inside a newborn 
process can now conflict with the addresses returned by 
MapViewOfFileEx(..., NULL).  These sets must have been disjoint in 
previous versions of windows, and postgres was relying on that behavior.


One straightforward "fix" is to specify a hardcoded address to 
MapViewOfFileEx instead of NULL.  This address should be carefully 
selected such that it is in an area disjoint from the portions of the 
address space that are potentially reserved in a newborn process, and 
also unlikely to be in use inside the postmaster when it first maps the 
shared memory.  This is pretty trivial to do for a particular 
version/configuration of Windows.  However, I see no future-proof 
solution (besides making the shared segment position independent).  If 
the hardcoded address is not available, you can always fall back on the 
current behavior.


On 64-bit versions of Windows, processes that do not use more than 4G or 
so of address space seem to always have a huge hole from about  
8000  ...  0700 .  Note that you cannot reserve 
addresses above 8TB, so it would need to go somewhere in this hole, 
above 4G is probably preferable.


32-bit Windows 8 also exists.  We haven't been testing on it, and so I 
can't confirm that the problem exists there.  Assuming it does, 32-bit 
processes are likely to be trickier since address space is more scarce.  
In practice, it appears that there is usually a big hole from 1000 
... 7000.


There is a security problem with the fix I outline above.  It bypasses 
ASLR to a limited degree, si

Re: [BUGS] BUG #7520: regexp_matches does not work as expected

2012-09-04 Thread Tom Lane
sbasu...@soft-gator.com writes:
> I am using regexp_matches in a function like this
> create or replace function test (v_string   in text)
> returns varchar as $$
> declare
> i_strings   text[];
> i_stringtext[];

> i_strings :=
> regexp_matches(v_string,E'[a-zA-Z0-9:\\s\\-\\.#%]*:[A-Za-z0-9\\s\\-\\.#%]+','g');

> postgresql complains:
> ERROR:  query "SELECT
> regexp_matches(v_string,E'[a-zA-Z0-9:\\s\\-\\.#%]*:[A-Za-z0-9\\s\\-\\.#%]+','g')"
> returned more than one row

As per David Johnston's response on pgsql-general, this isn't a bug:
simple assignments in plpgsql expect a simple source value, not a rowset.

Personally I'd just iterate over the regexp_matches result directly and
not bother trying to store it into a variable, for example

for string_var in select regexp_matches(...) loop ...

regards, tom lane


-- 
Sent via pgsql-bugs mailing list (pgsql-bugs@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-bugs