Re: [HACKERS] Logical to physical page mapping

2012-10-30 Thread Markus Wanner
On 10/29/2012 12:05 PM, Robert Haas wrote: > OK, I'll stop babbling now... Not perceived as babbling here. Thanks for that nice round-up of options and ideas around the torn page problem. Regards Markus Wanner -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make chang

Re: [HACKERS] WIP checksums patch

2012-10-30 Thread jesper
> On 10/1/12 12:22 PM, Josh Berkus wrote: >>> >Perhaps we don't allow this to be turned per page, but rather per >>> >cluster, and per-cluster would require the entire cluster to be >>> >rewritten. >> We dicussed this last year, and options which require a total rewrite of >> the database in order

Re: [HACKERS] [PERFORM] out of memory

2012-10-30 Thread Tatsuo Ishii
> i have sql file (it's size are 1GB ) > when i execute it then the String is 987098801 bytr too long for encoding > conversion error occured . > pls give me solution about You hit the upper limit of internal memory allocation limit in PostgreSQL. IMO, there's no way to avoid the error except yo

[HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread crocket
MySQL permits a connection to access multiple databases. But Postgresql restricts a connection to one database. I think postgresql database connection is somewhat limited. Is it an old and decrepit design? or does it deserve some appreciations? -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread Will Crawford
On 30 October 2012 12:37, crocket wrote: > MySQL permits a connection to access multiple databases. > But Postgresql restricts a connection to one database. > I think postgresql database connection is somewhat limited. > > Is it an old and decrepit design? or does it deserve some appreciations? A

Re: [HACKERS] What are the advantages of not being able to access multiple databases with one connection?

2012-10-30 Thread Hannu Krosing
On 10/30/2012 01:37 PM, crocket wrote: MySQL permits a connection to access multiple databases. But Postgresql restricts a connection to one database. I think postgresql database connection is somewhat limited. Is it an old and decrepit design? or does it deserve some appreciations? It's an old

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: > Heikki Linnakangas escribió: > > Hmm. I was thinking that making this work in a non-backend context > > would be too hard, so I didn't give that much thought, but I guess > > there isn't many dependencies to backend functions after all

[HACKERS] Re: [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Alvaro Herrera
Andres Freund escribió: > On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: > > Heikki Linnakangas escribió: > > Andres commented elsewhere about reading xlog records, processing them > > as they came in, and do a running CRC while we're still reading it. I > > think this is a mistake

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > Andres Freund escribió: > > On Monday, October 29, 2012 08:58:53 PM Alvaro Herrera wrote: > > > Heikki Linnakangas escribió: > > > > > > Andres commented elsewhere about reading xlog records, processing them > > > as they came in, an

Re: [HACKERS] September 2012 commitfest

2012-10-30 Thread Greg Stark
On Mon, Oct 29, 2012 at 3:27 PM, Alvaro Herrera wrote: > * tuplesort memory usage: grow_memtuples > Greg Stark signed up for this I'll commit this later this week. I looked at it briefly at the conference but I think it actually does need some minor tweaks. > * Trim trailing NULL columns > J

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Andres Freund writes: > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: >> Am I the only one who finds this rather bizarre? Maybe this was okay >> when xlog data would only come from WAL files stored in the data >> directory at recovery, but if we're now receiving these from a remo

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Alvaro Herrera
Tom Lane escribió: > Andres Freund writes: > > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > >> Am I the only one who finds this rather bizarre? Maybe this was okay > >> when xlog data would only come from WAL files stored in the data > >> directory at recovery, but if we're no

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Tom Lane
Alvaro Herrera writes: > And yeah, I was thinking in one sum for the header and another one for > the data. I don't think it's worth the space. > If we're using CRC to detect end of WAL, what sense does it > make to have to read the whole record if we can detect the end by just > looking at the

Re: [HACKERS] [PATCH 3/8] Add support for a generic wal reading facility dubbed XLogReader

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 04:24:21 PM Alvaro Herrera wrote: > Tom Lane escribió: > > Andres Freund writes: > > > On Tuesday, October 30, 2012 03:20:03 PM Alvaro Herrera wrote: > > >> Am I the only one who finds this rather bizarre? Maybe this was okay > > >> when xlog data would only come from

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hi, On 29/10/12 21:14, Peter Geoghegan wrote: > I have a few initial observations on this. Thanks for your feedback. > > * I think you should be making the new GUC PGC_INTERNAL on platforms > where MAP_HUGETLB is not defined or available. See also, > effective_io_concurrency. This gives sane err

Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hi, On 29/10/12 16:33, Tom Lane wrote: > > I created a patch which implements MAP_HUGETLB for sysv shared memory > > segments > > (PGSharedMemoryCreate). It is based on tests of Tom Lane and Andres Freund, > > I > > added error handling, huge page size detection and a GUC variable. > > My recoll

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, Oh man, first I didn't sent the email to the list and now I forgot the attachment. I should really get some sleep, sorry for any inconveniences :( Greetings, CK diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b4fcbaf..66ed10f 100644 --- a/doc/src/sgml/config.sgml +++

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 08:20:33 PM Christian Kruse wrote: > Hey, > > Oh man, first I didn't sent the email to the list and now I forgot the > attachment. I should really get some sleep, sorry for any > inconveniences :( +#ifdef MAP_HUGETLB +# ifdef __ia64__ +#define PG_HUGETLB_BASE_ADD

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, On 30/10/12 20:33, Andres Freund wrote: > +#ifdef MAP_HUGETLB > +# ifdef __ia64__ > +#define PG_HUGETLB_BASE_ADDR (void *)(0x8000UL) > +#define PG_MAP_HUGETLB (MAP_HUGETLB|MAP_FIXED) > +# else > > Not your fault, but that looks rather strange to me. The level of > docum

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, ok, I think I implemented all of the changes you requested. All but the ia64 dependent, I have to do more research for this one. Greetings, CK diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index b4fcbaf..66ed10f 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/co

[HACKERS] [PATCH] PL/Python: Add spidata to all spiexceptions

2012-10-30 Thread Oskari Saarenmaa
PL/Python maps Python SPIError exceptions with 'spidata' attribute into SQL errors. PL/Python also creates classes in plpy.spiexceptions for all known errors but does not initialize their spidata, so when a PL/Python function raises such an exception it is not recognized properly and is always rep

Re: [HACKERS] Extensions Documentation

2012-10-30 Thread Peter Eisentraut
On Fri, 2012-10-26 at 09:09 -0700, David E. Wheeler wrote: > On Oct 26, 2012, at 5:27 AM, Peter Eisentraut wrote: > > The advantage that these programming language ecosystems have is that > > they can implement the processors for the documentation format in the > > language itself, so it's easy t

[HACKERS] Re: [COMMITTERS] pgsql: Preserve intermediate .c files in coverage mode

2012-10-30 Thread Peter Eisentraut
On Sun, 2012-10-28 at 11:10 -0400, Tom Lane wrote: > Peter Eisentraut writes: > > Preserve intermediate .c files in coverage mode > > > > The introduction of the .y -> .c pattern rule causes some .c files such > > as bootparse.c to be considered intermediate files in the .y -> .c -> .o > > rule ch

Re: [HACKERS] Extensions Documentation

2012-10-30 Thread David E. Wheeler
On Oct 30, 2012, at 2:08 PM, Peter Eisentraut wrote: >> True, which is why I was thinking of something relatively light-weight >> and self-contained like sundown. > > That's a markdown library, which transforms markdown to HTML, right? So > what would we do with the HTML? Put it into the HTML

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
On 10/29/12 6:40 AM, Chris Corbyn wrote: > What's the use case of this? It sounds like it will just create a maintenance > nightmare where some stuff you expect to lookup in in postgresql.conf is > actually hiding in the .auto file. Assuming only super users/sysadmins would > have the ability to

Re: [HACKERS] [COMMITTERS] pgsql: Preserve intermediate .c files in coverage mode

2012-10-30 Thread Tom Lane
Peter Eisentraut writes: > On Sun, 2012-10-28 at 11:10 -0400, Tom Lane wrote: >> [ blink... ] I'd vote for making them precious all the time. No such >> behavioral change was discussed or agreed to, > This is standard, default make behavior. It only showed up here because > the coverage proces

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Christopher Browne
On Tue, Oct 30, 2012 at 5:25 PM, Josh Berkus wrote: > On 10/29/12 6:40 AM, Chris Corbyn wrote: >> What's the use case of this? It sounds like it will just create a >> maintenance nightmare where some stuff you expect to lookup in in >> postgresql.conf is actually hiding in the .auto file. Assumi

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
> I should think that doing this requires heading back towards there > being a single unique configuration stream, and over the course of > several versions, deprecating the INCLUDE directive. Oh, maybe I should take a closer look at Amit's proposal then. I thought we planned to make use of the

[HACKERS] Limiting the number of parameterized indexpaths created

2012-10-30 Thread Tom Lane
I looked into the complaint of unreasonable planner runtime in bug #7626, http://archives.postgresql.org/pgsql-bugs/2012-10/msg00232.php In the given example, the indexed relation "foo" has join clauses with 30 other relations. The code that I added in commit 3b8968f25232ad09001bf35ab4cc59f5a5011

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Hannu Krosing
On 10/29/2012 03:14 PM, Amit Kapila wrote: On Monday, October 29, 2012 7:11 PM Chris Corbyn > What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postgresql.conf is actually hiding in the .auto file. Assuming only

Re: [HACKERS] Re: [HACKERS] Patch für MAP_HUGETLB for mmap() shared memory

2012-10-30 Thread Christian Kruse
Hey, On Tuesday 30 October 2012 20:33:18 Andres Freund wrote: > +#ifdef MAP_HUGETLB > +# ifdef __ia64__ > +#define PG_HUGETLB_BASE_ADDR (void *)(0x8000UL) > +#define PG_MAP_HUGETLB (MAP_HUGETLB|MAP_FIXED) > +# else > > Not your fault, but that looks rather strange to me. The

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Alvaro Herrera
Christopher Browne escribió: > On Tue, Oct 30, 2012 at 5:25 PM, Josh Berkus wrote: > > On 10/29/12 6:40 AM, Chris Corbyn wrote: > >> What's the use case of this? It sounds like it will just create a > >> maintenance nightmare where some stuff you expect to lookup in in > >> postgresql.conf is ac

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Tom Lane
Josh Berkus writes: >> I should think that doing this requires heading back towards there >> being a single unique configuration stream, and over the course of >> several versions, deprecating the INCLUDE directive. > Oh, maybe I should take a closer look at Amit's proposal then. I > thought we

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Andres Freund
On Tuesday, October 30, 2012 11:24:20 PM Tom Lane wrote: > The fact that this isn't being done by a large number of > people (is anybody at all actually doing it?) suggests to me that maybe > the demand isn't all that great. It might also be that the idea of implementing that yourself is quite sca

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Josh Berkus
Tom, > I'm not convinced we ever *had* a consensus on this. There were > proposals, but I'm not sure a majority ever bought into any one of 'em. > The whole problem of intermixing manual editing and programmatic editing > is just a big can of worms, and not everybody is prepared to give up the >

Re: [HACKERS] Caching for stable expressions with constant arguments v6

2012-10-30 Thread Josh Berkus
Marti, > Sadly some other things intervened and I have not had the time to > return to hacking on this patch. But I am hopeful I can get it into > shape during the 9.3 cycle. Hey, are you going to work on this for 9.3? I really could use the feature ... -- Josh Berkus PostgreSQL Experts Inc. h

Re: [HACKERS] Move postgresql_fdw_validator into dblink

2012-10-30 Thread Shigeru Hanada
Sorry for long absence. On Sat, Oct 20, 2012 at 4:24 PM, Kohei KaiGai wrote: > IIRC, the reason why postgresql_fdw instead of pgsql_fdw was > no other fdw module has shorten naming such as ora_fdw for > Oracle. > However, I doubt whether it is enough strong reason to force to > solve the technica

Re: [HACKERS] Multiple --table options for other commands

2012-10-30 Thread Josh Kupershmidt
On Sun, Oct 28, 2012 at 4:30 PM, Josh Kupershmidt wrote: > I see there's already a TODO for allowing pg_restore to accept > multiple --table arguments[1], but would folks support adding this > capability to various other commands which currently accept only a > single --table argument, such as cl

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 4:14 AM Josh Berkus wrote: > Tom, > > > I'm not convinced we ever *had* a consensus on this. There were > > proposals, but I'm not sure a majority ever bought into any one of > 'em. > > The whole problem of intermixing manual editing and programmatic > editing > > i

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 3:58 AM Andres Freund wrote: > On Tuesday, October 30, 2012 11:24:20 PM Tom Lane wrote: > > The fact that this isn't being done by a large number of > > people (is anybody at all actually doing it?) suggests to me that > maybe > > the demand isn't all that great. > >

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-10-30 Thread Amit Kapila
On Wednesday, October 31, 2012 3:32 AM Hannu Krosing wrote: On 10/29/2012 03:14 PM, Amit Kapila wrote: On Monday, October 29, 2012 7:11 PM Chris Corbyn > What's the use case of this? It sounds like it will just create a maintenance nightmare where some stuff you expect to lookup in in postg

[HACKERS] Problem Observed in behavior of Create Index Concurrently and Hot Update

2012-10-30 Thread Amit Kapila
There seems to be a problem in behavior of Create Index Concurrently and Hot Update in HEAD code . Please see the below testcase Step-1 --- Client-1 Create table t1(c1 int, c2 int, c3 int); insert into t1 values(1,2,3); Step-2 --- Client - 2 update t1 set c2=4; where c1