Re: [HACKERS] small bug in recoveryStopsHere()

2011-04-14 Thread Jaime Casanova
On Thu, Apr 14, 2011 at 1:30 PM, Robert Haas wrote: > I discovered while fooling around the other night that the named > restore point patch introduced a small bug into recoveryStopsHere(): > the test at the top of the function now lets through two > resource-manager IDs rather than one, but the r

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Tom Lane
Robert Haas writes: > On Fri, Apr 1, 2011 at 5:13 AM, Thom Brown wrote: >> On 1 April 2011 12:57, Shigeru HANADA wrote: >>> NOT NULL constraint on foreign table is just declaration and can't >>> force data integrity. And I noticed that CREATE FOREIGN TABLE >>> document doesn't mention that seri

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
David Fetter writes: > On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: >> ... I think a lot of this ultimately >> traces to the extensible, data-type-agnostic design philosophy. The >> fact that we don't know what an integer is until we look in pg_type, >> and don't know what an "=" ope

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Pavel Stehule
2011/4/15 Jim Nasby : > On Apr 14, 2011, at 4:20 PM, Kevin Grittner wrote: >> Tom Lane wrote: >>> Robert Haas writes: >> So far the most promising proposal I've seen seems to be to let id mean the parameter called id only when it can't refer to anything in the query. >> >>> Yeah, I

Re: [HACKERS] using a lot of maintenance_work_mem

2011-04-14 Thread Jim Nasby
On Apr 9, 2011, at 9:23 PM, Stephen Frost wrote: > Actually, Tom has a point in that work_mem can be set above 1GB (which > is where I had it set previously..). I didn't think it'd actually do > anything given the MaxAlloc limit, but suprisingly, it does (at least, > under 8.4). I'm currently try

Re: [HACKERS] Proposal for GSoC : ADJ dashboard (Administration related software)

2011-04-14 Thread Jim Nasby
On Apr 14, 2011, at 1:45 PM, Robert Haas wrote: > On Wed, Apr 6, 2011 at 11:06 AM, Erdinc Akkaya wrote: >> Project Title: ADJ Dashboard >> Name : Erdinc AKKAYA >> Email: erdinc.akk...@gmail.com >> >> Synopsis >> AnyDBJSP is a database monitoring and reporting solution with a browser >> based >> i

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Jim Nasby
On Apr 14, 2011, at 4:20 PM, Kevin Grittner wrote: > Tom Lane wrote: >> Robert Haas writes: > >>> So far the most promising proposal I've seen seems to be to let >>> id mean the parameter called id only when it can't refer to >>> anything in the query. > >> Yeah, I've come round to that positio

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Noah Misch
On Thu, Apr 14, 2011 at 11:23:49AM -0700, Robert Haas wrote: > On Thu, Apr 14, 2011 at 5:18 AM, Noah Misch wrote: > >> I guess my gut feeling is that it would make more sense to forbid it > >> outright for 9.1, and we can look at relaxing that restriction later > >> if we're so inclined. > >> > >>

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
On Thursday 14 April 2011 23:10:41 Tom Lane wrote: > Noah Misch writes: > > On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: > >> It shouldn't be > >> terribly difficult to come up with some kind of hash function based > >> on, say, the first two characters of the keyword that would be

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Kevin Grittner
Tom Lane wrote: > Robert Haas writes: >> So far the most promising proposal I've seen seems to be to let >> id mean the parameter called id only when it can't refer to >> anything in the query. > Yeah, I've come round to that position too. I think allowing > parameter names to be checked onl

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 14, 2011 at 1:55 PM, Tom Lane wrote: >> Hmm, what I read Dimitri to be proposing is that we *require* parameter >> names to be qualified with the function name.  I don't recall hearing >> that before.  It would solve the problem perhaps, but I think the moans >>

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Merlin Moncure
On Thu, Apr 14, 2011 at 4:10 PM, Tom Lane wrote: > Noah Misch writes: >> On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: >>> It shouldn't be >>> terribly difficult to come up with some kind of hash function based >>> on, say, the first two characters of the keyword that would be a lo

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Noah Misch writes: > On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: >> It shouldn't be >> terribly difficult to come up with some kind of hash function based >> on, say, the first two characters of the keyword that would be a lot >> faster than what we're doing now. > I'd look at `g

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 1:55 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine >> wrote: >>> Unless we make it so that no such version ever exists.  Meaning that the >>> code works fine as is or using WHERE id = developer_lookup.id.  AS id >>> can't

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 12:45 PM, Noah Misch wrote: >> It shouldn't be >> terribly difficult to come up with some kind of hash function based >> on, say, the first two characters of the keyword that would be a lot >> faster than what we're doing now. > > I'd look at `gperf', which generates code f

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine > wrote: >> Unless we make it so that no such version ever exists.  Meaning that the >> code works fine as is or using WHERE id = developer_lookup.id.  AS id >> can't ever be the parameter in this case, you're just fine. >>

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Andres Freund writes: > Where I am with you is that its quite possible that it will not make sense > (performancewise) for all contexts. Which is quite annoying. The mcxt stuff was designed from day one to support multiple types of contexts, so it wouldn't be very hard at all to have different c

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
On Thursday 14 April 2011 22:21:26 Alvaro Herrera wrote: > Excerpts from Andres Freund's message of jue abr 14 17:08:34 -0300 2011: > > Hi, > > > > On Thursday 14 April 2011 16:43:16 Tom Lane wrote: > > > I doubt that it's possible to make AllocSetAlloc radically cheaper. > > > > I actually doubt

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Merlin Moncure
On Thu, Apr 14, 2011 at 2:08 AM, Greg Smith wrote: > This week several list regulars here waded into the MySQL Convention.  I > decided to revisit PostgreSQL vs. MySQL performance using the sysbench > program as part of that.  It's not important to what I'm going to describe > to understand exactl

Re: [HACKERS] WAL, xl_heap_insert and tuple oid mystry

2011-04-14 Thread jagan
Hi, > > > Where is the OID of tuple stored in a WAL record of a > tuple? If not with xl_heap_header, where is it stored? Is it > stored at all? > > It's stored in the "tuple data" portion. I see it now. I was having alignment issues, which I resolved. Thanks for your help. I am still not sure

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Greg Smith
Heikki Linnakangas wrote: In this case you could just use prepared statements and get rid of all the parser related overhead, which includes much of the allocations. Trying that gives me around 9200 TPS instead of 5500 on my laptop, so a pretty big gain here. Will have to include that in my n

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Alvaro Herrera
Excerpts from Andres Freund's message of jue abr 14 17:08:34 -0300 2011: > Hi, > > On Thursday 14 April 2011 16:43:16 Tom Lane wrote: > > I doubt that it's possible to make AllocSetAlloc radically cheaper. > I actually doubt your doubt. I think you could add some SLAB like interface > for common

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Andres Freund
Hi, On Thursday 14 April 2011 16:43:16 Tom Lane wrote: > I doubt that it's possible to make AllocSetAlloc radically cheaper. I actually doubt your doubt. I think you could add some SLAB like interface for common allocation sizes making it significantly faster for some uses (because allocation/de

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Heikki Linnakangas writes: > There's one very low-hanging fruit here, though. I profiled the pgbench > case, with -M prepared, and found that like in Greg Smith's profile, > hash_seq_search pops up quite high in the list. Those calls are coming > from LockReleaseAll(), where we scan the local l

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Noah Misch
On Thu, Apr 14, 2011 at 11:15:00AM -0700, Robert Haas wrote: > It's fairly far down in the noise on this particular profile, but in > the low-hanging fruit department, I think we should fix > ScanKeywordLookup to use a smarter algorithm that is more like O(1) > rather than O(lg n) in the number of

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Heikki Linnakangas
On 14.04.2011 17:43, Tom Lane wrote: Greg Smith writes: samples %image name symbol name 53548 6.7609 postgres AllocSetAlloc 32787 4.1396 postgres MemoryContextAllocZeroAligned 26330 3.3244 postgres base_yypars

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 11:37 AM, Dimitri Fontaine wrote: > Robert Haas writes: >> rhaas=# CREATE OR REPLACE FUNCTION developer_lookup(id integer) >> RETURNS text AS $$SELECT name FROM developer WHERE id = $1$$ LANGUAGE >> sql STABLE; >> >> Now, when this person attempts to recreate this function

Re: [HACKERS] proposal: a validator for configuration files

2011-04-14 Thread Robert Haas
On Mon, Apr 4, 2011 at 2:03 PM, Alexey Klyukin wrote: > Here's the update of Selena's patch, which also shows all errors in > configuration parameters (as well as parser errors) during reload. You should add this here: https://commitfest.postgresql.org/action/commitfest_view/open On a quick gla

Re: [HACKERS] Proposal for GSoC : ADJ dashboard (Administration related software)

2011-04-14 Thread Robert Haas
On Wed, Apr 6, 2011 at 11:06 AM, Erdinc Akkaya wrote: > Project Title: ADJ Dashboard > Name : Erdinc AKKAYA > Email: erdinc.akk...@gmail.com > > Synopsis > AnyDBJSP is a database monitoring and reporting solution with a browser > based > interface. ADJ dashboard mainly will be written for database

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Robert Haas
On Fri, Apr 1, 2011 at 1:29 AM, Shigeru HANADA wrote: > In addition to the 2nd GRANT above, "GRANT SELECT (colour) ON stuff TO > user_a" (omitting TABLE) will succeed too because parser assumes that > the target object is a regular table if object type was TABLE or > omitted. This inconsistent beh

Re: [HACKERS] Foreign table permissions and cloning

2011-04-14 Thread Robert Haas
On Fri, Apr 1, 2011 at 5:13 AM, Thom Brown wrote: > On 1 April 2011 12:57, Shigeru HANADA wrote: >> NOT NULL constraint on foreign table is just declaration and can't >> force data integrity.  And I noticed that CREATE FOREIGN TABLE >> document doesn't mention that serial and bigserial can't be u

Re: [HACKERS] WIP: Allow SQL-language functions to reference parameters by parameter name

2011-04-14 Thread Dimitri Fontaine
Robert Haas writes: > rhaas=# CREATE OR REPLACE FUNCTION developer_lookup(id integer) > RETURNS text AS $$SELECT name FROM developer WHERE id = $1$$ LANGUAGE > sql STABLE; > > Now, when this person attempts to recreate this function on a > hypothetical version of PostgreSQL that thinks "id" is amb

Re: [DOCS] [HACKERS] Uppercase SGML entity declarations

2011-04-14 Thread Robert Haas
On Tue, Apr 5, 2011 at 6:55 PM, Josh Kupershmidt wrote: > On Mon, Apr 4, 2011 at 3:02 PM, Robert Haas wrote: >> In theory, we have >> documentation that explains this: >> >> http://www.postgresql.org/docs/current/static/docguide-toolsets.html > > While we're on the subject.. > > Attached is a pat

[HACKERS] small bug in recoveryStopsHere()

2011-04-14 Thread Robert Haas
I discovered while fooling around the other night that the named restore point patch introduced a small bug into recoveryStopsHere(): the test at the top of the function now lets through two resource-manager IDs rather than one, but the remainder of the function tests only the record_info flag and

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 5:18 AM, Noah Misch wrote: >> I guess my gut feeling is that it would make more sense to forbid it >> outright for 9.1, and we can look at relaxing that restriction later >> if we're so inclined. >> >> Much as with the problem Tom fixed in commit >> eb51af71f241e8cb199790de

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 7:26 AM, A.M. wrote: > From what I understood, the primary purpose of the SysV check was to protect > the shared memory from multiple stompers. The interlock was a neat > side-effect. Not really - the purpose of the interlock is to protect the underlying data files. The

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Robert Haas
On Thu, Apr 14, 2011 at 7:43 AM, Tom Lane wrote: > I doubt that it's possible to make AllocSetAlloc radically cheaper. > I think the more likely route to improvement there is going to be to > find a way to do fewer pallocs.  For instance, if we had more rigorous > rules about which data structures

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread David Fetter
On Thu, Apr 14, 2011 at 10:43:16AM -0400, Tom Lane wrote: > > I doubt that it's possible to make AllocSetAlloc radically cheaper. > I think the more likely route to improvement there is going to be to > find a way to do fewer pallocs. For instance, if we had more rigorous > rules about which data

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Martijn van Oosterhout
On Thu, Apr 14, 2011 at 10:26:33AM -0400, A.M. wrote: > 1) the SysV nattch method's primary purpose is to protect the shmem > region. This is no longer necessary in my patch because the shared > memory in unlinked immediately after creation, so only the initial > postmaster and its children have ac

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 11:37 PM, Tom Lane wrote: > "A.M." writes: >> 1. As long one keeps SysV shared memory around, the postgresql project >> has to maintain the annoying platform-specific document on how to >> configure the poorly named kernel parameters. > > No, if it's just a small area, I don

Re: [HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Tom Lane
Greg Smith writes: > samples %image name symbol name > 53548 6.7609 postgres AllocSetAlloc > 32787 4.1396 postgres MemoryContextAllocZeroAligned > 26330 3.3244 postgres base_yyparse > 21723 2.7427 postgres

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 14, 2011, at 8:22 AM, Florian Weimer wrote: > * Tom Lane: > >> Well, the fundamental point is that "ignoring NFS" is not the real >> world. We can't tell people not to put data directories on NFS, >> and even if we did tell them not to, they'd still do it. And NFS >> locking is not trus

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread A.M.
On Apr 13, 2011, at 9:30 PM, Robert Haas wrote: > On Wed, Apr 13, 2011 at 6:11 PM, A.M. wrote: >>> I don't see why we need to get rid of SysV shared memory; needing less >>> of it seems just as good. >> >> 1. As long one keeps SysV shared memory around, the postgresql project has >> to maintai

Re: [HACKERS] pg_dump --binary-upgrade vs. ALTER TYPE ... DROP ATTRIBUTE

2011-04-14 Thread Noah Misch
On Wed, Apr 13, 2011 at 11:46:45PM -0400, Tom Lane wrote: > Robert Haas writes: > > If we adopt the elsewhere-proposed approach of forbidding the use of > > rowtypes to create typed tables, the circularity-checking logic here > > can become simpler. I think it's not actually water-tight right now

Re: [HACKERS] POSIX shared memory redux

2011-04-14 Thread Florian Weimer
* Tom Lane: > Well, the fundamental point is that "ignoring NFS" is not the real > world. We can't tell people not to put data directories on NFS, > and even if we did tell them not to, they'd still do it. And NFS > locking is not trustworthy, because the remote lock daemon can crash > and resta

Re: [HACKERS] Typed table DDL loose ends

2011-04-14 Thread Noah Misch
On Wed, Apr 13, 2011 at 07:57:29PM -0700, Robert Haas wrote: > On Sat, Apr 9, 2011 at 6:57 PM, Noah Misch wrote: > > While looking at the typed table/pg_upgrade problem, I ran into a few > > smaller > > problems in the area. ?I'm not envisioning a need for much code shift to fix > > them, but the

Re: [HACKERS] Itanium HP-UX build failure, register stack

2011-04-14 Thread Andrew Dunstan
On 04/14/2011 02:04 AM, Josh Berkus wrote: On 4/13/11 5:46 AM, Heikki Linnakangas wrote: On 13.04.2011 14:22, Andrew Dunstan wrote: I wish we could get some buildfarm coverage for HPUX. I've whined about this in the past, but nobody's ever made an offer to provide suitable platform(s) that I

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Heikki Linnakangas
On 14.04.2011 10:15, Pavel Stehule wrote: Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), "%s%s\n", row[0], row[1]);

Re: [HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello > > The way COPY works is that PQputCopyData just sends the data to the server, > and the server will buffer it in its internal buffer and processes it when > it feels like it. The PQputCopyData() calls don't even need to match line > boundaries. > Yes, it is current behave - then document

[HACKERS] PQputCopyData dont signal error

2011-04-14 Thread Pavel Stehule
Hello I have a problem with PQputCopyData function. It doesn't signal some error. while ((row = mysql_fetch_row(res)) != NULL) { snprintf(buffer, sizeof(buffer), "%s%s\n", row[0], row[1]); copy_result = PQputCopyData

[HACKERS] Single client performance on trivial SELECTs

2011-04-14 Thread Greg Smith
This week several list regulars here waded into the MySQL Convention. I decided to revisit PostgreSQL vs. MySQL performance using the sysbench program as part of that. It's not important to what I'm going to describe to understand exactly what statements sysbench runs here or how to use it, b