[HACKERS] Fwd: Extended query protocol and exact types matches.

2010-12-09 Thread Dmitriy Igrishin
-- Forwarded message -- From: Dmitriy Igrishin Date: 2010/12/10 Subject: Fwd: Extended query protocol and exact types matches. To: postgres list Hey sql@, -- Forwarded message -- From: Dmitriy Igrishin Date: 2010/12/9 Subject: Extended query protocol and exact

Re: [HACKERS] To Signal The postmaster

2010-12-09 Thread aaliya zarrin
Hi All, Can anybody tell after finding the trigger file what steps does postgres follow? When and how it will set the postgres recovery mode to false? On Thu, Dec 9, 2010 at 3:51 PM, Fujii Masao wrote: > On Wed, Dec 8, 2010 at 6:22 PM, Fujii Masao wrote: > > On Wed, Dec 8, 2010 at 4:59 PM, H

[HACKERS] Anyone for SSDs?

2010-12-09 Thread Vaibhav Kaushal
Hi all, Most of you already know I am new to this list and newer to any OSS development. However, while browsing the source code (of 9.0.1) I find that there is only one way to store relations on disk - the magnetic disk. This came suddenly in my mind so I am asking the experts here. Considerin

Re: [HACKERS] serializable read only deferrable

2010-12-09 Thread Dan Ports
On Tue, Dec 07, 2010 at 10:14:24AM -0600, Kevin Grittner wrote: > > Essentially, instead of adding dependencies as you go along and > > abort once you hit a conflict, SERIALIZABLE READ ONLY DEFERRED > > transactions would assume the worst case from the start and thus > > be able to bypass the more

[HACKERS] SynchRep; wait-forever and shutdown

2010-12-09 Thread Fujii Masao
Hi, In previous discussion, some people wanted the "wait-forever" option which blocks all the transactions on the master until sync'd standby has appeared, in order to reduce the risk of data loss in synchronous replication. What I'm not clear is; How does smart or fast shudown advance while all

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Greg Smith
Josh Berkus wrote: Did you rerun test_sync with O_DIRECT entabled, using my patch? The figures you had from test_fsync earlier were without O_DIRECT. No--I was just focused on testing the changes that had already been committed. The use of O_DIRECT in the server but not test_fsync could

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Josh Berkus
Greg, > This is interesting, because test_fsync consistently reported a rate of > about half this when using open_datasync instead of the equal > performance I'm getting from the database. I'll see if I can reproduce > that further, but it's no reason to be concerned about the change that's > bee

Re: [HACKERS] [GENERAL] Postgresql 9.1 pg_last_xact_replay_timestamp limitations

2010-12-09 Thread Fujii Masao
On Fri, Dec 10, 2010 at 1:24 AM, Gabi Julien wrote: > On Wednesday 08 December 2010 21:58:46 you wrote: >> On Thu, Dec 9, 2010 at 1:37 AM, Gabi Julien >> wrote: >> > slave# /etc/init.d/postgresql start >> > slave# psql -hlocalhost my_db -c "select pg_last_xact_replay_timestamp(), >> > now() as

Re: [HACKERS] Instrument checkpoint sync calls

2010-12-09 Thread Greg Smith
Jeff Janes wrote: In my test cases, the syncs that the backends were doing were almost always to the same file that the checkpoint writer was already choking on (so they are entangled simply by virtue of that). So very quickly all the backends hit the same wall and thunked to a halt. This is pr

Re: [HACKERS] Final(?) proposal for wal_sync_method changes

2010-12-09 Thread Greg Smith
Since any Windows refactoring has been postponed for now (I'll get back to performance checks on that platform later), during my testing time this week instead I did a round of pre-release review of the change Tom has now committed. All looks good to me, including the docs changes. I confirme

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kenneth Marshall
On Thu, Dec 09, 2010 at 05:18:57PM -0500, Tom Lane wrote: > I wrote: > > We're throwing away one tuple at a time as we advance forward through > > the tuplestore, and moving 10+ tuple pointers each time. Ugh. > > This code was all right when written, because (IIRC) the mergejoin > > case was a

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kevin Grittner
Tom Lane wrote: > Do you have reason to think that anybody is likely to exercise > window functions in HEAD, beyond what the regression tests do, in > the next couple of months? Not specifically, no. From the description (not having read the patch) I was somewhat concerned that it might affec

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
"Kevin Grittner" writes: > Tom Lane wrote: >> I'm not sure whether or not to back-patch this into 9.0 and 8.4. >> The code in tuplestore.c hasn't changed at all since 8.4, so >> there's not much risk of cross-version bugs, but if I did miss >> anything we could be shipping a buggy version next w

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Kevin Grittner
Tom Lane wrote: > I'm not sure whether or not to back-patch this into 9.0 and 8.4. > The code in tuplestore.c hasn't changed at all since 8.4, so > there's not much risk of cross-version bugs, but if I did miss > anything we could be shipping a buggy version next week. > Thoughts? Is there a

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
I wrote: > We're throwing away one tuple at a time as we advance forward through > the tuplestore, and moving 10+ tuple pointers each time. Ugh. > This code was all right when written, because (IIRC) the mergejoin > case was actually the only caller. But it's not all right for > WindowAgg's l

Re: [HACKERS] Extensions, patch v16

2010-12-09 Thread David E. Wheeler
On Dec 9, 2010, at 12:34 PM, Dimitri Fontaine wrote: > - add support for 'relocatable' boolean property in the control file, > as discussed on list > > this controls what happens at create extension time, by doing a > relocation of the extension objects when the extension is relocatable >

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread Andrew Dunstan
On 12/09/2010 03:36 PM, BRUSSER Michael wrote: No, we do not use --with-system-tzdata option. I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues. The problem occurs on all our UNIX platforms. Is there anything I could d

Re: [HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Tom Lane
Jie Li writes: > I'm new to window functions. Recently I run some simple queries but > surprised to find percent_rank is so slower than rank, could anybody tell me > why? Huh, interesting. I can reproduce this with toy data, such as create table inventory1 (inv_date_sk int, inv_item_sk int); in

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 2:51 PM, Kevin Grittner wrote: > Tom Lane wrote: >> If the constraint name is not specified, we should certainly use >> the existing index name, not randomly rename it. > > +1 +1 -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread BRUSSER Michael
No, we do not use --with-system-tzdata option. I looked at the makefile and at the output of pg_config. We may need to do some cleanup there, but I did not pick any clues. The problem occurs on all our UNIX platforms. Is there anything I could do to shed more light on it? I can post the output

Re: [HACKERS] [PATCH] Revert default wal_sync_method to fdatasync on Linux 2.6.33+

2010-12-09 Thread Josh Berkus
On 12/6/10 6:13 PM, Tom Lane wrote: > Josh Berkus writes: >> OK, patch coming then. Right now test_fsync aborts when O_DIRECT fails. >> What should I have it do instead? > > Report that it fails, and keep testing the other methods. Patch attached. Includes a fair amount of comment cleanup, si

Re: [HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread Tom Lane
"BRUSSER Michael" writes: > Initdb fails for me when host machine has no access to the Postgres build > location. > LOG: could not open directory .../install/share/timezone": No such file or > directory Moving the install tree works for me. Did you do something odd with the --with-system-tzd

Re: [HACKERS] BufFreelistLock

2010-12-09 Thread Jim Nasby
On Dec 8, 2010, at 11:44 PM, Jeff Janes wrote: >>> For the clock sweep algorithm, I think you could access >>> nextVictimBuffer without any type of locking. >> >> This is wrong, mainly because you wouldn't have any security against two >> processes decrementing the usage count of the same buffer b

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Kevin Grittner
Tom Lane wrote: > If the constraint name is not specified, we should certainly use > the existing index name, not randomly rename it. +1 -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgs

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Tom Lane
Gurjeet Singh writes: > But I still hold a bias towards renaming the index to match constraint name > (with a NOTICE), rather than require that the constraint name match the > index name, because the constraint name is optional and when it is not > provided system has to generate a name and we hav

Re: [HACKERS] Patch to add a primary key using an existing index

2010-12-09 Thread Gurjeet Singh
On Sun, Dec 5, 2010 at 2:09 PM, Peter Eisentraut wrote: > On fre, 2010-12-03 at 15:27 -0500, Robert Haas wrote: > > On Fri, Dec 3, 2010 at 2:56 PM, r t wrote: > > > What exactly was the objection to the following --> > > > ALTER TABLE table_name ADD PRIMARY KEY (column_list) USING index_name; >

Re: XLog vs SSD [Was: Re: [HACKERS] random write in xlog?]

2010-12-09 Thread James Cloos
> "JJ" == Jeff Janes writes: JJ> So PG always writing 8K at a time is unlikely to make a difference JJ> than if it wrote a smaller amount. Ah. Somehow I was thinking of the xlog files' 16M filesize rather than the internal 8k block size If it is only writing 8k blocks then there is pro

[HACKERS] initdb failure with Postgres 8.4.4

2010-12-09 Thread BRUSSER Michael
Initdb fails for me when host machine has no access to the Postgres build location. LOG: could not open directory .../install/share/timezone": No such file or directory LOG: could not open directory .../install/share/timezone": No such file or directory WARNING: could not open directory .../

Re: [HACKERS] Optimize PL/Perl function argument passing [PATCH]

2010-12-09 Thread Tim Bunce
On Wed, Dec 08, 2010 at 09:21:05AM -0800, David E. Wheeler wrote: > On Dec 8, 2010, at 9:14 AM, Tim Bunce wrote: > > >> Do you have any more improvements in the pipeline? > > > > I'd like to add $arrayref = decode_array_literal('{2,3}') and > > maybe $hashref = decode_hstore_literal('x=>1, y=>2')

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Merlin Moncure
On Thu, Dec 9, 2010 at 11:56 AM, Dimitri Fontaine wrote: > Merlin Moncure writes: >> On Wed, Dec 8, 2010 at 8:57 AM, Tatsuo Ishii wrote: >>> >>> Then execute the huge SELECT: >>> http://codezine.jp/static/images/article/1629/html/sql.html >> >> benchmark what you've got against this (ported to p

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Dimitri Fontaine
Merlin Moncure writes: > On Wed, Dec 8, 2010 at 8:57 AM, Tatsuo Ishii wrote: >> >> Then execute the huge SELECT: >> http://codezine.jp/static/images/article/1629/html/sql.html > > benchmark what you've got against this (ported to postgres by marcin mank): > http://www.pastie.org/684163 It that t

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Andrew Dunstan
On 12/09/2010 10:05 AM, Tom Lane wrote: I think what we need to do is make fix_dependencies build a reverse lookup list of all the objects dependent on each TOC object, so that the searching behavior in reduce_dependencies can be eliminated outright. That will take O(N) time and O(N) extra spa

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 10:05 AM, Tom Lane wrote: > I wrote: >> One fairly simple, if ugly, thing we could do about this is skip calling >> reduce_dependencies during the first loop if the TOC object is a blob; >> effectively assuming that nothing could depend on a blob.  But that does >> nothing a

Re: [HACKERS] Solving sudoku using SQL

2010-12-09 Thread Alvaro Herrera
Excerpts from Jan Urbański's message of mié dic 08 17:11:44 -0300 2010: > I'm pleasantly surprised that the SA code as it stands today, setting > the equlibrium factor to 8 and temperature reduction factor to 0.4, the > query takes 1799.662 ms in total. That's 5x better than Oracle :-) -- Álvar

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Tom Lane
I wrote: > One fairly simple, if ugly, thing we could do about this is skip calling > reduce_dependencies during the first loop if the TOC object is a blob; > effectively assuming that nothing could depend on a blob. But that does > nothing about the point that we're failing to parallelize blob >

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 9, 2010 at 12:28 AM, Tom Lane wrote: >> * Mark BLOB TOC entries as SECTION_DATA, or somehow otherwise make them >> parallelizable.  Also break the BLOBS data item apart into an item per >> BLOB, so that that part's parallelizable.  Maybe we should combine the >>

[HACKERS] Why percent_rank is so slower than rank?

2010-12-09 Thread Jie Li
Hi all, I'm new to window functions. Recently I run some simple queries but surprised to find percent_rank is so slower than rank, could anybody tell me why? The table schema: test=# \d inventory1 Table "public.inventory1" Column| Type | Modifiers -

[HACKERS] PS display and standby query conflict

2010-12-09 Thread Fujii Masao
Hi, When I created the conflict between recovery and many read-only transactions in the standby server for test purpose, I found that the keyword "waiting" disappeared from PS display for just a moment even though the conflict had not been resolved yet. This seems strange to me. This problem happ

Re: [HACKERS] [PERFORM] Slow BLOBs restoring

2010-12-09 Thread Robert Haas
On Thu, Dec 9, 2010 at 12:28 AM, Tom Lane wrote: > Vlad Arkhipov writes: >> 08.12.2010 22:46, Tom Lane writes: >>> Are you by any chance restoring from an 8.3 or older pg_dump file made >>> on Windows?  If so, it's a known issue. > >> No, I tried Linux only. > > OK, then it's not the missing-data

Re: [HACKERS] Hot Standby tuning for btree_xlog_vacuum()

2010-12-09 Thread Simon Riggs
Just wanted to say thanks for the review, since I haven't yet managed to fix and commit this. I expect to later this month. On Mon, 2010-09-27 at 23:06 -0400, Robert Haas wrote: > On Thu, Apr 29, 2010 at 4:12 PM, Simon Riggs wrote: > > Simple tuning of btree_xlog_vacuum() using an idea I had a wh

Re: [HACKERS] To Signal The postmaster

2010-12-09 Thread Fujii Masao
On Wed, Dec 8, 2010 at 6:22 PM, Fujii Masao wrote: > On Wed, Dec 8, 2010 at 4:59 PM, Heikki Linnakangas > wrote: >> For 9.1, we should think of a better way to do this, perhaps using SIGUSR1 >> to wake up. Maybe we won't even need the trigger file anymore. > > If we use SIGUSR1, the mechanism to

Re: [HACKERS] On-the-fly index tuple deletion vs. hot_standby

2010-12-09 Thread Simon Riggs
On Fri, 2010-12-03 at 21:43 +0200, Heikki Linnakangas wrote: > On 29.11.2010 08:10, Noah Misch wrote: > > I have a hot_standby system and use it to bear the load of various reporting > > queries that take 15-60 minutes each. In an effort to avoid long pauses in > > recovery, I set a vacuum_defer_c