Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Heikki Linnakangas
On 17.01.2011 22:33, Tom Lane wrote: Peter Eisentraut writes: On mån, 2011-01-17 at 07:35 +0100, Magnus Hagander wrote: In fact, aren't there cases where the *length test* also fails? Currently, two text values are only equal of strcoll() considers them equal and the bits are the same. So

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Itagaki Takahiro
On Tue, Jan 18, 2011 at 05:39, Tom Lane wrote: > I haven't looked at this patch, but it seems to me that it would be > reasonable to conclude A != B if the va_extsize values in the toast > pointers don't agree. It's a very light-weight alternative of memcmp the byte data, but there is still the s

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Mark Kirkwood
On 18/01/11 18:04, Tom Lane wrote: David Fetter writes: Who's the copyright holder(s)? If it's all individual contributors, Red Hat policy is not in play. Sorry David, it was written on the company's dime. However, I doubt that Red Hat derives any value from this useful product being excl

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 08:21, Fujii Masao wrote: > On Tue, Jan 18, 2011 at 4:15 PM, Heikki Linnakangas > wrote: >> I also find it weird that incoming replication connections are logged by >> default. In the standby, we also log "streaming replication successfully >> connected to primary", which

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/17/2011 07:58 PM, Kääriäinen Anssi wrote: The issue I saw was this: assume you have an extension foo, containing one function, test(). CREATE EXTENSION foo; DROP FUNCTION test(); -- restricted due to dependency ALTER FUNCTION test() RENAME TO test2; DROP FUNCTION test2(); -- not restrict

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: > Robert Haas wrote: > >> a quick-and-dirty attempt to limit the amount of I/O caused by hint >> bits. I'm still very interested in knowing what people think about >> that. > > I found the elimination of the response-time spike promising. I > d

Re: [HACKERS] Confusing comment in TransactionIdIsInProgress

2011-01-18 Thread Heikki Linnakangas
On 18.01.2011 07:15, Jim Nasby wrote: Shouldn't the comment read "If first time through"? /* * If not first time through, get workspace to remember main XIDs in. We * malloc it permanently to avoid repeated palloc/pfree overhead. */ if (xids == NULL)

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/17/2011 06:53 PM, Dimitri Fontaine wrote: Usability review: The patch implements a way to create extensions. While the patch is labeled extensions support for pg_dump, it actually implements more. It implements a new way to package and install extension, and changes contrib extensions to

Re: [HACKERS] Determining client_encoding from client locale

2011-01-18 Thread Susanne Ebrecht
On 17.01.2011 20:18, Peter Eisentraut wrote: That may be worth investigating, but I don't think it's related to the present patch. As I already said - not at all. The patch was ok for me. Susanne -- Susanne Ebrecht - 2ndQuadrant PostgreSQL Development, 24x7 Support, Training and Services www

Re: [HACKERS] Warning compiling pg_dump (MinGW, Windows XP)

2011-01-18 Thread Pavel Golub
Hello, Andrew. You wrote: AD> On 01/17/2011 03:51 PM, Tom Lane wrote: >> Andrew Dunstan writes: >>> On 01/17/2011 07:18 AM, Pavel Golub wrote: So you think I should just ignore these warnings? Because I can't remember the same behaviour on 8.x branches... >>> We've had them all along

Re: [HACKERS] MingW + GCC 4.5.2 + Relocate libpq.dll = SegFault

2011-01-18 Thread Pavel Golub
Hello, Charlie. Can you please express your opinion about my request "Warning compiling pg_dump (MinGW, Windows XP)" to pgsql-hackers on Thu, 13 Jan 2011. Do you have the same warnings using MinGW environment? You wrote: CS> I'm compiling postgresql 9.0.2 using msys + mingw + gcc 4.5.2 (latest

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Fujii Masao
On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao wrote: > Though I haven't seen the core part of the patch (i.e., > ReceiveTarFile, etc..) yet, > here is the comments against others. Here are another comments: When I untar the tar file taken by pg_basebackup, I got the following messages: $ t

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread tv
> On Jan 17, 2011, at 6:36 PM, Tomas Vondra wrote: >> 1) Forks are 'per relation' but the distinct estimators are 'per >> column' (or 'per group of columns') so I'm not sure whether the file >> should contain all the estimators for the table, or if there should >> be one fork for each estimat

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 11:42 AM, Dimitri Fontaine wrote: I've fixed the case by having the code remember the function's extension if any, and restore it along with the other dependencies. The only question here is should CREATE OR REPLACE be allowed. I just realized this could present a new problem. If I

Re: [HACKERS] Replication logging

2011-01-18 Thread Fujii Masao
On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander wrote: >> We should treat log_disconnections the same? > > We could keep it a boolean, but then only log disconnections for the > cases that are mentioned in log_connections? > > It doesn't make sense to log disconnection for a connection we didn't

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen writes: > Is this supposed to be used mainly by contrib and PGXN extensions? When I > saw the documentation, I immediately thought that this is a nice way to > package my application's stored procedures. If this is not one of the > intended usages, it should be documented. I can s

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Cédric Villemain
2011/1/18 Greg Smith : > Bruce Momjian wrote: >> >> Should we be writing until 2:30 then sleep 30 seconds and fsync at 3:00? >> > > The idea of having a dead period doing no work at all between write phase > and sync phase may have some merit.  I don't have enough test data yet on > some more funda

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Cédric Villemain
2011/1/18 Fujii Masao : > On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao wrote: >> Though I haven't seen the core part of the patch (i.e., >> ReceiveTarFile, etc..) yet, >> here is the comments against others. > > Here are another comments: > > > When I untar the tar file taken by pg_basebackup, I

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 12:11 PM, Anssi Kääriäinen wrote: The only question here is should CREATE OR REPLACE be allowed. I just realized this could present a new problem. If I am not mistaken, when loading from dump, you suddenly get the extension's version back, not the one you defined in CREATE OR REPLAC

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen writes: > The only question here is should CREATE OR REPLACE be allowed. I just Yes. Think ALTER EXTENSION UPGRADE, the next patch in the series (already proposed for this CF too). > realized this could present a new problem. If I am not mistaken, when > loading from dump, you

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen writes: > Ok, verified at least for CREATE OR REPLACE, ALTER FUNCTION RENAME and ALTER > FUNCTION SET search_path. You will get the extensions version back when > restoring from plain sql dump, not the CORed function, rename is lost and > same for search_path. I suspect this is a

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Greg Smith
Robert Haas wrote: Idea #4: For ext3 filesystems that like to dump the entire buffer cache instead of only the requested file, write a little daemon that runs alongside of (and completely indepdently of) PostgreSQL. Every 30 s, it opens a 1-byte file, changes the byte, fsyncs the file, and close

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 03:14, Fujii Masao wrote: > On Mon, Jan 17, 2011 at 10:50 PM, Magnus Hagander wrote: >>> +       printf(_("  -D, --pgdata=directory   receive base backup into >>> directory\n")); >>> +       printf(_("  -T, --tardir=directory    receive base backup into tar >>> files\n"

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 10:49, Fujii Masao wrote: > On Mon, Jan 17, 2011 at 10:30 PM, Fujii Masao wrote: >> Though I haven't seen the core part of the patch (i.e., >> ReceiveTarFile, etc..) yet, >> here is the comments against others. > > Here are another comments: Thanks! These are all good and

Re: [HACKERS] auto-sizing wal_buffers

2011-01-18 Thread Greg Smith
Fujii Masao wrote: +/* Minimum setting used for a lower bound on wal_buffers */ +#define XLOG_BUFFER_MIN4 Why didn't you use XLOG_BUFFER_MIN instead of XLOGbuffersMin? XLOG_BUFFER_MIN is not used anywhere for now. That's a typo; will fix. + if (XLOGbu

Re: [HACKERS] auto-sizing wal_buffers

2011-01-18 Thread Greg Smith
Josh Berkus wrote: I think we can be more specific on that last sentence; is there even any *theoretical* benefit to settings above 16MB, the size of a WAL segment? Certainly there have been no test results to show any. There was the set Marti just reminded about. The old wording suggested

Re: [HACKERS] Warning compiling pg_dump (MinGW, Windows XP)

2011-01-18 Thread Andrew Dunstan
On 01/18/2011 04:40 AM, Pavel Golub wrote: AD> We could add -Wno-format to the flags. That makes it shut up, but maybe AD> we don't want to use such a sledgehammer. I want to understand the only thing. Are these warnings really dangerous? Or I should just ignore them? As I pointed out

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Anssi Kääriäinen
On 01/18/2011 01:03 PM, Dimitri Fontaine wrote: I'd appreciate a list of yet-to-fix items. What I have is the search_path issue where CREATE EXTENSION foo; can leave it changed for the current session, I intend to fix that later today. Other than that, I have no further already agreed on code f

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Alvaro Herrera
Excerpts from Dimitri Fontaine's message of mar ene 18 07:01:55 -0300 2011: > Anssi Kääriäinen writes: > >> It used to work this way with \i, obviously. Should the extension patch > >> care about that and how? Do we want to RESET search_path or to manually > >> manage it like we do for the clie

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Mon, Jan 17, 2011 at 16:27, Simon Riggs wrote: > On Mon, 2011-01-17 at 16:20 +0100, Magnus Hagander wrote: >> On Mon, Jan 17, 2011 at 16:18, Robert Haas wrote: >> > On Mon, Jan 17, 2011 at 8:55 AM, Magnus Hagander >> > wrote: >> >> Hmm. I don't like those names at all :( >> > >> > I agree.  

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 12:40, Magnus Hagander wrote: > On Tue, Jan 18, 2011 at 10:49, Fujii Masao wrote: > Yeah that sounds like a good idea. Shouldn't be too hard to do (will > reuqire a backend patch as well, of course). Should we use "-f" for > fast? Though that may be an unfortunate overload

Re: [HACKERS] REVIEW: Extensions support for pg_dump

2011-01-18 Thread Dimitri Fontaine
Anssi Kääriäinen writes: > On 01/18/2011 01:03 PM, Dimitri Fontaine wrote: >> I'd appreciate a list of yet-to-fix items. What I have is the >> search_path issue where CREATE EXTENSION foo; can leave it changed for >> the current session, I intend to fix that later today. After some reading of b

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 08:40:50 -0300 2011: > On Tue, Jan 18, 2011 at 10:49, Fujii Masao wrote: > > +                                       fprintf(stderr, _("%s: could not > > write to file '%s': %m\n"), > > > > %m in fprintf is portable? > > Hmm. I just assumed

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Alvaro Herrera
Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: > > Hello Postgres Hackers, > > In reference to this todo item about clustering system table indexes, > > ( http://archives.postgresql.org/pgsql-hackers/2004-05/msg00989.php ) > I have been studying the system ta

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Alvaro Herrera
Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: > > Hello Postgres Hackers, BTW whatever you do, don't start a new thread by replying to an existing message and just changing the subject line. It will mess up the threading for some readers, and some might not even see you

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Merlin Moncure
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby wrote: > On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: >> Robert Haas  wrote: >> >>> a quick-and-dirty attempt to limit the amount of I/O caused by hint >>> bits. I'm still very interested in knowing what people think about >>> that. >> >> I found th

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread David Fetter
On Tue, Jan 18, 2011 at 09:14:41PM +1300, Mark Kirkwood wrote: > On 18/01/11 18:04, Tom Lane wrote: > >David Fetter writes: > >>Who's the copyright holder(s)? If it's all individual > >>contributors, Red Hat policy is not in play. > >Sorry David, it was written on the company's dime. > > However

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > Ok, thanks for clarifying. I've updated to use strerror(). Guess it's > time for another patch, PFA :-) Thanks ... Message nitpick: + if (compresslevel > 0) + { + fprintf(stderr, + _("%s: this bu

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera wrote: > Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > >> Ok, thanks for clarifying. I've updated to use strerror(). Guess it's >> time for another patch, PFA :-) > > Thanks ...  Message nitpick: > +   if (compresslevel >

Re: [HACKERS] review: FDW API

2011-01-18 Thread Shigeru HANADA
I've (hopefully) fixed issues above. Please find attached patches. == patch list == 1) 20110118-no_fdw_perm_check.patch - This patch is not included in last post. This had been proposed on 2011-01-05 first, but maybe has not been reviewd yet. I re-propose this patch for SQL standard conforman

Re: [HACKERS] SQL/MED - file_fdw

2011-01-18 Thread Shigeru HANADA
n call even if the planning is not for EXPLAIN. I'll try to defer generating explainInfo until EXPLAIN VERBOSE really uses it. It might need new hook point in expalain.c, though. Regards, -- Shigeru Hanada 20110118-file_fdw.patch.gz Description: Binary data -- Sent via pgsql-hackers mail

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 10:56, Fujii Masao wrote: > On Tue, Jan 18, 2011 at 5:17 PM, Magnus Hagander wrote: >>> We should treat log_disconnections the same? >> >> We could keep it a boolean, but then only log disconnections for the >> cases that are mentioned in log_connections? >> >> It doesn't

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Tom Lane
David Fetter writes: > I'm guessing there's a Policy® at Red Hat that software made on its > dime be GPL (v2, I'd guess), and that getting an exception would > involve convening its board or similarly drastic action. It's company policy, and while it *might* be possible to get an exception, the e

Re: [HACKERS] Replication logging

2011-01-18 Thread Tom Lane
Magnus Hagander writes: > Is there *any* usecase for setting them differently though? I can't believe we're still engaged in painting this bikeshed. Let's just control it off log_connections and have done. BTW, what about log_disconnections --- does a walsender emit a message according to that?

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Itagaki Takahiro writes: > On Tue, Jan 18, 2011 at 05:39, Tom Lane wrote: >> I haven't looked at this patch, but it seems to me that it would be >> reasonable to conclude A != B if the va_extsize values in the toast >> pointers don't agree. > It's a very light-weight alternative of memcmp the by

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Tom Lane
Magnus Hagander writes: >>> Actually, after some IM chats, I think pg_streamrecv should be >>> renamed, probably to pg_walstream (or pg_logstream, but pg_walstream >>> is a lot more specific than that) >> pg_stream_log >> pg_stream_backup > Those seem better. > Tom, would those solve your conce

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >> It's a very light-weight alternative of memcmp the byte data, >> but there is still the same issue -- we might have different >> compressed results if we use different algorithm for TOASTing. > > Which makes it a lightweight waste of cycles. > >

Re: [HACKERS] Replication logging

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas wrote: > I also find it weird that incoming replication connections are logged by > default. In the standby, we also log "streaming replication successfully > connected to primary", which serves much of the same debugging purpose. Oh, good point

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera wrote: > Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: >> >> Hello Postgres Hackers, >> >> In reference to this todo item about clustering system table indexes, >> ( http://archives.postgresql.org/pgsql-hackers/2004-05/msg00

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >> No, I don't think so.  Has any evidence been submitted that that part of >> the patch is of benefit? > I think you might be mixing up what's actually in the patch with > another idea that was proposed but isn't actually i

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Is there a value to this test_fsync test? > > > > Compare open_sync with different sizes: > > (This is designed to compare the cost of one large > > sync'ed write and two smaller sync'ed writes.) > > open_sync 16k write

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 11:15 AM, Tom Lane wrote: >>> No, I don't think so.  Has any evidence been submitted that that part of >>> the patch is of benefit? > >> I think you might be mixing up what's actually in the patch

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane wrote: > David Fetter writes: >> I'm guessing there's a PolicyŽ at Red Hat that software made on its >> dime be GPL (v2, I'd guess), and that getting an exception would >> involve convening its board or similarly drastic action. > > It's company policy,

Re: [HACKERS] texteq/byteaeq: avoid detoast [REVIEW]

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 11:44 AM, Tom Lane wrote: >> Oh, I misread Itagaki-san's comment to imply that that *was* in the >> patch.  Maybe I should go read it. > Perhaps. :-) > While you're at it you might commit it. :-) Yeah, as penance I'll take this one.

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:31, Tom Lane wrote: > Magnus Hagander writes: Actually, after some IM chats, I think pg_streamrecv should be renamed, probably to pg_walstream (or pg_logstream, but pg_walstream is a lot more specific than that) > >>> pg_stream_log >>> pg_stream_backup >

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Cédric Villemain
2011/1/18 Magnus Hagander : > On Tue, Jan 18, 2011 at 17:31, Tom Lane wrote: >> Magnus Hagander writes: > Actually, after some IM chats, I think pg_streamrecv should be > renamed, probably to pg_walstream (or pg_logstream, but pg_walstream > is a lot more specific than that) >> p

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 4:53 AM, wrote: > So the most important question is how to intercept the new/updated rows, > and where to store them. I think each backend should maintain it's own > private list of new records and forward them only in case of commit. Does > that sound reasonable? At the

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Cédric Villemain
2011/1/18 Robert Haas : > On Tue, Jan 18, 2011 at 10:53 AM, Tom Lane wrote: >> David Fetter writes: >>> I'm guessing there's a PolicyŽ at Red Hat that software made on its >>> dime be GPL (v2, I'd guess), and that getting an exception would >>> involve convening its board or similarly drastic act

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 8:24 AM, Merlin Moncure wrote: > a few weeks back I hacked an experimental patch that removed the hint > bit action completely. the results were very premature and/or > incorrect, but my initial findings suggested that hint bits might not > be worth the cost from performance st

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: > On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >> - Forks are very possibly a more efficient way to deal with TOAST than >> having separate tables. There's a fair amount of overhead we pay for the >> current setup. > > That seems like an inte

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: > On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>> - Forks are very possibly a more efficient way to deal with TOAST than >>> having separate tables. There's a fair amount of overhead we pa

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Tom Lane
Robert Haas writes: > On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: > On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: >>> - Forks are very possibly a more efficient way to deal with TOAST than >>> having separate tables. There's a fair a

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:24 AM, Robert Haas wrote: > On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >>> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: - Forks are very possibly a more efficient way to deal with TOAST than having s

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Jim Nasby
On Jan 18, 2011, at 11:32 AM, Tom Lane wrote: > Robert Haas writes: >> On Tue, Jan 18, 2011 at 12:23 PM, Jim Nasby wrote: >>> On Jan 17, 2011, at 8:11 PM, Robert Haas wrote: >> On Mon, Jan 17, 2011 at 7:56 PM, Jim Nasby wrote: - Forks are very possibly a more efficient way to deal with TOAS

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:47 AM, Jim Nasby wrote: > On Jan 16, 2011, at 4:37 PM, Kevin Grittner wrote: >> Robert Haas  wrote: >> >>> a quick-and-dirty attempt to limit the amount of I/O caused by hint >>> bits. I'm still very interested in knowing what people think about >>> that. >> >> I found th

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 9:24 AM, Merlin Moncure wrote: > a few weeks back I hacked an experimental patch that removed the hint > bit action completely.  the results were very premature and/or > incorrect, but my initial findings suggested that hint bits might not > be worth the cost from performan

Re: [HACKERS] estimating # of distinct values

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:32 PM, Jim Nasby wrote: >> How's that different from what vacuum does on a TOAST table now? > > TOAST vacuum is currently an entirely separate vacuum. It might run at the > same time as the main table vacuum, but it still has all the work that would > be associated wit

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:03 PM, Magnus Hagander wrote: > So it'd be pg_receive_wal and pg_receive_base_backup then? Votes from > others? (it's easy to rename so far, so I'll keep plugging away under > the name pg_basebackup based on Fujii-sans comments until such a time > as we have a reasonable

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Jan 18, 2011, at 6:35 AM, Alvaro Herrera wrote: >> > > Wow, this is really old stuff. I don't know if this is really of any > benefit, given that these catalogs are loaded into syscaches anyway. The benefit is educational primarily. I was looking for a todo list item that would expose me

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Simone Aiken
On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera wrote: >> Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: >>> >> >Hello Postgres Hackers, >>> >> >In reference to this todo item about clustering system table indexes, >>> ( http://archives.postgresql.org/pgsql-hackers/2004

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Alvaro Herrera
Excerpts from Magnus Hagander's message of mar ene 18 11:53:55 -0300 2011: > On Tue, Jan 18, 2011 at 15:49, Alvaro Herrera > wrote: > > Excerpts from Magnus Hagander's message of mar ene 18 10:47:03 -0300 2011: > > > >> Ok, thanks for clarifying. I've updated to use strerror(). Guess it's > >> tim

Re: [HACKERS] Re: new patch of MERGE (merge_204) & a question about duplicated ctid

2011-01-18 Thread Greg Smith
David Fetter wrote: I think I haven't communicated clearly what I'm suggesting, which is that we ship with both an UPSERT and a MERGE, the former being ugly, crude and simple, and the latter festooned with dire warnings about isolation levels and locking. I don't know that I completely agree

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Tom Lane
Robert Haas writes: >>> I think you may be confused about what the patch does - currently, >>> pages with hint bit changes are considered dirty, period. >>> Therefore, they are written whenever any other dirty page would be >>> written: by the background writer cleaning scan, at checkpoints, >>> a

Re: [HACKERS] psql: Add \dL to show languages

2011-01-18 Thread Andreas Karlsson
Hi Josh, Nope, I do not have any better ideas than "DO Blocks?". Everything looks good with the exception one bug now. \dL foo * QUERY ** SELECT l.lanname AS "Name", pg_catalog.pg_get_userbyid(l.lanowner) as "Owner", l.lanpltrusted AS "Trusted" FROM pg_catalog.pg_la

Re: [HACKERS] pg_basebackup for streaming base backups

2011-01-18 Thread Simon Riggs
On Tue, 2011-01-18 at 18:03 +0100, Magnus Hagander wrote: > So it'd be pg_receive_wal and pg_receive_base_backup then? OK for me. Maybe even pg_receive_wal_stream Don't see any reason why command names can't be long. We have many function names already that long. -- Simon Riggs htt

Re: [HACKERS] [PERFORM] pgbench to the MAXINT

2011-01-18 Thread Greg Smith
Euler Taveira de Oliveira wrote: (i) If we want to support and scale factor greater than 21474 we have to convert some columns to bigint; it will change the test. From the portability point it is a pity but as we have never supported it I'm not too worried about it. Why? Because it will use big

[HACKERS] Performance bug in DO blocks

2011-01-18 Thread Tom Lane
I just noticed that if you execute the same DO command over and over within a session, it gets slower and slower. And if you keep it up you'll notice the backend's RAM consumption bloating too. The cause appears to be that we leak the cached plans created for any SQL statements or expressions wit

Re: [HACKERS] test_fsync open_sync test

2011-01-18 Thread Bruce Momjian
Greg Smith wrote: > Bruce Momjian wrote: > > Is there a value to this test_fsync test? > > > > Compare open_sync with different sizes: > > (This is designed to compare the cost of one large > > sync'ed write and two smaller sync'ed writes.) > > open_sync 16k write

[HACKERS] PgEast: 2011, 2nd call for papers

2011-01-18 Thread Joshua D. Drake
Hey folks, PgEast is being held in NYC this year from 03/22-03-25. Get your papers in, the deadline is soon! http://www.postgresqlconference.org/ Joshua D. Drake -- PostgreSQL.org Major Contributor Command Prompt, Inc: http://www.commandprompt.com/ - 509.416.6579 Consulting, Training, Support,

Re: [HACKERS] Replication logging

2011-01-18 Thread Magnus Hagander
On Tue, Jan 18, 2011 at 17:33, Robert Haas wrote: > On Tue, Jan 18, 2011 at 2:15 AM, Heikki Linnakangas > wrote: >> I also find it weird that incoming replication connections are logged by >> default. In the standby, we also log "streaming replication successfully >> connected to primary", which

Re: [HACKERS] Spread checkpoint sync

2011-01-18 Thread Josh Berkus
> To be frank, I really don't care about fixing this behavior on ext3, > especially in the context of that sort of hack. That filesystem is not > the future, it's not possible to ever really make it work right, and > every minute spent on pandering to its limitations would be better spent > elsew

Re: [HACKERS] texteq/byteaeq: avoid detoast

2011-01-18 Thread Tom Lane
Noah Misch writes: > texteq, textne, byteaeq and byteane detoast their arguments, then check for > equality of length. Unequal lengths imply the answer trivially; given equal > lengths, the functions proceed to compare the actual bytes. We can skip > detoasting entirely when the lengths are uneq

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote: > Robert Haas writes: I think you may be confused about what the patch does - currently, pages with hint bit changes are considered dirty, period. Therefore, they are written whenever any other dirty page would be written: by th

Re: [HACKERS] Replication logging

2011-01-18 Thread Josh Berkus
All, Just speaking as someone who does a lot of log-crunching for performance review, I don't find having a separate log_connections option for replication terribly useful. It's easy enough for me to just log all connections and filter for the type of connections I want. Even in cases where ther

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Noah Misch writes: > On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >> I am sending a updated version with little bit more comments. But I am >> sure, so somebody with good English have to edit my comments. >> Minimally I hope, so your questions will be answered. > Thanks. I edit

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Heikki Linnakangas
On 18.01.2011 21:16, Robert Haas wrote: On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane wrote: While I was trying to performance-test the texteq patch, it occurred to me that this proposed hint-bit change has got a serious drawback. To wit, that it will totally destroy reproducibility of any perform

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
Heikki Linnakangas wrote: > There's a few remaining TODO comments in the code, which obviously > need to be resolved one way or another Not all of these are "must haves" for 9.1. Here's how they break down: The two in predicate_internals.h mark places which would need to be touched if we fu

Re: [HACKERS] limiting hint bit I/O

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:00 PM, Heikki Linnakangas wrote: > On 18.01.2011 21:16, Robert Haas wrote: >> >> On Tue, Jan 18, 2011 at 1:32 PM, Tom Lane  wrote: >>> >>> While I was trying to performance-test the texteq patch, it occurred to >>> me that this proposed hint-bit change has got a serious d

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 3:18 PM, Kevin Grittner wrote: > That's all of them. Our existing code has plenty of TODOs in it already, so I see no problem with continuing to comment places where future enhancements are possible, as long as they don't reflect deficiencies that are crippling in the pres

Re: [HACKERS] SSI patch version 12

2011-01-18 Thread Kevin Grittner
"Kevin Grittner" wrote: > If my back-of-the-envelope math is right, a carefully constructed > pessimal load could need up to (max_connections / 2)^2 -- so 100 > connections could conceivably require 2500 structures, although > such a scenario would be hard to create. Current "picked from > thin

[HACKERS] test_fsync label adjustments

2011-01-18 Thread Bruce Momjian
I have modified test_fsync to use test labels that match wal_sync_method values, and and added more tests for open_sync with different sizes. This should make the program easier for novices to understand. Here is a test run for Ubuntu 11.04: $ ./test_fsync 2000 operations per tes

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Bruce Momjian
Robert Haas wrote: > On Tue, Jan 18, 2011 at 8:35 AM, Alvaro Herrera > wrote: > > Excerpts from Simone Aiken's message of dom ene 16 02:11:26 -0300 2011: > >> > >> Hello Postgres Hackers, > >> > >> In reference to this todo item about clustering system table indexes, > >> ( http://archives.postgre

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > One of the reasons our client wants GIN for the integer[] column so bad is > because recreating the GiST integer[] index is quite painful. Before I duped > the table, I was just dropping and recreating the index on the original > table. It was great to create the GI

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Pavel Stehule
2011/1/18 Tom Lane : > Noah Misch writes: >> On Sun, Jan 16, 2011 at 06:49:27PM +0100, Pavel Stehule wrote: >>> I am sending a updated version with little bit more comments. But I am >>> sure, so somebody with good English have to edit my comments. >>> Minimally I hope, so your questions will be a

Re: [HACKERS] pg_filedump moved to pgfoundry

2011-01-18 Thread Mark Kirkwood
On 19/01/11 05:51, Robert Haas wrote: I'm not sure why they'd care, but it certainly doesn't seem worth spending the amount of time arguing about it that we are. David and Mark are, of course, free to spend their time petitioning Red Hat for relicensing if they are so inclined, but they aren't

Re: [HACKERS] ToDo List Item - System Table Index Clustering

2011-01-18 Thread Robert Haas
On Tue, Jan 18, 2011 at 12:16 PM, Simone Aiken wrote: > When I'm learning a new system I like to first learn how to use it, > second learn its data model, third start seriously looking at the code. > So that Todo is ideal for my learning method. Sure - my point is just that we usually have as a c

Re: [HACKERS] test_fsync label adjustments

2011-01-18 Thread A.M.
On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > I have modified test_fsync to use test labels that match wal_sync_method > values, and and added more tests for open_sync with different sizes. > This should make the program easier for novices to understand. Here is > a test run for Ubuntu 11

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread Tom Lane
"David E. Wheeler" writes: > These numbers are a bit crazy-making, but the upshot is that Gist is > slow out of the gate, but with data cached, it's pretty speedy. With > indexscan and bitmapscan disabled, these queries all took 300-400 > ms. So GIN was never better performing than a table scan.

Re: [HACKERS] Re: patch: fix performance problems with repated decomprimation of varlena values in plpgsql

2011-01-18 Thread Tom Lane
Pavel Stehule writes: > 2011/1/18 Tom Lane : >> I looked at this patch and found it fairly awkward.  What is the point >> of adding an additional flag to every variable, as opposed to just >> forcibly detoasting during assignment? > But detoasting on assignment isn't enought: > for i in array_l

Re: [HACKERS] Fixing GIN for empty/null/full-scan cases

2011-01-18 Thread David E. Wheeler
On Jan 18, 2011, at 1:39 PM, Tom Lane wrote: > At the moment my opinion is that gist__int_ops is too broken to be > usable, and it's also too uncommented to be fixable by anyone other > than the original author. That seems to jibe with your comments from last year: http://archives.postgresql.o

  1   2   >