Re: [HACKERS] Compression and on-disk sorting

2006-06-06 Thread Jim C. Nasby
On Fri, May 26, 2006 at 09:21:44PM +0100, Simon Riggs wrote: > On Fri, 2006-05-26 at 14:47 -0500, Jim C. Nasby wrote: > > > But the meat is: > > -- work_mem -- > > Scale 20002 > > not compressed 150

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread ITAGAKI Takahiro
Simon Riggs <[EMAIL PROTECTED]> wrote: > Itagaki had suggested adding options to heaps also, so clearly we'd need > to add that to pg_class, rather than pg_index in that case. Yes, I want to add options tables not only indexes. There is pg_index for indexes, but is not pg_table for tables, so I ad

Re: [HACKERS] DROP INHERITS

2006-06-06 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > > I suppose it makes more sense to optimize this on the basis of what's used > > in > > the planner and executor rather than ALTER TABLE commands though. > > No, definitely not. Syscaches only exist to support hard-wired lookups > in the backend C code.

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Koichi Suzuki
I've once proposed a patch for 64bit transaction ID, but this causes some overhead to each tuple (XMIN and XMAX). Pgbench with 64bit transaction ID has to pay about a couple of percent of performance. If 64bit transaction ID is a reasonable fix, I've already posted this patch. Anyone can

Re: [HACKERS] DROP INHERITS

2006-06-06 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > I have a question about pg_depends, namely how do you identify the dependency > tied to a given relationship. Specifically to handle DROP INHERITS. Is there > any other reason there might be a dependency between two tables other than > inheritance? If there

[HACKERS] DROP INHERITS

2006-06-06 Thread Greg Stark
I have a question about pg_depends, namely how do you identify the dependency tied to a given relationship. Specifically to handle DROP INHERITS. Is there any other reason there might be a dependency between two tables other than inheritance? If there was how would you tell the dependencies apart

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
> Mark Woodward wrote: >> OK, here's my problem, I have a nature study where we have about 10 >> video >> cameras taking 15 frames per second. >> For each frame we make a few transactions on a PostgreSQL database. > > Maybe if you grouped multiple operations on bigger transactions, the I/O > saving

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Mark Woodward
> Hello, I would like to know where in the source-code of postgres is > located the code of the aggregate functions min, max, avg. > I wish to develop more statistical aggregate functions, and I prefer to > use C than to write then in the PL/R. There is a library in "contrib" called "intagg." I wr

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Tom Lane <[EMAIL PROTECTED]> writes: >> And two, that upper plan nodes seem much more affected than lower >> ones. That makes sense because the execution cycle of an upper node >> will involve touching more userspace data than a lower node, and >> therefore

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Christopher Browne
Martha Stewart called it a Good Thing when [EMAIL PROTECTED] (Andrew Dunstan) wrote: > Christopher Browne wrote: >> I haven't been monitoring CVS HEAD, but you can be sure this will get >> tried out when 8.2 gets anywhere vaguely close to relese... >> > > The whole point of having a buildfarm is t

Re: [HACKERS] More thoughts about planner's cost estimates

2006-06-06 Thread Ron Mayer
Tom Lane wrote: One objection to this is that after moving "off the gold standard" of 1.0 = one page fetch, there is no longer any clear meaning to the cost estimate units; you're faced with the fact that they're just an arbitrary scale. I'm not sure that's such a bad thing, though. It seems

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > And two, that upper plan nodes seem much more affected than lower > ones. That makes sense because the execution cycle of an upper node > will involve touching more userspace data than a lower node, and > therefore more of the flushed TLB entries will need

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Martijn van Oosterhout writes: > This doesn't make any sense at all. How can a sampling run that only > sampled 7% of the actual tuples, end up with a actual measured time > that's more than 50% of the actual final runtime? AFAICS, the only conclusion is that the sampled executions are in fact ta

Re: [HACKERS] SERIAL problems?

2006-06-06 Thread Tom Lane
Zoltan Boszormenyi <[EMAIL PROTECTED]> writes: > What are the "existing problems"? Please read the archives; this has been discussed recently. There's a lot of disagreement about what ALTER should allow and what pg_dump should do with an altered sequence. > I am asking because I am experimenting

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Martijn van Oosterhout
On Tue, Jun 06, 2006 at 04:50:28PM -0400, Tom Lane wrote: > But that still leaves me with a problem because my machine is clearly > overestimating the correction needed. I added some printouts and got > > raw totaltime = 0.370937 > per_iter = 0.000156913, SampleOverhead = 3.28e-06 > adj totaltime

[HACKERS] SERIAL problems?

2006-06-06 Thread Zoltan Boszormenyi
Hi, I just saw these in the TODO list: o %Disallow changing DEFAULT expression of a SERIAL column? This should be done only if the existing SERIAL problems cannot be fixed. o %Disallow ALTER SEQUENCE changes for SERIAL sequences because pg_dump does not dump the changes What are the "ex

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > If that's the case, then maybe a more sopdisticated method of measuring > the overhead would work. I think you missed the point: the time spent in gettimeofday() itself is not the major overhead of EXPLAIN ANALYZE. At least it appears that this is the

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 04:50:28PM -0400, Tom Lane wrote: > I have a theory about this, and it's not pleasant at all. What I > think is that we have a Heisenberg problem here: the act of invoking > gettimeofday() actually changes what is measured. That is, the > runtime of the "second part" of Ex

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > The whole point of having a buildfarm is that we shouldn't have to wait, > we should be able to see very quickly if we have broken something. And in fact kookaburra seems happy ... regards, tom lane ---

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
I wrote: > The problem looks to be an underestimation of SampleOverhead, and on > reflection it's clear why: what CalculateSampleOverhead is measuring > isn't the total overhead, but the time between the two gettimeofday > calls. Which is probably about half the true overhead. On further thought,

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 12:58:10PM -0400, Travis Cross wrote: > >Do you *know* that the disk drive will not lie about write > >complete? > > "Know" is such a strong word ;) Honestly, I have very little idea. > I understand the nature of the problem this presents, as I've read > the very fine P

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Tom Lane
Martijn van Oosterhout writes: > Wierd. Can you get the output of *instr in each call of > InstrEndLoop? Preferably after it does the calculation but before it > clears the values. So we get an idea of number of samples and what it > guesses. SampleOverhead would be good too. The problem looks to

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Andrew Dunstan
Christopher Browne wrote: I haven't been monitoring CVS HEAD, but you can be sure this will get tried out when 8.2 gets anywhere vaguely close to relese... The whole point of having a buildfarm is that we shouldn't have to wait, we should be able to see very quickly if we have broken someth

Re: [HACKERS] adding new field

2006-06-06 Thread Martijn van Oosterhout
On Tue, Jun 06, 2006 at 09:44:04PM +0530, ranbeer makin wrote: > hey, > let me clarify something. > > 1. Whenever you run a query, its result or some other thing gets cached. I > want to flush this cache? Also let me know when this cache gets flushed > automatically. What cache? Query results

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Alvaro Herrera
Rod Taylor wrote: > With one potential snafu -- it blocks new SELECTs against the parent > table while truncate runs on the child (happens with constraint > exclusion as well). > > If your transactions are short then it won't be an issue. If you have > mixed length transactions (many short which

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Rod Taylor
On Tue, 2006-06-06 at 13:53 -0400, Christopher Browne wrote: > Clinging to sanity, [EMAIL PROTECTED] (Hannu Krosing) mumbled into her beard: > > Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: > >> OK, here's my problem, I have a nature study where we have about 10 video > >> c

Re: [HACKERS] Connection Broken with Custom Dicts for TSearch2

2006-06-06 Thread John Jawed
Since we are on the topic, is there a timeline/plans for openfts being brought into core? If not, I'll continue my work on bringing it into Gentoo Portage. John On 6/5/06, Oleg Bartunov wrote: On Mon, 5 Jun 2006, Teodor Sigaev wrote: > > > Teodor Sigaev wrote: >> Sorry, it isn't mentioned on

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Christopher Browne
In an attempt to throw the authorities off his trail, pgman@candle.pha.pa.us (Bruce Momjian) transmitted: > Tom Lane wrote: >> Bruce Momjian writes: >> > Joachim Wieland wrote: >> >> Can someone please explain why in include/utils/datetime.h (struct >> >> datetkn) >> >> there is a check for _AIX

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Christopher Browne
Clinging to sanity, [EMAIL PROTECTED] (Hannu Krosing) mumbled into her beard: > Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: >> OK, here's my problem, I have a nature study where we have about 10 video >> cameras taking 15 frames per second. >> For each frame we make a few t

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Joshua D. Drake
Roberto Rezende de Assis wrote: Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. I wish to develop more statistical aggregate functions, and I prefer to use C than to write then in the PL/R. http://projects.commandp

Re: [HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Roberto Rezende de Assis wrote: Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. They're in src/backend/utils/adt/numeric.c I wish to develop more statistical aggregate functions, and I prefer to

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Ian Barwick wrote: On 6/6/06, Tom Lane <[EMAIL PROTECTED]> wrote: Travis Cross <[EMAIL PROTECTED]> writes: > I'm noticing that a handful (4-16) of rows with duplicate columns > (uid,token) are sneaking into the table every day despite the > primary key constraint. Corrupt index, looks like ...

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Tom Lane wrote: Travis Cross <[EMAIL PROTECTED]> writes: I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. Corrupt index, looks like ... you might try reindexing the index. I probably should

[HACKERS] AGREGATE FUNCTIONS

2006-06-06 Thread Roberto Rezende de Assis
Hello, I would like to know where in the source-code of postgres is located the code of the aggregate functions min, max, avg. I wish to develop more statistical aggregate functions, and I prefer to use C than to write then in the PL/R. Thanks

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Joachim Wieland wrote: > >> Can someone please explain why in include/utils/datetime.h (struct datetkn) > >> there is a check for _AIX that either initializes a char* pointer or a char > >> array? > > > Wow, that is strange. We could remove it for 8.2

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
I wrote: > It looks like a workaround for some ancient compiler problem. [ digs > for awhile... ] Very ancient: we inherited that hack from Berkeley, In fact, now that I know where to look, I find the same thing in the postgres-v4r2 tarball, which means the hack is pre-1994. I don't have anythi

Re: [HACKERS] adding new field

2006-06-06 Thread ranbeer makin
hey, let me clarify something. 1. Whenever you run a query, its result or some other thing gets cached. I want to flush this cache? Also let me know when this cache gets flushed automatically. 2. After adding a new field in Resdom structure and making necessary changes in outfuncs.c, createfu

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Tom Lane
Bruce Momjian writes: > Joachim Wieland wrote: >> Can someone please explain why in include/utils/datetime.h (struct datetkn) >> there is a check for _AIX that either initializes a char* pointer or a char >> array? > Wow, that is strange. We could remove it for 8.2 and see how testing goes. It

Re: [HACKERS] AIX check in datetime.h

2006-06-06 Thread Bruce Momjian
Wow, that is strange. We could remove it for 8.2 and see how testing goes. --- Joachim Wieland wrote: > Can someone please explain why in include/utils/datetime.h (struct datetkn) > there is a check for _AIX that either ini

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE

2006-06-06 Thread Travis Cross
Kenneth Marshall wrote: We have used postgresql 7.4, 8.0, and 8.1 with DSPAM and have never had a single problem like you are describing. In the past on this mailing list, these sorts of issues have been caused by hardware problems on the DB server in some cases. Good luck with tracking it down.

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Simon Riggs
On Tue, 2006-06-06 at 10:27 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Why not implement an array of option parameters on pg_class, so both > > heaps and indexes can be given additional parameters? That way you > > wouldn't need a specific relfillfactor attribute. That wou

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Ian Barwick
On 6/6/06, Tom Lane <[EMAIL PROTECTED]> wrote: Travis Cross <[EMAIL PROTECTED]> writes: > I'm noticing that a handful (4-16) of rows with duplicate columns > (uid,token) are sneaking into the table every day despite the > primary key constraint. Corrupt index, looks like ... you might try reinde

Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 04:47:40PM +0200, Harald Fuchs wrote: > In article <[EMAIL PROTECTED]>, > Andrew Dunstan <[EMAIL PROTECTED]> writes: > > > Mark Woodward wrote: > >> Tom had posted a question about file compression with copy. I thought > >> about it, and I want to through this out and see i

Re: [HACKERS] COPY (query) TO file

2006-06-06 Thread Harald Fuchs
In article <[EMAIL PROTECTED]>, Andrew Dunstan <[EMAIL PROTECTED]> writes: > Mark Woodward wrote: >> Tom had posted a question about file compression with copy. I thought >> about it, and I want to through this out and see if anyone things it is a >> good idea. >> >> Currently, the COPY command o

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > On Tue, Jun 06, 2006 at 09:48:43AM -0400, Tom Lane wrote: >>> psql --cursor -c "select ..." | myprogram >>> there would be no very good way for myprogram to find out that it'd >>> been sent an incomplete result due to error partway through the SELECT. >

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Jim C. Nasby
On Tue, Jun 06, 2006 at 09:48:43AM -0400, Tom Lane wrote: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > ??hel kenal p??eval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: > >> Note of course that such a thing would push the incomplete-result > >> problem further upstream. For instance in (hypoth

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Why not implement an array of option parameters on pg_class, so both > heaps and indexes can be given additional parameters? That way you > wouldn't need a specific relfillfactor attribute. That would allow us to > keep CREATE TABLE free of additional keywo

Re: [HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Tom Lane
Travis Cross <[EMAIL PROTECTED]> writes: > I'm noticing that a handful (4-16) of rows with duplicate columns > (uid,token) are sneaking into the table every day despite the > primary key constraint. Corrupt index, looks like ... you might try reindexing the index. I don't believe that the PANIC y

Re: [HACKERS] That EXPLAIN ANALYZE patch still needs work

2006-06-06 Thread Martijn van Oosterhout
On Mon, Jun 05, 2006 at 11:02:33PM -0400, Tom Lane wrote: > Just got this rather surprising result: > The "Total runtime" is correct AFAICT, which puts the top node's "actual > time" rather far out in left field. This is pretty repeatable on my old > slow HPPA machine. A new Xeon shows less of

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > Ühel kenal päeval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: >> Note of course that such a thing would push the incomplete-result >> problem further upstream. For instance in (hypothetical --cursor >> switch) >> psql --cursor -c "select ..." |

Re: [HACKERS] Why do we want to %Remove behavior of postmaster -o

2006-06-06 Thread Tom Lane
Peter Eisentraut <[EMAIL PROTECTED]> writes: > Am Samstag, 3. Juni 2006 04:27 schrieb Tom Lane: >> Actually, the TODO item is very badly worded. The idea is to get rid of >> the spelling differences between postmaster and postgres options, and >> then there will be no need for '-o' because you'll

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Alvaro Herrera
Mark Woodward wrote: > OK, here's my problem, I have a nature study where we have about 10 video > cameras taking 15 frames per second. > For each frame we make a few transactions on a PostgreSQL database. Maybe if you grouped multiple operations on bigger transactions, the I/O savings could be en

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Hannu Krosing
Ühel kenal päeval, T, 2006-06-06 kell 08:42, kirjutas Mark Woodward: > OK, here's my problem, I have a nature study where we have about 10 video > cameras taking 15 frames per second. > For each frame we make a few transactions on a PostgreSQL database. > We want to keep about a years worth of data

Re: [HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Heikki Linnakangas
On Tue, 6 Jun 2006, Mark Woodward wrote: OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. I would suggest doing all the inserts of one frame in one transaction. Ma

Re: [HACKERS] [PERFORM] psql -A (unaligned format) eats too much

2006-06-06 Thread Hannu Krosing
Ühel kenal päeval, E, 2006-06-05 kell 14:10, kirjutas Tom Lane: > Neil Conway <[EMAIL PROTECTED]> writes: > > On Mon, 2006-06-05 at 19:17 +0200, Zoltan Boszormenyi wrote: > >> The general case cannot be applied for all particular cases. > >> E.g. you cannot use cursors from shell scripts > > > Thi

[HACKERS] How to avoid transaction ID wrap

2006-06-06 Thread Mark Woodward
OK, here's my problem, I have a nature study where we have about 10 video cameras taking 15 frames per second. For each frame we make a few transactions on a PostgreSQL database. We want to keep about a years worth of data at any specific time. We have triggers that fire is something interesting is

Re: [HACKERS] fillfactor using WITH syntax

2006-06-06 Thread Simon Riggs
On Tue, 2006-06-06 at 11:45 +0900, ITAGAKI Takahiro wrote: > Hi Hackers, > > I'm rewriting fillfactor patch, per the following discussion, > http://archives.postgresql.org/pgsql-hackers/2006-03/msg00287.php > Now fillfactor can be set using WITH syntax: > - CREATE INDEX index ON table USING

[HACKERS] AIX check in datetime.h

2006-06-06 Thread Joachim Wieland
Can someone please explain why in include/utils/datetime.h (struct datetkn) there is a check for _AIX that either initializes a char* pointer or a char array? Is there any advantage of a char-array except for warnings of some compilers if the initilization string is too long? Apart from that I dou

[HACKERS] Duplicate rows sneaking in despite PRIMARY KEY / UNIQUE constraint

2006-06-06 Thread Travis Cross
I have a table that I am using to store email token data for DSPAM. I'm noticing that a handful (4-16) of rows with duplicate columns (uid,token) are sneaking into the table every day despite the primary key constraint. The server currently processes a few thousand emails per day, and this parti

Re: [HACKERS] Why do we want to %Remove behavior of postmaster -o

2006-06-06 Thread Peter Eisentraut
Am Samstag, 3. Juni 2006 04:27 schrieb Tom Lane: > Actually, the TODO item is very badly worded. The idea is to get rid of > the spelling differences between postmaster and postgres options, and > then there will be no need for '-o' because you'll just say what you > want --- that is, "-o -foo" an