Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > Could someone please quantify how much bang we might get for what seems > like quite a lot of bucks? > I appreciate the need for speed, but the saving here strikes me as > marginal at best, unless my instincts are all wrong (quite possible) Two bytes

Re: [HACKERS] Assert failure found in 8.1RC1

2005-11-02 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > Ran with both for an hour with no problem, where I could produce the ASSERT > failure within minutes for the non patched version. Great. I'll go ahead and commit the smaller fix into HEAD and the back branches, and hold the larger fix for 8.2. It's cu

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Dave Cramer
Tuna is running Dave-Cramers-Computer:~ davec$ uname -a Darwin Dave-Cramers-Computer.local 8.2.0 Darwin Kernel Version 8.2.0: Fri Jun 24 17:46:54 PDT 2005; root:xnu-792.2.4.obj~3/RELEASE_PPC Power Macintosh powerpc gcc --version powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 20041026 (Apple Co

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Andrew Dunstan
[patches removed] Tom Lane wrote: Simon Riggs <[EMAIL PROTECTED]> writes: It seems straightforward enough to put in an additional test, similar to the ones already there so that if its too big for a decimal we make it a float straight away - only a float can be that big in that case. After

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Jim C. Nasby
On Wed, Nov 02, 2005 at 06:12:37PM -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > It seems straightforward enough to put in an additional test, similar to > > the ones already there so that if its too big for a decimal we make it a > > float straight away - only a float can be

Re: [HACKERS] Assert failure found in 8.1RC1

2005-11-02 Thread Robert Creager
On Wed, 02 Nov 2005 15:19:44 -0500 Tom Lane <[EMAIL PROTECTED]> wrote: > Robert Creager <[EMAIL PROTECTED]> writes: > > TRAP: FailedAssertion("!(shared->page_number[slotno] == pageno && > > shared->page_status[slotno] == SLRU_PAGE_READ_IN_PROGRESS)", File: "slru.c", > > Line: 309) > > http://arch

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > It seems straightforward enough to put in an additional test, similar to > the ones already there so that if its too big for a decimal we make it a > float straight away - only a float can be that big in that case. After > that I can't really see what the p

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Simon Riggs
On Wed, 2005-11-02 at 15:09 -0500, Tom Lane wrote: > [ thinks for a moment... ] Actually, neither proposal is going to get > off the ground, because the parser's handling of numeric constants is > predicated on the assumption that type NUMERIC can handle any valid > value of FLOAT8, and so we can

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
Idar Tollefsen <[EMAIL PROTECTED]> writes: > *blush* Found it. > Any one the following flags will produce the described problems (alone > or in combination): > -faltivec > -mabi=altivec > -maltivec > -mcpu= Fascinating. I tried this on my own machine, and found out that -faltivec causes Darwin'

Re: slru.c race condition (was Re: [HACKERS] TRAP: FailedAssertion("!((itemid)->lp_flags

2005-11-02 Thread Jim C. Nasby
On Wed, Nov 02, 2005 at 02:04:09PM -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > BTW, that's a reversal from what I was originally arguing for, which was > > due to the performance penalty associated with --enable-cassert. My > > client is now running with Tom's suggestion

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
Tom Lane wrote: Also it would be worth showing in more detail exactly what happens when you try to build the (unpatched) sources. *blush* Found it. Any one the following flags will produce the described problems (alone or in combination): -faltivec -mabi=altivec -maltivec -mcpu= I have thes

Re: [HACKERS] ODBC link to other databases

2005-11-02 Thread Josh Berkus
Edward, > To help with my work, I've created a basic plugin that allows Postgres > to retreive data from tables on other databases via ODBC. So far I've > only tested it against MS SQL Server 2000. It's rather crude at the > moment, as the only operation supported is a SELECT * query, and it > cur

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
Idar Tollefsen <[EMAIL PROTECTED]> writes: > I'll fire up grep in the morning and search for headers defining bool, > true and false. Also it would be worth showing in more detail exactly what happens when you try to build the (unpatched) sources. regards, tom lane -

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
Tom Lane wrote: Well, I updated to Xcode 2.1 and ... it still works. So that's not the explanation. I suppose it was a long shot. But thanks for the effort. I'm currently wondering if you have any nonstandard versions of system headers floating about, perhaps in /usr/local/include. Well...

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Simon Riggs
On Wed, 2005-11-02 at 15:09 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > I wasn't trying to claim the bit assignment made sense. My point was > > that the work to mangle the two fields together to make it make sense > > looked like it would take more CPU (since the standard

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Andrew Dunstan
Bruce Momjian wrote: Andrew Dunstan wrote: There is no fseeko in the Windows libraries, nor any provision in the mingw headers that I can see for a 64 bit off_t. So we would need to roll our own to some extent - I think we need more than just a bit of configure cleverness. However, the

[HACKERS] ODBC link to other databases

2005-11-02 Thread Edward Di Geronimo Jr.
Hello all, To help with my work, I've created a basic plugin that allows Postgres to retreive data from tables on other databases via ODBC. So far I've only tested it against MS SQL Server 2000. It's rather crude at the moment, as the only operation supported is a SELECT * query, and it curre

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
If tuna is upgraded to the latest version of the buildfarm script it will report config output even on sucessful builds, which will contain gcc info. IE: http://lnk.nu/pgbuildfarm.org/5ii.pl (search for 'configure:2078'). She is apparently not. I can't find any stage logs in there. But thanks f

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Bruce Momjian
Andrew Dunstan wrote: > There is no fseeko in the Windows libraries, nor any provision in the > mingw headers that I can see for a 64 bit off_t. So we would need to > roll our own to some extent - I think we need more than just a bit of > configure cleverness. > > However, there is a Windows li

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Andrew Dunstan
Tom Lane wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: There is no fseeko in the Windows libraries, nor any provision in the mingw headers that I can see for a 64 bit off_t. So we would need to roll our own to some extent - I think we need more than just a bit of configure cleverness

Re: [HACKERS] Assert failure found in 8.1RC1

2005-11-02 Thread Robert Creager
On Wed, 02 Nov 2005 15:37:05 -0500 Tom Lane <[EMAIL PROTECTED]> wrote: > Robert Creager <[EMAIL PROTECTED]> writes: > > I can reproduce very quickly. Looks like I should try the patch in 248 > > first to see if it fixes 8.1RC1? > > Excellent. Yes, the second patch is higher priority, but please

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Martijn van Oosterhout
On Wed, Nov 02, 2005 at 12:53:07PM -0600, Jim C. Nasby wrote: > > This is one of those issues where we need to run tests and take input. > > We cannot decide this sort of thing just by debate alone. So, I'll leave > > this as a less potentially fruitful line of enquiry. > > Is it worth comming up

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Tom Lane
Andrew Dunstan <[EMAIL PROTECTED]> writes: > There is no fseeko in the Windows libraries, nor any provision in the > mingw headers that I can see for a 64 bit off_t. So we would need to > roll our own to some extent - I think we need more than just a bit of > configure cleverness. > However, th

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Andrew Dunstan
Magnus Hagander wrote: Andrew Dunstan <[EMAIL PROTECTED]> writes: Hmm. Windows reports an offset size of 4 bytes on a dump I just made ... is that relevant? What governs it? [ looks again... ] Hm, that is a 40Gb dump Kevin is complaining of, is

Re: [HACKERS] Assert failure found in 8.1RC1

2005-11-02 Thread Tom Lane
Robert Creager <[EMAIL PROTECTED]> writes: > TRAP: FailedAssertion("!(shared->page_number[slotno] == pageno && > shared->page_status[slotno] == SLRU_PAGE_READ_IN_PROGRESS)", File: "slru.c", > Line: 309) http://archives.postgresql.org/pgsql-hackers/2005-10/msg01385.php If you can reproduce the fai

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > I wasn't trying to claim the bit assignment made sense. My point was > that the work to mangle the two fields together to make it make sense > looked like it would take more CPU (since the standard representation of > signed integers is different for +ve an

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Simon Riggs
On Wed, 2005-11-02 at 13:46 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote: > >> I don't think it'd be worth having 2 types. Remember that the weight is > >> measured in base-10k digits. Suppose for instance > >>sign

[HACKERS] Assert failure found in 8.1RC1

2005-11-02 Thread Robert Creager
Hey all, While trying to get a reproducible test case for my CS storm problem (see http://archives.postgresql.org/pgsql-hackers/2005-10/msg00585.php), I upgraded to 8.1RC1 and encountered the following assert: TRAP: FailedAssertion("!(shared->page_number[slotno] == pageno && shared->page_status[

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
Idar Tollefsen <[EMAIL PROTECTED]> writes: > I believe 5026 came as part of the Xcode 2.1 package when I upgraded it from > Xcode 2.0. Xcode et. al. can be downloaded from ADC > (http://developer.apple.com/, choose "Log in"). Well, I updated to Xcode 2.1 and ... it still works. So that's not th

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > On Wed, Nov 02, 2005 at 06:05:19PM +0100, Idar Tollefsen wrote: >> Good catch tough. Could you try the Xcode 2.1 upgrade and see if you get >> the same errors I'm seeing? Or could anyone tell us what GCC version "tuna" >> is running? > If tuna is upgr

Re: slru.c race condition (was Re: [HACKERS] TRAP: FailedAssertion("!((itemid)->lp_flags

2005-11-02 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > BTW, that's a reversal from what I was originally arguing for, which was > due to the performance penalty associated with --enable-cassert. My > client is now running with Tom's suggestion of commenting out > CLOBBER_FREED_MEMORY and MEMORY_CONTEXT_CHECK

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Jim C. Nasby
On Wed, Nov 02, 2005 at 06:05:19PM +0100, Idar Tollefsen wrote: > Tom Lane wrote: > >I notice however that you seem to have a different version of gcc: > > > >>gcc --version: > >>powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 (Apple Computer, Inc. build > >>5026) > >>Copyright (C) 2005 Free Software

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Jim C. Nasby
On Wed, Nov 02, 2005 at 08:48:25AM +, Simon Riggs wrote: > On Tue, 2005-11-01 at 18:15 -0500, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > Anybody like to work out a piece of SQL to perform data profiling and > > > derive the distribution of values with trailing zeroes? > >

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote: >> I don't think it'd be worth having 2 types. Remember that the weight is >> measured in base-10k digits. Suppose for instance >> sign1 bit >> weight 7 bits (-64 .. +63) >>

Re: slru.c race condition (was Re: [HACKERS] TRAP: FailedAssertion("!((itemid)->lp_flags

2005-11-02 Thread Jim C. Nasby
On Wed, Nov 02, 2005 at 07:03:57AM -0500, Greg Stark wrote: > > I would bet that ninety percent of the Asserts in the existing code are on > > conditions that could represent, at worst, corruption of backend-local or > > even transaction-local data structures. Taking down the entire database > > cl

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Simon Riggs
On Tue, 2005-11-01 at 17:55 -0500, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > FWIW, most databases I've used limit NUMERIC to 38 digits, presumably to > > fit length info into 1 or 2 bytes. So there's something to be said for a > > small numeric type that has less overhead and

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
Tom Lane wrote: I notice however that you seem to have a different version of gcc: gcc --version: powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 (Apple Computer, Inc. build 5026) Copyright (C) 2005 Free Software Foundation, Inc. Mine says powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 20041026 (Ap

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
Idar Tollefsen <[EMAIL PROTECTED]> writes: > Trying to build 8.1RC1 on the above configuration fails because it seems to > have > defined bool, but still doesn't seem to know what bool is. Well, having just updated my laptop to 10.4.3, I tried it again, using the same configure switches you ment

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
Tom Lane wrote: OS X 10.4.what-exactly? And did you install the 10.4 Xcode tools? PG builds fine for me on 10.4.2, and there's at least one 10.4.something machine in the buildfarm, so it's not broken in general. With all due respect, please read my original posting again. uname output is ther

Re: [HACKERS] Retrieving all possible join trees from PLANNER..

2005-11-02 Thread Tom Lane
Gayathri TK <[EMAIL PROTECTED]> writes: > How do i efficiently retrieve this information? You don't, because the planner doesn't actually calculate all possible join trees --- it wouldn't run in a reasonable amount of time if it tried. It only considers trees that are built from subtrees that app

Re: [HACKERS] pg_restore [archiver] file offset in dump file

2005-11-02 Thread Kevin Grittner
I think you're on to something, Magnus. From a Windows cmd prompt the size matches Linux: 11/01/2005 12:35 PM40,874,747,876 dtr.dump >From an msys command prompt, ti does not look good: -rw-r--r--1 Administ Administ 18446744071634626532 Nov 1 12:35 dtr.dump I believe we got the lates

[HACKERS] Retrieving all possible join trees from PLANNER..

2005-11-02 Thread Gayathri TK
Hello all, I am using Postgres to do my Masters' thesis work. Given a query, i want to know all possible join plan trees for that query. This information can be got from src/backend/optimizer/path/allpaths.c/make_one_rel_by_joins() method. [ie] more specifically, I need joinitems[levels_need

Re: [HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Tom Lane
Idar Tollefsen <[EMAIL PROTECTED]> writes: > Trying to build 8.1RC1 on the above configuration fails OS X 10.4.what-exactly? And did you install the 10.4 Xcode tools? PG builds fine for me on 10.4.2, and there's at least one 10.4.something machine in the buildfarm, so it's not broken in general.

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Magnus Hagander
> > Andrew Dunstan <[EMAIL PROTECTED]> writes: > >> Hmm. Windows reports an offset size of 4 bytes on a dump I > just made > >> ... is that relevant? What governs it? > > > > [ looks again... ] Hm, that is a 40Gb dump Kevin is > complaining of, > > isn't it. Do our Windows builds have LARGEF

Re: [HACKERS] PGXS on VPATH?

2005-11-02 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > Is PGXS on VPATH builds supported? Probably not in the way you're thinking. IIRC pgxs.mk explicitly resets VPATH, and I think it has to do that because any VPATH embedded in Makefile.global would represent what was done at the time of building the PG i

[HACKERS] 8.1RC1 fails to build on OS X (10.4)

2005-11-02 Thread Idar Tollefsen
Hello, uname -a: Darwin fulcrum.local 8.3.0 Darwin Kernel Version 8.3.0: Mon Oct 3 20:04:04 PDT 2005; root:xnu-792.6.22.obj~2/RELEASE_PPC Power Macintosh powerpc gcc --version: powerpc-apple-darwin8-gcc-4.0.0 (GCC) 4.0.0 (Apple Computer, Inc. build 5026) Copyright (C) 2005 Free Software Found

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Pollard, Mike
I am not able to quickly find your numeric format, so I'll just throw this in. MaxDB (I only mention this because the format and algorithms are now under the GPL, so they can be reviewed by the public) uses a nifty number format that allows the use memcpy to compare two numbers when they are in t

Re: [HACKERS] pg_restore [archiver] file offset in dump file is too

2005-11-02 Thread Andrew Dunstan
Tom Lane said: > Andrew Dunstan <[EMAIL PROTECTED]> writes: >> Hmm. Windows reports an offset size of 4 bytes on a dump I just made >> ... is that relevant? What governs it? > > [ looks again... ] Hm, that is a 40Gb dump Kevin is complaining of, > isn't it. Do our Windows builds have LARGEFILE s

Re: [HACKERS] [ANNOUNCE] PostgreSQL 8.1.0 Release Candidate 1

2005-11-02 Thread Josh Rehman
Am I a bad person for running postresql under windows? I'd like to help you test but I'd a)need a windows binary and b)a list of features to test (perhaps a simple changelog would suffice.) I could be missing the binary, but I don't think so. On 10/30/05, Marc G. Fournier <[EMAIL PROTECTED]> wrot

[HACKERS] PGXS on VPATH?

2005-11-02 Thread Alvaro Herrera
Hi, Is PGXS on VPATH builds supported? I was just testing tsearch2 and it isn't working. Note that I'm first standing on the builddir and it doesn't work; then I'm back at the srcdir and it does work. 07:02 jovan tsearch2 2$ pwd /pgsql/build/00orig/contrib/tsearch2 07:02 jovan tsearch2 0$ LAN

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Mike Rylander
On 11/2/05, Simon Riggs <[EMAIL PROTECTED]> wrote: > On Tue, 2005-11-01 at 18:15 -0500, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > Anybody like to work out a piece of SQL to perform data profiling and > > > derive the distribution of values with trailing zeroes? > > > > Don't

Re: slru.c race condition (was Re: [HACKERS] TRAP: FailedAssertion("!((itemid)->lp_flags

2005-11-02 Thread Greg Stark
Tom Lane <[EMAIL PROTECTED]> writes: > Greg Stark <[EMAIL PROTECTED]> writes: > > I happen to think that except for the rare assertion that has major > > performance impact all the assertions should be on in production builds. The > > goal of assertions is to catch corruption quickly and that's s

Re: [HACKERS] Reducing the overhead of NUMERIC data

2005-11-02 Thread Simon Riggs
On Tue, 2005-11-01 at 18:15 -0500, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Anybody like to work out a piece of SQL to perform data profiling and > > derive the distribution of values with trailing zeroes? > > Don't forget leading zeroes. And all-zero (we omit digits entirely