Re: [HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Bruce Momjian
Jim C. Nasby wrote: > On Sun, Oct 15, 2006 at 06:19:12PM -0400, Tom Lane wrote: > > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > > Take a look at the failure output... > > > > > ERROR: could not load library > > > "/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so": dlopen > > > (/

Re: [HACKERS] constraints in query plans

2006-10-15 Thread Jeremy Drake
On Sun, 15 Oct 2006, Tom Lane wrote: > Jeremy Drake <[EMAIL PROTECTED]> writes: > > CREATE TABLE test_domain ( > > fkey integer not null, > > kinteger not null, > > x1 integer not null, > > x2 integer, > > mp m_or_p not null > > ); > > > CREATE INDEX test_domain_k_x1_x2_m ON test_do

Re: [HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Jim C. Nasby
On Sun, Oct 15, 2006 at 06:19:12PM -0400, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: > > Take a look at the failure output... > > > ERROR: could not load library > > "/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so": dlopen > > (/home/buildfarm/buildfarm/HEAD/inst

Re: [HACKERS] constraints in query plans

2006-10-15 Thread Tom Lane
Jeremy Drake <[EMAIL PROTECTED]> writes: > CREATE TABLE test_domain ( > fkey integer not null, > kinteger not null, > x1 integer not null, > x2 integer, > mp m_or_p not null > ); > CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x1, x2) WHERE mp = 'm'; > CREATE INDEX test_doma

[HACKERS] constraints in query plans

2006-10-15 Thread Jeremy Drake
I set up the following experiment: CREATE DOMAIN m_or_p AS "char" CHECK (VALUE = 'm' OR VALUE = 'p'); CREATE TABLE test_domain ( fkey integer not null, kinteger not null, x1 integer not null, x2 integer, mp m_or_p not null ); CREATE INDEX test_domain_k_x1_x2_m ON test_domain (k, x

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Harvell F
On 15 Oct 2006, at 19:55, [EMAIL PROTECTED] wrote: On Mon, Oct 16, 2006 at 07:00:20AM +0930, Shane Ambler wrote: [EMAIL PROTECTED] wrote: As a thought experiment, I'm not seeing the benefit. I think if you could prove a benefit, then any proof you provided could be used to improve the already

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Jeremy Drake
On Sun, 15 Oct 2006, [EMAIL PROTECTED] wrote: > On Sun, Oct 15, 2006 at 08:31:36PM +0530, Merlin Moncure wrote: > > On 10/15/06, Anon Mous <[EMAIL PROTECTED]> wrote: > > > Would it be possible to combine a special memcache implementation of > > > memcache with a Postgresql interface wrapper? > > h

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Mon, Oct 16, 2006 at 07:00:20AM +0930, Shane Ambler wrote: > [EMAIL PROTECTED] wrote: > >As a thought experiment, I'm not seeing the benefit. I think if you > >could prove a benefit, then any proof you provided could be used to > >improve the already existing caching layers, and would apply equa

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Sun, Oct 15, 2006 at 02:39:36PM -0700, Josh Berkus wrote: > > Using memcache, I've had problems with consistency brought right to > > the front. Both of these have failed me: > > 1) When updating a PostgreSQL record, I invalidate the memcache record. > >If another process comes along

Re: [HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Tom Lane
"Jim C. Nasby" <[EMAIL PROTECTED]> writes: > Take a look at the failure output... > ERROR: could not load library > "/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so": dlopen > (/home/buildfarm/buildfarm/HEAD/inst/lib/postgresql/plpython.so) failed: > /usr/local/lib/python2.5/config

Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns

2006-10-15 Thread David Fetter
On Sun, Oct 15, 2006 at 04:50:15PM -0500, Andrew Dunstan wrote: > Tom Lane wrote: > > I wrote: > >> It looks to me like basically everywhere in plperl.c that does > >> newSVpv() should follow it with > >> > >> #if PERL_BCDVERSION >= 0x5006000L > >> if (GetDatabaseEncoding() == PG_UTF8)

Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns

2006-10-15 Thread Tom Lane
"Andrew Dunstan" <[EMAIL PROTECTED]> writes: > I am also wondering, now that it's been raised, if we need to issue a "use > utf8;" in the startup code, so that literals in the code get the right > encoding. Good question. I took care to ensure that the code strings passed to Perl are marked as UT

Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns

2006-10-15 Thread Andrew Dunstan
Tom Lane wrote: > I wrote: >> It looks to me like basically everywhere in plperl.c that does newSVpv() >> should follow it with >> >> #if PERL_BCDVERSION >= 0x5006000L >> if (GetDatabaseEncoding() == PG_UTF8) >> SvUTF8_on(sv); >> #endif > > Experimentation proved that th

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Josh Berkus
Mark, > Using memcache, I've had problems with consistency brought right to > the front. Both of these have failed me: > > 1) When updating a PostgreSQL record, I invalidate the memcache record. >If another process comes along in parallel before I commit, notices >that the memc

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Shane Ambler
[EMAIL PROTECTED] wrote: As a thought experiment, I'm not seeing the benefit. I think if you could prove a benefit, then any proof you provided could be used to improve the already existing caching layers, and would apply equally to read-only or read-write pages. For example, why not be able to h

Re: [HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Jim C. Nasby
On Sun, Oct 15, 2006 at 10:39:49PM +0200, Peter Eisentraut wrote: > Jim C. Nasby wrote: > > Currently, the FreeBSD ports make the following change when building > > python: > > > > --- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004 > > +++ src/pl/plpython/MakefileTue Dec 28 23:32:

Re: [HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Peter Eisentraut
Jim C. Nasby wrote: > Currently, the FreeBSD ports make the following change when building > python: > > --- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004 > +++ src/pl/plpython/MakefileTue Dec 28 23:32:16 2004 > @@ -9,7 +9,7 @@ > # shared library. Since there is no official way

Re: [HACKERS] [PATCHES] New shared memory hooks proposal (was Re: pre_load_libraries)

2006-10-15 Thread Tom Lane
Marc Munro <[EMAIL PROTECTED]> writes: > I am content that what you suggest is the right way to go. I will work > on a new patch immediately, unless you would prefer to do this yourself. I've already got some of the relevant changes made in my local copy, so I might as well just go ahead and fini

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Mon, Oct 16, 2006 at 05:14:59AM +0930, Shane Ambler wrote: > [EMAIL PROTECTED] wrote: > >On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote: > >>You could setup a table in memory to contain known popular data, you > >>could also use this to create a temporary table in memory to speed

[HACKERS] Threaded python on FreeBSD

2006-10-15 Thread Jim C. Nasby
Currently, the FreeBSD ports make the following change when building python: --- src/pl/plpython/Makefile.orig Fri Nov 19 20:23:01 2004 +++ src/pl/plpython/MakefileTue Dec 28 23:32:16 2004 @@ -9,7 +9,7 @@ # shared library. Since there is no official way to determine this # (at least n

Re: [HACKERS] [PATCHES] New shared memory hooks proposal (was Re:

2006-10-15 Thread Marc Munro
On Sat, 2006-10-14 at 14:55 -0400, Tom Lane wrote: > Marc Munro <[EMAIL PROTECTED]> writes: > > The attached patch provides add-ins with the means to register for > > shared memory and LWLocks. > > I finally got around to reviewing this patch, and realized that it's got > a pretty fundamental desi

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Tom Lane
Shane Ambler <[EMAIL PROTECTED]> writes: > [EMAIL PROTECTED] wrote: >> None of this avoids the cost of query planning, or query execution. > No but you can avoid costly disk access and still have the postgres > level of integrity and integration that memcached doesn't offer. If you're just tryin

Re: [HACKERS] Not quite there on timezone names in timestamp input

2006-10-15 Thread Tom Lane
Martijn van Oosterhout writes: > On Sat, Oct 14, 2006 at 07:42:18PM -0400, Tom Lane wrote: >> On investigation, the problem seems to lie with ParseDateTime(), which >> is being quite finicky about what it will believe is a timezone name. > Last time I wrote some code with the zic database, I let

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Shane Ambler
[EMAIL PROTECTED] wrote: On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote: You could setup a table in memory to contain known popular data, you could also use this to create a temporary table in memory to speed up multiple intermediate calculations without touching disks. I'm no

Re: [HACKERS] Asynchronous I/O Support

2006-10-15 Thread Martijn van Oosterhout
On Sun, Oct 15, 2006 at 02:26:12PM -0400, Neil Conway wrote: > On Sun, 2006-10-15 at 19:56 +0200, Martijn van Oosterhout wrote: > > Sure, I even implemented it once. Didn't get any faster. > > Did you just do something akin to s/read/aio_read/ etc., or something > more ambitious? I think that real

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Sun, Oct 15, 2006 at 06:43:43PM +0100, Lexington Luthor wrote: > With a bit of careful planning (and a few SELECT FOR UPDATE queries to > prevent deadlock), having perfect consistency and correct caching is > possible. I didn't respond directly to this claim of yours. SELECT FOR UPDATE is on

Re: [HACKERS] [BUGS] BUG #2683: spi_exec_query in plperl returns column names which are not marked as UTF8

2006-10-15 Thread Tom Lane
I wrote: > It looks to me like basically everywhere in plperl.c that does newSVpv() > should follow it with > > #if PERL_BCDVERSION >= 0x5006000L > if (GetDatabaseEncoding() == PG_UTF8) > SvUTF8_on(sv); > #endif Experimentation proved that this was insufficient to fix V

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Mon, Oct 16, 2006 at 03:08:39AM +0930, Shane Ambler wrote: > Just throwing some ideas around - > What if we could do something like > CREATE TABLESPACE myramcache LOCATION MEMORY(2GB); > CREATE TABLE mycache ( > id as integer, data as varchar(50)) > USING TABLESPACE myramcache; > INSERT INTO myc

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Sun, Oct 15, 2006 at 06:43:43PM +0100, Lexington Luthor wrote: > [EMAIL PROTECTED] wrote: > >Using memcache, I've had problems with consistency brought right to > >the front. Both of these have failed me: > >1) When updating a PostgreSQL record, I invalidate the memcache record. > > If

Re: [HACKERS] Asynchronous I/O Support

2006-10-15 Thread Neil Conway
On Sun, 2006-10-15 at 19:56 +0200, Martijn van Oosterhout wrote: > Sure, I even implemented it once. Didn't get any faster. Did you just do something akin to s/read/aio_read/ etc., or something more ambitious? I think that really taking advantage of the ability to have multiple I/O requests outsta

Re: [HACKERS] Asynchronous I/O Support

2006-10-15 Thread Luke Lonergan
Martijn, On 10/15/06 10:56 AM, "Martijn van Oosterhout" wrote: > Have enough systems actually got to the point of actually supporting > async I/O that it's worth implementing? I think there are enough high end applications / systems that need it at this point. The killer use-case we've identif

Re: [HACKERS] Asynchronous I/O Support

2006-10-15 Thread Martijn van Oosterhout
On Sun, Oct 15, 2006 at 04:16:07AM +0530, Raja Agrawal wrote: > Postgre8.1 doesn't seem to support asynchronous I/O. Has its design > been thought off already? Sure, I even implemented it once. Didn't get any faster. At that point I realised that my kernel didn't actually support async I/O, and th

Re: [HACKERS] Not quite there on timezone names in timestamp input

2006-10-15 Thread Martijn van Oosterhout
On Sat, Oct 14, 2006 at 07:42:18PM -0400, Tom Lane wrote: > On investigation, the problem seems to lie with ParseDateTime(), which > is being quite finicky about what it will believe is a timezone name. > In particular it won't accept digits as part of the name. That can > probably be fixed but I'

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Lexington Luthor
[EMAIL PROTECTED] wrote: Using memcache, I've had problems with consistency brought right to the front. Both of these have failed me: 1) When updating a PostgreSQL record, I invalidate the memcache record. If another process comes along in parallel before I commit, notices that

[HACKERS] Asynchronous I/O Support

2006-10-15 Thread Raja Agrawal
Postgre8.1 doesn't seem to support asynchronous I/O. Has its design been thought off already? To tried doing with a simple example: For a Index Nest loop join: Fetch the outer tuples in an array, and then send all the corresponding inner-tuple fetch requests asynchronously. Hence while the IO is

Re: [HACKERS] postgres database crashed

2006-10-15 Thread Ashish Goel
But the same code worked when I inserted around 2500 images in the database. After that it started crashing. So , I don't think it's because of error in the code. Can u suggest some other possible reasons and also why is it crashing at call to memcpy().Tom Lane <[EMAIL PROTECTED]> wrote: Ashish Goe

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Shane Ambler
Merlin Moncure wrote: On 10/15/06, Anon Mous <[EMAIL PROTECTED]> wrote: Would it be possible to combine a special memcache implementation of memcache with a Postgresql interface wrapper? have you seen http://people.freebsd.org/~seanc/pgmemcache/ merlin Now you got me thinkin ;-P Just thr

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Andrew Dunstan
Alvaro Herrera wrote: > [EMAIL PROTECTED] wrote: > >> Using memcache, I've had problems with consistency brought right to >> the front. Both of these have failed me: >> >> 1) When updating a PostgreSQL record, I invalidate the memcache >> record. >>If another process comes along in para

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Alvaro Herrera
[EMAIL PROTECTED] wrote: > Using memcache, I've had problems with consistency brought right to > the front. Both of these have failed me: > > 1) When updating a PostgreSQL record, I invalidate the memcache record. >If another process comes along in parallel before I commit, notices >

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Sun, Oct 15, 2006 at 08:31:36PM +0530, Merlin Moncure wrote: > On 10/15/06, Anon Mous <[EMAIL PROTECTED]> wrote: > > Would it be possible to combine a special memcache implementation of > > memcache with a Postgresql interface wrapper? > have you seen > http://people.freebsd.org/~seanc/pgmemcach

Re: [HACKERS] postgres database crashed

2006-10-15 Thread Andrew Dunstan
Did you follow Tom's suggestion of trying with a postgres configured with --enable-cassert ? cheers andrew Ashish Goel wrote: > But the same code worked when I inserted around 2500 images in the > database. After that it started crashing. So , I don't think it's because > of error in the code.

Re: [HACKERS] postgres database crashed

2006-10-15 Thread Ashish Goel
But the same code worked when I inserted around 2500 images in the database. After that it started crashing. So , I don't think it's because of error in the code. Can u suggest some other possible reasons and also why is it crashing at call to memcpy(). I have also checked the memory allocations ,

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread Merlin Moncure
On 10/15/06, Anon Mous <[EMAIL PROTECTED]> wrote: Would it be possible to combine a special memcache implementation of memcache with a Postgresql interface wrapper? have you seen http://people.freebsd.org/~seanc/pgmemcache/ merlin ---(end of broadcast)---

Re: [HACKERS] Postgresql Caching

2006-10-15 Thread mark
On Sun, Oct 15, 2006 at 03:41:25AM -0700, Anon Mous wrote: > However, the problem is surmountable and has been figured out by Oracle, > although I don't know how they did it: > > http://www.oracle.com/technology/products/ias/joc/index.html I'm pretty sure this is application-side caching. The ap

[HACKERS] Postgresql Caching

2006-10-15 Thread Anon Mous
Hi I may have a workable idea on a way to add caching to Postgres without disturbing the MVCC functionality. Caching, as I've been reading can provide an amazing and sometimes almost unbelievable performance boost to a database based application, especially for data that is rarely modified. ht

Re: [HACKERS] SQL functions, INSERT/UPDATE/DELETE RETURNING, and

2006-10-15 Thread Markus Schaber
Hi, Tom, Tom Lane wrote: > "Jim C. Nasby" <[EMAIL PROTECTED]> writes: >> The specific concern I have is large result sets, like 10s or 100s of MB >> (or more). We just added support for not buffering those in psql, so it >> seems like a step backwards to have the backend now buffering it (unless >

Re: [HACKERS] GROUP BY on a large table -- an idea

2006-10-15 Thread Markus Schaber
Hi, Dawid, Dawid Kuroczko wrote: > The hybrid approach means: sort as much as you can without spilling to > disk, then aggregate and store aggregate state variables in safe place > (like a "tree" above), get more tuples from the table, sort them, update > aggregate state variables, lather, rince,