[HACKERS] ODBC

2005-06-01 Thread Zahid Khan
hi I want to know that does ODBC executes all statement as simple query as i have't found any thing as parse/bind/execute messeges in ODBC case.Or where can i find the implementation of these messeges on ODBC side Thx Zahid K __ Do you Yahoo!?

Re: [HACKERS] Interval->day proposal

2005-06-01 Thread Michael Glaesemann
On Jun 1, 2005, at 1:42 PM, Michael Glaesemann wrote: -- v8.0.3 test=# select '25 hours'::interval; interval 1 day 01:00:00 (1 row) -- new interval code test=# select '25 hours'::interval; interval -- 25:00:00 (1 row) I'll be digging into the spec later and post w

Re: [HACKERS] Deadlock with tsearch2 index ...

2005-06-01 Thread Hannu Krosing
On T, 2005-05-31 at 14:41 -0300, Marc G. Fournier wrote: > Just want to make sure that this is, in fact, what is expected: > > client1: begin; > client1: update articles set some_col = where id = ; > client2: update articles set some_col2 = where id = ; > client1: update articles set some_col3 =

Re: [HACKERS] Consumer-grade vs enterprise-grade disk drives

2005-06-01 Thread Hannu Krosing
On T, 2005-05-31 at 17:08 +1200, Mark Kirkwood wrote: > Luke Lonergan wrote: > > Tom, > > > > This is a story that is evolving. Anyone else use StorageReview? Great > > comprehensive drive benchmarks: > > http://www.storagereview.com/ > > > > Check the comparisons between 15K RPM SCSI drives

[HACKERS] regarding storage in postgres

2005-06-01 Thread Nithin Sontineni
Hi, i want to know how postgres will store a relation in pages and where can i see the code related to this in the source code.Plz help me . S.Nithin __ Discover Yahoo! Use Yahoo! to plan a weekend, have fun online and more. Check it

Re: [HACKERS] ddl triggers

2005-06-01 Thread Simon Riggs
On Tue, 2005-05-31 at 11:13 -0400, Tom Lane wrote: > Barnali <[EMAIL PROTECTED]> writes: > > I was wondering whether it will be useful to extend postgreSQL support > > to ddl triggers. > > This has been proposed and rejected before ... see the archives. Could we keep track of NOT TODO items also

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Tue, 2005-05-31 at 22:47 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Recent test results have shown a substantial performance improvement > > (+25%) if WAL logging is disabled for large COPY statements. > > How much of that is left after we fix the 64-bit-CRC issue? We

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Greg Stark
Neil Conway <[EMAIL PROTECTED]> writes: > On Wed, 2005-06-01 at 00:40 -0400, Alvaro Herrera wrote: > > This doesn't work for COPY, but maybe for CREATE TABLE AS we could log > > the fact that the command was executed, so the replayer could execute > > the same command again. > > > > Of course, t

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Christopher Kings-Lynne
There are some other arguments in favour of a LOAD command Alon? We already have LOAD, so you'll have to choose something else :) Chris ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscr

Re: [HACKERS] Physical Tlist optimization still possible?

2005-06-01 Thread Simon Riggs
On Tue, 2005-05-31 at 22:31 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > If a table is created as WITHOUT OIDS, is it still possible to take > > advantage of the physical tlist optimization? > > Um ... it depends. There are cases where you have to have an OID, > and cases

Re: [HACKERS] regarding storage in postgres

2005-06-01 Thread Hans-Jürgen Schönig
Nithin Sontineni wrote: Hi, i want to know how postgres will store a relation in pages and where can i see the code related to this in the source code.Plz help me . S.Nithin __ Discover Yahoo! Use Yahoo! to plan a weekend, have fu

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Greg Stark
Greg Stark <[EMAIL PROTECTED]> writes: > For CREATE TABLE AS in the non-PITR case you don't really need to WAL log the > records at all. If it fails in the middle you just drop the table. When it > completes you do a checkpoint before acknowledging the COMMIT. > > I think this is already done fo

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Simon Riggs
On Tue, 2005-05-31 at 22:36 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > Hmmm. I seem to recall asking myself why xl_prev existed if it wasn't > > used, but passed that by. Damn. > > I couldn't believe it'd been overlooked this long, either. It's the > sort of thing that y

Re: [HACKERS] Can we simplify win32 threading code

2005-06-01 Thread Magnus Hagander
> I have simplified the code according to the discussion. Now > there is no pipe or signaling threads, process access shared > memory directly to pass signals. > Seems everything works fine except pg_ctl. I now have two > choices to fix it: > > (1) Record Shared memory name (it is already there

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Mark Cave-Ayland
> -Original Message- > From: Tom Lane [mailto:[EMAIL PROTECTED] > Sent: 31 May 2005 17:27 > To: Greg Stark > Cc: Mark Cave-Ayland (External); 'Manfred Koizar'; 'Bruce > Momjian'; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] Cost of XLogInsert CRC calculations (cut) > The odds

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Hannu Krosing
On K, 2005-06-01 at 09:16 +0100, Simon Riggs wrote: > On Tue, 2005-05-31 at 22:47 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > Recent test results have shown a substantial performance improvement > > > (+25%) if WAL logging is disabled for large COPY statements. > > > >

Re: [HACKERS] Tablespace-level Block Size Definitions

2005-06-01 Thread Zeugswetter Andreas DAZ SD
> The problem I see with this proposal is that the buffer manager knows > how to handle only a equally-sized pages. And the shared memory stuff > gets sized according to size * num_pages. So what happens if a certain > tablespace A with pagesize=X gets to have a lot of its pages cached, > evicti

Re: [HACKERS] Quick-and-dirty compression for WAL backup blocks

2005-06-01 Thread Mark Cave-Ayland
> Date: Tue, 31 May 2005 16:26:20 -0400 > From: Tom Lane <[EMAIL PROTECTED]> > To: pgsql-hackers@postgreSQL.org > Subject: Quick-and-dirty compression for WAL backup blocks > Message-ID: <[EMAIL PROTECTED]> (cut) > ... make the WAL writing logic aware of the layout > of buffer pages --- specific

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 16:34 +0800, Christopher Kings-Lynne wrote: > > There are some other arguments in favour of a LOAD command Alon? > > We already have LOAD, so you'll have to choose something else :) Its annoying, I grant you. :-) LOAD 'library' would still need to be the default. LOAD

Re: [HACKERS] Tablespace-level Block Size Definitions

2005-06-01 Thread Dawid Kuroczko
On 6/1/05, Zeugswetter Andreas DAZ SD <[EMAIL PROTECTED]> wrote: > You could create a separate bufferpool per page size. Of course that > has other disadvantages. > > Is it really so difficult to create and attach another shmem segment ? Well, I don't think it is much different from having two da

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 04:44 -0400, Greg Stark wrote: > Greg Stark <[EMAIL PROTECTED]> writes: > > > For CREATE TABLE AS in the non-PITR case you don't really need to WAL log > > the > > records at all. If it fails in the middle you just drop the table. When it > > completes you do a checkpoint be

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Hannu Krosing
On K, 2005-06-01 at 00:01 +0100, Simon Riggs wrote: > Recent test results have shown a substantial performance improvement > (+25%) if WAL logging is disabled for large COPY statements. This is to > be expected, though has a price attached: losing the ability to crash > recover data loaded in this

Re: [HACKERS] Tablespace-level Block Size Definitions

2005-06-01 Thread Hannu Krosing
On K, 2005-06-01 at 14:00 +0200, Dawid Kuroczko wrote: > On 6/1/05, Zeugswetter Andreas DAZ SD <[EMAIL PROTECTED]> wrote: > > You could create a separate bufferpool per page size. Of course that > > has other disadvantages. > > > > Is it really so difficult to create and attach another shmem segme

Re: [HACKERS] Deadlock with tsearch2 index ...

2005-06-01 Thread Marc G. Fournier
On Wed, 1 Jun 2005, Oleg Bartunov wrote: We'd love to start tomorrow working on GiST, but we have some current obligations we need to sort out. It will probably take more then a day to do fund raising, which is what Christopher is suggesting ... but they need some sort of 'value' for a goal

Re: [HACKERS] Oracle Style packages on postgres

2005-06-01 Thread Bruce Momjian
Josh Berkus wrote: > Bruce, > > > > >Added to TODO: > > > > > > > >* Add the features of packages > > > > o Make private objects accessable only to objects in the same > > > > schema > > > > o Allow current_schema.objname to access current schema objects > > > > o Add se

Re: [HACKERS] Consumer-grade vs enterprise-grade disk drives

2005-06-01 Thread Christopher Browne
[EMAIL PROTECTED] ("Jeffrey W. Baker") wrote: > On Mon, 2005-05-30 at 21:38 -0700, Luke Lonergan wrote: > After reading this very comprehensive benchmark: > > http://print.tweakers.net/?reviews/557 > > I purchased one of the Areca controllers with a large battery-backed > cache. It is unholy fast.

Re: [HACKERS] Deadlock with tsearch2 index ...

2005-06-01 Thread Oleg Bartunov
On Wed, 1 Jun 2005, Marc G. Fournier wrote: On Wed, 1 Jun 2005, Oleg Bartunov wrote: We'd love to start tomorrow working on GiST, but we have some current obligations we need to sort out. It will probably take more then a day to do fund raising, which is what Christopher is suggesting ... b

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Simon Riggs wrote: > On Wed, 2005-06-01 at 04:44 -0400, Greg Stark wrote: > > Greg Stark <[EMAIL PROTECTED]> writes: > > > > > For CREATE TABLE AS in the non-PITR case you don't really need to WAL log > > > the > > > records at all. If it fails in the middle you just drop the table. When it > > >

Re: [HACKERS] Physical Tlist optimization still possible?

2005-06-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > Where I'm leading to is this: If the physical tlist optimization is a > considerable gain, and it required OIDs, then changing the system > default to WITHOUT OIDS will cause us to lose the optimization. Yes? It doesn't require OIDs, though. What it requi

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > Sorry to followup to my own message, but it occurs to me that COPY could be > made to automatically do this for the case of an empty destination table too. Not unless you are proposing to change COPY to acquire a lock strong enough to lock out other writers

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > On Wed, 2005-06-01 at 16:34 +0800, Christopher Kings-Lynne wrote: > There are some other arguments in favour of a LOAD command Alon? >> >> We already have LOAD, so you'll have to choose something else :) > Its annoying, I grant you. :-) > LOAD 'libra

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > If I/O is a problem, then surely you will agree that PreAllocXLog is > still required and should not be run by a backend? It is still required, but it isn't run by backends --- it's fired off during checkpoints. I think there was some discussion recently

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Tom Lane
"Mark Cave-Ayland" <[EMAIL PROTECTED]> writes: > I'm still a little nervous about dropping down to CRC32 from CRC64 and so > was just wondering what the net saving would be using one CRC64 across the > whole WAL record? None to speak of; the startup/teardown time is trivial. It's the per-byte cos

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Jochem van Dieten
On 01 Jun 2005 04:44:24 -0400, Greg Stark wrote: > Greg Stark writes: >> >> For CREATE TABLE AS in the non-PITR case you don't really need to WAL log the >> records at all. If it fails in the middle you just drop the table. When it >> completes you do a checkpoint before acknowledging the COMMIT.

Re: [HACKERS] Quick-and-dirty compression for WAL backup blocks

2005-06-01 Thread Tom Lane
"Mark Cave-Ayland" <[EMAIL PROTECTED]> writes: > I also noticed your comment above that mentioned that compression would be > less effective as the pages became more full. Would changing the loading > factor of database pages have an effect here, as I would have thought that > the WAL would be fsyn

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Tom Lane wrote: > Greg Stark <[EMAIL PROTECTED]> writes: > > Sorry to followup to my own message, but it occurs to me that COPY could be > > made to automatically do this for the case of an empty destination table > > too. > > Not unless you are proposing to change COPY to acquire a lock strong >

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Jochem van Dieten wrote: > On 01 Jun 2005 04:44:24 -0400, Greg Stark wrote: > > Greg Stark writes: > >> > >> For CREATE TABLE AS in the non-PITR case you don't really need to WAL log > >> the > >> records at all. If it fails in the middle you just drop the table. When it > >> completes you do a c

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Bruce Momjian writes: > Tom Lane wrote: >> Not unless you are proposing to change COPY to acquire a lock strong >> enough to lock out other writers to the table for the duration ... > Well, if the table is initally empty, what harm is there in locking the > table? You cannot *know* whether it is

Re: [HACKERS] Quick-and-dirty compression for WAL backup blocks

2005-06-01 Thread Alvaro Herrera
On Wed, Jun 01, 2005 at 10:12:34AM -0400, Tom Lane wrote: > "Mark Cave-Ayland" <[EMAIL PROTECTED]> writes: > > I also noticed your comment above that mentioned that compression would be > > less effective as the pages became more full. Would changing the loading > > factor of database pages have an

[HACKERS] fdatasync failed, I/O error

2005-06-01 Thread Brusser, Michael
Title: fdatasync failed, I/O error Our customer is running Postgresql v. 7.3.4 on Linux(Redhat 3.0 Ent.) The application crashed and I am trying to understand what could have caused this kind of problems:    fdatasync of log file 0, segment 2 failed: input/output error    PGSTAT: Error clo

Re: [HACKERS] Interval->day proposal

2005-06-01 Thread Tom Lane
Michael Glaesemann <[EMAIL PROTECTED]> writes: > On Jun 1, 2005, at 1:42 PM, Michael Glaesemann wrote: >> -- v8.0.3 >> test=# select '25 hours'::interval; >> interval >> >> 1 day 01:00:00 >> (1 row) >> >> -- new interval code >> test=# select '25 hours'::interval; >> interval >> -

Re: [HACKERS] Interval->day proposal

2005-06-01 Thread Tom Lane
Michael Glaesemann <[EMAIL PROTECTED]> writes: > I've started working on this change, and one difference has shown up > immediately in the regression tests. v8.0.3 currently returns: >SELECT INTERVAL '10 years -11 month -12 days +13:14' AS "9 years..."; >9 years... >-

Re: [HACKERS] Can we simplify win32 threading code

2005-06-01 Thread Tom Lane
"Magnus Hagander" <[EMAIL PROTECTED]> writes: > Why not just use the pid in teh name, and have one segment per backend? Being used only for signals you mean? That might work. I dislike fooling around with the contents of postmaster.pid, as that will inject platform-specific code into places wher

Re: [HACKERS] ODBC

2005-06-01 Thread Tom Lane
Zahid Khan <[EMAIL PROTECTED]> writes: > I want to know that does ODBC executes all statement > as simple query as i have't found any thing as > parse/bind/execute messeges in ODBC case. Probably; the ODBC code long predates the V3 protocol, and I don't think it's been updated. But you're asking

Re: [HACKERS] ddl triggers

2005-06-01 Thread Jaime Casanova
> Could we keep track of NOT TODO items also? > > The question is where do you put this idea if you reject it? :-) > And of course a link to the thread, or an explanation on why it was rejected. That way if someone can solve the reason for reject it maybe a new discussion can be made. -- regard

[HACKERS] Google's Summer of Code ...

2005-06-01 Thread Marc G. Fournier
In case nobody has seen this: http://code.google.com/summerofcode.html Might be something to help fun, say, the GiST stuff? Just need to find a student that could be mentor'd/directed/brought up to speed? Marc G. Fournier Hub.Org Networking Services (http://www.hub.o

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Dawid Kuroczko
On 6/1/05, Hans-Jürgen Schönig <[EMAIL PROTECTED]> wrote: > Personally I don't think that it is a good idea to do that. > People will tend to corrupt their systems because they want speed > (sometimes without thinking about the consequences). > > I can only think of one scenario where nologging wo

Re: [HACKERS] fdatasync failed, I/O error

2005-06-01 Thread Tom Lane
"Brusser, Michael" <[EMAIL PROTECTED]> writes: > Our customer is running Postgresql v. 7.3.4 on Linux(Redhat 3.0 Ent.) > The application crashed and I am trying to understand what could have caused > this kind of problems: >fdatasync of log file 0, segment 2 failed: input/output error >PGS

Re: [HACKERS] regarding storage in postgres

2005-06-01 Thread Tom Lane
=?ISO-8859-1?Q?Hans-J=FCrgen_Sch=F6nig?= <[EMAIL PROTECTED]> writes: > Nithin Sontineni wrote: >> i want to know how postgres will store a >> relation in pages and where can i see the code related >> to this in the source code.Plz help me . > You can find the code in src/backend/storage. Also, re

Re: [HACKERS] fdatasync failed, I/O error

2005-06-01 Thread Alvaro Herrera
On Wed, Jun 01, 2005 at 10:27:04AM -0400, Brusser, Michael wrote: > > Our customer is running Postgresql v. 7.3.4 on Linux(Redhat 3.0 Ent.) > The application crashed and I am trying to understand what could have caused > this kind of problems: > >fdatasync of log file 0, segment 2 failed: inp

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Jonah H. Harris
Marc, If you wanted to submit PostgreSQL to Google for this, I'd be willing to work with any potentials. Should someone choose to work on PostgreSQL projects, the money can go to the group. If anyone sees a problem with submitting under the PostgreSQL Global Development Group, I can do it u

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 10:18 -0400, Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Not unless you are proposing to change COPY to acquire a lock strong > >> enough to lock out other writers to the table for the duration ... > > > Well, if the table is initally empty, what harm is

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Hannu Krosing <[EMAIL PROTECTED]> writes: > I think this should be a decision done when creating a table, just like > TEMP tables. So you always know if a certain table is or is not > safe/replicated/recoverable. > This has also the advantage of requiring no changes to actual COPY and > INSERT comm

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Bruce Momjian writes: > Jochem van Dieten wrote: >> Why only on an empty table? What is the problem with bypassing WAL on >> any table as long as all files of that table are fsync'ed before >> commit? > Because adding rows to a table might modify existing pages, and if the > COPY fails, you have

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > 4. Optionally, we set a flag on the table showing the whole table is > frozen. Anybody writing to this table subsequently will spoil this flag. > If the flag is set, all forms of VACUUM will return success immediately > without performing a scan (since it i

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Jochem van Dieten
On 6/1/05, Bruce Momjian wrote: > Jochem van Dieten wrote: >> >> Why only on an empty table? What is the problem with bypassing WAL on >> any table as long as all files of that table are fsync'ed before >> commit? > > Because adding rows to a table might modify existing pages, and if the > COPY fa

Re: [HACKERS] Can we simplify win32 threading code

2005-06-01 Thread Magnus Hagander
> > Why not just use the pid in teh name, and have one segment > per backend? > > Being used only for signals you mean? That might work. That was my idea. We'll end up using three global namespace objects (mutex+event+shared memory) instead of one (named pipe), but as we're not talking thousand

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 11:31 -0400, Tom Lane wrote: > Hannu Krosing <[EMAIL PROTECTED]> writes: > > I think this should be a decision done when creating a table, just like > > TEMP tables. So you always know if a certain table is or is not > > safe/replicated/recoverable. > > This has also the advan

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Alvaro Herrera
On Wed, Jun 01, 2005 at 06:00:28PM +0100, Simon Riggs wrote: > On Wed, 2005-06-01 at 11:31 -0400, Tom Lane wrote: > > Perhaps it could work to use an ALTER TABLE command to flip the state. > > But I'm not really seeing the point compared to treating it as a COPY > > option. I do not believe that

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > - it is an option to COPY and CREATE TABLE AS, not GUC, not ALTER TABLE AFAICS it could just happen automatically for CREATE TABLE AS; there's no need for an option there, any more than there is for CREATE INDEX. The only reason it needs to be an expli

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Alon Goldshuv
I have been working on improving the COPY command performance and as a result also came up with other thoughts that may possibly be better off implemented in a new command (i.e LOAD DATA) rather than adding them to the existing COPY. The improvements I made were in the COPY parsing logic - changin

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 11:55 -0400, Tom Lane wrote: > Simon Riggs <[EMAIL PROTECTED]> writes: > > 4. Optionally, we set a flag on the table showing the whole table is > > frozen. Anybody writing to this table subsequently will spoil this flag. > > If the flag is set, all forms of VACUUM will return

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Robert Treat
We have a whole TODO list of potential items for people to work on, and we have several member of this list that work for universities in some capacity, so istm we ought to give it a shot. I would think this is something that the foundation would be good to be involved in as well, as it would c

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Oleg Bartunov
I think we definitely should submit PostgreSQL to Google. Oleg On Wed, 1 Jun 2005, Jonah H. Harris wrote: Marc, If you wanted to submit PostgreSQL to Google for this, I'd be willing to work with any potentials. Should someone choose to work on PostgreSQL projects, the money can go t

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Alvaro Herrera
On Wed, Jun 01, 2005 at 06:55:46PM +0100, Simon Riggs wrote: > On Wed, 2005-06-01 at 11:55 -0400, Tom Lane wrote: > > Simon Riggs <[EMAIL PROTECTED]> writes: > > > 4. Optionally, we set a flag on the table showing the whole table is > > > frozen. Anybody writing to this table subsequently will spoi

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Simon Riggs <[EMAIL PROTECTED]> writes: > If the server crashes, we replay WAL. If we see a load start message, we > truncate the relation and note that a load has started. If there is WAL > data for the tuples, we replay it. If WAL replay ends without the load > transaction having successfully com

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Jonah H. Harris
My participation would be limited to the actual number of newcomers. I feel comfortable helping 2 or 3 people; any more than that would impact my clients and consulting time. Although, it would never hurt to have additional mentors :). Robert Treat wrote: We have a whole TODO list of poten

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Alvaro Herrera <[EMAIL PROTECTED]> writes: > On Wed, Jun 01, 2005 at 06:55:46PM +0100, Simon Riggs wrote: >> We're holding the table lock and will continue to do so until end of >> transaction. No transaction with an earlier id will ever see the data we >> load because of the lock. > Suppose you l

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Marc G. Fournier
Are there any restrictions on 'mentor groups'? For instance, could we create a 'mentors' mailing list, closed subscriptions, that contained those willing to actively mentor, so that those being mentor'd have multiple ppl's knowledge's to feed upon? On Wed, 1 Jun 2005, Jonah H. Harris wrote:

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Marc G. Fournier
One concern that was raised off of this list was the steep learning curve to start with, to get to the point of beign able to do anything ... Do we have any 'students' that are already up to speed, enough so that they'd be able to accomplish something significant over a 2-3 month period? O

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Greg Stark
It occurs to me that at least on some OSes the WAL logs are being synced with O_SYNC or its ilk. In those cases the writes should be guaranteed to be written out in the order postgres wrote them. So if the tail end of the WAL entry is there (is there any sort of footer?) then the entire entry must

Re: [HACKERS] Cost of XLogInsert CRC calculations

2005-06-01 Thread Tom Lane
Greg Stark <[EMAIL PROTECTED]> writes: > It occurs to me that at least on some OSes the WAL logs are being synced with > O_SYNC or its ilk. In those cases the writes should be guaranteed to be > written out in the order postgres wrote them. So if the tail end of the WAL > entry is there (is there a

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Jonah H. Harris
Not sure about the current 'students'. I need to take a look and see what kind of somewhat easy stuff is on the TODO (I haven't seen it in awhile). I like your suggestion for the mailing list. Marc G. Fournier wrote: One concern that was raised off of this list was the steep learning curv

Re: [HACKERS] Consumer-grade vs enterprise-grade disk drives

2005-06-01 Thread Luke Lonergan
Hannu, > RAID10 > > "Throughput report Y-axis is type of test X-axis is number of processes" > "Record size = 8 Kbytes " > "Output is in ops/sec" > " Initial write "1352.90 > "Rewrite " 413.31 > RAID5 > > "Throughput report Y-axis is type of test X-axis is number of processes"

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 14:24 -0400, Tom Lane wrote: > Alvaro Herrera <[EMAIL PROTECTED]> writes: > > On Wed, Jun 01, 2005 at 06:55:46PM +0100, Simon Riggs wrote: > >> We're holding the table lock and will continue to do so until end of > >> transaction. No transaction with an earlier id will ever se

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Oleg Bartunov
On Wed, 1 Jun 2005, Marc G. Fournier wrote: One concern that was raised off of this list was the steep learning curve to start with, to get to the point of beign able to do anything ... Do we have any 'students' that are already up to speed, enough so that they'd be able to accomplish somet

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 14:14 -0400, Tom Lane wrote: > So what I'm thinking is we need no special WAL entries for this. What > we need is just an operating mode of COPY in which it doesn't WAL-log > its inserts, but instead fsyncs before completion, much like index build > does. For safety it must

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Meredith L. Patterson
Marc G. Fournier wrote: > Do we have any 'students' that are already up to speed, enough so that > they'd be able to accomplish something significant over a 2-3 month period? Well, I suppose now might be a good time to de-lurk. Hi, my name's Meredith, I'm a PhD student at the University of Iowa, I

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Robert Treat
Funny how things come full circle... in previous threads someone mentioned a few low hanging fruit items, we should start there, and maybe get one of the more hard-core hackers to go through the TODO list and mark up some other items. My take on the mentor thing was that the students could p

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Marc G. Fournier
'k, since there appears to be interest, I've tried to read a bit more up on it, specifically the FAQs ... seems simple enough, but someone else may have found a link that I haven't ... Basically, developer (ie. Ramy Hassan) would go to: http://code.google.com/soc_application.html Fi

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Jochem van Dieten wrote: > On 6/1/05, Bruce Momjian wrote: > > Jochem van Dieten wrote: > >> > >> Why only on an empty table? What is the problem with bypassing WAL on > >> any table as long as all files of that table are fsync'ed before > >> commit? > > > > Because adding rows to a table might mo

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 15:50 -0500, Meredith L. Patterson wrote: > Marc G. Fournier wrote: > > Do we have any 'students' that are already up to speed, enough so that > > they'd be able to accomplish something significant over a 2-3 month period? > > Well, I suppose now might be a good time to de-lu

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Marc G. Fournier
On Wed, 1 Jun 2005, Robert Treat wrote: Funny how things come full circle... in previous threads someone mentioned a few low hanging fruit items, we should start there, and maybe get one of the more hard-core hackers to go through the TODO list and mark up some other items. My take on the ment

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Alon Goldshuv wrote: > 9) allow for Simon's WAL bypass. > > I have surely missed some problems that hide behind the idea, but these > points make me believe that LOAD DATA is a good idea. The community is unlikely to add a new LOAD DATA command that does _almost_ everything COPY does. We are muc

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > Tom Lane wrote: > >> Not unless you are proposing to change COPY to acquire a lock strong > >> enough to lock out other writers to the table for the duration ... > > > Well, if the table is initally empty, what harm is there in locking the > > table? >

Re: [HACKERS] Google's Summer of Code ...

2005-06-01 Thread Meredith L. Patterson
Simon Riggs wrote: > Is it possible that you could put sufficient of the application into > PostgreSQL to genericise some features? Stonebraker's Third Wave was > *all* about putting data intensive operations closer to where the data > is stored/accessed. And just like that, a lightbulb goes off i

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Luke Lonergan
Bruce, > The problem with a new command is that it becomes unclear when you > should use COPY and when LOAD DATA, and it confuses users, and has > maintenance overhead. If Bizgres wants a new command name, go for it, > but it is unlikely that the community release is going to go in that > directi

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
Bruce Momjian writes: > One idea would be to look at the table file size first. If it has zero > blocks, lock the table and if it still has zero blocks, do the no-WAL > copy. I think that's a bad idea. It would make the behavior unpredictable --- sometimes a COPY will take an exclusive lock, an

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Tom Lane
"Luke Lonergan" <[EMAIL PROTECTED]> writes: > One of the reasons to consider a LOAD DATA command is that we can isolate > the need for performance improvements and special syntax from the concerns > of preserving the legacy behavior of COPY for use as the primary mechanism > for DUMP and RESTORE.

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 10:35 -0700, Alon Goldshuv wrote: > I have been working on improving the COPY command performance > Around 40% for 15 column (mixed types) table. > Around 90% for 1 column table. Thats very cool. > 2) A modified command syntax for introducing a direct single row error > h

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Simon Riggs
On Wed, 2005-06-01 at 18:32 -0400, Tom Lane wrote: > Bruce Momjian writes: > > One idea would be to look at the table file size first. If it has zero > > blocks, lock the table and if it still has zero blocks, do the no-WAL > > copy. > > I think that's a bad idea. It would make the behavior unp

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Luke Lonergan wrote: > Bruce, > > > The problem with a new command is that it becomes unclear when you > > should use COPY and when LOAD DATA, and it confuses users, and has > > maintenance overhead. If Bizgres wants a new command name, go for it, > > but it is unlikely that the community release

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian writes: > > One idea would be to look at the table file size first. If it has zero > > blocks, lock the table and if it still has zero blocks, do the no-WAL > > copy. > > I think that's a bad idea. It would make the behavior unpredictable > --- sometimes a COPY w

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Simon Riggs wrote: > I'm not clear from all of those options whether we still need a LOAD > command, based upon other issues/comments raised on this thread. > > However, there are some other arguments for why it might be a good idea > to have a LOAD DATA command separate from COPY. Certainly long

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Joe Conway
Simon Riggs wrote: On Wed, 2005-06-01 at 10:35 -0700, Alon Goldshuv wrote: 2) A modified command syntax for introducing a direct single row error handling. By direct I mean - a row that if rejected from within the COPY 5) allow an ERRORLIMIT to allow control of aborting a load after a certain

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Alon Goldshuv
Bruce, The patch is not there to show that something is "broken" is it there to show how things could be done in another way, which may or may not be more desireable. > COPY works as designed. The idea that some guy we have never heard of > is going to appear and rewrite COPY's processing and te

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Luke Lonergan
Tom, > > ... and instead, define some new behavior that will soon be considered > broken legacy code itself? I'll not argue further about whether to have a separate LOAD command. That's not as important as fixing the performance issues in the data load path in PostgreSQL to me. However, I find i

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Alon Goldshuv wrote: > Bruce, > > The patch is not there to show that something is "broken" is it there to > show how things could be done in another way, which may or may not be more > desireable. Sure, we are always looking for ways to improve things. > > COPY works as designed. The idea that

Re: [HACKERS] NOLOGGING option, or ?

2005-06-01 Thread Bruce Momjian
Luke Lonergan wrote: > Tom, > > > > ... and instead, define some new behavior that will soon be considered > > broken legacy code itself? > > I'll not argue further about whether to have a separate LOAD command. > That's not as important as fixing the performance issues in the data load > path in

Re: [HACKERS] [PATCHES] Adding \x escape processing to COPY, psql, backend

2005-06-01 Thread Bruce Momjian
Patch applied. Thanks for the COPY \x patch. --- Bruce Momjian wrote: > Tom Lane wrote: > > Bruce Momjian writes: > > > Here is a new version of the three \x hex support patches. I have added > > > \x for psql variables,

  1   2   >