[PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread ingrid martinez
I want to know, how can I improve the performance of postgres, I have a java class thar inserts register every 30 min but is very slow. The process of postgres consume the 78% of CPU.   I have in /etc/system   set shmsys:shminfo_shmmax=0x5000set shmsys:shminfo_shmmni=0x100set shmsys:sh

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Fri, 8 Aug 2003, mixo wrote: > I have just installed redhat linux 9 which ships with Pg > 7.3.2. Pg has to be setup so that data inserts (blobs) should > be able to handle at least 8M at a time. Nothing has to be done to tune postgresql to handle this, 8 Meg blobs are no problem as far as I k

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Bruce Momjian
Rod Taylor wrote: -- Start of PGP signed section. > On Fri, 2003-08-08 at 14:53, Andrew Sullivan wrote: > > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote: > > > > > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS > > > (ext2?) with the partition that

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Shridhar Daithankar
On 11 Aug 2003 at 23:42, Ron Johnson wrote: > On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote: > > > Well, yeah. But given the Linux propensity for introducing major > > > features in "minor" releases (and thereby introducing all the > > > attendant bugs), I'd think twice about using _

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread scott.marlowe
On Wed, 13 Aug 2003, ingrid martinez wrote: > Floes table looks like this > > Table "flows" > Column| Type | Modifiers > --+--+--- > flidload | bigint | n

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Josh Berkus
Jeff, > Informix, etc. have spent a lot of time and money working on it. > They also have the advantage of having many paid fulltime > developers who are doing this for a job, not as a weekend hobby > (Compared to the what? 2-3 full time PG developers). I think 4-6 full-time, actually, plus about

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Christopher Browne
[EMAIL PROTECTED] (Josh Berkus) writes: >> The other advantage (which I hinted to above) with raw disks is being able >> to optimize queries to take advantage of it. Informix is multithreaded >> and it will spawn off multiple "readers" to do say, a seq scan (and merge >> the results at the end). >

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Ron Johnson
On Wed, 2003-08-13 at 09:37, Tom Lane wrote: > Jeff <[EMAIL PROTECTED]> writes: > > On Tue, 12 Aug 2003, Christopher Browne wrote: > >> Are you _certain_ that's still true? Have you a metric that shows > >> Informix being 10x faster on a modern system? That would be quite > >> surprising... > >

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Christopher Browne
Martha Stewart called it a Good Thing [EMAIL PROTECTED] ("Gregory S. Williamson")wrote: > FWIW, Informix can be run using a "cooked" (Unix) file for storing > data or it uses "raw" disk space and bypasses the ordinary (high > level) UNIX controllers and does its own reads/writes. About 10 > times f

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Shridhar Daithankar
On 5 Aug 2003 at 14:15, Peter Childs wrote: > On Tue, 5 Aug 2003, Shridhar Daithankar wrote: > > > On 5 Aug 2003 at 8:09, Jeff wrote: > > > > I would suggest autovacuum daemon which is in CVS contrib works for 7.3.x as > > well.. Or schedule a vacuum analyze every 15 minutes or so.. > >

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread Yaroslav Mazurak
Hi, All! Richard Huxton wrote: On Thursday 07 August 2003 17:30, Yaroslav Mazurak wrote: Richard Huxton wrote: On Thursday 07 August 2003 09:24, Yaroslav Mazurak wrote: PG's memory use can be split into four areas (note - I'm not a developer so this could be wrong). 1. Shared memory - vital

Re: [PERFORM] query/table design help

2003-08-14 Thread Shridhar Daithankar
On Tuesday 05 August 2003 15:03, Ara Anjargolian wrote: > I have a table > permissions > with the fields (party_id integer, permission varchar, key_name varchar, > key_value integer) > for which I need to a query to see if a person has permission to carry out > a particular action. > The query look

[PERFORM] query/table design help

2003-08-14 Thread Ara Anjargolian
I have a table permissions with the fields (party_id integer, permission varchar, key_name varchar, key_value integer) for which I need to a query to see if a person has permission to carry out a particular action. The query looks like: SELECT 1 FROM permissions WHERE party_id in (4, 7, 11, 26)

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Andrew Sullivan
On Mon, Aug 11, 2003 at 08:47:07AM -0600, scott.marlowe wrote: > This isn't saying to not use jounraling, but I would definitely test it > under load first to make sure it's not gonna lose data or get corrupted. Well, yeah. But given the Linux propensity for introducing major features in "minor"

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Matthew T. O'Connor
On Wed, 2003-08-06 at 00:45, Tom Lane wrote: > For core code, the answer would be a big NYET. We do not do feature > additions in point releases, only bug fixes. While contrib code is more > under the author's control than the core committee's control, I'd still > say that you'd be making a big m

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread Shridhar Daithankar
On 6 Aug 2003 at 15:42, Yaroslav Mazurak wrote: > >>sort_mem = 131072 > > This sort_mem value is *very* large - that's 131MB for *each sort* that gets > > done. I'd suggest trying something in the range 1,000-10,000. What's probably > > happening with the error above is that PG is allocating ridi

Re: [PERFORM] How to force PostgreeSQL to work faster?

2003-08-14 Thread Paul Thomas
On 11/08/2003 09:59 Serge Dorofeev wrote: Hi PostrgeSQL team, My PostrgeSQL installed as part of CYGWIN (Windows XP). I have compared performance PostrgeSQL to MS SQL (I used a little Java program with number of inserts in table). MS SQL is faster in 12 times :-( It's very strange results. Guys w

[PERFORM] Index correlation (was: Moving postgresql.conf tunables into 2003... )

2003-08-14 Thread Manfred Koizar
On Thu, 07 Aug 2003 19:31:52 -0400, Tom Lane <[EMAIL PROTECTED]> wrote: >The correlation is between index order and heap order --- that is, are >the tuples in the table physically in the same order as the index? >The better the correlation, the fewer heap-page reads it will take to do >an index sca

[PERFORM] Perfomance Tuning

2003-08-14 Thread mixo
I have just installed redhat linux 9 which ships with Pg 7.3.2. Pg has to be setup so that data inserts (blobs) should be able to handle at least 8M at a time. The machine has two P III 933MHz CPU's, 1.128G RAM (512M*2 + 128M), and a 36 Gig hd with 1 Gig swap and 3 equal size ext3 partitions. What

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > Since pg_autovaccum is a contrib module does that mean I can make functional > changes that will be included in point release of 7.4? Well, the bar is lower for contrib stuff than for core, but you'd better get such changes in PDQ, I'd say ...

Re: [PERFORM] Filesystems WAS: Perfomance Tuning

2003-08-14 Thread Rod Taylor
> specific drives might be a valuble capability. i know that i could go into > /var/lib/pgsql/data/base and fan the contents out, but this is unweildy and > impractical. has any consideration been given to providing a way to manage > such a deployment? The ability to take various database objects

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread Stephan Szabo
On Wed, 6 Aug 2003, Sebastien Lemieux wrote: > On Wed, 6 Aug 2003, Tom Lane wrote: > > > Sebastien Lemieux <[EMAIL PROTECTED]> writes: > > > All the time is taken at the commit of both transaction. > > > > Sounds like the culprit is foreign-key checks. > > > > One obvious question is whether you h

[PERFORM] ext3 block size

2003-08-14 Thread Wilson A. Galafassi Jr.
hello. my database size is 5GB. what is the block size recommend? thanks wilson  

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Matthew T. O'Connor
On Tue, 2003-08-05 at 12:49, Bruce Momjian wrote: > > Well, the bar is lower for contrib stuff than for core, but you'd better > > get such changes in PDQ, I'd say ... > > The contrib stuff is usually at the control of the author, so you can > make changes relatively late. However, the later the

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Tom Lane
"Matthew T. O'Connor" <[EMAIL PROTECTED]> writes: > ... My really question > was can I make large changes to a contrib module to a point release, > meaning, 7.4.0 will have what is in beta, but 7.4.1 would have a much > improved version. Does that sound possible? For core code, the answer would b

Re: [PERFORM] Odd performance results - more info

2003-08-14 Thread Medora Schauer
> Medora Schauer wrote: > > I would greatly appreciate it if someone could run this > code in their > > environment and let me know if you get results similiar to mine. > > The INT test results in execution times of 11 - 50+ secs increasing > > each time the test is run. The FLOAT test executi

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Josh Berkus
Ron, > If the databases are on different machines, maybe the postgres.conf > or pg_hba.conf files are different, and the buffer counts is affect- > ing the optimizer? The databases are on the same machine, using the same postmaster. -- -Josh Berkus Aglio Database Solutions San Francisco ---

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Shridhar Daithankar
On 5 Aug 2003 at 9:18, Jeff wrote: > As for the pg_dumping of it. I suppose it would work on this table as it > is only a couple million rows and not terribly big data-wise. The other > tables in this db are rather big and a load is not fast. (It is about > 8GB). You need to dump only those table

[PERFORM] Some vacuum & tuning help

2003-08-14 Thread Jeff
I've been trying to search through the archives, but it hasn't been successful. We recently upgraded from pg7.0.2 to 7.3.4 and things were happy. I'm trying to fine tune things to get it running a bit better and I'm trying to figure out how vacuum output correlates to tuning parameters. Here's th

Re: [PERFORM] Moving postgresql.conf tunables into 2003...

2003-08-14 Thread Tom Lane
Sean Chittenden <[EMAIL PROTECTED]> writes: >> If you CLUSTER on an index and then ANALYSE, you get a correlation of >> 1.0 (== optimum) for the first column of the index. > Correlating of what to what? Of data to nearby data? Of data to > related data (ie, multi-column index?)? Of related data

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Matthew T. O'Connor
From: "Christopher Browne" <[EMAIL PROTECTED]> > Shridhar Daithankar wrote: > > I agree, specifying per table thresholds would be good in autovacuum.. > > Which begs the question of what the future direction is for pg_autovacuum. This is a good question. > There would be some merit to having pg_

Re: [PERFORM] PostgreSql under Linux

2003-08-14 Thread Neil Conway
On Wed, Aug 06, 2003 at 03:03:41PM -0300, Wilson A. Galafassi Jr. wrote: > I'm installing Postgresql under linux for better performance and i want to know how > is the best configuration. > 1. What is the best linux distribuition for better performance? The Linux distribution itself isn't that i

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread Tom Lane
Sebastien Lemieux <[EMAIL PROTECTED]> writes: > All the time is taken at the commit of both transaction. Sounds like the culprit is foreign-key checks. One obvious question is whether you have your foreign keys set up efficiently in the first place. As a rule, the referenced and referencing colu

Re: [PERFORM] about performance of postgreSQL

2003-08-14 Thread Chris Travers
Hi Xin; PostgreSQL is configured to run on virutally anything out of the box. The reason for this is that, like Oracle, the database manager will not start if it cannot allocate sufficient resources. We take the approach of ensuring that it will start so you can tune it. I would recomment trying

Re: [PERFORM] How to force PostgreeSQL to work faster?

2003-08-14 Thread Tomka Gergely
Hi! Please send me the test db and the queries, with precise information maybe the developers can help. -- Tomka Gergely "S most - vajon barbárok nélkül mi lesz velünk? Ők mégiscsak megoldás voltak valahogy..." ---(end of broadcast)--- TIP 6: Ha

[PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread Sebastien Lemieux
Hi, I'm running on Redhat 7.2 with postgresql 7.3.2 and I have two schema in the same database 'db' and 'db_dev'. Both contain a set of >20 tables for a total of less than 50 Mb of data each (on the order of 50k rows in total). Once in a while (often these days!), I need to synchronize the dev

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Christopher Browne
Matthew T. O'Connor wrote: > Fair point, my only concern is that a backend integrated > pg_autovacuum would be radically different from the current libpq > based client application. Unfortunately, a "configurable-via-tables" pg_autovacuum is also going to be quite different from the current "uncon

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On 11 Aug 2003, Ron Johnson wrote: > On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote: > > > Well, yeah. But given the Linux propensity for introducing major > > > features in "minor" releases (and thereby introducing all the > > > attendant bugs), I'd think twice about using _any_ Linu

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread Andrew Sullivan
On Mon, Aug 11, 2003 at 10:58:18PM -0400, Christopher Browne wrote: > 1. Nobody has gone through any formal proofs, and there are few > systems _anywhere_ that are 100% reliable. I think the problem is that ext2 is known to be not perfectly crash safe. That is, fsck on reboot after a crash can

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Tom Lane wrote: > "scott.marlowe" <[EMAIL PROTECTED]> writes: > > On Wed, 6 Aug 2003, Tom Lane wrote: > >> One obvious question is whether you have your foreign keys set up > >> efficiently in the first place. As a rule, the referenced and > >> referencing columns should have

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Christopher Kings-Lynne
> So, group the inserts in transactions with maybe 1000 commands each. It > will go much faster. It can then cache the rows and in the end just make > sure all 1000 have been written out on disk. More than that, he should be using COPY - it's 10x faster than even grouped inserts. Chris

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Fri, 8 Aug 2003, Andrew Sullivan wrote: > On Fri, Aug 08, 2003 at 09:40:20AM -0700, Jonathan Gardner wrote: > > > > Redhat puts ext3 on by default. Consider switching to a non-journaling FS > > (ext2?) with the partition that holds your data and WAL. > > I would give you exactly the opposite

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Jonathan Gardner
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Friday 08 August 2003 03:28, mixo wrote: > I have just installed redhat linux 9 which ships with Pg > 7.3.2. Pg has to be setup so that data inserts (blobs) should > be able to handle at least 8M at a time. The machine has > two P III 933MHz CPU's,

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Sean Chittenden
> > > Well, yeah. But given the Linux propensity for introducing major > > > features in "minor" releases (and thereby introducing all the > > > attendant bugs), I'd think twice about using _any_ Linux feature > > > until it's been through a major version (e.g. things introduced in > > > 2.4.x won

[PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread Yaroslav Mazurak
Hi All! I have installed PostgreSQL 7.3.2 on FreeBSD 4.7, running on PC with CPU Pentium II 400MHz and 384Mb RAM. Problem is that SQL statement (see below) is running too long. With current WHERE clause 'SUBSTR(2, 2) IN ('NL', 'NM') return 25 records. With 1 record, SELECT time is about 50

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Still, they are differences. Attached. Actually, it was mainly "cases" that I wanted to know about --- specifically, whichever columns are in "idx_cases_tgroup". Also, which of the trial_groups columns is the pkey? regards, tom la

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Gregory S. Williamson
FWIW, Informix can be run using a "cooked" (Unix) file for storing data or it uses "raw" disk space and bypasses the ordinary (high level) UNIX controllers and does its own reads/writes. About 10 times faster and safer. Of course, itmay have taken a lot of programmer time to make that solid. But

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread ingrid martinez
The query that execute is only inserts, I use a batch of 300 and then do commit. insert into FLOWS values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) and postgresql.conf looks like this # # PostgreSQL configuration file # - # # This file consists of lines of the form #

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread Rod Taylor
On Wed, 2003-08-13 at 11:17, ingrid martinez wrote: > The query that execute is only inserts, I use a batch of 300 and then do > commit. > > insert into FLOWS values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Any foreign keys on this table? Triggers or Rules? What kind of hardware do you have?

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread Josh Berkus
People: > On Mon, Aug 11, 2003 at 10:58:18PM -0400, Christopher Browne wrote: > > 1. Nobody has gone through any formal proofs, and there are few > > systems _anywhere_ that are 100% reliable. > > I think the problem is that ext2 is known to be not perfectly crash > safe. That is, fsck on reboot

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Josh Berkus
Tom, > Partly. The numbers suggest that in ANALYZE's default sample of 3000 > rows, it's only finding about a dozen non-null tgroup_ids (yielding the > 0.996 null_frac value); and that in one case all dozen are different and > in the other case there are two duplicates. It would help if you > bo

Re: [PERFORM] partitioning for postgresql

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Wilson A. Galafassi Jr. wrote: > hello!!! > what is suggested partitioning schema for postgresql?? > the size of my db is 5BG and i have 36GB scsi disk! The first recommendation is to run Postgresql on a RAID set for reliability. I'm assuming you're building a machine and

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Ron Johnson
On Tue, 2003-08-12 at 13:39, Bruce Momjian wrote: > OK, I got some hard evidence. Here is a discussion on the Linux kernel > mailing list with postings from Allen Cox (ac Linux kernels) and Stephen > Tweedie (ext3 author). > > http://www.tux.org/hypermail/linux-kernel/1999week14/subject.htm

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread scott.marlowe
More than likely you are suffering from an affliction known as type mismatch. This is listed as tip 9 here on the performance list (funny, it was sent at the bottom of your reply :-) What happens is that when you do: select * from some_table where id=123; where id is a bigint the query planne

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Neil Conway
On Tue, Aug 12, 2003 at 12:52:46AM -0400, Bruce Momjian wrote: I don't use Linux and was just repeating what I had heard from others, > and read in postings. I don't have any first-hand experience with ext2 > (except for a laptop I borrowed that wouldn't boot after being shut > off), but others o

[PERFORM] Analyze makes queries slow...

2003-08-14 Thread Stef
Hi all, I posted this problem on the sql list, and was referred to this list in stead. I have attached an sql statement that normally runs under 3 minutes. That is, until I vacuum analyze the database (or just the tables in the query), then the same query runs longer than 12 hours, and I have to k

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Andrew Sullivan
On Tue, Aug 12, 2003 at 02:39:19PM -0400, Bill Moran wrote: > Meaning ... just tell it a raw partition to keep the data on and > Postgre would create its own "filesystem" ... obviously, doing that > would allow Postgre to bypass all the failings of all filesystems > and rely entirely apon its own r

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Bill Moran
Shridhar Daithankar wrote: On 11 Aug 2003 at 23:42, Ron Johnson wrote: On Mon, 2003-08-11 at 19:50, Christopher Kings-Lynne wrote: Well, yeah. But given the Linux propensity for introducing major features in "minor" releases (and thereby introducing all the attendant bugs), I'd think twice abou

[PERFORM] On Linux Filesystems

2003-08-14 Thread Christopher Browne
Bruce Momjian commented: "Uh, the ext2 developers say it isn't 100% reliable" ... "I mentioned it while I was visiting Red Hat, and they didn't refute it." 1. Nobody has gone through any formal proofs, and there are few systems _anywhere_ that are 100% reliable. NASA has occasionally lost spa

Re: [PERFORM] Analyze makes queries slow...

2003-08-14 Thread Stef
On Mon, 11 Aug 2003 14:25:03 -0400 Tom Lane <[EMAIL PROTECTED]> wrote: => set enable_mergejoin to off; => explain analyze ... query ... => => If it finishes in a reasonable amount of time, send the explain output. Hi again, I did this on the 7.3.1 database, and attached the output.

[PERFORM] Odd performance results - more info

2003-08-14 Thread Medora Schauer
I didn't get any helpful responses to my previous email so I thought I would try again, this time with example code. Below is my orignal email and code for a dead simple ASCII menu driven application that demonstrates the problem. The app starts up with a menu of 4 items to create the test data

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Also, there doesn't seem to be any way in 7.2 for me to find out what the > current statistics target for a column is. What am I missing? There still isn't a handy command for it --- you have to look at pg_attribute.attstattarget for the column.

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread scott.marlowe
On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > Hi All! > > Shridhar Daithankar wrote: > > > On 7 Aug 2003 at 10:05, Yaroslav Mazurak wrote: > > >>>It needs to reflect how much cache the system is using - try the "free" > >>>command to see figures. > > >>I'm not found "free" ut

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread Richard Huxton
On Thursday 07 August 2003 17:30, Yaroslav Mazurak wrote: > Hi All! > > > First, thanks for answers! > > Richard Huxton wrote: > > On Thursday 07 August 2003 09:24, Yaroslav Mazurak wrote: > >>>IIRC there is a limit on filesystem cache on freeBSD. 300MB by default. > >>>If that is the case, y

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread scott.marlowe
On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > scott.marlowe wrote: > > > On Thu, 7 Aug 2003, Yaroslav Mazurak wrote: > > >>Shridhar Daithankar wrote: > > > That's a nice theory, but it doesn't work out that way. About every two > > months someone shows up wanting postgresql to use all the mem

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread Andrew Sullivan
On Tue, Aug 12, 2003 at 09:36:21AM -0700, Josh Berkus wrote: > So it's a tradeoff with loss of performance vs. recovery time. In > a server room with redundant backup power supplies, "clean room" > security and fail-over services, I can certainly imagine that data > journalling would not be neede

[PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Josh Berkus
Folks: I have a live and a test database for an in-production system running on 7.2.4. The test database is a copy of the live one. They are running on the same copy of Postgres, on the same server. I use the test database to test changes before I apply them to the production system. Perio

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Peter Darley
Josh, I'm sure that you've thought of this, but it sounds like you may not have done an analyze in your new DB. Thanks, Peter Darley -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Josh Berkus Sent: Monday, August 11, 2003 3:48 PM To: Ron Johnson; PgS

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Rod Taylor wrote: >> On Fri, 2003-08-08 at 14:53, Andrew Sullivan wrote: >>> I would give you exactly the opposite advice: _never_ use a >>> non-journalling fs for your data and WAL. I suppose if you can >>> afford to lose some transactions, you can do w

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread Andrew Sullivan
On Wed, Aug 13, 2003 at 09:03:31AM -0500, ingrid martinez wrote: > I want to know, how can I improve the performance of postgres, I > have a java class thar inserts register every 30 min but is very > slow. What does the query do? How is postgres configured? A -- Andrew Sullivan

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Christopher Kings-Lynne
> Well, yeah. But given the Linux propensity for introducing major > features in "minor" releases (and thereby introducing all the > attendant bugs), I'd think twice about using _any_ Linux feature > until it's been through a major version (e.g. things introduced in > 2.4.x won't really be stable

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Christopher Kings-Lynne
Try the pg_autovacuum daemon in CVS contrib dir. It works fine with 7.3. Chris - Original Message - From: "Jeff" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Tuesday, August 05, 2003 8:09 PM Subject: [PERFORM] Some vacuum & tuning help > I've been trying to search through the arc

Re: [PERFORM] Peformance of Update

2003-08-14 Thread Shridhar Daithankar
On 11 Aug 2003 at 19:33, [EMAIL PROTECTED] wrote: > Currently we are using postgresql 7.3 with Redhat linux 9. We find that > when we try to execute 200,000 update statement through JDBC, the performance > of degraded obviously for each update statement when comparing with less update > stateme

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread Andrew Sullivan
On Wed, Aug 13, 2003 at 10:17:45AM -0500, ingrid martinez wrote: > The query that execute is only inserts, I use a batch of 300 and then do > commit. > > insert into FLOWS values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Are there any foreign keys, &c? > > and > > postgresql.conf looks like thi

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > By everything I can measure, the live database and the test are > identical; yet the test does not think that idx_caseclients_case is > very accessable, and the live database knows it is. Let's see the pg_stats rows for case_clients in both databases. The

Re: [PERFORM] How can I Improve performance in Solaris?

2003-08-14 Thread ingrid martinez
the primary key is flidload - Original Message - From: "scott.marlowe" <[EMAIL PROTECTED]> To: "ingrid martinez" <[EMAIL PROTECTED]> Cc: "Andrew Sullivan" <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]> Sent: Wednesday, August 13, 2003 11:47 AM Subject: Re: [PERFORM] How can I Improve perform

Re: [PERFORM] Filesystems WAS: Perfomance Tuning

2003-08-14 Thread Richard Welty
On Tue, 12 Aug 2003 13:09:42 -0700 Josh Berkus <[EMAIL PROTECTED]> wrote: > This idea has been discussed numerous times on the HACKERS list, and is > a > (pretty much) closed issue. While Oracle and SQL Server use their own > filesystems, PostgreSQL will not because: ... > 2) The filesystem pro

Re: [PERFORM] Filesystems WAS: Perfomance Tuning

2003-08-14 Thread Christopher Kings-Lynne
> there are advantages to being able to split the database across a slew of > disk drives. if we accept the notion of using the native OS filesystem on > each, it would seem that being able to direct various tables and indices to > specific drives might be a valuble capability. i know that i could

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Bruce Momjian
Josh Berkus wrote: > Jeff, > > > Informix, etc. have spent a lot of time and money working on it. > > They also have the advantage of having many paid fulltime > > developers who are doing this for a job, not as a weekend hobby > > (Compared to the what? 2-3 full time PG developers). > > I think

Re: [PERFORM] Analyze makes queries slow...

2003-08-14 Thread Stef
Hi Tom, Thanks for responding. I got as much info as I could : On Mon, 11 Aug 2003 11:43:45 -0400 Tom Lane <[EMAIL PROTECTED]> wrote: => Could we see the results of "EXPLAIN ANALYZE", rather than just EXPLAIN, => for the un-analyzed case? Attached the output of this. => Also, what do you see

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Bruce Momjian
OK, I got some hard evidence. Here is a discussion on the Linux kernel mailing list with postings from Allen Cox (ac Linux kernels) and Stephen Tweedie (ext3 author). http://www.tux.org/hypermail/linux-kernel/1999week14/subject.html#start Search for "softupdates and ext2". Here is the

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread mixo
Thanks to everyone who responded. It's a pity that the discussion has gone the ext2 vs ext3 route. The main reason I asked my original question is that I am currently importing data into Pg which is about 2.9 Gigs. Unfortunately, to maintain data intergrity, data is inserted into a table one row at

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Andrew Sullivan wrote: > On Mon, Aug 11, 2003 at 10:58:18PM -0400, Christopher Browne wrote: > > 1. Nobody has gone through any formal proofs, and there are few > > systems _anywhere_ that are 100% reliable. > > I think the problem is that ext2 is known to be not perfectly

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Christopher Kings-Lynne
> On Tue, 2003-08-05 at 17:40, Christopher Browne wrote: > > Unfortunately, a "configurable-via-tables" pg_autovacuum is also going > > to be quite different from the current "unconfigurable" version. You don't need to create actual tables - just use 'virtual' tables, like the pg_settings one. T

Re: [PERFORM] How Many Inserts Per Transactions

2003-08-14 Thread scott.marlowe
On Tue, 5 Aug 2003, Trevor Astrope wrote: > I was wondering if anyone found a sweet spot regarding how many inserts to > do in a single transaction to get the best performance? Is there an > approximate number where there isn't any more performance to be had or > performance may drop off? > >

Re: [PERFORM] Filesystems WAS: Perfomance Tuning

2003-08-14 Thread Josh Berkus
Greg, > FWIW, Informix can be run using a "cooked" (Unix) file for storing data or > it uses "raw" disk space and bypasses the ordinary (high level) UNIX > controllers and does its own reads/writes. About 10 times faster and safer. > Of course, itmay have taken a lot of programmer time to make tha

Re: [PERFORM] EXTERNAL storage and substring on long strings

2003-08-14 Thread Tom Lane
Scott Cain <[EMAIL PROTECTED]> writes: > A few days ago, I asked for advice on speeding up substring queries on > the GENERAL mailing list. Joe Conway helpfully pointed out the ALTER > TABLE STORAGE EXTERNAL documentation. After doing the alter, > the queries got slower! Here is the background:

Re: [PERFORM] Analyze makes queries slow...

2003-08-14 Thread Tom Lane
Stef <[EMAIL PROTECTED]> writes: > I have attached an sql statement that normally runs under 3 minutes. > That is, until I vacuum analyze the database (or just the tables in the query), > then the same query runs longer than 12 hours, and I have to kill it. Could we see the results of "EXPLAIN ANA

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Tom Lane
Josh Berkus <[EMAIL PROTECTED]> writes: > Tom, >> Okay, here's our problem: >> >> live DB: tgroup_id n_distinct = -1 >> >> test DN: tgroup_id n_distinct = 11 >> >> The former estimate actually means that it thinks tgroup_id is a unique >> column, whereas the latter says there are only 11 disti

Re: [PERFORM] How to efficiently duplicate a whole schema?

2003-08-14 Thread scott.marlowe
On Wed, 6 Aug 2003, Tom Lane wrote: > Sebastien Lemieux <[EMAIL PROTECTED]> writes: > > All the time is taken at the commit of both transaction. > > Sounds like the culprit is foreign-key checks. > > One obvious question is whether you have your foreign keys set up > efficiently in the first pla

[PERFORM] about performance of postgreSQL

2003-08-14 Thread xin fu
Dear master:    I have learned postgreSQL for serveral days, now i meet some problems. when I use a TPCC(Transaction Processing Performance Council) test program to test the performance of postgreSQL , postgreSQL works very slowly, it almost need 1 minute to finish a transaction, and the CPU pe

Re: [PERFORM] Odd problem with performance in duplicate database

2003-08-14 Thread Josh Berkus
Tom, > Partly. The numbers suggest that in ANALYZE's default sample of 3000 > rows, it's only finding about a dozen non-null tgroup_ids (yielding the > 0.996 null_frac value); and that in one case all dozen are different and > in the other case there are two duplicates. It would help if you > bo

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Neil Conway wrote: > On Tue, Aug 12, 2003 at 12:52:46AM -0400, Bruce Momjian wrote: > I don't use Linux and was just repeating what I had heard from others, > > and read in postings. I don't have any first-hand experience with ext2 > > (except for a laptop I borrowed that wo

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Bruce Momjian
Uh, the ext2 developers say it isn't 100% reliable --- at least that is that was told. I don't know any personally, but I mentioned it while I was visiting Red Hat, and they didn't refute it. Now, the failure window might be quite small, but I have seen it happen myself, and have heard it from o

Re: [PERFORM] On Linux Filesystems

2003-08-14 Thread Bruce Momjian
As I remember, there were clear cases that ext2 would fail to recover, and it was known to be a limitation of the file system implementation. Some of the ext2 developers were in the room at Red Hat when I said that, so if it was incorrect, they would hopefully have spoken up. I addressed the com

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread Tom Lane
Yaroslav Mazurak <[EMAIL PROTECTED]> writes: > Current postgresql.conf settings (some) are: > max_locks_per_transaction = 16 This strikes me as a really bad idea --- you save little space by reducing it from the default, and open yourself up to unexpected failures. > wal_buffers = 256 Tha

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread scott.marlowe
On Tue, 12 Aug 2003, Christopher Kings-Lynne wrote: > > Well, yeah. But given the Linux propensity for introducing major > > features in "minor" releases (and thereby introducing all the > > attendant bugs), I'd think twice about using _any_ Linux feature > > until it's been through a major versi

Re: [PERFORM] Analyze makes queries slow...

2003-08-14 Thread Tom Lane
Stef <[EMAIL PROTECTED]> writes: > => And what PG version is this, exactly? > PostgreSQL 7.3.1 Ah, I think I see it: you are getting burnt by a mergejoin estimation bug that was fixed in 7.3.2. Please update (you might as well go to 7.3.4 while you're at it) and see if the results improve.

Re: [PERFORM] Some vacuum & tuning help

2003-08-14 Thread Neil Conway
On Wed, Aug 06, 2003 at 12:45:34AM -0400, Tom Lane wrote: > For core code, the answer would be a big NYET. We do not do feature > additions in point releases, only bug fixes. While contrib code is more > under the author's control than the core committee's control, I'd still > say that you'd be m

Re: [PERFORM] Perfomance Tuning

2003-08-14 Thread Ron Johnson
On Wed, 2003-08-13 at 10:46, Josh Berkus wrote: > Jeff, > [snip] > > The other advantage (which I hinted to above) with raw disks is being able > > to optimize queries to take advantage of it. Informix is multithreaded > > and it will spawn off multiple "readers" to do say, a seq scan (and merge

  1   2   >