Re: [PERFORM] Follow-Up: How to improve db performance with $7K?

2005-04-06 Thread Steinar H. Gunderson
On Tue, Apr 05, 2005 at 09:44:56PM -0700, Kevin Brown wrote: > Now, the performance is pretty bad considering the setup -- a RAID 5 > with five 73.6 gig SCSI disks (10K RPM, I believe). Reads through the > filesystem come through at about 65 megabytes/sec, writes about 35 > megabytes/sec (at least

Re: [PERFORM] Follow-Up: How to improve db performance with $7K?

2005-04-06 Thread Steinar H. Gunderson
On Wed, Apr 06, 2005 at 03:26:33PM +0200, PFC wrote: > Well, unless you have PCI 64 bits, the "standard" PCI does 133 MB/s > which is then split exactly in two times 66.5 MB/s for 1) reading from > the > PCI network card and 2) writing to the PCI harddisk controller. No wonder > y

Re: [PERFORM] Réf

2005-04-06 Thread Steinar H. Gunderson
On Wed, Apr 06, 2005 at 01:18:29PM -0400, Rod Taylor wrote: > Yeah, I think that can be done provided there is more than one worker. > My limit seems to be about 1000 transactions per second each with a > single insert for a single process (round trip time down the Fibre > Channel is large) but run

Re: [PERFORM] speed up query with max() and odd estimates

2005-04-26 Thread Steinar H. Gunderson
On Tue, Apr 26, 2005 at 03:16:57PM -0500, Matthew Nuzum wrote: > Seq Scan on usage_access (cost=0.00..1183396.40 rows=12713851 > width=116) (actual time=481796.22..481839.43 rows=3343 loops=1) That's a gross misestimation -- four orders of magnitude off! Have you considering doing this in two s

Re: [PERFORM] PGSQL Capacity

2005-05-09 Thread Steinar H. Gunderson
On Mon, May 09, 2005 at 09:22:40PM +0200, [EMAIL PROTECTED] wrote: > How can i know a capacity of a pg database ? > How many records my table can have ? > I saw in a message that someone have 50 000 records it's possible in a table ? > (My table have 8 string field (length 32 car)). > Thanks for yo

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Steinar H. Gunderson
On Fri, May 13, 2005 at 03:52:38PM +0300, Mindaugas Riauba wrote: > Tables are not big (2-15 rows each) but have very high > turnover rate - 100+ updates/inserts/deletes/selects per second. > So contents of database changes very fast. Problem is that when > pg_autovacuum does vacuum those

Re: [PERFORM] PostgreSQL strugling during high load

2005-05-13 Thread Steinar H. Gunderson
On Fri, May 13, 2005 at 05:45:45PM +0300, Mindaugas Riauba wrote: > But there's no checkpoint warnings in serverlog. And btw we are running > with fsync=off (yes I know the consequences). Just a note here; since you have battery-backed hardware cache, you probably won't notice that much of a slo

Re: [PERFORM] Is there any other way to do this?

2005-05-17 Thread Steinar H. Gunderson
On Tue, May 17, 2005 at 06:58:20PM -0400, Wei Weng wrote: > This time it worked! But VACUUM FULL requires an exclusive lock on the > table which I don't really want to grant. So my question is: why is VACUUM > ANALYZE didn't do the job? Is there any setting I can tweak to make a > VACUUM without

Re: [PERFORM] Optimizing for writes. Data integrity not critical

2005-05-19 Thread Steinar H. Gunderson
On Thu, May 19, 2005 at 05:21:07PM -0500, Steve Bergman wrote: > I'm doing the writes individually. Is there a better way? Combining > them all into a transaction or something? Batching them all in one or a few transactions will speed it up a _lot_. Using COPY would help a bit more on top of th

Re: [PERFORM] Fwd: Index on table when using DESC clause

2005-05-23 Thread Steinar H. Gunderson
On Mon, May 23, 2005 at 07:41:19PM +0200, Yves Vindevogel wrote: > However, when I query my db using for instance order by pages, > documentname, it is very fast. > If I use order by pages desc, documentname, it is not fast at > all, like it is not using the index properly at all. Make an

Re: [PERFORM] Need help to decide Mysql vs Postgres

2005-05-24 Thread Steinar H. Gunderson
On Tue, May 24, 2005 at 01:56:54PM -0400, Amit V Shah wrote: > I took a look at this. I have a few concerns with bizgres though -- I am > using jetspeed portal engine and Hibernate as my O/R Mapping layer. If you have problems with performance, you might want to look into using JDBC directly inste

Re: [PERFORM] sequential scan performance

2005-05-29 Thread Steinar H. Gunderson
On Sun, May 29, 2005 at 08:27:26AM -0500, Michael Engelhart wrote: > this is a query that our system needs to do a LOT. Is there any way > to improve the performance on this either with changes to our query > or by configuring the database deployment? We have an index on > city_name but w

Re: [PERFORM] Postgresql and xeon.

2005-05-30 Thread Steinar H. Gunderson
On Mon, May 30, 2005 at 09:19:40AM -0700, Josh Berkus wrote: > Search the archives of this list. This has been discussed ad nauseum. > www.pgsql.ru I must admit I still haven't really understood it -- I know that it appears on multiple operating systems, on multiple architectures, but most with

Re: [PERFORM] postgresql-8.0.1 performance tuning

2005-06-01 Thread Steinar H. Gunderson
On Wed, Jun 01, 2005 at 07:30:37AM +0200, Cosimo Streppone wrote: >>fsync = true > false Just setting fsync=false without considering the implications is a _bad_ idea... /* Steinar */ -- Homepage: http://www.sesse.net/ ---(end of broadcast)-

Re: [PERFORM] slow growing table

2005-06-06 Thread Steinar H. Gunderson
On Mon, Jun 06, 2005 at 09:48:26AM -0700, Jone C wrote: > When the table is new it's very fast, towards the end of the month > it's taking almost 10 times longer, yet I'm deleting and COPYing in > the same amount of data. Other operations on this table slow down, > too, that were fast before using

Re: [PERFORM] slow growing table

2005-06-06 Thread Steinar H. Gunderson
On Mon, Jun 06, 2005 at 07:00:37PM +0200, Steinar H. Gunderson wrote: > You might have a problem with index bloat. Could you try REINDEXing the > indexes on the table and see if that makes a difference? On second thought... Does a VACUUM FULL help? If so, you might want to increase yo

Re: [PERFORM] faster search

2005-06-10 Thread Steinar H. Gunderson
On Fri, Jun 10, 2005 at 01:45:05PM -0400, Clark Slater wrote: > Indexes: > "test_id" btree (id) > "test_plid" btree (productlistid) > "test_typeid" btree (typeid) > "test_plidtypeid" btree (productlistid, typeid) > > > explain analyze select * from test where productlistid=3 and typeid=9 > order

Re: [PERFORM] autovacuum suggestions for 500,000,000+ row tables?

2005-06-21 Thread Steinar H. Gunderson
On Tue, Jun 21, 2005 at 11:08:43PM +0100, Alex Stapleton wrote: > Bloody Debian stable. I might have to experiment with building from > source or using alien on debian to convert the rpms. Fun. Oh well. Or just pull in postgresql-8.0 from unstable; sid is close enough to sarge for it to work qui

Re: [PERFORM] plain inserts and deletes very slow

2005-07-03 Thread Steinar H. Gunderson
On Mon, Jul 04, 2005 at 12:45:37AM +0200, Enrico Weigelt wrote: > my application reads and writes some table quite often > (multiple times per second). these tables are quite small > (not more than 20 tuples), but the operations take quite a > long time (>300 ms!). Are you VACUUMing often enough?

Re: [PERFORM] Surprizing performances for Postgres on Centrino

2005-07-07 Thread Steinar H. Gunderson
On Thu, Jul 07, 2005 at 03:48:06PM +0200, Dawid Kuroczko wrote: > This is why AMD stopped giving GHz ratings and instead uses numbers > which indicate how their processor relate to Pentium 4s. For instance > AMD Athlon XP 1700+ is running at 1.45 GHz, but competes with > Pentium 4 1.7 GHz. Actual

Re: [PERFORM] Looking for tips

2005-07-19 Thread Steinar H. Gunderson
On Tue, Jul 19, 2005 at 03:01:00PM -0400, Tom Lane wrote: > You could possibly get some improvement if you can re-use prepared plans > for the queries; but this will require some fooling with the client code > (I'm not sure if DBD::Pg even has support for it at all). Newer versions has, when compi

Re: [PERFORM] Looking for tips

2005-07-19 Thread Steinar H. Gunderson
On Tue, Jul 19, 2005 at 03:16:31PM -0400, Tom Lane wrote: > Ah, but are they really prepared, or is DBD::Pg faking it by inserting > parameter values into the query text and then sending the assembled > string as a fresh query? They are really prepared. /* Steinar */ -- Homepage: http://www.se

Re: [PERFORM] Why hash join instead of nested loop?

2005-08-08 Thread Steinar H. Gunderson
On Mon, Aug 08, 2005 at 08:58:26PM -0400, Tom Lane wrote: > Hmph. There is something really strange going on here. I tried to > duplicate your problem in 7.4.*, thus: PostgreSQL 7.4.7 (Debian sarge): regression=# explain analyze select rtmessagestate.* from rtmessagestate,connection where (c

Re: [PERFORM] Performance pb vs SQLServer.

2005-08-14 Thread Steinar H. Gunderson
On Sun, Aug 14, 2005 at 07:27:38PM -0500, John Arbash Meinel wrote: > My guess is that this is part of a larger query. There isn't really much > you can do. If you want all 3.2M rows, then you have to wait for them to > be pulled in. To me, it looks like he'll get 88 rows, not 3.2M. Surely we must

Re: [PERFORM] Performance pb vs SQLServer.

2005-08-14 Thread Steinar H. Gunderson
On Sun, Aug 14, 2005 at 07:27:38PM -0500, John Arbash Meinel wrote: > If you are just trying to determine what the unique entries are for cod, > you probably are better off doing some normalization, and keeping a > separate table of cod values. Pah, I missed this part of the e-mail -- you can igno

Re: [PERFORM] Performance pb vs SQLServer.

2005-08-14 Thread Steinar H. Gunderson
On Sun, Aug 14, 2005 at 09:18:45PM -0400, Tom Lane wrote: > Not really. There's been some speculation about implementing index > "skip search" --- once you've verified there's at least one visible > row of a given index value, tell the index to skip to the next different > value instead of handing

Re: [PERFORM] index as large as table

2005-08-20 Thread Steinar H. Gunderson
On Sat, Aug 20, 2005 at 11:08:13PM +1000, Gavin Sherry wrote: > Of course. The idea is that, generally speaking, you're only interested in > a small portion of the data stored in the table. Indexes store extra data > so that they can locate the portion you're interested in faster. I think his ques

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-27 Thread Steinar H. Gunderson
On Fri, Aug 26, 2005 at 07:31:51PM -0400, Tom Lane wrote: > Or you could just play with the order of the filter conditions ... for > example, the date condition at the end is probably far cheaper to test > than the text comparisons, so if that's fairly selective it'd be worth > putting it first. T

Re: [PERFORM] Weird performance drop after VACUUM

2005-08-27 Thread Steinar H. Gunderson
On Sat, Aug 27, 2005 at 11:05:01AM -0400, Tom Lane wrote: > It could, but it doesn't really have enough information. We don't > currently have any model that some operators are more expensive than > others. IIRC the only sort of reordering the current code will do > in a filter condition list is

Re: [PERFORM] Massive performance issues

2005-09-01 Thread Steinar H. Gunderson
On Thu, Sep 01, 2005 at 10:13:59PM +0100, Matthew Sackman wrote: > Well that's the thing - on the queries where it decides to use the index > it only reads at around 3MB/s and the CPU is maxed out, whereas when it > doesn't use the index, the disk is being read at 60MB/s. So when it > decides to us

Re: [PERFORM] Massive performance issues

2005-09-01 Thread Steinar H. Gunderson
On Thu, Sep 01, 2005 at 06:42:31PM +0100, Matthew Sackman wrote: > flat_extra | character varying(100) | not null > number | character varying(100) | not null > street | character varying(100) | not null > locality_1 | character varying(100) | not

Re: [PERFORM] Query take 101 minutes, help, please

2005-09-07 Thread Steinar H. Gunderson
On Wed, Sep 07, 2005 at 12:22:27PM -0400, Christian Compagnon wrote: > I'm a newbie in postgresql, I've installed it on a Windows XP machine > ( I can't use linux, it's a company machine ), I'm courious why this > query takes so long It sounds like you've set work_mem too low; increasing it might

Re: [PERFORM] Advice on RAID card

2005-09-25 Thread Steinar H. Gunderson
On Sun, Sep 25, 2005 at 06:53:57PM +0200, PFC wrote: > Gonna investigate now if Linux software RAID5 is rugged enough. Can > always buy the a card later if not. Note that 2.6.13 and 2.6.14 have several improvements to the software RAID code, some with regard to ruggedness. You might want t

Re: [PERFORM] Comparative performance

2005-09-29 Thread Steinar H. Gunderson
On Thu, Sep 29, 2005 at 08:16:11AM -0400, Joe wrote: > I just tried using pg_pconnect() and I didn't notice any significant > improvement. PHP persistent connections are not really persistent -- or so I've been told. Anyhow, what was discussed here was pg_query, not pg_connect. You really want t

Re: [PERFORM] Is There Any Way ....

2005-09-29 Thread Steinar H. Gunderson
On Thu, Sep 29, 2005 at 07:21:08PM -0400, Lane Van Ingen wrote: > (1) Make a table memory-resident only ? You might want to look into memcached, but it's impossible to say whether it will fit your needs or not without more details. /* Steinar */ -- Homepage: http://www.sesse.net/

[PERFORM] Ultra-cheap NVRAM device

2005-10-03 Thread Steinar H. Gunderson
I thought this might be interesting, not the least due to the extremely low price ($150 + the price of regular DIMMs): http://www.tomshardware.com/storage/20050907/index.html Anybody know a good reason why you can't put a WAL on this, and enjoy a hefty speed boost for a fraction of the price of

Re: [PERFORM] Text/Varchar performance...

2005-10-05 Thread Steinar H. Gunderson
On Wed, Oct 05, 2005 at 12:21:35PM -0600, Cristian Prieto wrote: > Hello, just a little question, It's preferable to use Text Fields or > varchar(255) fields in a table? Are there any performance differences in the > use of any of them? They are essentially the same. Note that you can have varchar

Re: [HACKERS] [PERFORM] A Better External Sort?

2005-10-05 Thread Steinar H. Gunderson
On Wed, Oct 05, 2005 at 04:55:51PM -0700, Luke Lonergan wrote: > In COPY, we found lots of libc functions like strlen() being called > ridiculous numbers of times, in one case it was called on every > timestamp/date attribute to get the length of TZ, which is constant. That > one function call was

Re: [PERFORM] count(*) using index scan in "query often, update rarely" environment

2005-10-07 Thread Steinar H. Gunderson
On Fri, Oct 07, 2005 at 11:24:05AM +0200, Cestmir Hybl wrote: > Isn't it possible (and reasonable) for these environments to keep track of > whether there is a transaction in progress with update to given table and > if not, use an index scan (count(*) where) or cached value (count(*)) to > perform

Re: [PERFORM] Text/Varchar performance...

2005-10-10 Thread Steinar H. Gunderson
On Mon, Oct 10, 2005 at 06:28:23PM +0700, Ahmad Fajar wrote: > than you can index the field and you can gain better > perfomance in searching base on the fields, because the search uses the > index you have been created. That really depends on the queries. An index will help some queries (notably

Re: [PERFORM] Massive delete performance

2005-10-11 Thread Steinar H. Gunderson
On Tue, Oct 11, 2005 at 10:47:03AM +0300, Andy wrote: > So, I have a replication only with the tables that I need to send, then I > make a copy of this replication, and from this copy I delete all the data's > that are not needed. > > How can I increase this DELETE procedure because it is really

[PERFORM] Materializing a sequential scan

2005-10-19 Thread Steinar H. Gunderson
Hi, I'm using PostgreSQL 8.1 beta 3 (packages from Debian experimental), and I have a (rather complex) query that seems to take forever -- when the database was just installed, it took about 1200ms (which is quite good, considering that the 7.4 system this runs on today uses about the same time, b

Re: [PERFORM] Materializing a sequential scan

2005-10-20 Thread Steinar H. Gunderson
On Thu, Oct 20, 2005 at 12:37:25AM -0400, Tom Lane wrote: > That mdb_gruppekobling_transitiv_tillukning function looks awfully > grotty ... how many rows does it return, and how long does it take to > run by itself? How often does its temp table get vacuumed? A quick > band-aid might be to use TR

Re: [PERFORM] Materializing a sequential scan

2005-10-20 Thread Steinar H. Gunderson
On Thu, Oct 20, 2005 at 12:58:51AM -0400, Tom Lane wrote: > As-is, it's not doing anything for you ... certainly not enforcing > that the undergruppe_id be aktiv. Oops, yes, that's a bug -- thanks for noticing. (It does not matter particularily with the current data set, though.) /* Steinar */ --

Re: [PERFORM] What gets cached?

2005-10-21 Thread Steinar H. Gunderson
On Fri, Oct 21, 2005 at 07:34:30AM -0500, Martin Nickel wrote: > Let's say I do the same thing in Postgres. I'm likely to have my very > fastest performance for the first few queries until memory gets filled up. > The only time Postgres seems to take advantage of cached data is when I > repeat t

Re: [PERFORM] Need help in setting optimal configuration for a huge database.

2005-10-22 Thread Steinar H. Gunderson
On Sun, Oct 23, 2005 at 02:45:25AM +0530, Kishore B wrote: > Database *:* Postgresql 7.3 You definitely want to upgrade this if you can. > Memory : 2 GB For 2GB of RAM, your effective_cache_size (10) is a bit low (try doubling it), and sort_mem (2048) is probably a bit too low as well. /* S

Re: [PERFORM] Need help in setting optimal configuration for a huge database.

2005-10-22 Thread Steinar H. Gunderson
[please send replies to the list, not to me directly] On Sun, Oct 23, 2005 at 03:19:39AM +0530, Kishore B wrote: > *You definitely want to upgrade this if you can. > > > Memory : 2 GB > * > We can move upto 12 GB if need to be. I was referring to your PostgreSQL version, not your RAM. More RAM

Re: [PERFORM] Materializing a sequential scan

2005-10-23 Thread Steinar H. Gunderson
On Thu, Oct 20, 2005 at 12:37:25AM -0400, Tom Lane wrote: > That mdb_gruppekobling_transitiv_tillukning function looks awfully > grotty ... how many rows does it return, and how long does it take to > run by itself? How often does its temp table get vacuumed? A quick > band-aid might be to use TR

Re: [PERFORM] Need help in setting optimal configuration for a huge

2005-10-23 Thread Steinar H. Gunderson
On Sun, Oct 23, 2005 at 09:31:44AM -0700, Craig A. James wrote: > COUNT() -- There is no good substitute. What I do is create a new column, > "ROW_NUM" with an auto-incrementing sequence. Every time I insert a row, > it gets a new value. Unfortunately, this doesn't work if you ever delete a >

Re: [PERFORM] Problem analyzing explain analyze output

2005-10-23 Thread Steinar H. Gunderson
On Mon, Oct 24, 2005 at 01:53:59AM +0200, Guillaume Smet wrote: > I don't understand why I have the Nested Loop at line 19 with an actual > time of 254.292..257.328 because I can't find anywhere the line taking > this 254 ms. You don't have a nested loop with that time; however, you have > ->

Re: [PERFORM] What gets cached?

2005-10-24 Thread Steinar H. Gunderson
On Mon, Oct 24, 2005 at 11:09:55AM -0400, Alex Turner wrote: > Just to play devils advocate here for as second, but if we have an algorithm > that is substational better than just plain old LRU, which is what I believe > the kernel is going to use to cache pages (I'm no kernel hacker), then why > d

Re: [PERFORM] zero performance on query

2005-10-26 Thread Steinar H. Gunderson
On Tue, Oct 25, 2005 at 10:26:43PM -0600, Sidar López Cruz wrote: > look at this: > select count(*) from fotos where archivo not in (select archivo from > archivos) > Aggregate (cost=4899037992.36..4899037992.37 rows=1 width=0) > -> Seq Scan on fotos (cost=22598.78..4899037338.07 rows=261716 wi

Re: [PERFORM] zero performance on query

2005-10-26 Thread Steinar H. Gunderson
On Wed, Oct 26, 2005 at 08:05:21AM -0400, Merlin Moncure wrote: > select count(*) from fotos f where not exists (select archivo from archivos a > where a.archivo = f.archivo) This was an optimization before 7.4, but probably isn't anymore. /* Steinar */ -- Homepage: http://www.sesse.net/

Re: [PERFORM] Materializing a sequential scan

2005-10-26 Thread Steinar H. Gunderson
Hi, I finally found what I believe is the root cause for the hopeless performance, after a lot of query rewriting: > Subquery Scan mdb_effektiv_tilgang (cost=19821.69..4920621.69 rows=1 > width=48) >Filter: ((NOT (hashed subplan)) AND (NOT (subplan))) The problem here is simply that 8

Re: [PERFORM] Materializing a sequential scan

2005-10-26 Thread Steinar H. Gunderson
On Wed, Oct 26, 2005 at 07:06:15PM -0400, Tom Lane wrote: > AFAICS, subplan_is_hashable() is testing the same conditions in 7.4 and > HEAD, so this isn't clear. Want to step through it and see where it's > deciding not to hash? Line 639, ie.: 635 if (!optup->oprcanhash || optup->

Re: [PERFORM] Materializing a sequential scan

2005-10-26 Thread Steinar H. Gunderson
On Wed, Oct 26, 2005 at 07:53:02PM -0400, Tom Lane wrote: > I don't think you're getting a correct reading for optup, but OID > 2373 is timestamp = date: > > [...] > > My recollection is that there was no such operator in 7.4; probably in > 7.4 the IN ended up using timestamp = timestamp which is

Re: [PERFORM] Materializing a sequential scan

2005-10-26 Thread Steinar H. Gunderson
On Wed, Oct 26, 2005 at 08:51:03PM -0400, Tom Lane wrote: > I have some ideas in the back of my head about supporting > cross-data-type hashing. Essentially this would require that the hash > functions for two types be compatible in that they generate the same > hash value for two values that woul

Re: [PERFORM] Simple query: how to optimize

2005-10-28 Thread Steinar H. Gunderson
On Fri, Oct 28, 2005 at 03:40:40PM -0700, Roger Hand wrote: > You're first joining against the entire user table, then filtering out the > users > you don't need. That's just wrong, sorry -- the planner is perfectly able to push the WHERE down before the join. I'd guess the problem is the age()

Re: [PERFORM] multi-layered view join performance oddities

2005-10-30 Thread Steinar H. Gunderson
On Sun, Oct 30, 2005 at 06:16:04PM +0100, Svenne Krap wrote: > Nested Loop (cost=223.09..338.61 rows=1 width=174) (actual > time=20.213..721.361 rows=2250 loops=1) >Join Filter: (("outer".dataset_id = "inner".dataset_id) AND > ("outer".nb_property_type_id = "inner".nb_property_type_id)) >

Re: [PERFORM] 8.1beta3 performance

2005-10-31 Thread Steinar H. Gunderson
On Mon, Oct 31, 2005 at 05:16:46PM -0600, PostgreSQL wrote: > We're running 8.1beta3 on one server and are having ridiculous performance > issues. This is a 2 cpu Opteron box and both processors are staying at 98 > or 99% utilization processing not-that-complex queries. Prior to the > upgrade,

Re: [PERFORM] Joining views disables indexes?

2005-11-01 Thread Steinar H. Gunderson
On Tue, Nov 01, 2005 at 06:16:59PM -0500, Mitch Pirtle wrote: > I have a client that is testing an internal data platform, and they > were happy with PostgreSQL until they tried to join views - at that > time they discovered PostgreSQL was not using the indexes, and the > queries took 24 hours to e

Re: [PERFORM] Improving performance on multicolumn query

2005-11-09 Thread Steinar H. Gunderson
On Wed, Nov 09, 2005 at 01:08:07PM +0100, Jan Kesten wrote: > First, I'm using postgresql 7.4.7 on a 2GHz machine having 1.5GByte RAM > and I have a table with about 220 columns and 2 rows - and the first > five columns build a primary key (and a unique index). I forgot this, but it should be

Re: [PERFORM] Improving performance on multicolumn query

2005-11-09 Thread Steinar H. Gunderson
On Wed, Nov 09, 2005 at 01:08:07PM +0100, Jan Kesten wrote: > Now my problem: I need really many queries of rows using it's primary > key and fetching about five different columns but these are quite slow > (about 10 queries per second and as I have some other databases which > can have about 300 q

[PERFORM] WAL sync behaviour

2005-11-10 Thread Steinar H. Gunderson
Hi, We're having problems with our PostgreSQL server using forever for simple queries, even when there's little load -- or rather, the transactions seem to take forever to commit. We're using 8.1 (yay!) on a single Opteron, with WAL on the system two-disk (software) RAID-1, separate from the datab

Re: [PERFORM] WAL sync behaviour

2005-11-10 Thread Steinar H. Gunderson
On Thu, Nov 10, 2005 at 02:14:30PM +, Richard Huxton wrote: > You're beyond my area of expertise, but I do know that someone's going > to ask what filesystem this is (ext2/xfs/etc). Ah, yes, I forgot -- it's ext3. We're considering simply moving the WAL onto a separate partition (with data=wr

Re: [PERFORM] WAL sync behaviour

2005-11-10 Thread Steinar H. Gunderson
On Thu, Nov 10, 2005 at 12:44:15PM -0500, Tom Lane wrote: > Don't think so ... want to write something up? Hard part is to > figure out where to put it ... To be honest, I think we could use a "newbie's guide to PostgreSQL performance tuning". I've seen rather good guides for query tuning, and gu

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-16 Thread Steinar H. Gunderson
On Wed, Nov 16, 2005 at 11:06:25AM -0600, Scott Marlowe wrote: > There was a big commercial EMC style array in the hosting center at the > same place that had something like a 16 wide by 16 tall array of IDE > drives for storing pdf / tiff stuff on it, and we had at least one > failure a month in i

Re: [PERFORM] Help speeding up delete

2005-11-17 Thread Steinar H. Gunderson
On Thu, Nov 17, 2005 at 09:40:42AM +0800, Christopher Kings-Lynne wrote: > In my experience not many pgsql admins have test servers or the skills > to build up test machines with the latest pg_dump, etc. (Seriously.) > In fact, few realise at all that they should use the 8.1 dumper. Isn't your

Re: [PERFORM] Help speeding up delete

2005-11-17 Thread Steinar H. Gunderson
On Thu, Nov 17, 2005 at 11:07:42PM +0800, Christopher Kings-Lynne wrote: >> Isn't your distribution supposed to do this for you? Mine does these >> days... > A distribution that tries to automatically do a major postgresql update > is doomed to fail - spectacularly... Automatically? Well, you ca

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-20 Thread Steinar H. Gunderson
On Sat, Nov 19, 2005 at 08:13:09AM -0800, Luke Lonergan wrote: > Iowait is time spent waiting on blocking io calls. To be picky, iowait is time spent in the idle task while the I/O queue is not empty. It does not matter if the I/O is blocking or not (from userspace's point of view), and if the I/

Re: [PERFORM] Hardware/OS recommendations for large databases (

2005-11-20 Thread Steinar H. Gunderson
On Sun, Nov 20, 2005 at 09:22:41AM -0500, Greg Stark wrote: > I don't think that's true. If the syscall was preemptable then it wouldn't > show up under "iowait", but rather "idle". The time spent in iowait is time in > uninterruptable sleeps where no other process can be scheduled. You are confus

Re: [PERFORM] 8.1 count(*) distinct: IndexScan/SeqScan

2005-11-25 Thread Steinar H. Gunderson
On Thu, Nov 24, 2005 at 09:15:44PM -0600, Kyle Cordes wrote: > I have hit cases where I have a query for which there is a somewhat > "obvious" (to a human...) query plan that should make it possible to get > a query answer pretty quickly. Yet the query "never" finishes (or > rather, after hours

Re: [PERFORM] Open request for benchmarking input

2005-11-27 Thread Steinar H. Gunderson
On Sat, Nov 26, 2005 at 01:57:47PM -0500, Qingqing Zhou wrote: > Don't forget TPCC (data > memory, with intensive updates). So the benchmarks > in my mind include TPCC, TPCH and TPCW. I'm lost in all those acronyms, but am I right in assuming that none of these actually push the planner very hard

Re: [PERFORM] BLCKSZ

2005-12-06 Thread Steinar H. Gunderson
On Tue, Dec 06, 2005 at 01:40:47PM +0300, Olleg wrote: > I can't undestand why "bigger is better". For instance in search by > index. Index point to page and I need load page to get one row. Thus I > load 8kb from disk for every raw. And keep it then in cache. You > recommend 64kb. With your rec

Re: [PERFORM] Context switching and Xeon processors

2005-12-06 Thread Steinar H. Gunderson
On Tue, Dec 06, 2005 at 03:01:02PM -0600, Brandon Metcalf wrote: > We're running a dual Xeon machine with hyperthreading enabled and > PostgreSQL 8.0.3. The two single most important things that will help you with high rates of context switching: - Turn off hyperthreading. - Upgrade to 8.1.

Re: [PERFORM] 7.4.7 vs. 8.1

2005-12-12 Thread Steinar H. Gunderson
On Mon, Dec 12, 2005 at 09:54:30AM +0100, Szabolcs BALLA wrote: > When I use autovacuum (8.1) is it required to use "vacuum analyze" for > maintenance or autovacuum is enough? autovacuum should be enough. > We have 2 processors (hyperthread) and is it needed to configure the > psql to use it or

Re: [PERFORM] query from partitions

2005-12-13 Thread Steinar H. Gunderson
On Tue, Dec 13, 2005 at 06:18:19PM +0300, Ключников А.С. wrote: > select * from base > where id in (select id from device where id = 1 or id = 2) and > datatime between '2005-05-15' and '2005-05-17'; > 10 minits That's a really odd way of saying "1 or 2". It probably has to go through

Re: [PERFORM] Simple Join

2005-12-14 Thread Steinar H. Gunderson
On Wed, Dec 14, 2005 at 04:03:52PM -0600, Kevin Brown wrote: > -> Index Scan using paid_index on ordered_products > (cost=0.00..4954.79 rows=21759 width=16) (actual time=136.472..5966.275 > rows=18042 loops=1) >Index Cond: (paid = true) >Filter: (paid AN

Re: [PERFORM] PostgreSQL performance question. [OT]

2005-12-17 Thread Steinar H. Gunderson
On Sun, Dec 18, 2005 at 02:11:16AM +, Harry Jackson wrote: > The one thing that may be skewing these results is that this was > compiled and installed from source with > > ./configure CFLAGS='-O2' --with-openssl --enable-thread-safety > > I am not sure what the default Debian binary for 7.4.7

Re: [PERFORM] Speed of different procedural language

2005-12-21 Thread Steinar H. Gunderson
On Sun, Dec 18, 2005 at 01:10:21AM -, Ben Trewern wrote: > I know I should be writing these in C but that's a bit beyond me. I was > going to try PL/Python or PL/Perl or even PL/Ruby. Has anyone any idea > which language is fastest, or is the data access going to swamp the overhead > of sm

Re: [PERFORM] Speed of different procedural language

2005-12-21 Thread Steinar H. Gunderson
On Wed, Dec 21, 2005 at 02:24:42PM -0700, Michael Fuhr wrote: > The difference is clear only in specific cases; just because you > saw a 10x increase in some cases doesn't mean you can expect that > kind of increase, or indeed any increase, in others. I've seen > PL/pgSQL beat all other PL/* challe

Re: [PERFORM] Speed of different procedural language

2005-12-21 Thread Steinar H. Gunderson
On Wed, Dec 21, 2005 at 03:10:28PM -0700, Michael Fuhr wrote: >> That's funny, my biggest problems with PL/PgSQL have been (among others) >> exactly with large result sets... > Out of curiosity, do you have a simple test case? I'd be interested > in seeing what you're doing in PL/pgSQL that's cont

Re: [PERFORM] What's the best hardver for PostgreSQL 8.1?

2005-12-25 Thread Steinar H. Gunderson
On Sun, Dec 25, 2005 at 04:13:57AM -0800, David Lang wrote: > Thanks for the clarification, I knew that PATA didn't do hotswap, and I've > seen discussions on the linux-kernel list about SATA hotswap being worked > on, but I thought that scsi handled it. how recent a kernel have you had > proble

Re: [PERFORM] How import big amounts of data?

2005-12-29 Thread Steinar H. Gunderson
On Thu, Dec 29, 2005 at 10:48:26AM +0100, Arnau wrote: > Which is the best way to import data to tables? I have to import > 9 rows into a column and doing it as inserts takes ages. Would be > faster with copy? is there any other alternative to insert/copy? There are multiple reasons why yo

Re: [PERFORM] improving write performance for logging application

2006-01-03 Thread Steinar H. Gunderson
On Tue, Jan 03, 2006 at 04:44:28PM -0700, Steve Eckmann wrote: > Are there general guidelines for tuning the PostgreSQL server for this kind > of application? The suggestions I've found include disabling fsync (done), Are you sure you really want this? The results could be catastrophic in case of

Re: [PERFORM] [PERFORMANCE] Beetwen text and varchar field

2006-01-09 Thread Steinar H. Gunderson
On Mon, Jan 09, 2006 at 11:58:19AM +0100, TNO wrote: > what is the best for a char field with less than 1000 characters? > a text field or a varchar(1000) They will be equivalent. text and varchar are the same type internally -- the only differences are that varchar can have a length (but does not

Re: [PERFORM] When to vacuum a table?

2006-11-26 Thread Steinar H. Gunderson
On Sun, Nov 26, 2006 at 09:43:11AM -0300, Marcelo Costa wrote: > All its tables constantly manipulated (INSERT, UPDATE, DELETE) they need a > VACUUM Just a minor clarification here: INSERT does not create dead rows, only UPDATE and DELETE do. Thus, if you only insert rows, you do not need to vacuu

Re: [PERFORM] Performance of ORDER BY

2006-12-05 Thread Steinar H. Gunderson
On Tue, Dec 05, 2006 at 01:02:06PM -0500, Tom Lane wrote: > In 8.0 that might be counterproductively high --- we have seen cases > where more sort_mem = slower with the older sorting code. I concur > with Luke's advice that you should update to 8.2 (not 8.1) to get the > improved sorting code. By

Re: [PERFORM] Locking in PostgreSQL?

2006-12-06 Thread Steinar H. Gunderson
On Wed, Dec 06, 2006 at 08:26:45AM -0500, Dave Cramer wrote: > Apparently I've completely misunderstood MVCC then My > understanding is that unless you do a select ... for update then > update the rows will not be locked . The discussion was about updates, not selects. Selects do not in ge

Re: [PERFORM] File Systems Compared

2006-12-06 Thread Steinar H. Gunderson
On Wed, Dec 06, 2006 at 05:31:01PM +0100, Markus Schiltknecht wrote: >> Care to post these numbers *without* word wrapping? Thanks. > How is one supposed to do that? Care giving an example? This is a rather long sentence without any kind of word wrapping except what would be imposed on your own s

[PERFORM] [offtopic] Word wrapping

2006-12-06 Thread Steinar H. Gunderson
On Wed, Dec 06, 2006 at 06:45:56PM +0100, Markus Schiltknecht wrote: > Cool, thank you for the example :-) I thought the MTA or at least the the > mailing list would wrap mails at some limit. I've now set word-wrap to > characters (it seems not possible to turn it off completely in > thund

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 09:05:56AM -0200, Daniel van Ham Colchete wrote: > But, trust me on this one. It's worth it. You know what? I don't. > Think of this: PostgreSQL and GNU LibC use a lot of complex algorithms: > btree, hashes, checksums, strings functions, etc... And you have a lot of > ways

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:09:13AM -0200, Daniel van Ham Colchete wrote: >> You know what? I don't. > So test it yourself. You're making the claims, you're supposed to be proving them... > As I said, it is an example. Take floatpoint divisions. You have > plenty of ways of doing it: 387, MMX, SSE

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:17:06AM -0200, Daniel van Ham Colchete wrote: > I just remebered one case with MySQL. When I changed the distro from > Conectiva 10 (rpm-based ended brazilian distro) to Gentoo, a MySQL > operation that usually took 2 minutes to run, ended in 47 seconds. How do you know

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-11 Thread Steinar H. Gunderson
On Mon, Dec 11, 2006 at 11:31:48AM -0200, Daniel van Ham Colchete wrote: > What PostgreSQL benchmark software should I use??? Look up the list archives; search for "TPC". > I'll test PostgreSQL 8.1 on a Fedora Core 6 and on a Gentoo. I'll get > the same version FC6 uses and install it at my Gento

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-12 Thread Steinar H. Gunderson
On Tue, Dec 12, 2006 at 01:42:06PM +0100, Cosimo Streppone wrote: > "-O0" ~ 957 tps > "-O1 -mcpu=pentium4 -mtune=pentium4" ~ 1186 tps > "-O2 -mcpu=pentium4 -mtune=pentium4" ~ 1229 tps > "-O3 -mcpu=pentium4 -mtune=pentium4" ~ 1257 tps > "-O6 -mcpu=pentium4 -mtune=pentium4" ~ 1254 tps For the record

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-15 Thread Steinar H. Gunderson
On Fri, Dec 15, 2006 at 10:53:25AM +0100, Alexander Staubo wrote: > The difference is very slight. I'm going to run without -funroll- > loops and -pipe (which are not arch-related) to get better data. -pipe does not matter for the generated code; it only affects compiler speed. (It simply means t

Re: [PERFORM] New to PostgreSQL, performance considerations

2006-12-16 Thread Steinar H. Gunderson
On Sat, Dec 16, 2006 at 10:53:21AM -0500, Ron wrote: > AFAICT, no one has stated there would be a "blow-your-socks-off > dramatic performance improvement" for pg due to compilation > options. Just that there might be some, and there might be some that > are arch specific. FWIW, the original cl

Re: [PERFORM] Scaling concerns

2006-12-16 Thread Steinar H. Gunderson
On Sat, Dec 16, 2006 at 11:26:02AM -0600, tsuraan wrote: > Even an operation like "select count(*) from messages" can take minutes, > with a totally idle system. Postgres seems to be the most scalable Free > database out there, so I must be doing something wrong. Unqualified SELECT COUNT(*) FROM

Re: [PERFORM] GROUP BY vs DISTINCT

2006-12-20 Thread Steinar H. Gunderson
On Tue, Dec 19, 2006 at 11:19:39PM -0800, Brian Herlihy wrote: > Actually, I think I answered my own question already. But I want to > confirm - Is the GROUP BY faster because it doesn't have to sort results, > whereas DISTINCT must produce sorted results? This wasn't clear to me from > the docum

  1   2   3   4   >