Re: [PERFORM] [ADMIN] Benchmarking postgres on Solaris/Linux

2004-03-23 Thread matt
> Personally, I've been unimpressed by Dell/Xeon; I think the Sun might do > better than you think, comparitively.On all the Dell servers I've used > so > far, I've not seen performance that comes even close to the hardware > specs. It's true that any difference will be far less than the GHz r

Re: [PERFORM] [ADMIN] Benchmarking postgres on Solaris/Linux

2004-03-23 Thread matt
> Indeed, if our Suns actually diabled the broken hardware when they > died, fell over, and rebooted themselves, I'd certainly praise them > to heaven. But I have to say that the really very good reporting of > failing memory has saved me some headaches. Ha! Yes, it would seem the obvious thing

Re: [PERFORM] RAID or manual split?

2004-06-03 Thread matt
> It seems, that if I know the type and frequency of the queries a > database will be seeing, I could split the database by hand over > multiple disks and get better performance that I would with a RAID array > with similar hardware. Unlikely, but possible if you had radically different hardware f

Re: [PERFORM] Performance Bottleneck

2004-08-10 Thread matt
> Squid also takes away the work of doing SSL (presuming you're running it > on a different machine). Unfortunately it doesn't support HTTP/1.1 which > means that most generated pages (those that don't set Content-length) end > up forcing squid to close and then reopen the connection to the web >

Re: [PERFORM] PostgreSQL performance problem -> tuning

2003-08-14 Thread matt
> Mem: 71M Active, 23M Inact, 72M Wired, 436K Cache, 48M Buf, 208M Free > Swap: 368M Total, 2852K Used, 366M Free > > It's right that I can figure that I can use 384M (total RAM) - 72M > (wired) - 48M (buf) = 264M for PostgreSQL. > Hence, if I set effective_cache_size to 24M (3072 8K

[PERFORM] Hardware recommendations to scale to silly load

2003-08-27 Thread matt
ffer any pointers, anecdotes, etc? Any theoretical musings also more than welcome. Comments upon my sanity will be referred to my doctor. If the best price/performance option is a second hand 32-cpu Alpha running VMS I'd be happy to go that way ;-) Many thanks for reading this far. Matt

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> You probably, more than anything, should look at some kind of > superfast, external storage array Yeah, I think that's going to be a given. Low end EMC FibreChannel boxes can do around 20,000 IOs/sec, which is probably close to good enough. You mentioned using multiple RAID controllers as a b

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> Don't know how "cheap" they are. > > I have an app that does large batch updates. I found that if I dropped > the indexes, did the updates and recreated the indexes, it was faster > than doing the updates while the indexes were intact. Yeah, unfortunately it's not batch work, but real time fina

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
ew percent here, a few percent there, pretty soon you're talking serious improvements... Thanks Matt ---(end of broadcast)--- TIP 3: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly

Re: [PERFORM] Hardware recommendations to scale to silly load

2003-08-28 Thread matt
> Are you *sure* about that 3K updates/inserts per second xlates > to 10,800,000 per hour. That, my friend, is a WHOLE HECK OF A LOT! Yup, I know! > During the 1 hour surge, will SELECTs at 10 minutes after the > hour depend on INSERTs at 5 minutes after the hour? Yes, they do. It's a

Re: [PERFORM] Disappointing performance in db migrated from MS SQL

2004-02-13 Thread matt
> Josh, the disks in the new system should be substantially faster than > the old. Both are Ultra160 SCSI RAID 5 arrays, but the new system has > 15k RPM disks, as opposed to the 10k RPM disks in the old system. Spindle speed does not correlate with 'throughput' in any easy way. What controllers

[PERFORM] Prefetch

2005-05-10 Thread Matt Olson
warehouse, the observed performance is not what I would hope for. Regards, Matt Olson Ocean Consulting http://www.oceanconsulting.com/ ---(end of broadcast)--- TIP 1: subscribe and unsubscribe commands go to [EMAIL PROTECTED]

Re: [PERFORM] Prefetch

2005-05-10 Thread Matt Olson
o the list what I find and maybe do some _rough_ benchmarking. This is a production app, so I can't get too much in the way of the daily batches. -- Matt Olson Ocean Consulting http://www.oceanconsulting.com/ On Tuesday 10 May 2005 11:13 am, Greg Stark wrote: > Matt Olson writes: > &

[PERFORM] Prefetch

2005-05-15 Thread Matt Olson
r, in this set up, a data warehouse, the observed performance is not what I would hope for. Regards, Matt Olson Ocean Consulting http://www.oceanconsulting.com/ ---(end of broadcast)--- TIP 8: explain analyze is your friend

[PERFORM] pgsql upgrade

2006-12-04 Thread Matt Chambers
topped postgres from writing to data/base but the strange load pattern remains. (system is ~30% of the overall load, vs 3% before) So, my question is, what happened, and how can I get it back to the same load pattern 7.4.6 had, and the same pattern I had for 4 hours before it wen

[PERFORM] Sunfire X4500 recommendations

2007-03-22 Thread Matt Smiley
My company is purchasing a Sunfire x4500 to run our most I/O-bound databases, and I'd like to get some advice on configuration and tuning. We're currently looking at: - Solaris 10 + zfs + RAID Z - CentOS 4 + xfs + RAID 10 - CentOS 4 + ext3 + RAID 10 but we're open to other suggestions. >From

Re: [PERFORM] Sunfire X4500 recommendations

2007-03-23 Thread Matt Smiley
gle Postgres backend will see better than a single disk's seek rate, because the executor currently dispatches only 1 logical I/O request at a time. >>> Dimitri <[EMAIL PROTECTED]> 03/23/07 2:28 AM >>> On Friday 23 March 2007 03:20, Matt Smiley wrote: > My company is pur

Re: [PERFORM] Sunfire X4500 recommendations

2007-03-27 Thread Matt Smiley
Hi Dimitri, First of all, thanks again for the great feedback! Yes, my I/O load is mostly read operations. There are some bulk writes done in the background periodically throughout the day, but these are not as time-sensitive. I'll have to do some testing to find the best balance of read vs.

Re: [PERFORM] Sunfire X4500 recommendations

2007-03-29 Thread Matt Smiley
Hi David, Thanks for your feedback! I'm rather a newbie at this, and I do appreciate the critique. First, let me correct myself: The formulas for the risk of loosing data when you loose 2 and 3 disks shouldn't have included the first term (g/n). I'll give the corrected formulas and tables at

[PERFORM] db performance/design question

2007-09-12 Thread Matt Chambers
ta will be created. Considering a single table would grow to 10mil+ rows at max, and this machine will sustain about 25mbps of insert/update/delete traffic 24/7 - 365, will I be saving much by partitioning data like that? -- -Matt <http://twiki.spimageworks.com/twiki/bin/view/Software/CueDevelopment>

[PERFORM] Slowing UPDATEs inside a transaction

2011-03-03 Thread Matt Burke
Hi. I've only been using PostgreSQL properly for a week or so, so I apologise if this has been covered numerous times, however Google is producing nothing of use. I'm trying to import a large amount of legacy data (billions of denormalised rows) into a pg database with a completely different schem

Re: [PERFORM] Slowing UPDATEs inside a transaction

2011-03-04 Thread Matt Burke
Robert Haas wrote: > Old row versions have to be kept around until they're no longer of > interest to any still-running transaction. Thanks for the explanation. Regarding the snippet above, why would the intermediate history of multiply-modified uncommitted rows be of interest to anything, or is

Re: [PERFORM] Scaling further up

2004-03-15 Thread Matt Davies
Quoting Andrew Sullivan <[EMAIL PROTECTED]>: > On Fri, Mar 12, 2004 at 06:25:48PM -0500, Anjan Dave wrote: > > We upgraded from 8GB to 12GB RAM a month or so ago, but even in the > > past, I've never seen the system exhaust on it's system cache (~6GB, in > > 'top'), while it's swapping. > > > > S

Re: [PERFORM] [ADMIN] Benchmarking postgres on Solaris/Linux

2004-03-23 Thread Matt Clark
Dell will most likely beat the Sun by some distance, although what the Sun lacks in CPU power it may make up a bit in memory bandwidth/latency. Matt > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Subbiah, > Stalin > Sent: 23 March

Re: [PERFORM] [ADMIN] Benchmarking postgres on Solaris/Linux

2004-03-24 Thread Matt Clark
> > Now if these vendors could somehow eliminate downtime due to human error > > we'd be talking *serious* reliablity. > > You mean making the OS smart enough to know when clearing the arp > cache is a bonehead operation, or just making the hardware smart > enough to realise that the keyswitch real

Re: [PERFORM] Wierd context-switching issue on Xeon

2004-04-20 Thread Matt Clark
107 115842 54 14 32 2 1 0 23380 577680 105920 2145140 0 0 032 156 117549 50 16 35 HTH Matt > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Tom Lane > Sent: 20 April 2004 01:02 > To: [EMAIL PROTECTED] > Cc: Joe

Re: [PERFORM] Postgres over Linux NBD or NFS

2004-06-22 Thread Matt Clark
How about iSCSI? This is exactly what it's for - presenting a bunch of remote SCSI hardware as if it were local. There are several reference implementations on SourceForge from Intel, Cisco & others. I've never tried it myself, but I would if I had the need. And let's face it there are some v

Re: [PERFORM] Swapping in 7.4.3

2004-07-16 Thread Matt Clark
> This is normal. My personal workstation has been up for 16 > days, and it shows 65 megs used for swap. The linux kernel > looks for things that haven't been accessed in quite a while > and tosses them into swap to free up the memory for other uses. > > This isn't PostgreSQL's fault, or anyt

Re: [PERFORM] Performance Bottleneck

2004-08-08 Thread Matt Clark
> And this is exactly where the pgpool advantage lies. > Especially with the > TPC-W, the Apache is serving a mix of PHP (or whatever CGI > technique is > used) and static content like images. Since the 200+ Apache > kids serve > any of that content by random and the emulated browsers very mu

Re: [PERFORM] insert

2004-08-13 Thread Matt Clark
> > It is likely that you are missing an index on one of those foreign > > key'd items. > > I don't think that is too likely as a foreign key reference > must be a unique key which would have an index. I think you must be thinking of primary keys, not foreign keys. All one-to-many relationshi

Re: [PERFORM] Disabling transaction/outdated-tuple behaviour

2004-08-26 Thread Matt Clark
> Immediately after performing a vaccum, updates take upto 50 > milliseconds to occur, however the update performance > degrades over time, such that after a few hours of continuous > updates, each update takes about half a second. Regular > vacuuming improves the performance temporarily, but d

Re: [PERFORM] Table UPDATE is too slow

2004-08-31 Thread Matt Clark
> >That looks like poor database normalization, really. Are you > sure you > >don't want to split this into multiple tables instead of having 62 > >columns? > > > No, it is properly normalized. The data in this table is stock > fundamentals, stuff like 52 week high, ex-dividend date, etc, etc.

Re: [PERFORM] Caching of Queries

2004-09-23 Thread Matt Clark
> I've looked at PREPARE, but apparently it only lasts > per-session - that's worthless in our case (web based > service, one connection per data-requiring connection). That's a non-sequitur. Most 'normal' high volume web apps have persistent DB connections, one per http server process. Are yo

Re: [PERFORM] IBM P-series machines

2004-10-11 Thread Matt Clark
As for "vendor support" for Opteron, that sure looks like a trainwreck... If you're going through IBM, then they won't want to respond to any issues if you're not running a "bog-standard" RHAS/RHES release from Red Hat. And that, on Opteron, is preposterous, because there's plenty of the bits of

Re: [PERFORM] Which plattform do you recommend I run PostgreSQL for

2004-10-12 Thread Matt Clark
In the MySQL manual it says that MySQL performs best with Linux 2.4 with ReiserFS on x86. Can anyone official, or in the know, give similar information regarding PostgreSQL? I'm neither official, nor in the know, but I do have a spare moment! I can tell you that any *NIX variant on any modern

Re: [PERFORM] Opteron vs RHAT

2004-10-13 Thread Matt Clark
> >>trainwreck... If you're going through IBM, then they won't want to > >>respond to any issues if you're not running a > "bog-standard" RHAS/RHES > >>release from Red Hat. ...> To be fair, we keep on actually running into things that > _can't_ be backported, like fibrechannel drivers that

Re: [PERFORM] Select with qualified join condition / Batch inserts

2004-10-15 Thread Matt Clark
> SELECT cmp.WELL_INDEX, cmp.COMPOUND, con.CONCENTRATION > FROM SCR_WELL_COMPOUND cmp, SCR_WELL_CONCENTRATION con > WHERE cmp.BARCODE=con.BARCODE > AND cmp.WELL_INDEX=con.WELL_INDEX > AND cmp.MAT_ID=con.MAT_ID > AND cmp.MAT_ID = 3 >

Re: [PERFORM] Caching of Queries

2004-09-27 Thread Matt Clark
Basically you set a default in seconds for the HTML results to be cached, and then have triggers set that force the cache to regenerate (whenever CRUD happens to the content, for example). Can't speak for Perl/Python/Ruby/.Net/Java, but Cache_Lite sure made a believer out of me! Nice to have it

Re: [PERFORM] Caching of Queries

2004-09-27 Thread Matt Clark
It might be easiest to shove the caching logic into pgpool instead. ... When pg_pool is told to cache a query, it can get a table list and monitor for changes. When it gets changes, simply dumps the cache. It's certainly the case that the typical web app (which, along with warehouses, seems to

Re: [PERFORM] Caching of Queries

2004-09-27 Thread Matt Clark
More to the point though, I think this is a feature that really really should be in the DB, because then it's trivial for people to use. How does putting it into PGPool make it any less trivial for people to use? The answers are at  http://www2b.biglobe.ne.jp/~caco/pgpoo

Re: [PERFORM] Caching of Queries

2004-09-27 Thread Matt Clark
Any competently written application where caching results would be a suitable performance boost can already implement application or middleware caching fairly easily, and increase performance much more than putting result caching into the database would. I guess the performance increase is that

Re: [PERFORM] Caching of Queries

2004-09-30 Thread Matt Clark
If you're not using a connection pool of some kind then you might as well forget query plan caching, because your connect overhead will swamp the planning cost. This does not mean you have to use something like pgpool (which makes some rather questionable claims IMO); any decent web application la

Re: [PERFORM] Caching of Queries

2004-10-01 Thread Matt Clark
> OK, that'd work too... the point is if you're re-connecting > all the time it doesn't really matter what else you do for > performance. Yeah, although there is the chap who was asking questions on the list recently who had some very long-running code on his app servers, so was best off closing

Re: [PERFORM] Caching of Queries

2004-10-05 Thread Matt Clark
> I don't know what you are exactly referring to in above URL > when you are talking about "potential pitfalls of pooling". > Please explain more. Sorry, I wasn't implying that pgpool doesn't deal with the issues, just that some people aren't necessarily aware of them up front. For instance, p

[PERFORM] Speeding up this function

2004-10-19 Thread Matt Nuzum
Hello, I've thought it would be nice to index certain aspects of my apache log files for analysis. I've used several different techniques and have something usable now, but I'd like to tweak it one step further. My first performance optimization was to change the logformat into a CSV format. I pr

Re: [PERFORM] Speeding up this function

2004-10-19 Thread Matt Nuzum
On Tue, 19 Oct 2004 15:49:45 -0400, Jeremy Dunn <[EMAIL PROTECTED]> wrote: > > -Original Message- > > From: [EMAIL PROTECTED] > > [mailto:[EMAIL PROTECTED] On Behalf Of > > Matt Nuzum > > Sent: Tuesday, October 19, 2004 3:35 PM > > To: pgsql-perf

Re: [PERFORM] OS desicion

2004-10-20 Thread Matt Clark
You are asking the wrong question. The best OS is the OS you (and/or the customer) knows and can administer competently. The real performance differences between unices are so small as to be ignorable in this context. The context switching bug is not OS-dependent, but varys in severity acro

Re: [PERFORM] how much mem to give postgres?

2004-10-20 Thread Matt Clark
How would I turn that off? In the kernel config? Not too familiar with that. I have a 2 proc xeon with 4 gigs of mem on the way for postgres, so I hope HT isn't a problem. If HT is turned off, does it just not use the other "half" of the processor? Or does the processor just work as one unit? Y

Re: [PERFORM] OS desicion

2004-10-20 Thread Matt Clark
The real performance differences between unices are so small as to be ignorable in this context. <> Well, at least the difference between Linux and BSD. There are substantial tradeoffs should you chose to use Solaris or UnixWare. Yes, quite right, I should have said 'po

Re: [PERFORM] how much mem to give postgres?

2004-10-20 Thread Matt Clark
Hyperthreading is actually an excellent architectural feature that can give significant performance gains when implemented well and used for an appropriate workload under a decently HT aware OS. IMO, typical RDBMS streams are not an obviously appropriate workload, Intel didn't implement it partic

[PERFORM] Anything to be gained from a 'Postgres Filesystem'?

2004-10-21 Thread Matt Clark
, I'm not an FS guru...): * great append performance for the WAL? * optimised scattered writes for checkpointing? * Knowledge that FSYNC is being used for preserving ordering a lot of the time, rather than requiring actual writes to disk (so long as the writes eventually happen in order...)? Mat

Re: [PERFORM] Anything to be gained from a 'Postgres Filesystem'?

2004-10-21 Thread Matt Clark
> Looking at that list, I got the feeling that you'd want to > push that PG-awareness down into the block-io layer as well, > then, so as to be able to optimise for (perhaps) conflicting > goals depending on what the app does; for the IO system to be > able to read the apps mind it needs to hav

Re: [PERFORM] can't handle large number of INSERT/UPDATEs

2004-10-26 Thread Matt Clark
I don't have iostat on that machine, but vmstat shows a lot of writes to the drives, and the runnable processes are more than 1: 6 1 0 3617652 292936 279192800 0 52430 1347 4681 25 19 20 37 Assuming that's the output of 'vmstat 1' and not some other delay, 50MB/second of sus

Re: [PERFORM] can't handle large number of INSERT/UPDATEs

2004-10-26 Thread Matt Clark
and certainly anyone who's been around a computer more than a week or two knows which direction "in" and "out" are customarily seen from. regards, tom lane Apparently not whoever wrote the man page that everyone copied ;-) Interesting. I checked this on several machines. They actually

Re: [PERFORM] preloading indexes

2004-11-03 Thread Matt Clark
Title: Message The best way to get all the stuff needed by a query into RAM is to run the query.  Is it more that you want to 'pin' the data in RAM so it doesn't get overwritten by other queries?   -Original Message-From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [E

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
> I have a dual processor system that can support over 150 concurrent > connections handling normal traffic and load. Now suppose I setup > Apache to spawn all of it's children instantly, what will ... > This will spawn 150 children in a short order of time and as > this takes "Doctor, it h

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
> Apache::DBI overall works better to what I require, even if > it is not a > pool per sey. Now if pgpool supported variable rate pooling like > Apache does with it's children, it might help to even things > out. That > and you'd still get the spike if you have to start the webserver and >

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
> Case in point: A first time visitor hits your home page. A > dynamic page is generated (in about 1 second) and served > (taking 2 more seconds) which contains links to 20 additional The gain from an accelerator is actually even more that that, as it takes essentially zero seconds for Apache

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
> Correct the 75% of all hits are on a script that can take > anywhere from > a few seconds to a half an hour to complete.The script > essentially > auto-flushes to the browser so they get new information as it arrives > creating the illusion of on demand generation. This is more like a s

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
1- You have a query that runs for half an hour and you spoon feed the results to the client ? (argh) 2- Your script looks for new data every few seconds, sends a packet, then sleeps, and loops ? If it's 2 I have a readymade solution for you, just ask. I'm guessing (2) - PG do

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
These are CGI scripts at the lowest level, nothing more and nothing less. While I could probably embed a small webserver directly into the perl scripts and run that as a daemon, it would take away the portability that the scripts currently offer. If they're CGI *scripts* then they just use the

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
In your webpage include an iframe with a Javascript to refresh it every five seconds. The iframe fetches a page from the server which brings in the new data in form of generated JavaScript which writes in the parent window. Thus, you get a very short request every 5 seconds to fetch new

Re: [PERFORM] appropriate indexing

2004-11-04 Thread Matt Clark
- ITEM table will, grow, grow, grow (sold items are not deleted) WHERE PRODUCT.SECTION_USED_FK IS NOT NULL AND ITEM.STATUS=1 and (ITEM.KIND=2 or ITEM.KIND=3) Partial index on item.status ? ---(end of broadcast)--- TIP 1: subscribe and unsubscribe c

Re: [PERFORM] Better Hardware, worst Results

2004-11-04 Thread Matt Clark
[EMAIL PROTECTED] wrote: Citando Rod Taylor <[EMAIL PROTECTED]>: Please send an explain analyze from both. I'm sendin three explains. In the first the Dell machine didn't use existing indexes, so I turn enable_seqscan off (this is the second explain). The total cost dec

Re: [PERFORM] Better Hardware, worst Results

2004-11-04 Thread Matt Clark
All 3 plans have crappy estimates. Run ANALYZE in production, then send another explain analyze (as an attachment please, to avoid linewrap). Er, no other possible answer except Rod's :-) ---(end of broadcast)--- TIP 2: you can get off all lists a

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
Javascript is too powerful to turn for any random web page. It is only essential for web pages because people write their web pages to only work with javascript. Hmm... I respectfully disagree. It is so powerful that it is impossible to ignore when implementing a sophisticated app. And it is

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
A note though : you'll have to turn off HTTP persistent connections in your server (not in your proxy) or youre back to square one. I hadn't considered that. On the client side it would seem to be up to the client whether to use a persistent connection or not. If it does, then yeah, a

Re: [PERFORM] Restricting Postgres

2004-11-04 Thread Matt Clark
Pierre-Frédéric Caillaud wrote: check this marvelus piece of 5 minutes of work : http://boutiquenumerique.com/test/iframe_feed.html cela m'a fait le sourire :-) (apologies for bad french) M ---(end of broadcast)--- TIP 7: don't forget to increase yo

Re: [PERFORM] Strange (?) Index behavior?

2004-11-05 Thread Matt Clark
> For some reason it's a requirement that partial wildcard > searches are done on this field, such as "SELECT ... WHERE > field LIKE 'A%'" > > I thought an interesting way to do this would be to simply > create partial indexes for each letter on that field, and it > works when the query matche

[PERFORM] What is the difference between these?

2004-11-05 Thread Matt Nuzum
To me, these three queries seem identical... why doesn't the first one (simplest to understand and write) go the same speed as the third one? I'll I'm trying to do is get statistics for one day (in this case, today) summarized. Table has ~25M rows. I'm using postgres 7.3.? on rh linux 7.3 (note

Re: [PERFORM] Strange (?) Index behavior?

2004-11-05 Thread Matt Clark
> With that many rows, and a normal index on the field, > postgres figures the best option for say "I%" is not an index > scan, but a sequential scan on the table, with a filter -- > quite obviously this is slow as heck, and yes, I've run > analyze several times and in fact have the vacuum anal

Re: [PERFORM] Strange (?) Index behavior?

2004-11-05 Thread Matt Clark
So there are really only 9000 rows out of 76 million starting with 'I'? How about combining some techniques - you could create an index on the first two chars of the field (should be selective enough to give an index scan), select from that, and select the actual data with the like clause. I

Re: [PERFORM] ext3 journalling type

2004-11-08 Thread Matt Clark
> Am I right to assume that "writeback" is both fastest and at > the same time as safe to use as ordered? Maybe any of you > did some benchmarks? It should be fastest because it is the least overhead, and safe because postgres does it's own write-order guaranteeing through fsync(). You should

Re: [PERFORM] Speed in V8.0

2004-12-24 Thread Matt Clark
Another man working to the bitter end this Christmas! There could be many reasons, but maybe first you should look at the amount of RAM available? If the tables fit in RAM on the production server but not on the dev server, then that will easily defeat the improvement due to using the native DB v

Re: [PERFORM] Swapping on Solaris

2005-01-19 Thread Matt Casters
this command like "vmstat 10". (ignore the first line) Keep an eye on the "pi" and "po" parameters. (kilobytes paged in and out) HTH, Matt -- Matt Casters <[EMAIL PROTECTED]> i-Bridge bvba, http://www.kettle.be Fonteinstraat 70,

Re: [PERFORM] Swapping on Solaris

2005-01-19 Thread Matt Clark
This page may be of use: http://www.serverworldmagazine.com/monthly/2003/02/solaris.shtml From personal experience, for god's sake don't think Solaris' VM/swap implementation is easy - it's damn good, but it ain't easy! Matt Kevin Schroeder wrote: I think it's pr

[PERFORM]

2005-01-20 Thread Matt Casters
#x27;t find any documents immediately, are there any fine manuals to read on data warehouse performance tuning on PostgreSQL? Thanks in advance for any help you may have, I'll do my best to keep pgsql-performance up to date on the results. Best regards, Matt -- Matt Casters <[EMAIL PROTECTE

Re: [PERFORM]

2005-01-20 Thread Matt Casters
laris level to get maximum performance out of the system. Mmmm. This is going to be a though one to crack. Perhaps it will be possible to get some extra juice out of placing the indexes on the smaller disks (150G) and the data on the bigger ones? Thanks! Matt -Oorspronkelijk bericht

Re: [PERFORM]

2005-01-20 Thread Matt Casters
Joshua, Actually that's a great idea! I'll have to check if Solaris wants to play ball though. We'll have to see as we don't have the new disks yet, ETA is next week. Cheers, Matt -Oorspronkelijk bericht- Van: Joshua D. Drake [mailto:[EMAIL PROTECTED] Verzonden:

[PERFORM] DWH on Postgresql

2005-01-20 Thread Matt Casters
u may have, I'll do my best to keep pgsql-performance up to date on the results.   Best regards,   Matt ___ Matt Casters i-Bridge bvba, http://www.kettle.be Fonteinstraat 70, 9400 OKEGEM, Belgium Tel. 054/25.01.37 GSM 0486/97.29.37    

Re: [PERFORM]

2005-01-21 Thread Matt Casters
in the table partitions. (and in doing so is causing less disk IO) BTW, internally, Oracle sees partitions as tables too. Even the "Union all" system that MS SQL Server uses works fine as long as the optimiser supports it to prune correctly. Cheers, Matt -- Matt Casters <[EMAIL

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-21 Thread Matt Clark
Presumably it can't _ever_ know without being explicitly told, because even for a plain SELECT there might be triggers involved that update tables, or it might be a select of a stored proc, etc. So in the general case, you can't assume that a select doesn't cause an update, and you can't be su

Re: [PERFORM]

2005-01-21 Thread Matt Casters
a view. But there is one thing > missing. You can't just insert into super_foo and aquire the "correct > partition". You will still have to insert into the correct underlying > table. "Real" partitioning will take care of correct parti

Re: [PERFORM] PostgreSQL clustering VS MySQL clustering

2005-01-21 Thread Matt Clark
not to say that pgpool couldn't make a good guess in the majority of cases! M Joshua D. Drake wrote: Matt Clark wrote: Presumably it can't _ever_ know without being explicitly told, because even for a plain SELECT there might be triggers involved that update tables, or it might be

[PERFORM] Query plan excluding index on view

2008-04-03 Thread Matt Klinker
I'm trying to fine tune this query to return in a reasonable amount of time and am having difficulties getting the query to run the way I'd like. I have a couple of semi-related entities that are stored in individual tables, say, A and B. There is then a view created that pulls together the commo

Re: [PERFORM] Query plan excluding index on view

2008-04-03 Thread Matt Klinker
mpany_id on company c (cost=0.00..3.06 rows=1 width=517) Index Cond: (c.id = outer.listing_fid) On Thu, Apr 3, 2008 at 7:19 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Matt Klinker" <[EMAIL PROTECTED]> writes: > > I new I'd forget something! I've t

Re: [PERFORM] Query plan excluding index on view

2008-04-04 Thread Matt Klinker
ex Scan using pk_company_id on company c (cost=0.00..9.67 rows=1 width=424) Index Cond: (c.id = xref.listing_fid) On Thu, Apr 3, 2008 at 11:49 PM, Tom Lane <[EMAIL PROTECTED]> wrote: > "Matt Klinker" <[EMAIL PROTECTED]> writes: > > Sorry for not including this extra

Re: [SOLVED] [PERFORM] Query plan excluding index on view

2008-04-07 Thread Matt Klinker
wrote: > "Matt Klinker" <[EMAIL PROTECTED]> writes: > > --Joined View: > > CREATE OR REPLACE VIEW directory_listing AS > > SELECT school.id, school.name, school.description, 119075291 AS > > listing_type_fid > >FROM school > > UNION

Re: [PERFORM] limit clause breaks query planner?

2008-09-02 Thread Matt Smiley
Hi David, Early in this thread, Pavel suggested: > you should partial index > > create index foo(b) on mytable where a is null; Rather, you might try the opposite partial index (where a is NOT null) as a replacement for the original unqualified index on column A. This new index will be ignor

Re: [PERFORM] limit clause breaks query planner?

2008-09-03 Thread Matt Smiley
"Tom Lane" <[EMAIL PROTECTED]> writes: > "Matt Smiley" <[EMAIL PROTECTED]> writes: > > So an Index Scan is always going to have a higher cost estimate than > > an equivalent Seq Scan returning the same result rows (unless > > random_

Re: [PERFORM] limit clause breaks query planner?

2008-09-04 Thread Matt Smiley
"Tom Lane" <[EMAIL PROTECTED]> writes: > I'm not sure offhand whether the existing correlation stats would be of use > for > it, or whether we'd have to get ANALYZE to gather additional data. Please forgive the tangent, but would it be practical to add support for gathering statistics on an arbi

Re: [PERFORM] inaccurate stats on large tables

2008-09-08 Thread Matt Smiley
column, reanalyze the table, and recheck the selectivity estimate: alter table paliasorigin alter column origin_id set statistics 20; analyze paliasorigin; explain analyze select * from paliasorigin where origin_id=20; Good luck! Matt -- Sent via pgsql-performance mailing list (pgsql-performance

Re: [PERFORM] too many clog files

2008-09-09 Thread Matt Smiley
type: ROLLBACK PREPARED 'my_prepared_transaction1'; Hope this helps! Matt -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] too many clog files

2008-09-09 Thread Matt Smiley
Alvaro Herrera wrote: > Move the old clog files back where they were, and run VACUUM FREEZE in > all your databases. That should clean up all the old pg_clog files, if > you're really that desperate. Has anyone actually seen a CLOG file get removed under 8.2 or 8.3? How about 8.1? I'm probably

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-05 Thread Matt Burke
Arjen van der Meijden wrote: > Afaik the Perc 5/i and /e are more or less rebranded LSI-cards (they're > not identical in layout etc), so it would be a bit weird if they > performed much less than the similar LSI's wouldn't you think? I've recently had to replace a PERC4/DC with the exact same ca

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-05 Thread Matt Burke
Glyn Astill wrote: > Did you try flashing the PERC with the LSI firmware? > > I tried flashing a PERC3/dc with LSI firmware, it worked fine but I > saw no difference in performance so I assumed it must be somethign > else on the board that cripples it. No, for a few reasons: 1. I read somewhere

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Rajesh Kumar Mallah wrote: >> I've checked out the latest Areca controllers, but the manual >> available on their website states there's a limitation of 32 disks >> in an array... > > Where exactly is there limitation of 32 drives. the datasheet of > 1680 states support upto 128drives using en

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Scott Carey wrote: > You probably don’t want a single array with more than 32 drives anyway, > its almost always better to start carving out chunks and using software > raid 0 or 1 on top of that for various reasons. I wouldn’t put more than > 16 drives in one array on any of these RAID cards, they

Re: [PERFORM] suggestions for postgresql setup on Dell 2950 , PERC6i controller

2009-02-06 Thread Matt Burke
Glyn Astill wrote: >> Stupid question, but why do people bother with the Perc line of >> cards if the LSI brand is better? It seems the headache of trying >> to get the Perc cards to perform is not worth any money saved. > > I think in most cases the dell cards actually cost more, people end > u

Re: [PERFORM] Slow-ish Query Needs Some Love

2010-02-02 Thread Matt White
On Feb 2, 6:06 am, Edgardo Portal wrote: > On 2010-02-02, Matt White wrote: > > > I have a relatively straightforward query that by itself isn't that > > slow, but we have to run it up to 40 times on one webpage load, so it > > needs to run much faster than it d

  1   2   >