Re: [PERFORM] Best replication solution?

2009-04-07 Thread Andrew Sullivan
-- were planned in from the beginning. Londiste aimed to be simpler, so it would be interesting to see whether those features could be incorporated without the same complication. A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.

Re: [PERFORM] Best replication solution?

2009-04-07 Thread Andrew Sullivan
duplicate key of slony > meta-data were as this was a duplicate key of one of my table's primary > key. This really ought to be impossible -- Slony just speaks standard SQL statements between nodes. But I won't say there's no possible bug there. Your best bet is the Slony lis

Re: [PERFORM] Best replication solution?

2009-04-06 Thread Andrew Sullivan
s also much handier as reference material than Slony's (which is, in my experience, a little hard to navigate if you don't already know the system pretty well). A -- Andrew Sullivan a...@crankycanuck.ca -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Occasional Slow Commit

2008-10-28 Thread Andrew Sullivan
arget parameter. 0.7 might help. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Slow updates, poor IO

2008-09-26 Thread Andrew Sullivan
sue exactly? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] Choosing a filesystem

2008-09-11 Thread Andrew Sullivan
irectory. But maybe I'm > completely wrong. Can you please advise how to create logical partitions? I would listen to yourself before you listen to the expert. You sound right to me :) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- S

Re: [PERFORM] select on 22 GB table causes "An I/O error occured while sending to the backend." exception

2008-08-28 Thread Andrew Sullivan
s a solution that's as > bad or worse than the problem it's trying to solve. Ok, but the danger is that the OOM killer kills your postmaster. To me, this is a cure way worse than the disease it's trying to treat. YMMD &c. &c. A -- Andrew Sullivan [EMAI

Re: [PERFORM] select on 22 GB table causes "An I/O error occured while sending to the backend." exception

2008-08-27 Thread Andrew Sullivan
pshot of this is that postgres tends to be a big target for the OOM killer, with seriously bad effects to your database. So for good Postgres operation, you want to run on a machine with the OOM killer disabled. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -

Re: [PERFORM] Autovacuum does not stay turned off

2008-08-26 Thread Andrew Sullivan
em, when are you vacuuming? A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance

Re: [PERFORM] The state of PG replication in 2008/Q2?

2008-08-21 Thread Andrew Sullivan
rmal way, and is broken by applications doing DDL as part of the regular operation. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your su

Re: [PERFORM] Mailing list hacked by spammer?

2008-07-18 Thread Andrew Sullivan
machines all over the Internet to send spam is hardly "hacking the list".) A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667 4564 x104 http://www.commandprompt.com/ -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://w

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
re aren't any other > transactions no MVCC bloat seems to occur and updates are faster. Are you on 8.3? That may be HOT working for you. MVCC doesn't get turned off if there are no other transactions (it can't: what if another transaction starts part way through yours?). A -- Andrew

Re: [PERFORM] ??: Postgresql update op is very very slow

2008-06-26 Thread Andrew Sullivan
there no options (algorithms) for adaptively choosing different > update strategies that do not incur the full MVCC overhead? How would you pick? But one thing you could do is create the table with a non-standard fill factor, which might allow HOT to work its magic. A -- Andrew Sullivan [EMAIL P

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
ot;free software" that includes the cost of a few administrators, the accounting people want to know why the free software costs so much. > If you depend on your systems, though, you should never deploy any > change, no matter how innocuous it seems, without testing. I agree comple

Re: [PERFORM] Hardware vs Software RAID

2008-06-25 Thread Andrew Sullivan
ace to check that things will work before you deploy to production. (The other way to say that, of course, is "Linux is only free if your time is worth nothing." Substitute your favourite free software for "Linux", of course. ;-) ) A -- Andrew Sullivan

Re: [PERFORM] Which hardware ?

2008-06-17 Thread Andrew Sullivan
an issue. The power of the system is hard to know about in the context (with only 8Go of memory, I don't consider this a powerful box at all, note). But why wouldn't it be on the same network? You're using the network stack anyway, note: JVMs can't go over domain sockets. A

Re: [PERFORM] Which hardware ?

2008-06-17 Thread Andrew Sullivan
there will be a JDK 1.6 installed > too) . . .I think this is the real mistake. Get a separate database box. It's approximately impossible to tune a box correctly for both your application and your database, in my experience. A -- Andrew Sullivan [EMAIL PROTECTED] +1 503 667

Re: [PERFORM] Replication Syatem

2008-04-28 Thread Andrew Sullivan
On Mon, Apr 28, 2008 at 07:48:48PM +0530, Gauri Kanekar wrote: > Slony don't do automatic failover. And we would appreciate a system with > automatic failover :( No responsible asynchronous system will give you automatic failover. You can lose data that way. A -- Andrew Sull

Re: [PERFORM] Replication Syatem

2008-04-28 Thread Andrew Sullivan
the > decision of having a replication sytsem. So any suggestion on that :). I think you will find that no replication system will solve your underlying problems. That said, I happen to work for a company that will sell you a replication system to work with 8.1 if you really want it. A -- An

Re: [PERFORM] Oddly slow queries

2008-04-16 Thread Andrew Sullivan
On Wed, Apr 16, 2008 at 11:48:21PM +0200, Thomas Spreng wrote: > What I meant is if there are no INSERT's or UPDATE's going on it > shouldn't > affect SELECT queries, or am I wrong? CHECKPOINTs also happen on a time basis. They should be short in that case, but they still have to happen. --

Re: [PERFORM] count * performance issue

2008-03-11 Thread Andrew Sullivan
On Tue, Mar 11, 2008 at 02:19:09PM +, Matthew wrote: > of rows with IS NULL, then someone changes a row, then you find the count > of rows with IS NOT NULL. Add the two together, and there may be rows that > were counted twice, or not at all. Only if you count in READ COMMITTED. A -- Sen

Re: [PERFORM] Disable WAL completely

2008-02-19 Thread Andrew Sullivan
On Tue, Feb 19, 2008 at 02:48:55PM +, Matthew wrote: > If there's not much write traffic, the WAL won't be used much anyway. You still have checkpoints. > If you really don't care much about the integrity, then the best option is > probably to put the WAL on ramfs. Um, that will cause the

Re: [PERFORM] analyze

2008-01-29 Thread Andrew Sullivan
On Tue, Jan 29, 2008 at 04:28:45PM +0200, Adrian Moisey wrote: > > Seriously though, how do I try measure this? Is autovacuum not going to work for your case? A ---(end of broadcast)--- TIP 9: In versions below 8.0, the planner will ignore your

Re: [PERFORM] Best way to index IP data?

2008-01-11 Thread Andrew Sullivan
On Fri, Jan 11, 2008 at 02:38:27PM -0800, Steve Atkins wrote: > I don't think there's ambiguity about what an dotted-quad without a > netmask > means, and hasn't been for a long time. Am I missing something? Well, maybe. The problem is actually that, without a netmask under CIDR, the address al

Re: [PERFORM] Best way to index IP data?

2008-01-11 Thread Andrew Sullivan
On Fri, Jan 11, 2008 at 05:02:36PM -0500, Michael Stone wrote: > networks), but there's a conspicuous lack of a type for (hosts). I > suppose if you really are sure that you want to store hosts and not > networks Well, part of the trouble is that in the CIDR world, an IP without a netmask can

Re: [PERFORM] Best way to index IP data?

2008-01-11 Thread Andrew Sullivan
On Fri, Jan 11, 2008 at 10:19:51AM -0500, Tom Lane wrote: > Given that the world is going to IPv6 in a few years whether you like it > or not, that seems pretty darn short-sighted to me. Indeed. Even ARIN has finally started to tell people that IPv4 is running out. There are currently significan

Re: function body actors (was: [PERFORM] viewing source code)

2007-12-21 Thread Andrew Sullivan
On Fri, Dec 21, 2007 at 12:40:05AM -0500, Tom Lane wrote: > whether there is a useful policy for it to implement. Andrew Sullivan > argued upthread that we cannot get anywhere with both keys and encrypted > function bodies stored in the same database (I hope that's an adequate &g

Re: [HACKERS] function body actors (was: [PERFORM] viewing source code)

2007-12-21 Thread Andrew Sullivan
On Fri, Dec 21, 2007 at 12:09:28AM -0500, Merlin Moncure wrote: > Maybe a key management solution isn't required. If, instead of > strictly wrapping a language with an encryption layer, we provide > hooks (actors) that have the ability to operate on the function body > when it arrives and leaves p

Re: [PERFORM] viewing source code

2007-12-20 Thread Andrew Sullivan
On Thu, Dec 20, 2007 at 05:04:33PM -0500, Merlin Moncure wrote: > right, right, thanks for the lecture. I am aware of various issues > with key management. Sorry to come off that way. It wasn't my intention to lecture, but rather to try to stop dead a cure that, in my opinion, is rather worse th

Re: [PERFORM] viewing source code

2007-12-20 Thread Andrew Sullivan
On Thu, Dec 20, 2007 at 03:24:34PM -0600, Roberts, Jon wrote: > > Actually, PostgreSQL already has column level security for pg_stat_activity. Not exactly. pg_stat_activity is a view. But I think someone suggested upthread experimenting with making pg_proc into a view, and making the real tab

Re: [PERFORM] viewing source code

2007-12-20 Thread Andrew Sullivan
On Thu, Dec 20, 2007 at 01:45:08PM -0600, Roberts, Jon wrote: > Businesses use databases like crazy. Non-technical people write their own > code to analyze data. The stuff they write many times is as valuable as the > data itself and should be protected like the data. They don't need or want > m

Re: [PERFORM] viewing source code

2007-12-20 Thread Andrew Sullivan
On Thu, Dec 20, 2007 at 03:35:42PM -0500, Merlin Moncure wrote: > > Key management is an issue but easily solved. Uber simple solution is > to create a designated table holding the key(s) and use classic > permissions to guard it. Any security expert worth the title would point and laugh at th

Re: [PERFORM] 7.4 Checkpoint Question

2007-11-29 Thread Andrew Sullivan
During this one checkpoint, I'm seeing transactions running 2-3 seconds. > During this time, writes are < 5/minute. > What gives? pg_dump? Remember that it has special locks approximately equivalent (actually eq? I forget) with SERIALIZABLE mode, which makes things rather differen

Re: [PERFORM] Query only slow on first run

2007-11-27 Thread Andrew Sullivan
> run too? Probably by buying much faster disk hardware. You'll note that the query plans you posted are the same, except for the actual time it took to get the results back. That tells me you have slow storage. On subsequent runs, the data is cached, so it's fast. A -- Andrew Sulliva

Re: [PERFORM] Curious about dead rows.

2007-11-14 Thread Andrew Sullivan
On Wed, Nov 14, 2007 at 11:58:23AM -0500, Andrew Sullivan wrote: > No, every statement in psql is a transaction. Even SELECT. Every statement Err, to be clearer, "Every statement in psql is _somehow_ part of a transaction; if you don't start one explicitly, the statement runs on

Re: [PERFORM] Curious about dead rows.

2007-11-14 Thread Andrew Sullivan
l do it You could grant superuser status to your user (or just connect as postgres user) for the time being, while debugging this. A -- Andrew Sullivan Old sigs will return after re-constitution of blue smoke ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PERFORM] Curious about dead rows.

2007-11-14 Thread Andrew Sullivan
STER will do everything you need. But are you sure there are _no other_ transactions open when you do that? This could cause problems, and CLUSTER's behaviour with other open transactions is not, um, friendly prior to the current beta. A -- Andrew Sullivan Old sigs will return after re-con

Re: [PERFORM] Curious about dead rows.

2007-11-13 Thread Andrew Sullivan
ting a > primary key, so it should be impossible anyhow. I thought you were doing INSERTs? It's not true that the output of the sequence is the only way -- if you insert directly, it will happily insert into that column. But it should cause an error to show in the log, which is what'

Re: [PERFORM] Curious about dead rows.

2007-11-13 Thread Andrew Sullivan
onflict with used sequence values. That should cause errors that you'd get in the log, presuming that you have the log level set correctly. A -- Andrew Sullivan Old sigs will return after re-constitution of blue smoke ---(end of broadcast)--- TI

Re: [PERFORM] Curious about dead rows.

2007-11-13 Thread Andrew Sullivan
rows, something is producing them. Either INSERT is firing a trigger that is doing something there (you won't see an UPDATE in that case), or else something else is causing INSERTs to fail. A -- Andrew Sullivan Old sigs will return after re-constitution of blue smoke

Re: [PERFORM] Curious about dead rows.

2007-11-12 Thread Andrew Sullivan
On Sat, Nov 10, 2007 at 09:22:58PM -0500, Jean-David Beyer wrote: > > > > So, there are NO failed inserts, and no updates? Cause that's what > > I'd expect to create the dead rows. > > > So would I. Hence the original question. Foreign keys with cascadi

Re: [PERFORM] server performance issues - suggestions for tuning

2007-08-28 Thread Andrew Sullivan
at due to large numbers of failed vacuums, however, I suspect your problem is I/O. Vacuum churns through the disk very aggressively, and if you're close to your I/O limit, it can push you over the top. A -- Andrew Sullivan | [EMAIL PROTECTED] "The year's penultimate month"

Re: [PERFORM] best use of an EMC SAN

2007-07-11 Thread Andrew Sullivan
ur WAL is near to its I/O limits, the only way you're going to get your redundancy back is to go noticably slower :-( > will lose a very little bit in comparison. Andrew Sullivan had a > somewhat similar finding a few years ago on some old Solaris hardware > that unfortunately isn&#x

Re: [PERFORM] TIMING A QUERY ???

2007-07-11 Thread Andrew Sullivan
on the order of hours for the EXPLAIN ANALYSE to return, I assumed that the problem is one of impatience and not clock cycles. After all, the gettimeofday() additional overhead is still not going to come in on the order of minutes without a _bursting_ huge query plan. A -- Andrew Sulliva

Re: [PERFORM] Two questions.. shared_buffers and long reader issue

2007-07-11 Thread Andrew Sullivan
size? > On a dedicated postgres server with 4 Giga RAM. Is there any rule of > thumb? > Actually I set it to +-256M. There has been Much Discussion of this lately on this list. I suggest you have a look through the recent archives on that topic. A -- Andrew Sullivan | [EMAIL PRO

Re: [PERFORM] TIMING A QUERY ???

2007-07-11 Thread Andrew Sullivan
r? > ...can I use \timing??? I don't get any time when using the > \timing option... How so? It returns Time: N ms at the end of output for me. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably becom

Re: [PERFORM] WALL on controller without battery?

2007-07-11 Thread Andrew Sullivan
where the battery is. Even if it's slower (and I don't know whether it will be), I assume that having the right data more slowly is better than maybe not having the data at all, quickly. A -- Andrew Sullivan | [EMAIL PROTECTED] The plural of anecdote is not data.

Re: [PERFORM] best use of an EMC SAN

2007-07-11 Thread Andrew Sullivan
ds on your SAN and its hard- and firm-ware, as well as its ability to interact with the OS. I think the best answer is "sometimes yes". A -- Andrew Sullivan | [EMAIL PROTECTED] However important originality may be in some fields, restraint and adherence to procedure emerge as the mor

Re: [PERFORM] vacuum a lot of data when insert only

2007-06-21 Thread Andrew Sullivan
r update to the table). A -- Andrew Sullivan | [EMAIL PROTECTED] Unfortunately reformatting the Internet is a little more painful than reformatting your hard drive when it gets out of whack. --Scott Morris ---(end of broadcast)--- TIP 9:

Re: [PERFORM] Performance query about large tables, lots of concurrent access

2007-06-21 Thread Andrew Sullivan
to get faster disk that 6 drives in RAID5, even if they're 15,000 RPM. The rotation speed is the least of your problems in many RAID implementations. A -- Andrew Sullivan | [EMAIL PROTECTED] "The year's penultimate month" is not in truth a good way of saying Novem

Re: [PERFORM] Volunteer to build a configuration tool

2007-06-21 Thread Andrew Sullivan
y a working low-level part of your design to get an undemonstrated benefit and probably a whole lot of new bugs? A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Po

Re: [PERFORM] Performance query about large tables, lots of concurrent access

2007-06-20 Thread Andrew Sullivan
ob. You probably need more I/O, and actually more CPU wouldn't hurt, because then you could run three VACUUMs on three separate tables (on three separate disks, of course) and not have to switch them off and on the CPU. A -- Andrew Sullivan | [EMAIL PROTECTED] A certain description of men

Re: [PERFORM] Performance query about large tables, lots of concurrent access

2007-06-20 Thread Andrew Sullivan
each table -- like maybe in a loop -- would be better for your case. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Postmodernism. --B

Re: [PERFORM] Replication

2007-06-20 Thread Andrew Sullivan
at problem. Another part of the problem was that, for high-contention workloads like the ones we happened to be working on, an optimistic approach like Postgres-R is probably always going to be a loser. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes sh

Re: [GENERAL] [pgsql-advocacy] [PERFORM] [ADMIN] Postgres VS Oracle

2007-06-18 Thread Andrew Sullivan
On Mon, Jun 18, 2007 at 02:38:32PM -0400, Andrew Sullivan wrote: > I've picked -advocacy. Actually, I _had_ picked advocacy, but had an itchy trigger finger. Apologies, all. A -- Andrew Sullivan | [EMAIL PROTECTED] A certain description of men are for getting out of debt, yet are aga

Re: [GENERAL] [pgsql-advocacy] [PERFORM] [ADMIN] Postgres VS Oracle

2007-06-18 Thread Andrew Sullivan
, we'd be violating it, and since we're not, we can't possibly know about it, right ;-) But there are some materials about why to use Postgres on the website: http://www.postgresql.org/about/advantages A -- Andrew Sullivan | [EMAIL PROTECTED] When my information changes, I alte

Re: [GENERAL] [pgsql-advocacy] [PERFORM] [ADMIN] Postgres VS Oracle

2007-06-18 Thread Andrew Sullivan
least limit it to one list? A -- Andrew Sullivan | [EMAIL PROTECTED] Everything that happens in the world happens at some place. --Jane Jacobs ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http

Re: [PERFORM] How much ram is too much

2007-06-11 Thread Andrew Sullivan
arge one? In the past, that wasn't the case for relatively small buffers; with the replacement of single-pass LRU, that has certainly changed, but I'd be surprised if anyone tested a buffer as large as 32G. A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern

Re: [PERFORM] [ADMIN] reclaiming disk space after major updates

2007-06-08 Thread Andrew Sullivan
ase works differently, by taking an exclusive lock, but the basic conceptual problem is the same. A -- Andrew Sullivan | [EMAIL PROTECTED] Unfortunately reformatting the Internet is a little more painful than reformatting your hard drive when it gets out of whack. --Scott Morris

Re: [PERFORM] Getting Slow

2007-06-07 Thread Andrew Sullivan
-- first thing I'd look at is to see whether you are in fact hitting 100% of your I/O capacity and, if so, what your options are for getting more room there. A -- Andrew Sullivan | [EMAIL PROTECTED] "The year's penultimate month" is not in truth a good way

Re: [PERFORM] VERY slow queries at random

2007-06-07 Thread Andrew Sullivan
nly get > told about the slow query *after* it has completed and postgres has told > me so by logging a slow query entry in my logs? You can't :( A -- Andrew Sullivan | [EMAIL PROTECTED] This work was visionary and imaginative, and goes to show that visionary

Re: [PERFORM] VERY slow queries at random

2007-06-06 Thread Andrew Sullivan
On Wed, Jun 06, 2007 at 09:20:54PM +0200, Gunther Mayer wrote: > > What the heck could cause such erratic behaviour? I suspect some type of > resource problem but what and how could I dig deeper? Is something (perhaps implicitly) locking the table? That will cause this. A -- Andrew

control of benchmarks (was: [PERFORM] Thousands of tables)

2007-06-06 Thread Andrew Sullivan
er system that I've used that certainly had a similar issue, but I couldn't show you the data to prove it. Everyone who used it knew about it, though. A -- Andrew Sullivan | [EMAIL PROTECTED] A certain description of men are for getting out of debt, yet are against al

Re: [PERFORM] Thousands of tables versus on table?

2007-06-06 Thread Andrew Sullivan
row enough hardware money at it. But it seems a waste to re-implement something that's already apparently working for you in favour of something more expensive that you don't seem to need. A -- Andrew Sullivan | [EMAIL PROTECTED] When my information changes, I alter my con

Re: [PERFORM] Thousands of tables versus on table?

2007-06-06 Thread Andrew Sullivan
read-only data segments (maybe partitions, maybe something else) would help, so I know for sure that someone is working on a problem like this, but I don't think it's the sort of thing that's going to come any time soon. A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when compu

Re: [PERFORM] Vacuum takes forever

2007-05-30 Thread Andrew Sullivan
lity, introduced so that _other_ transactions don't get I/O starved. ("Make vacuum fast" isn't in most cases an interesting goal.) A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. Th

Re: [PERFORM] ECC RAM really needed?

2007-05-27 Thread Andrew Sullivan
worth storing correctly, and so doing things to improve the chances of correct storage is a good idea. A -- Andrew Sullivan | [EMAIL PROTECTED] Everything that happens in the world happens at some place. --Jane Jacobs ---(end of broadcast)

Re: [PERFORM] CPU Intensive query

2007-05-19 Thread Andrew Sullivan
On Fri, May 18, 2007 at 03:26:08PM -0700, Abu Mushayeed wrote: > Also, this query ran today and it already finished. Today it was > IO intensive. Are you entirely sure that it's not a coincidence, and something _else_ in the system is causing the CPU issues? A -- Andr

Re: [PERFORM] 121+ million record table perf problems

2007-05-18 Thread Andrew Sullivan
he > minimum go to a RAID1). Workload will primarily be comprised of queries I bet that single disk is your problem. Iostat is your friend, I'd say. A -- Andrew Sullivan | [EMAIL PROTECTED] Everything that happens in the world happens at some place.

Re: [PERFORM] Background vacuum

2007-05-17 Thread Andrew Sullivan
duling will really hurt. This means that, to use CPU scheduling safely, you have to be really sure that you know what the other transactions are doing. A -- Andrew Sullivan | [EMAIL PROTECTED] Information security isn't a technological problem. It's an economics problem.

Re: [PERFORM] When to vacuum a table?

2006-11-26 Thread Andrew Sullivan
On Sun, Nov 26, 2006 at 09:24:29AM -0500, Rod Taylor wrote: > attempt and fail a large number of insert transactions then you will > still need to vacuum. And you still need to vacuum an insert-only table sometimes, because of the system-wide vacuum requirement. A -- Andrew Su

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-29 Thread Andrew Sullivan
-performance charter now, so if anyone wants to pursue this, I urge you to take it to the Slony list.) A -- Andrew Sullivan | [EMAIL PROTECTED] Windows is a platform without soap, where rats run around in open sewers. --Daniel Eran ---(end of broadcast)--

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-29 Thread Andrew Sullivan
7;t rely on a pg_dump of a replica giving you a dump that, when restored, actually works. A -- Andrew Sullivan | [EMAIL PROTECTED] Everything that happens in the world happens at some place. --Jane Jacobs ---(end of broadcast)--- TIP

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-29 Thread Andrew Sullivan
way, don't use pg_dump on a replica. There's a tool that comes with slony that will allow you to take consistent, restorable dumps from replicas if you like. (And you might as well throw away the dumpfiles from the replicas that you have. They won't work when you restore them.) A

Re: [PERFORM] VACUUMs take twice as long across all nodes

2006-10-26 Thread Andrew Sullivan
gs showing increased time too? Are your targets getting further behind? 3. Your backups "from the slave" aren't done with pg_dump, right? But I suspect Slony has a role here, too. I'd look carefully at the slony tables -- especially the sl_log and pg_listen things, which

Re: [HACKERS] [PERFORM] Hints proposal

2006-10-12 Thread Andrew Sullivan
the use-cases I hear for a statement-level hints system fall into this latter category. A -- Andrew Sullivan | [EMAIL PROTECTED] Unfortunately reformatting the Internet is a little more painful than reformatting your hard drive when it gets out of whack. --Sco

Re: [PERFORM] Postgres locking up?

2006-09-29 Thread Andrew Sullivan
queries in question. The next thing I'd look for is OS-level performance problems. A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now.

Re: [PERFORM] slow queue-like empty table

2006-09-28 Thread Andrew Sullivan
ting from it, but all your other transactions depend on knowing the value of the "unprocessed queue", the design just doesn't work under PostgreSQL. It turns out to be impossible to keep the table vacuumed well enough for high performance. A -- Andrew Sullivan | [EMAIL PROTECTED]

Re: [PERFORM] Update on high concurrency OLTP application and Postgres 8 tuning

2006-09-20 Thread Andrew Sullivan
ss frequently. That's a good thing just because ANALYSE will impose an I/O load. A -- Andrew Sullivan | [EMAIL PROTECTED] A certain description of men are for getting out of debt, yet are against all taxes for raising money to pay it off. --Alexander Hamilton ---

Re: [PERFORM] Some queries starting to hang

2006-06-06 Thread Andrew Sullivan
none". (That said, I appreciate that there's precious little reason to spend a lot of work optimising a feature that is mostly there to counteract bad management practices.) A -- Andrew Sullivan | [EMAIL PROTECTED] When my information changes, I alter my conclusions. What do you do

Re: [PERFORM] Some queries starting to hang

2006-06-06 Thread Andrew Sullivan
ptom," might be helpful to users. Because the impatient simply won't wait for the full report to come back, and therefore they'll end up flying blind instead. (Note that "the impatient" is not always the person logged in and executing the commands.) A -- Andrew Sulli

Re: [PERFORM] vacuuming problems continued

2006-06-05 Thread Andrew Sullivan
hinking about strategies and am still a bit lost. Our > apps are up 24/7 and we didn't code for the eventuality of having the > db going offline for maintenance... we live and learn! You shouldn't need to, with anything after 7.4, if your vacuum regimen is right. There's some

Re: [PERFORM] Some queries starting to hang

2006-06-05 Thread Andrew Sullivan
ate or nonexistent ANALYZE stats, missing > custom adjustments of statistics target settings, etc. But even the nested loop shouldn't be a "never returns" case, should it? For 1800 rows? (I've _had_ bad plans that picked nestloop, for sure, but they're usually for tens

Re: [PERFORM] Some queries starting to hang

2006-06-05 Thread Andrew Sullivan
pushing the processor up to 99.9% active). Are there any locks preventing the query from completing? I can't recall how you check in 7.3, but if nothing else, you can check with ps for something WAITING. A -- Andrew Sullivan | [EMAIL PROTECTED] Unfortunately reformatting the Internet

Re: [PERFORM] Optimizing a huge_table/tiny_table join

2006-05-25 Thread Andrew Sullivan
x27;t change _at all_? Are you sure no VACUUMs or anything are happening automatically? > Anyway, I take it that there is no way to bypass the optimizer and > instruct PostgreSQL exactly how one wants the search performed? No, there isn't. A -- Andrew Sullivan | [EMAIL PROTECTED]

Re: [PERFORM] Postgres 7.4 and vacuum_cost_delay.

2006-05-04 Thread Andrew Sullivan
ot of time on trying to emulate the new features in 7.4. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably become the textbook definition of Postmodernism. --Brad Holland --

Re: [PERFORM] Investigating IO Saturation

2006-01-24 Thread Andrew Sullivan
uot;standard" as of > 8.0... And it doesn't work very well without changes to buffering. You need both pieces to get it to work. A -- Andrew Sullivan | [EMAIL PROTECTED] In the future this spectacle of the middle classes shocking the avant- garde will probably b

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
On Tue, Jan 17, 2006 at 11:43:14AM -0500, Chris Browne wrote: > [EMAIL PROTECTED] (Andrew Sullivan) writes: > > Because nothing that runs automatically should ever take an exclusive > > lock on the entire database, > That's a bit more than what autovacuum would probabl

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
actly the right settings for any generic workload yet under 8.1 (although probably people know them well enough for particular workloads). A -- Andrew Sullivan | [EMAIL PROTECTED] This work was visionary and imaginative, and goes to show that visionary and imaginative work need not end up well.

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
rdware is fixed and cannot be changed," is the first optimisation I'd make. Heck, I gave away a box to charity only two weeks ago that would solve your problem better than automatically issuing VACUUM FULL. A -- Andrew Sullivan | [EMAIL PROTECTED] Information security isn't a technol

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
't happen at the same time, because the bits might move out from under the SELECT while it's running. Concurrency is hard, and race conditions are easy, to implement. A -- Andrew Sullivan | [EMAIL PROTECTED] A certain description of men are for getting out of debt, yet are against all t

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
eally empty, the performance effect is positive. If you have VACUUM FULLed table, inserts have to extend the table before inserting, whereas in a table with some space reclaimed, the I/O effect of having to allocate another disk page is already done. A -- Andrew Sullivan | [EMAIL PROTECT

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
may have to fiddle with it from time to time. A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now. --J.D. Baldwin ---(end of bro

Re: [PERFORM] Autovacuum / full vacuum

2006-01-17 Thread Andrew Sullivan
ses to show that the tiny loss of memory to FSM is worth (a) an exclusive lock and (b) the loss of efficiency you get from having some preallocated pages in tables. A -- Andrew Sullivan | [EMAIL PROTECTED] The fact that technology doesn't work is no bar to success in the marketpla

Re: [PERFORM] new to postgres (and db management) and performance already a problem :-(

2006-01-17 Thread Andrew Sullivan
iew of storage, not the point of view of the user). A -- Andrew Sullivan | [EMAIL PROTECTED] The plural of anecdote is not data. --Roger Brinner ---(end of broadcast)--- TIP 6: explain analyze is your friend

Re: [PERFORM] new to postgres (and db management) and performance already a problem :-(

2006-01-16 Thread Andrew Sullivan
me EXPLAIN ANALYSE queries to understand that. A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern prose is away from concreteness. --George Orwell ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster

Re: [PERFORM] 15,000 tables

2005-12-02 Thread Andrew Sullivan
really wedded to this design? (I have a feeling that something along the lines of what Tom Lane said would be a better answer -- I think you need to be more clever, because I don't think this will ever work well, on any system.) A -- Andrew Sullivan | [EMAIL PROTECTED] This work was visionary an

Re: [PERFORM] weird performances problem

2005-11-17 Thread Andrew Sullivan
achine?). Is this a time, for example, when logrotate is killing your I/O with file moves? A -- Andrew Sullivan | [EMAIL PROTECTED] I remember when computers were frustrating because they *did* exactly what you told them to. That actually seems sort of quaint now. --J.D. B

Re: [PERFORM] Help tuning postgres

2005-10-18 Thread Andrew Sullivan
? If it's very large compared to the data you have stored in there, you may want to ask if you're "leaking" space from the free space map (because of that table turnover, which seems pretty severe). A -- Andrew Sullivan | [EMAIL PROTECTED] The whole tendency of modern

Re: [PERFORM] Help tuning postgres

2005-10-13 Thread Andrew Sullivan
occasional REINDEX to solve; I forget which version you said you were using). The painful part about tuning a production system is really that you have to keep about 50 variables juggling in your head, just so you can uncover the one thing that you have to put your finger on to make it all play

  1   2   3   >