Re: [PERFORM] pg_stat_all_indexes understand

2015-09-09 Thread Bruce Momjian
te.  This causes the > usage counts to get incremented.  Even when it doesn't end up using the merge > join. And it will be documented in 9.5: commit 7e9ed623d9988fcb1497a2a8ca7f676a5bfa136f Author: Bruce Momjian Date: Thu Mar 19 22:38:12 2015 -0400

Re: [PERFORM] intel s3500 -- hot stuff

2014-12-09 Thread Bruce Momjian
nowing which blocks to request before actually needing them. With a bitmap scan, that is easy --- I am unclear how to do it for other scans. We already have kernel read-ahead for sequential scans, and any index scan that hits multiple rows will probably already be using a bitmap heap scan. --

Re: [PERFORM] intel s3500 -- hot stuff

2014-12-06 Thread Bruce Momjian
rt many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review. It seems that time has come. -

Re: [PERFORM] Turn off Hyperthreading! WAS: 60 core performance with 9.3

2014-08-21 Thread Bruce Momjian
ut then this was improved, and it might have gotten bad again. I am afraid results are based on the type of CPU, so I am not sure we can know a general answer. I know I asked Greg Smith, and I assume he would know. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Turn off Hyperthreading! WAS: 60 core performance with 9.3

2014-08-21 Thread Bruce Momjian
On Thu, Aug 21, 2014 at 02:02:26PM -0700, Josh Berkus wrote: > On 08/20/2014 07:40 PM, Bruce Momjian wrote: > > On Wed, Aug 20, 2014 at 12:13:50PM -0700, Josh Berkus wrote: > >> On a read-write test, it's 10% faster with HT off as well. > >> > >> Further,

Re: [PERFORM] Turn off Hyperthreading! WAS: 60 core performance with 9.3

2014-08-20 Thread Bruce Momjian
-threading. I am also unclear exactly what you tested, as I didn't see it mentioned in the email --- CPU type, CPU count, and operating system would be the minimal information required. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterpr

Re: [PERFORM] SSI slows down over time

2014-04-09 Thread Bruce Momjian
t dropped by 23% during the same interval. And this was > actually one of the better sets of runs; I had a few last week that > dipped below 1ktps. > > I'm not sure what to make of this, thoughts? Seems it is time to ask on hackers. -- Bruce Momjian htt

Re: [PERFORM] PGSQL, checkpoints, and file system syncs

2014-04-09 Thread Bruce Momjian
flusher threads will start writeback. I think we want the flusher to be active, not necessarly the writing process. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsq

Re: [PERFORM] PGSQL, checkpoints, and file system syncs

2014-04-09 Thread Bruce Momjian
> of RAM. Most VM servers fit that profile, so I'm not surprised it's > hurting you. Agreed. The dirty kernel defaults Linux defaults are too high for systems with large amounts of memory. See sysclt -a | grep dirty for a list. -- Bruce Momjian h

Re: [PERFORM] Why shared_buffers max is 8GB?

2014-04-09 Thread Bruce Momjian
cessing more than 8GB of data in a short period of time. Add to that the problem if potentially dirtying all the buffers and flushing it to a now-smaller kernel buffer cache, and you can see why the 8GB limit is recommended. I do think this merits more testing against the current Postgres source

Re: [PERFORM] Explain analyze time overhead

2013-12-10 Thread Bruce Momjian
o calculate this-? Finally, for testing purposes, I > have disabled material and the query execution time dropped from 1 minute to > 12 second. The executable is not tied to any particular Postgres version, so you could get the 9.3 binary and just use that. --

Re: [PERFORM] Explain analyze time overhead

2013-12-10 Thread Bruce Momjian
l regardless of how > much work happens within the node. The original poster might also want to run pg_test_timing to get hardware timing overhead: http://www.postgresql.org/docs/9.3/static/pgtesttiming.html -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Performance bug in prepared statement binding in 9.2?

2013-09-10 Thread Bruce Momjian
indexes are created has changed between 8.4 -> 9.x? I don't know as you have not shown us exactly what is slower between versions --- you only said the bug appears or not in certain circumstances. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Performance bug in prepared statement binding in 9.2?

2013-09-09 Thread Bruce Momjian
know if this clarifies my earlier post. Yes, that is clear. So it is the seed data that is causing the problem? That is the only different I see from #2 and #3. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's imposs

Re: [PERFORM] Performance bug in prepared statement binding in 9.2?

2013-09-09 Thread Bruce Momjian
will correct the behavior. I do not understand what you are saying above. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (

Re: [PERFORM] Performance bug in prepared statement binding in 9.2?

2013-09-09 Thread Bruce Momjian
ap_hot_search_buffer OK, certainly looks like a HOT chain issue. I think there are two possibilities: 1) the heap or index file is different from a dump/restore vs. pg_upgrade 2) some other files is missing or changed between the two My guess is that the dump/r

Re: [PERFORM] Postgres upgrade, security release, where?

2013-04-01 Thread Bruce Momjian
www.postgresql.org/about/news/1454/ ) does not mention the > new release number, methinks there is plenty of room for confusion :( > > It might be an idea to update the "splash box" with details of the upcoming > release. I agree updating the "

Re: [PERFORM] Postgres upgrade, security release, where?

2013-04-01 Thread Bruce Momjian
10:25:12 MB > postgresql-9.2.3.tar.gz.md5 2013-02-07 58 > postgresql-9.2.3.tar.gz.md5 10:25:13 bytes Due to the security nature of the release, the source and binaries will only be publicly available on A

Re: [PERFORM] New server setup

2013-03-15 Thread Bruce Momjian
be showstopper for being able to take advantage of some > awesome I/O performance opportunities. Do you want to recreate the server if it loses power over an extra $100 per drive? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com

Re: [PERFORM] New server setup

2013-03-14 Thread Bruce Momjian
On Fri, Mar 15, 2013 at 10:37:55AM +1300, Mark Kirkwood wrote: > On 15/03/13 07:54, Bruce Momjian wrote: > >Only use SSDs with a BBU cache, and don't set SSD caches to > >write-through because an SSD needs to cache the write to avoid wearing > >out the chips ea

Re: [PERFORM] New server setup

2013-03-14 Thread Bruce Momjian
te-through because an SSD needs to cache the write to avoid wearing out the chips early, see: http://momjian.us/main/blogs/pgblog/2012.html#August_3_2012 -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossi

Re: [PERFORM] temp tablespaces and SSDs, etc..

2013-02-15 Thread Bruce Momjian
ike to be able to tell postgresql to > prefer them - in that order - until they each get full. IE, use tmpfs > until it reports ENOSPC and then fall back to SSD, finally falling > back to spinning rust. Is there a way to do this? Nope. -- Bruce Momjian http://momjian.us

Re: [PERFORM] Ubuntu 12.04 / 3.2 Kernel Bad for PostgreSQL Performance

2012-12-05 Thread Bruce Momjian
ar. Ah, that is interesting about 2.6. I had wondered how Debian stable would have performed, 2.6.32-5. This relates to a recent discussion about the appropriateness of Ubuntu for database servers: http://archives.postgresql.org/pgsql-performance/2012-11/msg00358.php Thanks. -- Bru

Re: [PERFORM] shared_buffers on ubuntu precise

2012-11-30 Thread Bruce Momjian
tayed on CentOS. Or Debian. Not sure what would justify use of Ubuntu as a server, except wanting to have the exact same OS as their personal computers. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for e

Re: SOLVED - RE: [PERFORM] Poor performance using CTE

2012-11-23 Thread Bruce Momjian
ll. All that said, there could be a narrow class of low hanging cases > (such as the OP's) that could be sniped...I'm just skeptical. Is thi -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible fo

Re: [PERFORM] Thousands databases or schemas

2012-11-15 Thread Bruce Momjian
tables. I was testing this as part of pg_upgrade performance improvements for large tables. We have a few other things we might try to improve for 9.3 related to caching, but that might not help in this case. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Bruce Momjian
On Wed, Nov 7, 2012 at 03:44:13PM -0300, Claudio Freire wrote: > On Wed, Nov 7, 2012 at 3:36 PM, Bruce Momjian wrote: > >> Bring both down. > >> pg_upgrade master > >> Bring master up > >> pg_upgrade slave > > > > Is there any reason to upgrade

Re: [PERFORM] How to upgrade from 9.1 to 9.2 with replication?

2012-11-07 Thread Bruce Momjian
sebackup) > Bring slave up Good ideas. I have applied the attached doc patch to pg_upgrade head and 9.2 docs to suggest using rsync as part of base backup. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossibl

Re: [PERFORM] Have: Seq Scan - Want: Index Scan - what am I doing wrong?

2012-10-16 Thread Bruce Momjian
ingle date wouldn't work. Have you read our FAQ on this matter? http://wiki.postgresql.org/wiki/FAQ#Why_are_my_queries_slow.3F_Why_don.27t_they_use_my_indexes.3F -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's im

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-10 Thread Bruce Momjian
ole index is in cache and that won't be true? Did I get that > right? Well, the real question is whether, while traversing the index, if some of the pages are going to be removed from the cache by other process cache usage. effective_cache_size is not figuring the cache will remain

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-10 Thread Bruce Momjian
On Wed, Oct 10, 2012 at 02:05:20PM -0300, Claudio Freire wrote: > On Wed, Oct 10, 2012 at 1:10 PM, Bruce Momjian wrote: > >> >shared_buffers = 10GB > >> > >> Generally going over 4GB for shared_buffers doesn't help.. some of > >> the overhead of bgw

Re: [PERFORM] Hyperthreading (was: Two identical systems, radically different performance)

2012-10-10 Thread Bruce Momjian
n't as > fool-proof as we thought? Gotta read the source. BRB Well, I have exactly the same setup here: new: 2x4-core Intex Xeon E5620 2.40 GHz Let me know if you want any tests run, on SSDs or magnetic disk. I do have hyperthreading enabled, and Greg Smith benchmarked my

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-10 Thread Bruce Momjian
it's a dedicated server) Why guess? Use 'free' to tell you the kernel cache size: http://momjian.us/main/blogs/pgblog/2012.html#May_4_2012 -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's imposs

Re: [PERFORM] shared_buffers/effective_cache_size on 96GB server

2012-10-10 Thread Bruce Momjian
21000 51 20949 > > So it did a little swapping, but only minor, still I should probably decrease > shared_buffers so there is no swapping at all. You might want to read my blog entry about swap space: http://momjian.us/main/blogs/pgblog/2012.html#July_25_2012

Re: [PERFORM] hardware advice - opinions about HP?

2012-10-02 Thread Bruce Momjian
t; switching. > > What about HP? If you need a big vendor, I think HP is a good choice. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-per

Re: [PERFORM] Bad query plan when the wrong data type is used

2012-09-01 Thread Bruce Momjian
exactly a small project, since you'd > > be talking about something like 36 new operators to cover all of int2, > > int4, int8.  But it's a straightforward extension. > > Interesting. Worth a TODO? Since we are discussing int2 casting, I wanted to bring up this other castin

Re: [PERFORM] pg_dump and thousands of schemas

2012-08-30 Thread Bruce Momjian
I replace them with single LOCK TABLE with multiple > > tables. With 100k tables LOCK statements took 13 minutes in total, now > > it only takes 3 seconds. Comments? > > Shall I commit to master and all supported branches? Was this applied? -- Bruce Momjian http://momjian

Re: [PERFORM] pg_dump and thousands of schemas

2012-08-30 Thread Bruce Momjian
On Thu, Aug 30, 2012 at 04:51:56PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Thu, May 31, 2012 at 09:20:43AM +0900, Tatsuo Ishii wrote: > >>> Ok, I modified the part of pg_dump where tremendous number of LOCK > >>> TABLE are issued. I replace them wit

Re: [PERFORM] High Disk write and space taken by PostgreSQL

2012-08-16 Thread Bruce Momjian
On Thu, Aug 16, 2012 at 06:07:26PM +0200, anara...@anarazel.de wrote: > > > Bruce Momjian schrieb: > > >On Thu, Aug 16, 2012 at 03:48:57PM +1000, Ondrej Ivanič wrote: > >> Hi, > >> > >> On 16 August 2012 15:40, J Ramesh Kumar > >wrote: >

Re: [PERFORM] High Disk write and space taken by PostgreSQL

2012-08-16 Thread Bruce Momjian
On Thu, Aug 16, 2012 at 10:53:21AM -0400, Bruce Momjian wrote: > On Thu, Aug 16, 2012 at 03:48:57PM +1000, Ondrej Ivanič wrote: > > Hi, > > > > On 16 August 2012 15:40, J Ramesh Kumar wrote: > > > As you said, MySQL with MyISAM is better choice for my app. Bec

Re: [PERFORM] High Disk write and space taken by PostgreSQL

2012-08-16 Thread Bruce Momjian
of an > unlogged table are also not replicated to standby servers. Any indexes > created on an unlogged table are automatically unlogged as well; > however, unlogged GiST indexes are currently not supported and cannot > be created on an unlogged table. I would set full_pag

Re: [PERFORM] Linux memory zone reclaim

2012-07-30 Thread Bruce Momjian
r specs: http://momjian.us/main/blogs/pgblog/2012.html#January_20_2012 I have 12 2GB DDR3 DIMs. Of course, my home server is ridiculously idle too. :-) -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's i

Re: [PERFORM] Linux memory zone reclaim

2012-07-26 Thread Bruce Momjian
t; wide-spread) Suggestions on whether that is necessary, or if just > disabling zone_reclaim is enough, are welcome from anyone who wants > to try and benchmark it. Should I be turning it off on my server too? It is enabled on my system. -- Bruce Momjian http://momjian.us

Re: [PERFORM] Drop statistics?

2012-07-03 Thread Bruce Momjian
> - > - regards, tom lane > - > > Awesome, thanks! One cool trick I have seen is to do the DELETE pg_statistic in a multi-statement transaction and then run query query, and roll it back. This allows the statistics to be preserved, and for only your query to see

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Bruce Momjian
the past to > >> fix various kinds of issues, and this is pretty non-invasive. > > > > I am not convinced either that this patch will still be useful after > > Jeff's fix goes in, ... > > But people on older branches are not going to GET Jeff's fix.

Re: [HACKERS] [PERFORM] pg_dump and thousands of schemas

2012-05-31 Thread Bruce Momjian
in very limited scenarios (--schema-only, perhaps). FYI, that is the pg_upgrade use-case, and pg_dump/restore time is reportedly taking the majority of time in many cases. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It'

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-25 Thread Bruce Momjian
e reported pg_upgrade took 45 minutes because of pg_dumpall --schema, which is quite long. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing

Re: [PERFORM] pg_dump and thousands of schemas

2012-05-24 Thread Bruce Momjian
t addressing them. It seems like addressing the overall > quadratic nature might be a globally better option, but addressing > just the problem with dumping one schema might be easier to kluge > together. Postgres 9.2 will have some speedups for pg_dump scanning large databases --- that might he

Re: [PERFORM] pg_upgrade

2011-12-08 Thread Bruce Momjian
ql version 8.4, server version 9.1. > Some psql features might not work. > Type "help" for help. > > Sorry my upgrade process has been an ugly mess :) You are using an 8.4.4 psql to connect to a 9.1.1 server. -- Bruce Momjian http://momjian.us Enterpr

Re: [PERFORM] pg_upgrade

2011-12-05 Thread Bruce Momjian
Tory M Blue wrote: > On Mon, Dec 5, 2011 at 10:31 AM, Tory M Blue wrote: > > On Mon, Dec 5, 2011 at 10:22 AM, Bruce Momjian wrote: > >>> But initial response to all this, is umm we have not really made a > >>> dump/restore unnecessary with the latest releases o

Re: [PERFORM] pg_upgrade

2011-12-05 Thread Bruce Momjian
Tory M Blue wrote: > On Mon, Dec 5, 2011 at 10:22 AM, Bruce Momjian wrote: > >> But initial response to all this, is umm we have not really made a > >> dump/restore unnecessary with the latest releases of Postgres than, as > >> I would have to think that there is a

Re: [PERFORM] pg_upgrade

2011-12-05 Thread Bruce Momjian
nge tablespace locations during the upgrade. In fact, we have had surprisingly few (zero) request for moving tablespaces, and now we are trying to implement this for Postgres 9.2. The normal API will be to have the user move the tablespace before the upgrade, but as I said before, it isn't e

Re: [PERFORM] pg_upgrade

2011-12-05 Thread Bruce Momjian
Nicholson, Brad (Toronto, ON, CA) wrote: > > -Original Message- > > From: Bruce Momjian [mailto:br...@momjian.us] > > Sent: Monday, December 05, 2011 10:24 AM > > To: Nicholson, Brad (Toronto, ON, CA) > > Cc: Tory M Blue; pgsql-performance@postgresql.org; M

Re: [PERFORM] pg_upgrade

2011-12-05 Thread Bruce Momjian
2011.html#June_15_2011_2 On a related note, Magnus is working on code for Postgres 9.2 that would allow for easier moving of tablespaces. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be

Re: [PERFORM] Intersect/Union X AND/OR

2011-12-05 Thread Bruce Momjian
ot; (cost=0.00..0.27 rows=1 > width=0)" > " -> Result (cost=0.00..0.26 rows=1 width=0)" > > The second plan is about 10 times faster than the first one. Well, there are usually several ways to execute a query internally, intsersect is using a differe

Re: [PERFORM] pg_upgrade

2011-12-03 Thread Bruce Momjian
Bruce Momjian wrote: > Tory M Blue wrote: > > On Sat, Dec 3, 2011 at 6:04 AM, Bruce Momjian wrote: > > > > > Well, I am not totally clear how you are moving things around, but I do > > > know pg_upgrade isn't happy to have the old and new cluster be very >

Re: [PERFORM] pg_upgrade

2011-12-03 Thread Bruce Momjian
Tory M Blue wrote: > On Sat, Dec 3, 2011 at 6:04 AM, Bruce Momjian wrote: > > > Well, I am not totally clear how you are moving things around, but I do > > know pg_upgrade isn't happy to have the old and new cluster be very > > different. > > > > Wha

Re: [PERFORM] pg_upgrade

2011-12-03 Thread Bruce Momjian
you didn't properly move the tablespace in the old cluster. We don't give you a very easy way to do that. You need to not only move the directory, but you need to update the symlinks in data/pg_tblspc/, and update the pg_tablespace system table. Did you do all of that? Does the 8.4 server

Re: [PERFORM] Intersect/Union X AND/OR

2011-12-02 Thread Bruce Momjian
etween them. I have seen rare cases where this is a win, so I mentioned it in that talk. Intersection is similarly possible for AND in WHERE clauses. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible f

Re: [PERFORM] SSD options, small database, ZFS

2011-11-22 Thread Bruce Momjian
msg00944.php) said to > limit shared_buffers max to 8 GB on Linux and leave the rest for OS > caching. Does the same advice hold on FreeBSD systems too? Hard to say. We don't know why this is happening but we are guessing it is the overhead of managing over one million shared buffers. P

Re: [PERFORM] SSD options, small database, ZFS

2011-11-22 Thread Bruce Momjian
his whole maze, and have some > data points to set expectations against. See > https://github.com/gregs1104/stream-scaling for the code and the samples. I can confirm that a Xeon E5620 CPU wants memory to be in multiples of 3, and a dual-CPU 5620 system needs memory in multiples of 6. (I inst

Re: [PERFORM] Allow sorts to use more available memory

2011-10-05 Thread Bruce Momjian
rs have pointed out, you can > do that by putting pgsql_tmp on a memory filesystem and letting the > sorts spill to the memory-based FS. It would be nice if the tempfs would allow us to control total temp memory usage, except it causes a failure rather than splilling to real disk. --

Re: [PERFORM] BBU still needed with SSD?

2011-07-18 Thread Bruce Momjian
getting very good performance while maintaining > > reliability. > > I'm not comparing SSD in SW RAID with rotating disks in HW RAID with > BBU though. I'm just comparing SSDs with or without BBU. I'm going to > get a couple of Intel 320s, just want to know if BB

Re: [PERFORM] Why we don't want hints Was: Slow count(*) again...

2011-02-16 Thread Bruce Momjian
1 and those correlations could be used to weigh how the single-column statistics should be combined. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-perf

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-04 Thread Bruce Momjian
. You need to state the case for hints independent of what other databases do, and indepdendent of fixing the problems where the optimizer doesn't match reatility. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com +

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-04 Thread Bruce Momjian
her throw away the effort invested in You want to reconsider using MySQL because Postgres doesn't have hints. Hard to see how that logic works. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for ever

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-04 Thread Bruce Momjian
aced +1 for Oracle being a "troublesome legacy product". -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (pgsql-performa

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Bruce Momjian
n the stove catches fire, > users is expected to report the issue and wait for a better stove to be > developed. It is a very rough analogy, but rather accurate one, too. That might be true. -- Bruce Momjian http://momjian.us EnterpriseDB htt

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-03 Thread Bruce Momjian
I wanted to say in a much better way . The settings are currently there to better model the real world (random_page_cost), or for testing (enable_seqscan). They are not there to force certain plans. They can be used for that, but that is not their purpose and they would not have been added if

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-02 Thread Bruce Momjian
y wings take dream ... I think this humorous video really nails it: http://www.youtube.com/watch?v=Km26gMI847Y Presidential Speechalist -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible f

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-02 Thread Bruce Momjian
gt; > stupid than a computer program, would not be a smart thing to do. > > Please, do not misunderestimate me. > > I remember when I used to only weigh that much. You are lucky to be > such a slim little guy! > > Oh, I guess I should add, :) Oh, wow, what a great retort.

Re: [HACKERS] [PERFORM] Slow count(*) again...

2011-02-02 Thread Bruce Momjian
hey actually tested it, they understood. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to

Re: [PERFORM] Questions on query planner, join types, and work_mem

2011-02-01 Thread Bruce Momjian
Bruce Momjian wrote: > Robert Haas wrote: > > On Wed, Aug 11, 2010 at 9:42 PM, Bruce Momjian wrote: > > > This confused me. ?If we are assuing the data is in > > > effective_cache_size, why are we adding sequential/random page cost to > > > the query cost rou

Re: [PERFORM] Questions on query planner, join types, and work_mem

2011-01-31 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Aug 11, 2010 at 9:42 PM, Bruce Momjian wrote: > > This confused me. ?If we are assuing the data is in > > effective_cache_size, why are we adding sequential/random page cost to > > the query cost routines? > > See the comments for index_pag

Re: [PERFORM] Running PostgreSQL as fast as possible no matter the consequences

2011-01-27 Thread Bruce Momjian
Jeff Janes wrote: > On Tue, Jan 25, 2011 at 5:32 PM, Bruce Momjian wrote: > > Robert Haas wrote: > >> On Wed, Jan 19, 2011 at 12:07 PM, Bruce Momjian wrote: > > >> > ? ? ? > >> > ?http://developer.postgresql.org/pgdocs/postgres/non-durability.htm

Re: [PERFORM] anti-join chosen even when slower than old plan

2011-01-26 Thread Bruce Momjian
that flavor but we haven't taken it to the > >>> logical conclusion. > >> > >> Is there a TODO here? > > > > it looks like, yes. > > "Modify the planner to better estimate caching effects"? Added to TODO. -- Bruce Momjian htt

Re: [PERFORM] Running PostgreSQL as fast as possible no matter the consequences

2011-01-25 Thread Bruce Momjian
Robert Haas wrote: > On Wed, Jan 19, 2011 at 12:07 PM, Bruce Momjian wrote: > > Chris Browne wrote: > >> gentosa...@gmail.com (A B) writes: > >> > If you just wanted PostgreSQL to go as fast as possible WITHOUT any > >> > care for your data (you acce

Re: [PERFORM] anti-join chosen even when slower than old plan

2011-01-19 Thread Bruce Momjian
d shouldn't be inventing a new one out of thin air. > The problem is to figure out what numbers to apply the M-L formula to. > > I've been thinking that we ought to try to use it in the context of the > query as a whole rather than for individual table scans; the current &g

Re: [PERFORM] anti-join chosen even when slower than old plan

2011-01-19 Thread Bruce Momjian
ay that reading 25% of effective_cache_size will be > more expensive *per-page* than reading 5% of effective_cache_size, > independently of what the total cluster size is. Late reply, but one idea is to have the executor store hit counts for later use by the optimizer. Only the executor kno

Re: [PERFORM] anti-join chosen even when slower than old plan

2011-01-19 Thread Bruce Momjian
you might want to look at my blog entry explaining why knobs are often not useful because they are only used by a small percentage of users (and confuse the rest): http://momjian.us/main/blogs/pgblog/2009.html#January_10_2009 -- Bruce Momjian http://momjian.us Enterprise

Re: [PERFORM] Running PostgreSQL as fast as possible no matter the consequences

2011-01-19 Thread Bruce Momjian
7;t care about durability: http://developer.postgresql.org/pgdocs/postgres/non-durability.html -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performanc

Re: [PERFORM] BBU Cache vs. spindles

2010-12-22 Thread Bruce Momjian
Bruce Momjian wrote: > Greg Smith wrote: > > Kevin Grittner wrote: > > > I assume that we send a full > > > 8K to the OS cache, and the file system writes disk sectors > > > according to its own algorithm. With either platters or BBU cache, > > > the

Re: [PERFORM] BBU Cache vs. spindles

2010-12-01 Thread Bruce Momjian
Pierre C wrote: > > > Is that true? I have no idea. I thought everything was done at the > > 512-byte block level. > > Newer disks (2TB and up) can have 4k sectors, but this still means a page > spans several sectors. Yes, I had heard about that. -- Bru

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
> clarification. Given that, it seems to me there are only two situations > where full_page_writes is safe to turn off: > > 1) The operating system block size is exactly the same database block > size, and all writes are guaranteed to be atomic to that block size. Is that true

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
patch, if you are using ZFS then you can turn off full-page writes, so full-page writes are still useful. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent vi

Re: [PERFORM] BBU Cache vs. spindles

2010-11-30 Thread Bruce Momjian
where I was assuming that an 8k-block based file system would write to the disk atomically in 8k segments, which of course it cannot. My bet is that even if you write to the kernel in 8k pages, and have an 8k file system, the disk is still accessed via 512-byte blocks, even with a BBU. -- Bruce

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > I assume we send a full 8k to the controller, and a failure during > > that write is not registered as a write. > > On what do you base that assumption? I assume that we send a full > 8K to the OS cache, and

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > If the write fails to the controller, the page is not flushed and > > PG does not continue. If the write fails, the fsync never > > happens, and hence PG stops. > > PG stops? This case at issue is when the OS c

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
OS cache > so that the window might occasionally be rather large? If the write fails to the controller, the page is not flushed and PG does not continue. If the write fails, the fsync never happens, and hence PG stops. -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] How does PG know if data is in memory?

2010-10-21 Thread Bruce Momjian
; we add more knobs which are this hard to tune correctly, we would > risk inundation with complaints from people to tried to use it and > made things worse. Agreed. Here is a blog entry that explains some of the tradeoffs of adding knobs: http://momjian.us/main/blogs/pgblog/2009.h

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Kevin Grittner wrote: > Bruce Momjian wrote: > > > With a BBU you can turn off full_page_writes > > My understanding is that that is not without risk. What happens if > the WAL is written, there is a commit, but the data page has not yet > been written to the controll

Re: [PERFORM] BBU Cache vs. spindles

2010-10-21 Thread Bruce Momjian
Scott Marlowe wrote: > On Wed, Oct 20, 2010 at 8:25 PM, Joshua D. Drake > wrote: > > On Wed, 2010-10-20 at 22:13 -0400, Bruce Momjian wrote: > >> Ben Chobot wrote: > >> > On Oct 7, 2010, at 4:38 PM, Steve Crawford wrote: > >> > > >> >

Re: [PERFORM] New wiki page on write reliability

2010-10-21 Thread Bruce Momjian
dded into there once > he's finished tweaking it. My presentation is done and is now on the wiki too: http://momjian.us/main/writings/pgsql/hw_selection.pdf -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's

Re: [PERFORM] Slow count(*) again...

2010-10-20 Thread Bruce Momjian
nd costs. If the row number is large, just round it to the nearest thousand and return it to the application as a count --- this is what Google does for searches (just try it). If the row count/cost are low, run the query and return an exact count. -- Bruce Momjian http://momjian.us Enter

Re: [PERFORM] BBU Cache vs. spindles

2010-10-20 Thread Bruce Momjian
something else? > > A BBU is, what, $100 or so? Adding one seems a no-brainer to me. > Dedicated WAL spindles are nice and all, but they're still spinning > media. Raid card cache is wy faster, and while it's best at bursty > writes, it sounds like bursty writes are precisely

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-16 Thread Bruce Momjian
requirement no matter how your array is set up underneath. Do we need to document this? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (p

Re: [PERFORM] Advice configuring ServeRAID 8k for performance

2010-08-13 Thread Bruce Momjian
Linux ext3 file system. Journaled file systems do improve boot speed after a crash. Should this be changed? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + --

Re: [PERFORM] Questions on query planner, join types, and work_mem

2010-08-11 Thread Bruce Momjian
ybe refine > the CPU cost model more, as all-in-RAM cases get more common. This confused me. If we are assuing the data is in effective_cache_size, why are we adding sequential/random page cost to the query cost routines? -- Bruce Momjian http://momjian.us EnterpriseDB

Re: [PERFORM] Testing Sandforce SSD

2010-08-11 Thread Bruce Momjian
entation: http://www.postgresql.org/docs/9.0/static/wal-reliability.html -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-performance mailing list (pgsql-perform

Re: [PERFORM] Need help in performance tuning.

2010-07-12 Thread Bruce Momjian
g in > performance on workloads where queries are big and expensive but there > are relatively few of them running at a time. Agreed. We certainly are going to have to go in that direction someday. We have TODO items for these. -- Bruce Momjian http://momjian.us EnterpriseDB

  1   2   3   4   5   >